Why Chatbots Are the Future of Big Data
By Tim Scargill
Almost exactly a year ago, Facebook launched the Messenger Platform, allowing developers to create their own chatbots. For many people that was the first time they had even heard of chatbots, but Facebook and others now proclaimed them as 'the next big thing', conversational assistants that would revolutionize how businesses communicate with their customers. But where are chatbots now? Do they still have a role to play? Here is how they are set to transform enterprise, and why they are the future of big data.
Chatbots are the New Apps?
In the wake of Facebook's announcement, many have repeated the mantra 'chatbots are the new apps' (a claim made by Microsoft's CEO Satya Nadella in March 2016). But despite solid statistics identifying the slowdown of the app market, the growing use of messenger apps, and the fact that there are already tens of thousands of Messenger bots out there, the revolution that was promised is yet to arrive. I mean, how often do you use a chatbot instead of an app?
The problem is that chatbots have yet to find their place in the world. Many are gimmicky, or simply provide the same functionality as an existing app with a different interface. This is a bad design approach to start with, and the truth is that for some tasks a graphical user interface is actually more intuitive anyway. The idea that chatbots will replace apps is a good headline, but it's unhelpful- to increase adoption rates developers have to create chatbots that do things apps can't, or at least concentrate on use cases that suit their conversational nature.
So where does that leave chatbots? Just as apps didn't replace websites, successful chatbots won't be built primarily to replace apps, but for the tasks to which they are best-suited. For example, the strength of a conversational assistant is its ability to replicate a human, so it is the human interactions between business and customer that they will replace. In fact, we are already seeing this in the growing use of chatbots in banking. The way chatbots can answer queries allows them to give the impression that a bank really knows their customer, restoring the 'personal touch' that has been lost in recent years. The same is true for other industries - it is in the areas of customer service, product selection advice (think expert recommendations about fashion, travel etc.) and customer relationship management that chatbots can currently provide real value.
Natural Language Processing
A successful chatbot involves a large amount of data exchange between business and customer. However, due to the nature of those interactions, the information provided by humans is complex, unstructured text. So how is that data analyzed to be sure that queries are responded to correctly? It is recent developments in natural language processing (NLP) that are making it possible. Just a few years ago it was seen as a fringe technology with frustratingly low accuracy, but big strides have brought it to the mainstream. Ready-made NPL platforms from the likes of IBM, Microsoft and Google are available to developers, ensuring that any chatbot can take advantage of them.
One example of how far we've come is how chatbots are being used to provide legal advice. Analyzing linguistically complicated text about a user's circumstances, with a sufficient level of accuracy to respond correctly, would have been impossible until recently. But Stanford student Joshua Browder has developed a 'robot lawyer' that is helping people challenge parking fines, apply for emergency housing and even claim asylum.
What is important for enterprise to consider is that the power of natural language processing can be leveraged not just to improve the functionality of a chatbot, but to further analyze the data being exchanged. That unstructured text is a potential goldmine of information, that (as long as data privacy issues are addressed) should be fully utilized.
Insights for Enterprise
There are a range of potential insights that may be gained through chatbots, spanning both business analytics and business intelligence. One emerging area of business analytics is predictive analytics, being able to predict what your customer wants before they ask for it. By analyzing the trends in conversations (along with the type of customer involved), a chatbot is able to make informed, targeted suggestions about what to buy, increasing the chance of a sale. Fashion retailers such as H&M and American Eagle are already doing this, through the messaging service Kik.
Those targeted recommendations are just part of the current trend in marketing towards personalization, as a way to improve the customer experience. A Gartner report on personalization found that there is an increasing demand for it among consumers, who are often willing to pay more for a service that doesn't just treat them like any other customer. Building up knowledge of someone through their chatbot interactions can help businesses achieve that, to the extent of only contacting them at the times they are active on a messaging service.
In terms of business intelligence, chatbots provide a fast, effective method for upper management to query aspects of their business and inform their decision making. Most significantly, management can make those queries directly, without using technical language. The ability to have the most up-to-date reporting (on KPIs and other metrics) available on demand at any time could be a key advantage.
Sentiment Analysis and Customer Management
But the opportunities extend beyond customer data analysis as we currently know it. Sentiment analysis (determining the feelings of a speaker from their text) is an established tool on social media, used to analyze opinions about a product or service. This can be employed in chatbot conversations to determine when a customer feels positive (implying that the service has been successful and met their needs) or negative (implying that they are dissatisfied with the level of service). That is particularly helpful in predicting when a someone is about to 'churn' (stop being a customer) - a special deal could be offered at that point to encourage them to remain for example.
In fact, sentiment analysis can help companies understand the feelings of a customer throughout their entire lifecycle. Knowledge of which parts of the service work for different people, and which parts of the service influence a decision to complete a transaction, is invaluable for businesses. Given that there is significant room for improvement in understanding the full spectrum of human emotions (made more complex by constantly changing vernacular), there is huge future potential in the insights that customer conversations give us.
It is clear that communicating with customers in a conversational format has advantages for enterprise, from the ability to learn much more about specific customers, to providing another source of information regarding general customer trends and opinions. In the future, as natural language processing and other related technologies develop, it will be possible for chatbots to be used for more and more tasks that are currently completed using websites or apps.
At a certain point their human-like interactions will become so advanced, and their use so widespread, that we won't even consciously think about the fact we are interacting with a machine and not a human (the equivalent of chatbots passing the Turing Test). As that happens, it is not unreasonable to assume we will become more trusting of them, and interact with them as if they were friendly, human assistants. This can only lead to our being willing to share even more valuable data with them.
There is a growing awareness of the importance of big data in enterprise, and it should be noted that the data we get from chatbot conversations will not replace that which we get from other sources (such as social media). However, in the coming years it will form a huge new source of big data, which businesses will have to take advantage of in order to stay competitive. Business analytics and intelligence based on natural language processing should be a key part of strategy going forward.