Making Live Chat Human: Optimization vs. Sentiment Analysis

In Artificial Intelligence, a lot of people extol the ability to automate repetitive tasks.

I’m more interested in how AI can enhance human decision making. At RapportBoost.AI, where I’m Co-Founder and Chief Data Scientist, we leverage Artificial Intelligence to help online brands communicate more effectively with their customers via live chat.

In this post I’m going to talk about three different AI technologies available to companies to enhance their live chat implementation.

Photo by James Pond on Unsplash

AI provides insight on a customer's emotions

First, I want to make clear that when I’m talking about how to apply Artificial Intelligence to improve Live Chat, I’m not talking about giving humans’ jobs to machines. [We've already seen that bots aren't quite ready to handle the job of a human.] I’m talking about how we can use AI to help live chat agents be better at their jobs.

When communicating using live chat, a crucial range of contextual clues normally present during human conversation are nowhere to be found. There’s no way for a live chat agent to read a customer’s body language, to take in environmental clues that may contribute to a customer’s emotional state, or to pick up on the conversational nuances that allow customer service representatives to build rapport with customers.

When implemented well, Artificial Intelligence technologies make up for this missing information by giving live chat agents recommendations on how to talk to the customer. Because it’s not just what the customer service agent says, it’s how they say it that ultimately determines the outcome of a live chat conversation.

Defining AI

Let’s start by defining the major language-based Artificial Intelligence technologies that you might consider applying to a chat analysis:

Live Chat Optimization: The act of analyzing your chat and messaging data in relation to Key Performance Indicators including sales, conversions, and customer satisfaction, and making suggestions to live chat agents about the language they use to lift KPIs. For example, RapportBoost.AI recently completed an analysis of chat language for a customer of ours and discovered that if they use 5% less formal language (e.g., Hello vs. hi, thank you vs. thanks), customers would upgrade their accounts 4.35% more and they would earn an additional $74K.

Natural Language Processing: Any method of processing the unstructured data present in natural language. In conversational commerce a live chat agent uses entire phrases. For example, a parts of speech (POS) tagger might take a sentence like “Hi, I’d like to inquire about the status of my order.” and break it down into its component parts of speech: Hi (noun), I (pronoun), ‘d (verb), like (verb), to, inquire (verb), about (preposition), the (determiner), status (noun), of (preposition), my (pronoun), order (noun).

Sentiment Analysis: A machine learning approach to taking unstructured language that has been labeled with its sentiment (e.g., positive, negative) and building classifiers that attempt to predict that sentiment. For example, Python’s NLTK text classification tool takes a sentence like “It was a pleasure speaking with you!” and classifies it as positive (0.8), negative (0.2) with a polarity of 0.6.

While Sentiment Analysis sounds great in theory, it’s important to be aware of the caveats. Although a variety of platforms such as Google Cloud Natural Language API and Microsoft Azure now make it relatively easy to engage in sentiment analysis, these tools have limited ability to lift KPIs. Many live chat apps now implement some form of sentiment analysis, either through predictive analysis based on previous conversations/interactions, or allowing the visitor to choose an emoticon before the conversation that indicates their emotional state.

One dirty little secret is that the majority of these sentiment analysis tools are based on the tags connected to movie reviews. As part of the Stanford Core NLP, researchers used language from movie reviews on Rotten Tomatoes and IMDB to develop a Sentiment Treebank. Although movie reviews are readily available sources of unstructured data that have been labeled with the writer’s intended sentiment, a sentiment analysis tool based on movie reviews is not the best fit for a brand looking to improve consumer experience.

Understanding immediate drivers of consumer behavior

Furthermore, live chat optimization isn’t just about analyzing sentiment, it’s about understanding the immediate drivers of consumer behavior — whether a consumer decides to make a buying decision and whether that consumer becomes a repeat customer. Large brands that are determined to build relationships with their customers need to know more than the customer’s emotional state. They need to understand the hundreds and thousands of factors that occur during a conversation that influence a consumer’s behavior. This difference is key.

We’ve done a variety of studies on conversational commerce that look at how different aspects of live chat agent behavior affect a customer’s Key Performance Indicators. Check out our study, How Much Does the Conversation in Conversational Commerce Matter?, for primary research on this topic.

We’ve found that sentiment accounts for 5 to 10% of the purchase outcome – not negligible but not huge. We have also utilized APIs that measure the customer’s personality and have found that those variables collectively account for 15 to 25% of the outcome –roughly 3x the influence of sentiment.

That’s especially true when we combine personality variables with other indicators like emoticon usage, spelling, grammar, and capitalization. For example, it’s possible to identify strict grammarians by analyzing live chat data and, for those people, it’s far more important for a live chat agent to be grammatically correct than it for an agent to monitor the customer’s emotional state.

Finally, there is huge variation in the efficacy of the sentiment APIs out on the market: not all sentiment analysis tools are created equal. Over the two years it took to build our platform, RapportBoost.AI tested a half-dozen of them. Some were designed incredibly well, and some were no better than guessing. Some of this may be attributable to the level of rigor that went into designing those tools, but some is the inevitable result of the fact that movie reviews are very different from chat data, which is far messier and often subtler.

If you’re thinking about how to turbo-charge your chats with customers to drive a specific behavior, sentiment analysis may be a part of the solution, but it cannot be the entire solution. Yes, brands do want happy customers, but they’re typically more interested in customers that will tell their friends about the amazing experience they just had engaging with a company’s front line agents. If we want to drive that behavior, we need to build our models off of that behavior instead of the emotional cues that partially drive it.

Interested in learning more about live chat optimization? Stop by Rapportboost.AI and talk with their team.