Text Mining: The Power of Words

“This phone stinks!”

“I don’t want to be a nuisance, but I wanted to let you know that I’ve had a lot of dropped calls over the last two weeks.”

“This is the worst phone I have ever used!”

Maybe you called your telecom provider in the last year because you were dissatisfied with your phone or service. Maybe you even used words or phrases similar to the examples above. Maybe since then you defected to a new cellular provider.

Not surprisingly, service centers get a lot more calls with complaints and criticisms than they get showers of praise. All telecom companies experience churn rates way above those of many other industries — it isn’t unheard of for 30 percent of an average telecom company’s customers to switch providers in the course of a year.

Warning Signs

In the struggle to retain customers, companies strive to know ahead of time who is at risk of leaving. Look at the snippets of service center conversations above — these are customers telling their supplier, before they make the decision to leave, how unhappy and dissatisfied they are.

So, can companies use this data to aid their retention efforts?

The simple answer is “yes,” but it’s only recently that this has happened. Historically, most notes from call center conversations have been ignored. Some may be read through for “forensic” reasons when particular problems have arisen, but the vast majority just sit there — unread.

Yet, here we have customers talking to us in their own words, giving us invaluable insights into their feelings and intentions.

Why haven’t companies exploited this valuable resource of customer insight in the past? Because, put simply, it has been too hard to do. To get value out of text comments, you have to read them, but with even moderately sized contact centers taking millions of calls per year, that’s an impractical approach.

Gathering Information

It’s only in the past few years that a technology has emerged that makes it feasible to extract the relevant information from these huge numbers of conversation notes that would otherwise gather dust. Text mining unlocks the value held in an organization’s free-text assets.

Approximately 80 percent of an organization’s data is contained in free-text form. If organizations were to rely solely on traditionally structured, column-and-row data, such as transaction records and campaign responses, critical business decisions might depend on only 20 percent of available data.

Although the first-generation tools were relatively primitive — little more than “smart search” — text mining technologies have vastly improved in recent years, both in their linguistic and statistical analytic approaches, and are now able to extract information with unprecedented accuracy.

This text extraction presents analysts with a clear picture of past and present data, highlighting concepts and opinions. Those concepts and opinions can even be combined now with emotion analysis extracted through advanced voice recognition systems.

As telecom competition increases, providers are continuing to try to get as complete a picture of their customers as they can — and they’re spending millions of dollars to do it.

Text mining has emerged as the next big application for understanding customer behavior by analyzing key textual information, such as call center logs, open-ended customer survey responses and, now, blogs.

Marketers can gain insights into why customers are leaving, and the key “concepts and opinions” extracted from text notes can even be fed into predictive models to generate churn “scores” that can be applied to current customers and estimate the likelihood of their leaving.

More Than CRM and Marketing

Text mining alone — though it’s a powerful tool that opens up new possibilities for CRM and marketing — isn’t enough, however. The best and most accurate decisions are based on a holistic view of the customer. Their call center conversations, while important, represent only a modest portion of all the customer information a telecom company can pull together.

How do these companies integrate and leverage their data assets — structured and unstructured?

Instead of simply finding and neatly displaying specific pieces of information out of large amounts of text, text mining takes this text and turns it into structured row-and-column data, which is then ready for analysis just like any other traditionally structured information.

Once the essential concepts and opinions have been mined out and put into a structured format, this newly structured data can be freely combined with the corresponding structured customer data coming from existing database stores.

This holistic data view will typically include behavioral information (such as patterns of calling and phone usage); demographic data, either self-declared or brought in based on ZIP code; attitudinal data obtained through surveys and other feedback solicitations; and data about the customer’s use of other channels, such as Web-based services.

The composite data is then fed through the same predictive modeling algorithms used for traditional data mining, creating accurate models of which customers are high value and which are about to defect. Once these models are deployed, they can provide real-time on-screen recommendations for agents as they talk to customers or provide marketers with focused customer segments.

The 360-Degree View of Customers

Text mining on its own is exciting and delivers previously unobtainable value by unlocking an organization’s unstructured data.

Its greatest value, though, comes when it isn’t held and analyzed as simply a silo of customer data. It is a key component in completing the 360-degree view of customers that ensures deeper insight and understanding, and an improved ability to predict customer behavior and drive interactions for optimum outcomes.

Incorporating text doesn’t simply add “a bit more data.” By giving insight into what customers say in their own words, it can even boost the performance of predictive models which were already delivering quite high accuracy from structured data.

Adding text from call centers has improved churn model accuracy by as much as 20 percent. In an industry like telecom, where each fraction of a percent improvement in customer retention can equate to millions saved each year, the ROI from adding text mining can be huge.

E-commerce — and everyone will benefit if it’s successful in that goal alone.

Colin Shearer is senior vice president of market strategy at SPSS.