Old Possum’s Book of Practical Big Data

OK, the title’s a cheap riff on the T.S. Elliot book of poems that inspired Cats. You have to start somewhere and that’s as good a place as any — but stay with me, this goes places. A big group of cats is called a “clowder.” What if we could access a clowder of big data?

Big Data has been taking up a big part of my conscious life lately, what with all the analytics vendors out there and so many companies trying to figure it all out. There are at least three issues that converge when you discuss big data — two that we know, and one that we don’t pay a lot of attention to just yet.

The two devils we know are physical storage and analytics, and many people stop there. Storage has largely been taken care of with dirt-cheap spindles, the cloud and other advances. Analytics is an old story that’s gotten better year in and year out. Huskier processors, bigger spindles, and in-memory databases have made real-time slicing and dicing easy.

Thinking Bigger

The other day I spoke with Alan Trefler, CEO of Pegasystems, who told me that software and hardware have advanced in equal measure over the years, to the point that today we can do quite a bit of analysis in very little time.

Batch pattern analysis gave us the input we needed to get predictive — to assign probabilities to situations based on prior experience — and that has given us the ability to stack rank ideas and offers, and generally to be able to say “this is the next best thing to do or offer — not always, but in this particular situation.” It gives us the ability to (sort of) be in the moment with customers.

Predictive modeling has been a great way to enable companies to better understand customers and their needs. Based on company-gathered and maintained big data, we can confidently deploy systems that suggest to employees what to do next. In case after case that I listened to in Orlando during Pega’s user meeting, Pegaworld, I heard of huge improvements in business process results, in part due to leveraging analytics and big data — so good for them.

But please pay attention! In these last few paragraphs, we’ve traversed the long path from data to information. Did you catch it? Maybe not. Analytics turns data into information, and people (usually) turn information into knowledge. Our systems serve up information, but our people — our employees — apply that information in customer-facing situations to make decisions that achieve desired outcomes. There’s nothing more important than good decisions in business today, which is why we are fixated on big data and analytics — but we can’t stop there.

It’s time to think bigger. What would it be like if we could amass more data than a single company typically captures for its analysis? Naturally, this assumes all the data is relevant to a set of business processes. It’s very hard to do something like this. Some vendors I’ve spoken with about this say that the data lives in many places, and consolidating it is not necessarily quick or easy.

One place where it might be easier to accomplish this kind of Major Big Data consolidation — can we start calling it Major Knowledge? — might be in SaaS applications. Of course, not just any Software as a Service will do, but those systems that operate on multitenant storage might have an important leg up. A company like Salesforce, NetSuite or Xactly might be a good place to look. In a recent conversation I had with Chris Cabrera, CEO of Xactly, I heard his team was thinking about what that would look like.

You may recall that Xactly focuses on incentive compensation management. Xactly collects data that focuses on sales people, deals, credit (for partial deals), compensation, and much more. If properly scrubbed so that all identifying data is removed, this database would be capable of revealing all the best practices information in sales by examining the way that people are compensated — no small accomplishment.

In the right hands, that type of information can become powerful knowledge. I know some of you are saying why not look at other data — like revenue, or stack ranking the reps, or any of a thousand things?

Too Many Slices of Pie

The answer is simple. Other data won’t give you the answer. Incentive compensation is an art at the moment (unless you already use a system like Xactly) and relying on a single data set might only reinforce bad practice. Capturing data from a wide body of knowledgeable sources is, after all, one of the hallmarks of crowdsourcing.

People do their jobs, and they get compensated, and someone is the top rep and someone else is at the bottom. However, these rankings don’t say anything about whether the incentive compensation was really an incentive, whether it caused people to modify behavior.

So what? you might add — but wouldn’t it be nice to know if the incentive were both effective and in the right proportion? What do others do? Effectively, what’s the best practice? Those are questions you can’t answer if all you’re looking at is your own data.

Sales comp is a big hairy issue. It’s estimated that in the United States alone companies spend UA$800 billion per year supporting sales-based incentive compensation.

It’s messy and full of 25 percent credit for this deal and 65 percent credit for that; it also has accelerators and clawbacks — so how do you get it right? For example, Xactly found that 75 percent of its customers were crediting five or fewer individuals on a deal, but some outliers credited up to 161 individuals on a single deal. Splitting a deal 161 ways can hardly be motivating.

Many are the stories of sales people who left a job because they felt underappreciated (i.e., undercompensated) or just thought they could get a better deal elsewhere. Having access to information based on such a huge volume of data might give every sales manager and HR or finance department the concrete information necessary to do a job that’s now largely guesswork.

Sales compensation is not the only area worth exploring with this approach, either. The same techniques can be applied to every job category — and if you’ve looked lately, you will have seen that many jobs are coming to have incentive compensation as a part of the package. In fact, an estimated 84 percent of companies today are now using some kind of incentive compensation outside the sales function.

If you ask me, there’s never been a greater need for the kind of data resource pooling and analysis I am proposing here, and it’s easily within our reach. We just have to think a bit different about the challenge at hand — how to attract and retain the best talent — because business today is less about your widgets, and more about the quality of people you have representing them.