Let’s Hear It for Metadata

It is a quirk of the data-information-knowledge continuum that data perceived by one person might be seen as information by another. The same goes for information and knowledge — the frame of reference is important. You might even say it’s all relative, and I don’t think you’d be wrong.

This conundrum is all brought into sharper relief by yet another slice of the range, and I don’t mean Big Data — that’s just marketing. The other division is metadata because it sits right on the cusp between data and information. For some people, such as the NSA, metadata is literally information.

The NSA sifts through phone records, for example, not always trying to read or interpret every phone call. Instead, it sifts through calls’ metadata — things like the number called, the country and the length of the call. They use the metadata as kind of a first sort through a mountain of call data to determine what calls are “interesting.”

Before I go any further, I have no inside knowledge of the NSA — I am just going by what I read. Also, I am using the NSA as an example to make a broader point about metadata, not because I’m trying to get on the news. OK?

Better Organizational Decisions

In any case, this example is relevant to our business lives and to how we process the mountains of customer data that we routinely collect from websites and social media. The metadata — or data about the data that we collect — can be very revealing. Things like an IP address or a phone number can be stand-ins for a company and its location and possibly even an identifier of an individual.

We instinctively dislike knowing that a little bit of data, or in this case metadata, can be so revealing about us, but we also rely on this kind of sleuthing to get our jobs done in an increasingly interconnected world. Very often collected data doesn’t get the job done: It doesn’t add up to the information we need on its own, but rather needs to be cross-referenced and completed. Surely and swiftly, a crumb can be turned into a loaf, something we can work with.

There’s a thriving industry that collects and catalogs commonly available data so that others can perform the cross-referencing and develop not just information, but the knowledge with which to make business decisions. Often we don’t give enough consideration to the data industry, at least not until we need something — then it becomes very valuable. Or, we focus on ETL only — extraction, transformation and loading, the round trip from dirty to clean.

That round trip merely gives you more confidence in what you already have, however. It doesn’t enhance anything. You might end up knowing the correct spelling of my name, for instance, but you still might not know if I have purchasing authority or if my credit is any good. That’s where cross-referencing can be so important.

In a very real sense, verifying metadata can do a lot to improve the information content of the data we collect and help drive better organizational decisions that are based on the knowledge inspired by the data we collect.

I think understanding the relationship between data, metadata and information might be a critical next step for many people trying to make sense of all the data we collect. Understanding its importance opens up the discussion and helps us to move beyond the simpler discussions of collection and storage, or even manipulating it with technology vs. using it to get something done.

New Ways to Compete

Finally, metadata is not simply useful in the early stages of collection and identification of business opportunities. Metadata is also driving how we develop and integrate business applications. In this case, the metadata in question is more related to process than it is to identifying people and business opportunities, but it’s still worth considering its place. Metadata is driving the need to integrate platforms instead of applications — there’s so much of it, and it’s so important to business processes, that simple interfaces can’t do the job.

Metadata is often associated with business processes in workflows. An order isn’t an order until there is metadata about inventory (is it in stock and ready to ship?) attached to it. Same goes for the customer’s creditworthiness — a lot goes into that simple line item on the invoice, even though it’s not all represented there.

So, it really is true that your data might be my information or even knowledge. This makes understanding the data revolution so important, and it gives clever companies new ways to compete.