A New Approach to Customer Data Integration, Part 1

Customer data integration (CDI) is becoming important for enterprises who want to target, acquire, develop and retain customers. In order to benefit from it, however, an enterprise needs to create a unified and comprehensive customer view from disparate data sources — including CRM, financial, product and external data services. Once integrated, unified customer views provide the entire organization with the ability to drive meaningful business action within and across operational systems.

As the demand for CDI grows, large application vendors have been trying to position themselves as the preferred solution providers, but their attempts often have been unsuccessful. Because these application vendors anchor their CDI solutions on their own applications (often in CRM), they lack the flexibility to become an enterprise-wide platform that delivers unified customer views across a diverse set of application systems and data repositories. Moreover, rigid frameworks sometimes limit the ability of an enterprise to adapt these solutions to the changes in data sources and IT environment that are routine in large organizations.

As a result, it is not surprising that, according to a recent survey by industry analyst firm The CDI Institute, more than 75 percent of IT professionals said that they were actively considering purchases “outside the family” to facilitate connectivity between customer-facing applications and processes.

Building and managing a unified customer view — across disparate data sources, applications and channels — has often proved to be a complex and costly exercise. In this two-part article, we will see how a neutral, rules-based approach can make the process much easier.

A Neutral Model

The most critical feature of a CDI solution is its ability to let organizations develop an enterprise-wide customer data model that can record all master customer data. In order to be effective across an organization, a CDI platform must support the broad range of data structures contained in multiple customer data sources.

Application vendors often take the approach of building an enterprise data model, often borrowed from one of their enterprise applications, in their CDI solutions. This approach requires an enterprise to adopt a pre-determined model with minor modifications. There are two key problems with this.

First, it is difficult to change or extend the data model. This makes it challenging to consolidate data from other applications and data sources that are outside the scope of the application vendor. In a large enterprise, where it is common to have customer data spread across 20 or 30 data sources, applications from a single vendor control at most 10 to 20 percent of these sources. The rest of the data sources, including most of the external ones, have to be mapped and transformed to feed data into the vendor’s customer data hub. Therefore, standardizing on the application vendor data model means more work, not less.

Second, since the application vendor data models were developed originally to support proprietary applications, these models are too complex and contain too many extraneous attributes not required for the CDI solution. This not only duplicates a lot of unnecessary data in the customer hub and impairs the performance of the system, but it also makes data mappings, data imports, and the addition of new data sources unnecessarily complex.

An alternate approach is to develop a CDI solution supporting a neutral, template-driven data model that does not lock an enterprise into a vendor-specific model. Instead, with easy-to-use tools, such a solution can conform to the specific sources that need to be integrated in the data hub. Ideally, such an approach will also offer a set of industry-specific data model templates that may be used as starting points by organizations.

This approach offers an organization many options for defining a data model for its customer hub, including:

• Using the pre-existing data model of a legacy hub
• Building a data model from scratch
• Incorporating and modifying an industry-standard model
• Selecting and modifying one of the generalized data model templates

In all four cases, the data model can be readily extended to incorporate any changes in the data sources or the addition of new data sources at any point without requiring coding.

Rules-Based Framework

In addition to supporting a neutral and flexible data model, a CDI solution should be based on a configurable framework that can easily adapt to the changing environments of organizations. A metadata-driven, rules-based framework enables the CDI solution to be easily customized to an organization’s specific data needs and adapt rapidly — without additional programming — as changes in data sources or business rules occur.

Metadata is key information about the data itself, such as who created it, where it came from, how it has changed over time, and to what extent the data is accurate and trustworthy. A metadata-driven framework captures, stores and uses metadata to ensure the highest quality, reliability and audit capability for all data stored in the system. This includes a complete history of data in each state (raw, cleansed and merged), cross-reference and lineage information for master data, as well as any changes, manual or automatic, made to the data. This framework provides a wealth of information for monitoring data quality on an ongoing basis.

To automate the consolidation of customer reference data in an intelligent and dynamic fashion, organizations need to apply repeatable business rules and conditions to their data match-and-merge processes. A rules-based configurable framework provides a rich set of tools that help to capture all these rules without custom coding.

Next week, we’ll examine why CDI systems need to ensure cell-level survivorship, manage different data types, and provide proper tools for all users.