blog | 4 min read

Tablestakes for Taming Messy Data

October 20, 2022

Teal datawave pattern

Let's face it — the data management space is confusing. It's flooded with solutions that market themselves as customer data platforms (CDPs) but are anything but. A true CDP will work with your messy data and help you use it effectively. Being able to use your data means three things: making sense of it, learning from it, and putting it into action. In other words, being able to use your data means getting value from it. We've broken down the bare minimum a buyer should look for when shopping around for a CDP and the bonus features that will make work run smoother.

The Basic Ingredients

Security: When customers trust brands with their personal information, brands have to uphold their side of the bargain and make sure that their information is kept secure while following privacy regulations.

Table stakes

Extra credit

- SOC2, CCPA, and GDPR compliant - Keep user logs - Have a Single sign-on (SS0)

- Scrambles PII  - Resource groups for different levels of data access - Traceability graphs for data lineage - Makes DSAR requests and delete requests simple to execute by identifying all of the places a person’s PII is present in the data

Enterprise-grade: Enterprise brands have billions of data points from millions of customers in their system. They need a CDP with the ability to handle massive amounts of data quickly and at scale.

Table stakes

Extra credit

- Benefits teams across the company including IT, Marketing, and Analysts - Beneficial and clear ROI model - Handles hundreds of billions of profile, behavior, and transaction records

- Choose between Azure or AWS hosting

Identity resolution: Delivering stellar customer experiences without quality identity resolution is impossible. It’s integral to any enterprise-grade data management strategy and done right, make valuable use cases that revolve around making sense of messy customer data possible.

Table stakes

Extra credit

- Deterministic ID Resolution - Digital identity support - PII identity support - Customizable merge rules

- ID resolution separated from merge - Transparent ID graph shows raw data and confidence of every link - Machine learning algorithm with probabilistic resolution  - Stable ID ensures your IDs don’t change between runs - Merge rules can include behaviors and custom algorithms

Integration: The ability to easily support integrations that work with your data ecosystem allows different team members to use data the way they need to. For example, team members who look at point-of-sale data inputs use that information differently than someone running a loyalty campaign.

Table stakes

Extra credit

- Integrates directly with raw data - Large library of pre-built connectors - No ETL required to integrate - Real-time stream integration from web applications - Integrates directly to and from data warehouses (Snowflake, Redshift, Big Query, Azure, etc.) - Integrates directly with SAAS platforms via API - Integrates with the three major cloud platforms (AWS, Azure, and GCP) and with marketing channels eg. social media, email, SMS

- Secure data sharing integration with AWS, Azure or Snowflake - Semantic tags handle data modeling, eliminating the need for complex custom data jobs - Premium services to model raw transactions into unified models - Converts data to data models required for marketing clouds via Profile Accelerator

Data access: Teams use data for different needs — your CDP should be able to create “context-relevant” customer data assets to meet the needs of each department, brand, or country. Democratized data access helps brands make better decisions, unlock cross-brand insights, and generate loyalty.

Table stakes

Extra credit

- Analyst access to data via SQL queries - Data access for reporting tools (Tableau, Looker, etc.) - Marketer friendly segmentation experience - Support for custom predictive model attributes to be stored on customer profiles - Download lists from the UI - Sync data to on-premise data warehouses

- Automatic data dictionaries via Data Explorer - A library of out-of-the-box predictive models - Segment briefs enable customizable visualizations on segments

Flexibility: To really unlock value from customer data, your CDP needs the ability to work with data in any format, from any vertical, in any amount, and those interactions should be customizable.

Table stakes

Extra credit

- Handles online and offline data - Stores data in customizable data models that benefit the resulting use case - Stores historical and legacy data - No limits to data type - Platform supports different verticals and industries (hospitality, retail, banking, etc.)

- Every rule and attribute can be customized - No limitations to data types - No fixed data model - ID resolution customizable for any level of identity information - Custom blocklists eliminate bad data patterns - Customizable ID resolution

Transparency: A CDP should come with a fully transparent user interface that allows you to delve into every connection to give you visibility into how identities are resolved. A straightforward process builds your team's trust and confidence in your data and gives them conviction in any marketing based on that information.

Table stakes

Extra credit

- Users can see rules used for merges and see data available - Tracked app usage - See automations and job history

- See all source data AND resolved/unified data - Automatic, searchable data dictionary via data explorer - Searchable ID graphs for every link established that shows original data and confidence - Workflow management and steward interface for easily resolving common problems

Change management: As businesses evolve, they need to add new information and change the way things fit together. A data solution needs to be able to make changes in isolation, review them before they go live, and most importantly, the ability to roll back changes in case something goes wrong. 

Table stakes

Extra credit

- Ability to roll back changes - Transparency when changes have impacts in other parts of the platform - “Drafts” to make changes to live systems rather than requiring something to be turned off

- Complete platform version control - Sandboxes > Make changes to a live production environment using production data and infrastructure > Out-of-the-box “staging” environments at the click of a button > Ability to validate changes safely in an isolated environment > Easy promotion to production > Easy to roll back if something unintentionally goes to production

Expertise: Standing up a CDP can be tricky at the beginning. The right data solution will come fully equipped with a client services team who will partner with you to overcome obstacles and unlock opportunities.

Table stakes

Extra credit

- Premium services to help turn data into strategy - Attributes for common marketing use cases - Pre-built models and algorithms for common use cases - Householding modeling

- Library of out-of-the-box data models derived from across the industry for common use cases > Models can be tailored, meaning customers don’t have to start from scratch - Service teams that have solved the problem for multiple enterprise orgs to lead and maximize time to value - Premium services to drive strategy and analytics once the data is ready - Out-of-the-box QA reports for profiles and transactions - Library of out-of-the-box QA queries for answering the most common data quality problems when customizing ID resolution

Working with your data

When you start with tools intentionally designed to help you get value from your data, you’re on your way to understanding your customers’ wants and needs, personalizing their experience, and growing your business. But to start seeing results from your data, you need the right CDP — one that recognizes that messy customer data is an inevitable fact of life. A CDP should come with the right capabilities to help you navigate the marketing landscape, and be able to work with messy data rather than seeking to “fix” the data mess. Learn more about how Amperity turns messy data into value here.