Where Do Duplicates Come From?

We all know that having duplicates within our customer base is a bad thing. But where do they come from?

Firstly lets be clear what we mean by “customer base”.

All businesses deal with people. And a person may be a customer. Or they may be a prospect. Or a business contact. Or a recipient of goods or services. Or a dormant customer. Or….you get the picture.

So the entity “person” should be considered as part of the businesses Master Data and managed accordingly. Alas this is not always the case so lets have a look at why a business may end up with duplicates within this important data.

1. You purchased them. Perhaps the Marketing Department have purchased a list of prospects from a third party supplier. If care is not taken to exclude existing customers it is likely that some people on this new list already exist within your customer data. Treating an active customer as if they were a prospect i.e. trying to sell them something they already have is never a good thing.

2. Mergers & Acquisition. Where businesses merge, the likelihood is that they will share some of the same customers. So if the process of integrating the businesses fails to identify the presence of the same person in both data sets then you will end up having multiple entries for the same customer with all the overheads that ensue.

3. Data entry points that are not “joined up”. This is the classic data silo situation where the different functions within the business operate independently. So Marketing may create a prospect entry in their database. Sales may create them as a customer in their database. Accounts do the same and so on.

4. Data entry points with inadequate validation. Customer data can be created through a website (on-line registration), via in-house services (telephone enquiries etc), batch data entry. Each data entry point ought to perform appropriate testing (may be auto or manual) to ensure that the person data being entered does not exist within the existing person “universe”. If they do not then they can be created anew. If they do already exist then either they are discarded or the status of the existing entry can be updated. But identifying duplicates within customer Name and Address data can be very tricky (see my previous blogs on this subject). So if care is not taken duplicate entries can “slip through the net”.

5. Movers. Most data matching software packages will utilise the customer address as part of the matching rules. So if an existing customer moves house and registers as a customer at their new address, the chances are they will be created as a new customer and all their previous purchase history will be lost. This can happen if you do not provide the customer with some sort of unique reference number (customer number, account number, loyalty scheme number) and incentivise them to use this when interacting with your business.

Customer (or person) data by its very nature is one of the most difficult to manage effectively. It tends to be captured by people (rather than machines). It contains valid variations (i.e. Bob / Robert). It will contain spelling mistakes (Andrew / Andrwe). Different sources will be structured differently. It changes over time – people move house, people change their name, Postcodes can change, contact details can change. So managing customer master data across an organisation is extremely challenging.

If your business is troubled by duplicates or you are concerned that you are not managing your customer data effectively then please get in touch. I will be able to help.



This entry was posted in data matching, Data Quality, Data Strategy, entity resolution, Golden Record, Project Management, Single Customer View, Uncategorized. Bookmark the permalink.