|Summary:||This article is for partners and provides guidance on how to audit their customer’s Dynamics 365 system for data duplication using Paribus Discovery.|
|Article Type:||How-To Guide, Technical Article|
|Related Product(s):||This article relates to the following products:
The purpose of this document is to provide partner guidance on how to audit your customer’s CRM system for data duplication using Paribus Discovery.
Paribus Discovery – The Identification & Review Process
Let’s first give you an understanding of how the system architecture manages this. In the data deduplication process, there are essentially two stages.
The first is to identify where you may have duplicate records in your system and once you have selected which records you want to merge, you then would then initiate the second stage of merging those records. Fig 1 illustrates how that review process is managed.
Fig 1 – The identification process
After installing the Paribus Discovery Client Software on your PC we have to establish the Source of the data for the merge.
A data extraction utility is used to extract Account, Contacts & Leads data from your online or on-premise CRM, in to a Source Extract Database. I
To enable the data matching review to be conducted a Control Database is created. This Control Database houses the output of the matching process, the Match Sets & Conditions (which can be modified) and the Match Sessions, which define which fields you are matching on and to what sensitivity.
Once the data within the Match Session has been reviewed it is time to run the Merge process. As outlined in Fig 2, this is enabled through running a plug-in for the respective CRM system in Paribus Discovery.
This plug-in connects enables the Control Database to update the Live Source Database with the changes selected, through a Direct Connection via the API.
Fig 2 – System Architecture for the Record Merge
Paribus Discovery Duplication Exercise – Stage 1
Request a Paribus Discovery trial via the ‘Try it for Free’ button on our website using the following link:
We will then be in touch asking you to confirm the details of the Customer, so that we can setup a Subscription relating to the trial license. Once the Subscription has been set-up you will be sent a license key, & using the installation instructions in the automated e-mail you will have been set you can install the software.
Paribus Discovery Duplication Exercise – Stage 2
The next stage of the Duplication process is to use the Data Extraction Tool (Fig 3). This file is stored under the Data Extraction Utility folder of the software you have installed.
Fig 3 – Data Extraction – Microsoft Dynamics Only
Advice on defining the service connection to enable this data extract can be found here.
Paribus Discovery Duplication Exercise – Stage 3
The next stage in the process is to recreate a Control Database (Fig 4). See page 25 of the installation instructions to learn how to do this.
Fig 4 – Create Control Database
During the process of setting up the Control Database it will need to be pointed to the Source Database/Source Extract Database (See Fig 5).
Fig 5 – Point Control Database to Source Database / Source Extract
Paribus Discovery Duplication Exercise – Stage 4
The next stage is to import the Match Definitions (Fig 6). You will need to import the Definitions appropriate for your CRM. See page 29 of the installation instructions on how to do this.
Fig 6 – Import Match Definitions
Please note: For this exercise, it is important to use the Standard Paribus Audit Definitions that have been supplied (Sessions 1,2,3 & 11) and run the reviews against the criteria already set (see below).
Session 01 – PARIBUS AUDIT: Account Name 95%, Address Composite 95% – No Filter
Session 02 – PARIBUS AUDIT: Account Name 90%, Address Composite 85% – No Filter
Session 03 – PARIBUS AUDIT: Account Name 80%, Address Composite 85% – No Filter
Session 11 – PARIBUS AUDIT: Contact Name 90%, Account Name 95%, Address Composite 95% – No Filter
Session 12 – PARIBUS PRODUCTION: Contact Name 90%, Account ID = – No Filter
This enables a direct comparison with the results from our study. A customer can clearly see where they compare with the average levels of Duplication, and can benchmark against the best that can be achieved.
In a real scenario the Accounts would be merged first, followed by the Contacts and also the criteria used would need to reflect how the Account hierarchy has been structured.
Paribus Discovery Duplication Exercise – Stage 5
The next step is to run the match sessions (Fig 7). See page 36 of the installation instructions on how to do this:
Fig 7 – Running Match Session
Then record the results of the four match sessions in the spreadsheet supplied, update the charts and the customer output template (Fig 8 and Fig 9).
Fig 8 – Recording Match Session Results
Fig 9 – Updating Charts
Paribus Discovery Duplication Exercise – Study Results
As a result of our study it was established that with the use of Paribus Discovery it is possible to reduce duplication percentages to 1/20thth of average duplication levels experienced.
Average duplication levels are circa 4%. This is compared with levels of 0.2% that are achievable through the use of Paribus Discovery.
The conclusions of the study also suggest that there is not necessarily a direct correlation between the age & size of the database and the duplication percentage. But, there does appear to be link between the company market type and the duplication percentage. With those dealing in Business to Consumer markets generally having a higher duplication percentage than those dealing on a Business to Business basis.
Paribus Discovery Duplication Exercise – Study Implications
The implications of this study suggest that the use of Paribus Discovery can considerably reduce the amount of duplicate records in a CRM system. And whilst ‘on the face of it’ a duplication percentage of 4% might be deemed acceptable, 4% of say 15,000 records, would mean that potentially there may be 600 records in your system where you have duplication.
With duplicate records in your system there is the potential for misaligning Service & Sales data against the wrong record, double counting Sales Opportunities in your pipeline, sending information to the wrong e-mail address, sending Marketing material to the same person multiple times or indeed sending irrelevant Marketing material to those customers of yours whom may already have the products or services you are trying to promote.
This may or not be deemed an acceptable business risk, but a company has the chance to establish what level of duplication they have in their system & assess that risk.
Against the backdrop of GDPR, there are also considerations about what exposures you may have as a result of having duplicate records in your system.
Whilst there are a number of factors involved in GDPR compliance, you may find it useful to understand what role Paribus Discovery can make in the identification of data held & its’ influence on meeting the requirement of the ‘Right to be Forgotten’ this blog refers to.