If you are reading this article you likely appreciate that poor data quality in business systems can have negative consequences. To avoid the financial costs and poor customer relations that can occur your goal is clear, you want Clean Data!
There are going to have to be two separate phases to achieving this goal.
Phase 1: Data cleansing
Phase 2: Maintaining clean data
There are many ways in which data can be in a poor state and the impact it has on a business will vary from one to another. For a business dispatching parcels the customer addresses must be correct but for a business dealing with customers purely electronically, their physical location is less important. Every business must determine the data that is key to them.
Some general recommendations on gathering key data and how to maintain good quality data can be found in these related articles: 10 Top Tips for Good Quality Data and Data Entry Standards – Guidelines and Examples. However, there is one cause of problems with data that is problematic to all businesses, that is duplicate data. In this article the focus is on how QGate’s two products namely “Paribus Discovery” and “Paribus Interactive” can clean the data and keep the data clean.
The two products line up neatly with the two phases we have identified. Paribus Discovery is a Cure to the data duplication and Paribus Interactive is a Preventative measure to stop the introduction of duplicates moving forward.
|Phase 1:||Data cleansing||Paribus Discovery||Cure|
|Phase 2:||Maintaining clean data||Paribus Interactive||Prevention|
Paribus Discovery is deduplication software. Within the product are some standard searches but the searches can be tailored to suit each business. The searches use a matching algorithm named QMatch+ that will recognize potential duplication, including those that would be missed through a standard SQL query. Potential duplicates where one of the records may have been misspelled, had abbreviations used or short names entered will be flagged up in the output from the search. Further details on the matching capabilities can be found in the What is the Paribus Data Matching Engine article.
The list of potential duplicates can be reviewed and then action taken to dedupe the system. For Microsoft Dynamics 365, Infor CRM the software can resolve the duplicates by merging the records and realigning related records. For other systems, an export of the results including the database key fields is available so that further action can be taken outside of Paribus Discovery. Further details are available here.
Once the duplicates have been removed the goal will have shifted to maintaining a duplicate free system. Paribus Interactive facilitates data entry for Microsoft Dynamics 365. At the point of entry, a check using the intelligence of the QMatch+ algorithm is run against all existing records and potential matches are made visible to the person seeking to add the record. At this point they can navigate to an existing record or push ahead with adding the new record. In this manner, the addition of duplicates is avoided. As there is no blocking mechanism, duplicates could still be added but there would have to be a conscious decision taken to do so. Periodic re-runs of Paribus Discovery searches could be included in the ongoing maintenance program to maintain a duplicate free system. Further details are available here.