Data quality rules ensure the best data quality for business partner data
Data quality rules are the hidden champions of data cleansing tools – these help improve the quality of the master data records in data quality tools. Based on different data quality dimensions, data stewards establish different quality rules to determine if a data set is of good or poor quality. Creating data quality rules can be very time consuming. An easy approach could be to adopt proven algorithms, which already work very well in the context of customer and vendor data within our data sharing community.
Today, we would like to introduce one of our virtual employees to you: Our data quality rule number 1297; "DQR-1297", for short.
"My name is DQR-1297, I have been working as a data quality rule for over 10 years in a team of more than 1300 brothers and sisters in the service of the CDQ Data Sharing Community to clean up our customers' data and keep it up to date and fit for purpose.
My main task is to identify invalid legal forms from customer and vendor organizations within all data sets. The real challenge is remembering all valid legal forms that an organization can possess around the globe, e.g., 'Fundação' is only valid for Portugal, 'EURL' for France and 'Enkelt bolag' for Sweden."
A powerful yet scalable server infrastructure provides CDQ with optimal performance and can also handle a large number of data records "just in time".
"Here is an example of my daily job: I take one of the 300,000 records from your B2B customer database. I look at the country the record is from–ah, Portugal. Then I pick up the legal form–very interesting, an LDA company. I compare it to my list of permitted legal codes in Portugal – LDA is on it! Check.
I take the next record, I find somebody from Finland. Their legal form QY is not allowed in Finland, but that’s probably a typo because OY actually does exist. I suggest replacing the legal form."
In the cloud engine of the CDQ Data Sharing Community, over 1,300 data quality rules such as DQR-1297 ensure that the customer and vendor master data of our customers always remains clean and up-to-date; even for bank data! In this way, they avoid negative impact from bad data quality like wrong deliveries, bank fraud or unpaid invoices.
"By assuring that there is no incorrect or missing legal form within the data records of our Data Sharing customers, me and my dear colleagues like DQR-1928 help them avoid having incorrect invoices, fewer tax issues and ensure correct shipping deliveries.
Data Sharing not only saves a lot of money, but it also protects the employees of our organization from any negative consequences, especially in the context of bank account data. It’s a great feeling to be part of this team!"
DQR-1297 currently works in a team with more than 1,300 data quality rules for the members of the CDQ Data Sharing Community. Our AI-based rules measure the actual gap between reality and IT (SAP, CRM, etc.) in data quality. To do this, however, the AI needs continuous feedback from this reality in order to constantly improve and develop itself.
"Sharing my work within our data sharing community is in my DNA because we data quality rules are created by learning from the community. The bigger the community and the more feedback we get, the faster we grow.
We use our knowledge to analyze the data in the cloud and send reports and updates to the community, which in turn provides us with feedback. This closes the circle and we become even better over time."