Community-based data quality rules save resources during development
By using ready-made data quality rules, the effort for implementation is reduced considerably because the resources for development, documentation and verification are largely eliminated.
Using AI/ML can even further optimize an existing set of data rules. For example, in so-called data quality rule mining, new data validation rules can be identified by pattern analysis of data sets, which ultimately leads to an improvement of existing rules.
On average, our member companies save over 85 man-days in designing and creating data quality rules
On average, our member companies use 30% of the approximately 1,700 data quality rules. This means that business and data management professionals spend a total of 2,275 man-hours on research, documentation and testing only. These can be saved by using the already tested ready-to-use CDQ data quality rules. Another benefit is that IT saves on implementation costs for each individual rule. These often amount to several hundred euros per rule.
Continuous savings in the maintenance of data validation rules
In addition, the companies save the annual maintenance costs for these rules, which amounts to about 85 man-days per year, in the following years.
Use of data quality rules within CDQ Data Management Services
Data quality rules form the basis of our data management services. Through their use in data validation or continuous data quality measurement, they also ensure ongoing data quality assurance for shared data among the data sharing community.
CDQ currently has more than 1,700 data quality rules, which are continually being improved upon through cooperation with the companies in our community. In this way, the effort for maintenance and further development is not only spread over several shoulders, but everyone can also benefit from the know-how of the fellow member companies.
Some rules are also checked against reference sources, such as European VAT numbers in special databases or, as in the example above, the postal codes in Germany. The community also maintains its own reference data for which there are no, or no trustworthy, external sources, such as official legal forms of businesses in a particular country.
If a company has special business requirements for a certain data format for which there is no explicit data quality rule yet, our software specialists work together with the customer to develop a fitting solution. In this way, we also enable customer-specific extensions, e.g., for individual data fields that are not otherwise used by any other member of the community.
What is Data Quality?
Data quality is a measure for the suitability of data for specific requirements in the business processes, where it is used. A low level of data quality will reduce the value of the data assets in the company because its usability is minimal. Companies are, therefore, striving to achieve the quality of data required by the business strategy using data quality management.
Data Quality Defines How Well-Suited Data Sets are for Intended Tasks
Data quality characterizes the degree of how given data objects satisfy the needs (fitness for use) of consuming business processes.
In a broader sense, it refers to both the quality of data content as well as the performance of the underlying data management processes.
Data quality measurement is used to assess the data quality level for selected quality dimensions that are relevant to the selected business uses. Typical examples for data quality dimensions are completeness, consistency, validity or timeliness (Fig. 1)
Data quality is Essential for the Value of Data
Poor data quality has a negative impact on the value of data (as reflected by the popular idea of "garbage in, garbage out"). In the digital economy, the role of data is changing. Data is changing from a secondary asset that supports business processes and decision-making even to a primary asset enabling digital business strategies and business models. Recent studies identify data management and data quality as two major pain points when it comes to launching business intelligence and advanced analytics/data science initiatives.
Do you need more information about data quality in the corporate area or would you like to talk to one of our data quality experts? Just feel free to contact us!
Fig. 1: Dimensions of Data Quality
Switzerland
Germany
