Data Quality

The problem is, product data is scattered across multiple data silos. Data is not consistent and changes in one system sometimes do not sync with other related systems. In addition, duplicated data and outdated data can completely offset your analysis, therefore any decision made based on inaccurate data might not be the best decision.

Data quality cannot be verified in isolation, we can only ensure data quality when we verify the data relations with the entire system. Verifying data relations in large scale databases can cause performance hit, increase resource consumptions and delay realtime operations.

Why having a Duplicate Checker is so important?

Duplicated product data can offset your analysis and affect your decision-making process. Worse of all, it can create all sorts of bugs. Duplicates can break the “single source of truth” promise and put your company back in the dark ages with the same data scattered.

How duplicate data appears?

The most common reason is human error, however, sometimes the root cause of the problem can be found in ERP, MES or other systems. You will be surprised to see how many errors your ERP and MES systems already have. Upon importing data, DUGAA Data Quality Checker leaves a human-readable detailed log and will explain why and where the problem exists.

Technical Challenges

Preventing duplicates in 1 to 1 field mapping is simple and it is offered out of the box from many database vendors. Simply set the desired field to unique when creating your database schema. A -> A

The problem begins when the device serial number appears as a component serial number or the component serial number appears as a sub-component of another product.
A -> B1
A -> B2 -> C1
A -> B2 -> C2
A -> B3 -> C1 -> D1

A simple device may have thousands of components and sub-components. The incoming data can be one device or a million devices at a time. This is where we used to hit the mathematically impossible number of queries (Query explosion).

To make the matters even harder, just imagine during the repair process, one component is taken out of a device and it is replaced with a new one, the same component will be refurbished and will continue its life in a totally different product.

What we can do in a single-digit millisecond

At DUGAA, we use our proprietary algorithms and technologies to tackle big-data query explosions. We ensure data is unique and consistent at the sub-component level.

  • We can process each device in a single-digit millisecond.
  • In case of any error, the system will generate a human-readable report and explain where exactly the duplicated record exists.
  • In addition, the system offers recommendations, explaining your options to resolve the duplicated problem based on various factors such as component dependencies and component lifecycle stage.