Reconciliation, Merging Current View

Inventory manager sources data to the reconciliation view nightly for all of the tables from the various data sources configured right after midnight. This data is used to for validating the incremental data which is sourced near-real time nightly. Any errors in the pipeline are addressed as part of the validation using the data. The reconciliation data is kept by default for 30-day period and this time period is configurable. It is expected to record the inventory counts and reconciliation fixes within that period. The data for reconciliation is sourced into the recon schema as configured in the target data store configuration.

The data from incremental view is merged into current view periodically for all of the configured tables by data source. This merge leverages the sourced date time, database action to effectively maintain the current view to reflect the state of the records in the source system. In addition, the date column is updated to reflect the time of the update in the current view so downstream data jobs can use that to drive processing logic i.e. select data changed for a particular time window.