Validating Data in Oracle Retail’s Markdown Optimization Solution

This document outlines the metodology followed in validating data for Oracle Retail’s Markdown Optimization solution before loading it into the system. MDO ha an inbult mechanism of validating the client data once loaded into the system. In this case the data validation and data loading happens simultaneously. In this approach we have tried to segregate data validation and data loading in MDO. This approach reduces time delays and would also ensure that only the correct data gets loaded into the system which in turn would reduce the system overload.

Current Data Validation Methodology

Currently the data loading methodology followed in MDO is illistrated in the picture below: The data in MDO comes in from of flat files (*.dat) and goes through the following stages before getting loaded into the system.

1. Flat files come in format as specified in the standard guide
2. Flat files loaded into staging tables using standard sqlldr script. In this stage only file format is verified i.e. validation happens only to check if the format of the file is correct and there are no special characters.
3. The standard load script is then run on the staged data.
4. In this step all the data is validated. Only the valid data gets loaded into the system. All the individual data goes into tables suffixed with _BAD. For example during the calendar load the staging table is ASH_CAL_TBL during load all the data gets loaded into CALENDARS_TBL whereas the bad datat goes into ASH_CAL_TBL_BAD.
5. Share the results of the Data Validation with the Client.

In this method the validation and loading of the data happens simultaneously. Hence the user gets to see the validation results only after the entire load procedure is done. If you would like to know more, check out our whitepaper below.