The Warehouse Builder
This is a user configured tool for consolidating a set of flat files into relational databases. It includes a tool for editing warehouse metadata. As with all the modules that comprised Naxian, it was written in Delphi and generated stored procedures to create and populate databases from user inputs filled in on a front end with additional information being sourced from the metadata. It was able to run interactively or in batch mode. Procedures were generated for MS-SQL6.5, MS-SQL7.0 or other database specific language, for example Oracle or Interbase, depending on a user selected option.
The user provided:
- Datastore name(s)
- Server name(s)
- Permanent and temporary database names
- Names to use for batch files (the generated code)
- File names (and paths) for logs, formats and errors
- Type of run – trickle feed or full update
When being used interactively, the Warehouse builder allowed:
- Formats or SQL code to be created independently from executing this code
- Viewing of log and error files, bulk storage use, load history etc.
- Metadata to be edited right down to individual variable and value level.The user can subdivide the warehouse tables into a set of GROUPS – each comprising a collection of tables. Lookup tables could, for example, be one group and data tables another.
Groups may be:
- Reloaded independently either from scratch, or incrementally.
Tables within a group may:
- Have their names, descriptions and data edited - i.e. their metadata may be modified
- Loaded independently
- Split for efficient loading
- Have handling of duplicate key values defined
- Log flushed for efficient mass storage space utilization
- Data page schema defined for optimising the speed of reloads
- Data within a Table can be previewed
- Key structures can be defined and modified
- Null or erroneous data value treatment defined.
The code is built in such a way that run times are minimised and the system can be tuned to only reload necessary files, further optimising performance. In batch mode, the user initially configures the system and metadata and then runs an executable file that generates the SQL code.
This is followed by running an executable that submits this SQL code to the database server and executes it.
Batch running does not require operator intervention and does not interact with desktop.
The NAXIAN software consisted of six components:
- Operational Data Store Loader – The Warehouse Builder
- Metabase Maintenance Module – The Mart Metabase Manager
- Data Mart Extractor and Manager – The Mart Builder
- OLAP Engine and Maintenance Module – The Maintenance Module
- Desktop OLAP Browser – The Browser
- WEB OLAP Report Writer and Menu Builder – The Web Writer
"The Information Age...is dead and gone, replaced by the information economy...The Information Age was about the building of databases. It was about the rise of…