Data warehouses (DWs) are complex computer systems whose main goal is to facilitate the decision making process of knowledge workers. ETL (Extraction-Transformation-Loading) processes are responsible for the extraction of data from heterogeneous operational data sources, their transformation (conversion, cleaning, normalization, etc.) and their loading into DWs. ETL processes are a key component of DWs because incorrect or misleading data will produce wrong business decisions, and therefore, a correct design of these processes at early stages of a DW project is absolutely necessary to improve data quality. However, not much research has dealt with the modeling of ETL processes. In this paper, we present our approach, based on the Unified Modeling Language (UML), which allows us to accomplish the conceptual modeling of these ETL processes. We provide the necessary mechanisms for an easy and quick specification of the common operations defined in these ETL processes such as, the integration of different data sources, the transformation between source and target attributes, the generation of surrogate keys and so on. Another advantage of our proposal is the use of the UML (standardization, ease-of-use and functionality) and the seamless integration of the design of the ETL processes with the DW conceptual schema.
ETL Scheduling is an operational process which is required to determine the sequence and time of execution of the various data flows (Jobs/Mappings).
The ETL schedule is dependent on the following
- Load Window of the ETL process flow.
- External Dependencies like
- Timeframe of the source data availability.
- Warehouse/Mart database Maintenance, like database backup time.
- Operating System Maintenance, like file system backup time.
- ETL’s inter process flow dependencies like conformed dimensions ETL before the subject area specific dimension ETL, Dimension table ETL before the Fact Table ETL etc.
ETL Scheduling integrates the various process flows and data flows (Jobs/Mappings) with respect to the sequence and time of events in the data warehouse.
RECENT COMMENTS