Design Approach to Handle Late Arriving Dimensions and Late Arriving Facts

Design Approach to Handle Late Arriving Dimensions and Late Arriving Facts

In the typical case for a data warehouse, dimensions are processed first and the facts are loaded later, with the assumption that all required dimension data is already in place. This may not be true in all cases because of nature of your business process or the source application behavior. Fact data also, can be sent from the source application to the warehouse way later than the actual fact data is created. In this article lets discusses several options for handling late arriving dimension and Facts.

Continue Reading

SOFT and HARD Deleted Records and Change Data Capture in Data Warehouse

SOFT and HARD Deleted Records and Change Data Capture in Data Warehouse

In our couple of prior  articles we spoke about change data capture, different techniques to capture change data and a change data capture frame work as well. In this article we will deep dive into different aspects for change data in Data Warehouse including soft and hard deletions in source systems.

Continue Reading

Surrogate Key in Data Warehouse, What, When and Why

Surrogate Key in Data Warehouse, What, When and Why

Surrogate keys are widely used and accepted design standard in data warehouses. It is sequentially generated unique number attached with each and every record in a Dimension table in any Data Warehouse. It join between the fact and dimension tables and is necessary to handle changes in dimension table attributes.

Continue Reading

SCD Type 4, a Solution for Rapidly Changing Dimension

SCD Type 4, a Solution for Rapidly Changing Dimension

SCD Type 2, is design to generate new records for every change of a dimension attribute, so that complete historical changes can be tracked correctly. When we have dimension attributes which changes very frequently, the dimension grow very rapidly causing considerable performance and maintenance issues. In this article lets see how we can handle this rapidly changing dimension issue using SCD Type 4.

Continue Reading

SCD Type 6, a Combination of SCD Type 1, 2 and 3

SCD Type 6, a Combination of SCD Type 1, 2 and 3

In couple of our previous articles, we discussed how to design and implement SCD Type1, Type 2 and Type 3. We always can not fulfill all the business requirements just by these basic SCD Types. So here lets see what is SCD Type 6 and what it offers beyond the basic SCD Types.

Continue Reading