Slowly Changing Dimension are all about how we manage changes to dimensional data over time in a data warehouse or analytics systems. They are useful for historical data tracking, trend analysis and versioning.

While building a data pipeline, the pipeline should be idempotent (it should produce same data, )

They are mainly 3 types of SCDs

  1. SCD Type 1 : Data will be overwritten by new data, no historical data
  2. SCD Type 2 : Tracks full history with new rows
  3. SCD Type 3 :

In SCD Type 1: We only have the latest information. Or we can say during the pipeline the existing rows will be overwritten with the update values. Hence no historical data is available