Data Integration Solutions: ETL Design

Surrogate Key in Data Warehouse, What, When and Why

Surrogate Key in Data Warehouse, What, When, Why and Why Not

Surrogate keys are widely used and accepted design standard in data warehouses. It is sequentially generated unique number attached with each and every record in a Dimension table in any Data Warehouse. It join between the fact and dimension tables and is necessary to handle changes in dimension table attributes.

Informatica PowerCenter on Grid for Greater Performance and Scalability

Informatica PowerCenter Workflows on Grid for Performance and Scalability

Informatica has developed a solution that leverages the power of grid computing for greater data integration scalability and performance. The grid option delivers the load balancing, dynamic partitioning, parallel processing and high availability to ensure optimal scalability, performance and reliability. In this article lets discuss how to setup Infrmatica Workflow to run on grid.

Time Zones Conversion and Standardization Using Informatica PowerCenter

When your data warehouse is sourcing data from multi-time zoned data sources, it is recommended to capture a universal standard time, as well as local times. Same goes with transactions involving multiple currencies. This design enables analysis on the local time along with the universal standard time. The time standardization will be done as part of the ETL, which loads the warehouse. In this article lets discuss about the implementation using Informatica PowerCenter.

Dynamically Changing ETL Calculations Using Informatica Mapping Variable

Quite often we deal with ETL logic, which is very dynamic in nature. Such as a discount calculation which changes every month or a special weekend only logic. There is a lot of practical difficulty in making such frequent ETL change into production environment. Best option to deal with this dynamic scenario is parametrization. In this article let discuss how we can make the ETL calculations dynamic.

Informatica Workflow Recovery with High Availability for Auto Restartable Jobs

Restartable ETL jobs are very crucial to job failure recovery, supportability and data quality of any ETL system. In one of our prior articles we discussed different design techniques for ETL restartability, independent of the ETL tool used. We can also implement restartability in an ETL job using Informatica PowerCenter workflow recovery capabilities. In this article lets see what is required to setup an informatica workflow for recovery.

User Defined Functions in Informatica PowerCenter

Re-Runnability for ETL Processes Which Uses Mapping Variables

Reusability is a great feature in Informatica PowerCenter which can be used by developers. Its general purpose is to reduce unnecessary coding which ultimately reduces development time and increases supportability. User Defined Functions is one among the reusability feature provided by Informatica PowerCenter. In this article lets understand User Defined Functions in detail.

Re-Runnability for Informatica ETL Processes Which Uses Mapping Variables

Informatica PowerCenter mapping variable can be effectively used to implement change data capture logic. The mapping variable is stored in the repository and its value is set to the new value only when the session execution is successful. This property makes the restartability of the ETL process easy. Since the last success point - variable value, is stored in repository, we cannot go back and reprocess an already processed data set. In this article lets see how we can handle overriding the mapping variable value.

SCD Type 4, a Solution for Rapidly Changing Dimension

SCD Type 2, is design to generate new records for every change of a dimension attribute, so that complete historical changes can be tracked correctly. When we have dimension attributes which changes very frequently, the dimension grow very rapidly causing considerable performance and maintenance issues. In this article lets see how we can handle this rapidly changing dimension issue using SCD Type 4.

Popular Posts

Random Posts

Posts Being Viewed