Data Integration Solutions: Performance Tips

Informatica Incremental Aggregation Implementation and Business Use Cases

Informatica PowerCenter Incrimental Aggregation

Incremental Aggregation is the perfect performance improvement technique to implement; when you have to do aggregate calculations on your incrementally changing source data. Rather than forcing the session to process the entire source data and recalculate the same data each time you run the session, incremental aggregation persist the aggregated value and adds the incremental changes to it. Lets see more details in this article.

Informatica Performance Tuning Guide, Performance Enhancements - Part 4

In our performance turning article series, so far we covered about the performance turning basics, identification of bottlenecks and resolving different bottlenecks. In this article we will cover different performance enhancement features available in Informatica PowerCener. In addition to the features provided by PowerCenter, we will go over the designs tips and tricks for ETL load performance improvement.

Informatica PowerCenter Load Balancing for Workload Distribution on Grid

Informatica PowerCenter Workflows runs on grid, distributes workflow tasks across nodes in the grid. It also distributes Session, Command, and predefined Event-Wait tasks within workflows across the nodes in a grid. PowerCenter uses load balancer to distribute workflows and session tasks to different nodes. This article describes, how to use load balancer to setup high workflow priorities and how to allocate resources.

Informatica PowerCenter on Grid for Greater Performance and Scalability

Informatica PowerCenter Workflows on Grid for Performance and Scalability

Informatica has developed a solution that leverages the power of grid computing for greater data integration scalability and performance. The grid option delivers the load balancing, dynamic partitioning, parallel processing and high availability to ensure optimal scalability, performance and reliability. In this article lets discuss how to setup Infrmatica Workflow to run on grid.

Informatica Performance Tuning Guide, Resolve Performance Bottlenecks - Part 3

In our previous article in the performance tuning series, we covered different approaches to identify performance bottlenecks. In this article we will cover the methods to resolve different performance bottlenecks. We will talk about session memory, cache memory, source, target and mapping performance turning techniques in detail.

Informatica Performance Tuning Guide, Identify Performance Bottlenecks - Part 2

In our previous article in the performance tuning series, we covered the basics of Informatica performance tuning process and the session anatomy. In this article we will cover the methods to identify different performance bottlenecks. Here we will use session thread statistics, session performance counter and workflow monitor properties to help us understand the bottlenecks.

Informatica Performance Tuning Guide, Tuning and Bottleneck Overview - Part 1

Informatica PowerCenter Performance Tuning A to Z Guide, Part I

Performance tuning process identifies the bottlenecks and eliminate it to get a better acceptable ETL load time. Tuning starts with the identification of bottlenecks in source, target, mapping and further to session tuning. It might need further tuning on the system resources on which the Informatica PowerCenter Services are running.

Dynamic Partitioning to Increase Parallelism Based on Resources Availability

Informatica PowerCenter session partition can be used to process data in parallel and achieve faster data delivery. Using Dynamic Session Partitioning capability, PowerCenter can dynamically decide the degree of parallelism. The Integration Service scales the number of session partitions at run time based on factors such as source database partitions or the number of CPUs on the node resulting significant performance improvement.

Informatica PowerCenter Pushdown Optimization a Hybrid ELT Approach

Informatica PowerCenter Pushdown Optimization an ELT Approach

Informatica Pushdown Optimization Option increases performance by providing the flexibility to push transformation processing to the most appropriate processing resource. Using Pushdown Optimization, data transformation logic can be pushed to source database or target database or through the PowerCenter server. This gives the option for the ETL architect to choose the best of the available resources for data processing.

Implementing Informatica PowerCenter Session Partitioning Algorithms

Implementing Different Informatica PowerCenter Session Partitioning Algorithms

Informatica PowerCenter Session Partitioning can be effectively used for parallel data processing and achieve faster data delivery. Parallel data processing performance is heavily depending on the additional hardware power available. In additional to that, it is important to choose the appropriate partitioning algorithm or partition type. In this article lets discuss the optimal session partition settings.

Informatica PowerCenter Partitioning for Parallel Processing and Faster Delivery

In addition to a better ETL design, it is obvious to have a session optimized with no bottlenecks to get the best session performance. After optimizing the session performance, we can further improve the performance by exploiting the under utilized hardware power. This refers to parallel processing and we can achieve this in Informatica PowerCenter using Partitioning Sessions.

Use Informatica Persistent Lookup Cache and Reduce Fact Table Load Time

Use Informatica Persistent Cache and Reduce Fact Table Load Time

In a matured data warehouse environment, it is very common to see fact tables with dozens of dimension tables linked to it. If we are using informatica to build this ETL process, we would expect to see dozens of lookup transformations as well; unless any other design techniques are used. Since lookup is the predominant transformation, turning this will help us gain some performance. Lets see how we can use persistent lookup cache for this performance improvement.

Design approach to Update Huge Tables Using Oracle MERGE

Design approach to Update Huge Tables in Informatica powercenter workflow

One of the issues we come across during the ETL design is "Update Large Tables". This is a very common ETL scenarion especially when you treat with large volume of data like loading an SCD Type 2 Dimension. We discussed about a design approach for this scenarion in one of our prior articles. Here in this updated article lets discuss a different approach to update Larger tables using Informatica Mapping.

Concurrent Workflows to Reduce Warehouse ETL Load Time

In large data integration projects, it is quite common to source data from multiple systems, sources, regions etc... As the number of data sources increases, the ETL load time also increases, because of the increasing data volume. One way to reduce the load time is by running different ETL process in parallel. Informatica PowerCenter's capability to run workflow concurrently can be used in such scenarios to reduce the ETL load time.

Update Without Update Strategy for Better Session Performance

You might have come across an ETL scenario, where you need to update a huge table with few records and occasional inserts. The straight forward approach of using LookUp transformation to identify the Inserts, Update and Update Strategy to do the Insert or Update may not be right for this particular scenario, mainly because of the LookUp transformation may not perform better and start degrading as the lookup table size increases.

Use CHECKSUM Number to Make the LookUp Transformation Simple

CHECKSUM Number in Informatica powercenter workflow

Lookup is one of the frequently used transformations in Informatica PowerCenter. This transformation is considered to be relatively simple, but at times this can become complex because of the amount of data to be cached, number of columns used in the lookup condition, checks on columns fetched from lookup table etc... In this article lets see how we can take care of couple of these issues using CHECKSUM number.

ETL Design approach to Update Huge Tables

One of the issue we come across during the ETL design is "Update Large Tables". This is a very common ETL scenarion especially when you treat with large volume of data like loading an SCD Type 2 dimension table. Here in this article lets discuss an approach to update Larger tables using Informatica Mapping.

Design Approach to Ensure High Data Availability During Data Refresh

Many times we create snapshot tables and do build reporting on top of it. Refreshing the underlying table became difficult, if the report needs to be available 24x7 with out any down time. The refresh process can be a major design bottleneck, if in case the refresh takes hours. In this article lets discuss a design approach to make the target table data highly available.

Popular Posts

Random Posts

Posts Being Viewed