Concurrent Workflows to Reduce Warehouse ETL Load Time

Johnson Cyriac Nov 13, 2012
|

Informatica concurrent workflow
In large data integration projects, it is quite common to source data from multiple systems, sources, regions etc... As the number of data sources increases, the ETL load time also increases, because of the increasing data volume. One way to reduce the load time is by running different ETL process in parallel. Informatica PowerCenter's capability to run workflow concurrently can be used in such scenarios to reduce the ETL load time.

What is Concurrent Workflows

A concurrent workflow is a workflow that can run as multiple instances concurrently. A workflow instance is a representation of a workflow. We can configure two types of concurrent workflows.

1. Allow concurrent workflows with the same instance name. Configure one workflow instance to run multiple times concurrently. Each instance has the same source, target, and variables parameters. The Integration Service identifies each instance by the run ID.
2. Configure unique workflow instances to run concurrently. Define each workflow instance name and configure a workflow parameter file for the instance. You can define different sources, targets, and variables in the parameter file.

Concurrent Workflows Configuration

For the demonstration, lets consider a scenario where we need to load daily transaction data from North America, Europe region.  These two files are expected to be available around the same time.

Here we will create one workflow to load the sales transaction data and the same work will be used to load both the file, which can execute concurrently.

Once the workflow is created, enable concurrent execution as shown in below image.
Informatica concurrent workflow configuration
Now Click on “Configure Concurrent Execution” and given the properties as in below image. Provide two different parameter files, which contains the source file information of corresponding region.
Informatica PowerCenter Concurrent Workflow parame
With that concurrent workflow configuration is done. Now to trigger the workflow, you can start the workflow using "Start Workflow Advanced" option as shown below.
Informatica PowerCenter Concurrent Workflow RunningChoose the workflow instance name from the pop up window and click OK to run the selected workflow instance.
Informatica PowerCenter Concurrent Workflow triggering
Form the workflow monitor you can see the running instance of the workflow. As shown in below image you can see the workflow run instance, which is running concurrently.
Informatica PowerCenter Concurrent Workflow RunningHope you enjoy this tutorial, Please let us know your comments and feedback.




About US Contact US Advertise Guest Post Terms and Conditions Privacy Policy Disclaimer

© 2012-2013 Data Intelligence Solution, All Rights Reserved
The contents in this site is copyrighted to Data intelligence Solution and may not be reproduced on other websites.