Datawarehousing - Etl Project Life Cycle _ Know Informatica
This document was uploaded by user and they confirmed that they have the permission to share it. If you are author or own the copyright of this book, please report to us by using this DMCA report form. Report DMCA
Overview
Download & View Datawarehousing - Etl Project Life Cycle _ Know Informatica as PDF for free.
More details
- Words: 1,629
- Pages: 5
Loading documents preview...
Home
Posts RSS
Comments RSS
Edit
Search here...
Know Informatica
DataWareHousing ETL Project Life Cycle Posted by Suresh at 1:08 AM Labels: Data Warehouse, ETL, Life Cycle
Me
The following are the different phases involved in a ETL project development life cycle.
S UR ES H
1) Requirement Gathering
VI EW M Y C OM PL ETE PROF I L E
a) Business Requirement Collection ( BRD ) b) System Requirement Collection ( SRD ) 2) Design Phase a) High Level Design Document ( HRD ) b) Low level Design Document ( LLD ) 3) Development Phase a) Mapping Design b) Code Review
Labels
c) Peer Review 4) Testing
Data Warehouse (3)
a) Unit Testing b) System Integration Testing.
ETL (1)
c) USer Acceptance Testing ( UAT ) 5) Pre Production 6) Production ( GoLive )
Requirement Gathering:
Fact Table (1)
Life Cycle (1)
Business Requirement Collection ( BRD ):
SQL (2)
> The business requirement gathering start by business Analyst, onsite technical lead and client business users. > In this phase,a Business Analyst prepares Business Requirement Document ( BRD ) (or) Business Requirement Specifications ( BRS ) > BR collection takes place at client location. > The o/p from BR Analysis are
Blog Archive
> BRS Business Requirement Specifications > SRS System Requirement Specifications
System Requirement Collection ( SRD ): Senior technical people (or) ETL architect will prepare the SRS which contains s/w and h/w details
▼ 2011 (6) ► March (3)
The SRS will includes a) O/S to be used ( windows or unix ) b) RDBMS required to build database ( oracle, Teradata etc ) c) ETL tools required ( Informatica,Datastage )
▼ February (3)
DataWareHousing ETL Project Life
d) OLAP tools required ( Cognos ,BO ) The SRS is also called as Technical Requirement Specifications ( TRS )
Design Phase: > The o/p from design and planning phase is a) HLD ( High Level Design ) Document
Powered by Blogger. Fact Table
Data Warehouse Definition
Search This Blog
b)LLD ( Low Level Design ) Document
Search
HLD ( High Level Design ) Document : An ETL Architect and DWH Architect participate in designing a solution to build a DWH. An HLD document is prepared based on Business Requirement.
LLD ( Low Level Design ) Document : Based on HLD,a senior ETL developer prepare Low Level Design Document The LLD contains more technical details of an ETL System. An LLD contains data flow diagram ( DFD ), details of source and targets of each mapping.
Resources
An LLD also contains information about full and incremental load. After LLD then Development Phase will start
1Keydata
Development Phase : Mapping: > Based an LLD, the ETL team will create mapping ( ETL Code ) > After designing the mappings, the code ( Mappings ) will be reviewed by developers.
Followers
Code Review : > Code Review will be done by developer.
Join this site
> In code review,the developer will review the code and the logic but not the data.
with Google Friend Connect
> The following activities takes place in code review
Members (2)
> You have to check the naming standards of transformation,mappings of data etc. > Source and target mapping ( Placed the correct logic or not in mapping ) Peer Review : > The code will reviewed by your team member ( third party developer )
Testing:
Already a member? Sign in
The following various types testing carried out in testing environment. 1) Unit Testing 2) Development Integration Testing 3) System Integration Testing
Add Subscribtion
4) User Acceptance Testing Unit Testing : > A unit test for the DWH is a white Box testing,It should check the ETL procedure and Mappings. > The following are the test cases can be executed by an ETL developer. 1) Verify data loss 2) No.of records in the source and target 3) Dataload/Insert 4) Dataload/Update 5) Incremental load 6) Data accuracy 7) verify Naming standards. 8) Verify column Mapping > The Unit Test will be carried by ETL developer in development phase. > ETL developer has to do the data validations also in this phase. Development Integration Testing > Run all the mappings in the sequence order. > First Run the source to stage mappings. > Then run the mappings related to dimensions and facts. System Integration Testing : > After development phase,we have to move our code to QA environment. > In this environment,we are giving readonly permission to testing people. > They will test all the workflows. > And they will test our code according to their standards. User Acceptance Testing ( UAT ) : > This test is carried out in the presence of client side technical users to verify the data migration from source to destination.
Production Environment : > Migrate the code into the GoLive environment from test environment ( QA Environment ).
Thanks: shivakrishnas 2 0 C OMME NTS : Anonymous said... thanks for such a useful post, really appreciate it. JUNE 14 , 2 011 AT 9 : 57 AM Anonymous said... Good post on ETL life cycle. Mahesh AUG US T 10, 2 011 AT 10: 4 0 PM Sandya said...
Posts Comments
Hi, This life cycle is really very clear and nice thank you for this. AUG US T 17, 2 011 AT 9 : 52 PM Suresh said... Thanks for your comments Sandya... O C TO B E R 14 , 2 011 AT 5: 16 PM Anonymous said... thanx for posting it helps alot to me .......... this life cycle is same for abinitio also? plz give replay to me my email id is [email protected]....... FE B RUARY 2 2 , 2 012 AT 1: 18 AM Alekhya said... Thanks for the Posting.....It helps me lot... Can u please say wat are the complex issues usually face while mapping.... Can u please share the information to my gmail account [email protected] MARC H 4 , 2 012 AT 7: 07 PM cmtopinka said... What's the difference between Unit testing and Data Validation? MAY 1, 2 012 AT 12 : 12 PM Anonymous said... Hi All, I am new in informatica, can you please share with me the sample documents like SRS, FS, HLD,LLD, etc on my mail id is [email protected] Thanks in advance. Gajraj D E C E MB E R 3 1, 2 013 AT 11: 17 PM Nitin said... thank You So Much this post is really very useful... JANUARY 2 1, 2 014 AT 4 : 02 AM Naveen Kumar said... Thank You Suresh FE B RUARY 5, 2 014 AT 9 : 04 AM sruthi said... Really a great work, it helped me a lot FE B RUARY 9 , 2 014 AT 11: 2 1 PM jake george said... ETL Testing Online Training, ONLINE TRAINING – IT SUPPORT – CORPORATE TRAINING http://www.21cssindia.com/courses/etltestingonlinetraining249.html The 21st Century Software Solutions of India offers one of the Largest conglomerations of Software Training, IT Support, Corporate Training institute in India +919000444287 +917386622889 Visakhapatnam,Hyderabad ETL Testing Online Training, ETL Testing Training, ETL Testing, ETL Testing Online Training| ETL Testing Training| ETL Testing| "Courses at 21st Century Software Solutions Talend Online Training Hyperion Online Training IBM Unica Online Training Siteminder Online Training SharePoint Online Training Informatica Online Training SalesForce Online Training Many more… | Call Us +917386622889 +919000444287 [email protected] Visit: http://www.21cssindia.com/courses.html"
S E PTE MB E R 3 0, 2 014 AT 9 : 2 4 PM Daniel Mason said... It was really a nice article and I was really impressed by reading this article We are also giving all software Course Online Training. The Data Warehousing Courses Online Training is one of the leading Online Training institute in the world. D E C E MB E R 2 6 , 2 014 AT 11: 18 AM jhansi rani said... thank u Suresh,.. really very usefull..give me ur mail id plz APRI L 2 , 2 015 AT 10: 10 PM venkatesan kalyanasundaram said... Thank u so much Mr Suresh for the wonderful article. can u send the sample document for BRD, SRD, HLD, LLD for reference ([email protected])
APRI L 6 , 2 015 AT 5: 08 AM Bapi said... Thank you Suresh, for all these Valuable info. Can you plzz provide me the relevant matrials and documents to my following email id: [email protected] JULY 6 , 2 015 AT 12 : 2 1 AM Cat Minds said... This information which you provided is very much useful for us.It was very interesting and useful for qa online training.We also providing ETL Testing Online Training institute in USA. AUG US T 10, 2 015 AT 11: 54 AM rajeshkanna yadhavan said... I'm happy to joining this blog, It is a very nice experience for me... Thank you for all your postings. Testing Training with Live Project AUG US T 2 8, 2 015 AT 9 : 4 5 PM Anonymous said... Hi Suresh, As am new for the INFORMATICA, it will be very helpful for me, thank you for this informative blog. And pls can you share the sample documents of BRD, SRD, etc... would be so thank full for that. thank you.. O C TO B E R 1, 2 015 AT 2 : 11 AM Unknown said... hi pls provide documents for hld and lld [email protected] D E C E MB E R 2 6 , 2 015 AT 6 : 4 7 PM
POS T A C OMME NT
Enter your comment...
Comment as:
Publish
Google Account
Preview
LINK S TO THIS POS T Create a Link Newer Post
Home
Subscribe to: Post Comments (Atom)
©Know Informatica Blogger theme by BloggerThemes Design by FreeThemeLayouts
Older Post
Posts RSS
Comments RSS
Edit
Search here...
Know Informatica
DataWareHousing ETL Project Life Cycle Posted by Suresh at 1:08 AM Labels: Data Warehouse, ETL, Life Cycle
Me
The following are the different phases involved in a ETL project development life cycle.
S UR ES H
1) Requirement Gathering
VI EW M Y C OM PL ETE PROF I L E
a) Business Requirement Collection ( BRD ) b) System Requirement Collection ( SRD ) 2) Design Phase a) High Level Design Document ( HRD ) b) Low level Design Document ( LLD ) 3) Development Phase a) Mapping Design b) Code Review
Labels
c) Peer Review 4) Testing
Data Warehouse (3)
a) Unit Testing b) System Integration Testing.
ETL (1)
c) USer Acceptance Testing ( UAT ) 5) Pre Production 6) Production ( GoLive )
Requirement Gathering:
Fact Table (1)
Life Cycle (1)
Business Requirement Collection ( BRD ):
SQL (2)
> The business requirement gathering start by business Analyst, onsite technical lead and client business users. > In this phase,a Business Analyst prepares Business Requirement Document ( BRD ) (or) Business Requirement Specifications ( BRS ) > BR collection takes place at client location. > The o/p from BR Analysis are
Blog Archive
> BRS Business Requirement Specifications > SRS System Requirement Specifications
System Requirement Collection ( SRD ): Senior technical people (or) ETL architect will prepare the SRS which contains s/w and h/w details
▼ 2011 (6) ► March (3)
The SRS will includes a) O/S to be used ( windows or unix ) b) RDBMS required to build database ( oracle, Teradata etc ) c) ETL tools required ( Informatica,Datastage )
▼ February (3)
DataWareHousing ETL Project Life
d) OLAP tools required ( Cognos ,BO ) The SRS is also called as Technical Requirement Specifications ( TRS )
Design Phase: > The o/p from design and planning phase is a) HLD ( High Level Design ) Document
Powered by Blogger. Fact Table
Data Warehouse Definition
Search This Blog
b)LLD ( Low Level Design ) Document
Search
HLD ( High Level Design ) Document : An ETL Architect and DWH Architect participate in designing a solution to build a DWH. An HLD document is prepared based on Business Requirement.
LLD ( Low Level Design ) Document : Based on HLD,a senior ETL developer prepare Low Level Design Document The LLD contains more technical details of an ETL System. An LLD contains data flow diagram ( DFD ), details of source and targets of each mapping.
Resources
An LLD also contains information about full and incremental load. After LLD then Development Phase will start
1Keydata
Development Phase : Mapping: > Based an LLD, the ETL team will create mapping ( ETL Code ) > After designing the mappings, the code ( Mappings ) will be reviewed by developers.
Followers
Code Review : > Code Review will be done by developer.
Join this site
> In code review,the developer will review the code and the logic but not the data.
with Google Friend Connect
> The following activities takes place in code review
Members (2)
> You have to check the naming standards of transformation,mappings of data etc. > Source and target mapping ( Placed the correct logic or not in mapping ) Peer Review : > The code will reviewed by your team member ( third party developer )
Testing:
Already a member? Sign in
The following various types testing carried out in testing environment. 1) Unit Testing 2) Development Integration Testing 3) System Integration Testing
Add Subscribtion
4) User Acceptance Testing Unit Testing : > A unit test for the DWH is a white Box testing,It should check the ETL procedure and Mappings. > The following are the test cases can be executed by an ETL developer. 1) Verify data loss 2) No.of records in the source and target 3) Dataload/Insert 4) Dataload/Update 5) Incremental load 6) Data accuracy 7) verify Naming standards. 8) Verify column Mapping > The Unit Test will be carried by ETL developer in development phase. > ETL developer has to do the data validations also in this phase. Development Integration Testing > Run all the mappings in the sequence order. > First Run the source to stage mappings. > Then run the mappings related to dimensions and facts. System Integration Testing : > After development phase,we have to move our code to QA environment. > In this environment,we are giving readonly permission to testing people. > They will test all the workflows. > And they will test our code according to their standards. User Acceptance Testing ( UAT ) : > This test is carried out in the presence of client side technical users to verify the data migration from source to destination.
Production Environment : > Migrate the code into the GoLive environment from test environment ( QA Environment ).
Thanks: shivakrishnas 2 0 C OMME NTS : Anonymous said... thanks for such a useful post, really appreciate it. JUNE 14 , 2 011 AT 9 : 57 AM Anonymous said... Good post on ETL life cycle. Mahesh AUG US T 10, 2 011 AT 10: 4 0 PM Sandya said...
Posts Comments
Hi, This life cycle is really very clear and nice thank you for this. AUG US T 17, 2 011 AT 9 : 52 PM Suresh said... Thanks for your comments Sandya... O C TO B E R 14 , 2 011 AT 5: 16 PM Anonymous said... thanx for posting it helps alot to me .......... this life cycle is same for abinitio also? plz give replay to me my email id is [email protected]....... FE B RUARY 2 2 , 2 012 AT 1: 18 AM Alekhya said... Thanks for the Posting.....It helps me lot... Can u please say wat are the complex issues usually face while mapping.... Can u please share the information to my gmail account [email protected] MARC H 4 , 2 012 AT 7: 07 PM cmtopinka said... What's the difference between Unit testing and Data Validation? MAY 1, 2 012 AT 12 : 12 PM Anonymous said... Hi All, I am new in informatica, can you please share with me the sample documents like SRS, FS, HLD,LLD, etc on my mail id is [email protected] Thanks in advance. Gajraj D E C E MB E R 3 1, 2 013 AT 11: 17 PM Nitin said... thank You So Much this post is really very useful... JANUARY 2 1, 2 014 AT 4 : 02 AM Naveen Kumar said... Thank You Suresh FE B RUARY 5, 2 014 AT 9 : 04 AM sruthi said... Really a great work, it helped me a lot FE B RUARY 9 , 2 014 AT 11: 2 1 PM jake george said... ETL Testing Online Training, ONLINE TRAINING – IT SUPPORT – CORPORATE TRAINING http://www.21cssindia.com/courses/etltestingonlinetraining249.html The 21st Century Software Solutions of India offers one of the Largest conglomerations of Software Training, IT Support, Corporate Training institute in India +919000444287 +917386622889 Visakhapatnam,Hyderabad ETL Testing Online Training, ETL Testing Training, ETL Testing, ETL Testing Online Training| ETL Testing Training| ETL Testing| "Courses at 21st Century Software Solutions Talend Online Training Hyperion Online Training IBM Unica Online Training Siteminder Online Training SharePoint Online Training Informatica Online Training SalesForce Online Training Many more… | Call Us +917386622889 +919000444287 [email protected] Visit: http://www.21cssindia.com/courses.html"
S E PTE MB E R 3 0, 2 014 AT 9 : 2 4 PM Daniel Mason said... It was really a nice article and I was really impressed by reading this article We are also giving all software Course Online Training. The Data Warehousing Courses Online Training is one of the leading Online Training institute in the world. D E C E MB E R 2 6 , 2 014 AT 11: 18 AM jhansi rani said... thank u Suresh,.. really very usefull..give me ur mail id plz APRI L 2 , 2 015 AT 10: 10 PM venkatesan kalyanasundaram said... Thank u so much Mr Suresh for the wonderful article. can u send the sample document for BRD, SRD, HLD, LLD for reference ([email protected])
APRI L 6 , 2 015 AT 5: 08 AM Bapi said... Thank you Suresh, for all these Valuable info. Can you plzz provide me the relevant matrials and documents to my following email id: [email protected] JULY 6 , 2 015 AT 12 : 2 1 AM Cat Minds said... This information which you provided is very much useful for us.It was very interesting and useful for qa online training.We also providing ETL Testing Online Training institute in USA. AUG US T 10, 2 015 AT 11: 54 AM rajeshkanna yadhavan said... I'm happy to joining this blog, It is a very nice experience for me... Thank you for all your postings. Testing Training with Live Project AUG US T 2 8, 2 015 AT 9 : 4 5 PM Anonymous said... Hi Suresh, As am new for the INFORMATICA, it will be very helpful for me, thank you for this informative blog. And pls can you share the sample documents of BRD, SRD, etc... would be so thank full for that. thank you.. O C TO B E R 1, 2 015 AT 2 : 11 AM Unknown said... hi pls provide documents for hld and lld [email protected] D E C E MB E R 2 6 , 2 015 AT 6 : 4 7 PM
POS T A C OMME NT
Enter your comment...
Comment as:
Publish
Google Account
Preview
LINK S TO THIS POS T Create a Link Newer Post
Home
Subscribe to: Post Comments (Atom)
©Know Informatica Blogger theme by BloggerThemes Design by FreeThemeLayouts
Older Post
Related Documents
Datawarehousing - Etl Project Life Cycle _ Know Informatica
January 2021 1
Etl Standards For Informatica
January 2021 1
Etl Informatica Developer Resume
March 2021 0
Informatica Etl Testing With Unix
January 2021 1
Etl Project Plan
January 2021 1