Datawarehouse Tools

Below are the Data warehousing tools that are available
ü    Database
ü    Hardware
ü    ETL (Extraction, Transformation, and Loading)
ü    OLAP (online analytical processing, or OLAP is an approach to answering multi-dimensional analytical (MDA) queries swiftly)
ü    Reporting

ü    Metadata  

Datastage Components

DataStage Designer:
A design interface used to create DataStage applications (known as jobs). Each job specifies the data sources, the transforms required, and the destination of the data. Jobs are compiled to create executables that are scheduled by the Director and run by the Server.
DataStage Director:
A user interface used to validate, schedule, run, and monitor DataStage jobs.
DataStage Manager:
A user interface used to view and edit the contents of the Repository.
DataStage Administrator:
A user interface used to configure DataStage projects and users.
Repository:
A central store that contains all the information required to build a data mart or data warehouse.
DataStage Server:
Runs executable jobs that extract, transform, and load data into a data warehouse.
DataStage Package Installer:
A user interface used to install packaged DataStage jobs and plug-ins

Client components of Datstage

Datastage consists of client and server components.
DataStage has four client components which are installed on any PC running Windows 2000 or Windows XP:

ü    DataStage Designer
ü    DataStage Director
ü    DataStage Manager
ü    DataStage Administrator

What is DataWarehouse?

A data warehouse is a central integrated database containing data from all the operational sources and archive systems in an organization. It contains a copy of transaction data specifically structured for query analysis. This database can be accessed by all users, ensuring that each group in an organization is accessing valuable, stable data
Data Warehouse
A Data Warehouse Is A Structured Repository of Historic Data.
It Is Developed in an Evolutionary Process by Integrating Data from Non-integrated Legacy Systems.
It Is Usually:
ü    Subject Oriented
ü    Integrated
ü    Time Variant
ü    Non-volatile

 Next: Under stand Why do we need a Data warehouse

Datastage Migration checklist

Category
Checkpoint
    SERVER INSTALLATION ACTIVTIES:
Set up the data stage server and ODBC connections.
    SERVER INSTALLATION ACTIVTIES:
Create login ids and passwords
    SERVER INSTALLATION ACTIVTIES:
Set up the database configuration on the server like tnsnames.ora   file in case of oracle.
    SERVER INSTALLATION ACTIVTIES:
In case of a PX version (PX 8.0 now), extra steps need to be taken care like node config file etc
     CLIENT INSTALLATION ACTIVITY:
Install the client components of Datastage on the individual machines and database client has to be set up as well. E.g. tnsnames.ora in case of Oracle.
COMPONENTS MIGRATION ACTIVITIES:
Get the inventory of the components to be migrated. (All active and functional jobs/processes & Static files on the existing server, if any)
COMPONENTS MIGRATION ACTIVITIES:
Prepare the mapping spreadsheet to be used in the transformer. It can be also derived from the existing processes.
COMPONENTS MIGRATION ACTIVITIES:
Convert the business logic from OWB format to Datastage transformation rules.
COMPONENTS MIGRATION ACTIVITIES:
Construct the Datastage jobs using the transformation rules.
COMPONENTS MIGRATION ACTIVITIES:
Error/exception handling needs to be taken care.
COMPONENTS MIGRATION ACTIVITIES:
If there is any in-built scheduling in OWB, then it may need to be shifted as well to Datastage level. Or another scheduling tool like ControlM may be put in place.
COMPONENTS MIGRATION ACTIVITIES:
If there is any user-defined procedure created in OWB for a specific purpose, then custom defined routines may have to be written in Datastage.
COMPONENTS MIGRATION ACTIVITIES:
Import the metadata from the database which makes the construction easier. Data modeling tool can be used here for metadata transfer.

Industry Verticals

§  BFSI – Banking, Finance, Securities, Insurance
§  Telecom
§  Retail and Distribution
§  Manufacturing and Process
§  Transportation
§  Healthcare
§  Utilities

§  Others

Project transition from one vendor – Activities for Primary Support

Support for Maintenance activities
New Vendor team to provide primary support
Perform analysis and fix the assigned problems and obtain Client acceptance on the methodology adopted by the maintenance team for completing the fix
Client to support and back up New Vendor or guidance, problem resolution, as escalation points if required for critical problems
Handle all tickets that are assigned for the supported systems
Handle on call support and Helpdesk
Review fixes, resolutions provided by New Vendor
Support for Development activities
Project estimations
Review and approval of estimates
Own and perform development activities
Participate in requirements gathering, Analysis and design activities
Ownership of requirements gathering, Analysis and design
Facilitate co-ordination of New Vendor activities with support groups
Refine Manuals
Update Induction Manual
Review and release Induction Manual
Update Operational Process Document
Review and release Operational Process Document
Sign-Off
Sign off cross over criteria
Update the Transition Project Plan based on the project progress and changes in the scope of the project.

Project transition from one vendor – Activities for Secondary Support

Support for Maintenance activities
Client team to provide Primary support
Nightly batch monitor and handle calls (New Vendor to observe and monitor)
New Vendor to provide Secondary support
Handle severity 1 and critical severity 2 problems
Assign lower priority severity 2 and severity 3 tickets to New Vendor
Provide fixes to assigned problem tickets
Provide consultation for problem resolution related to business knowledge
Review fixes for problems assigned to New Vendor
Support for Development activities
Own and perform development activities
Assign specific tasks to New Vendor – low level design, specifications coding, unit test
Perform assigned tasks – low level design, specifications coding, unit test
Collect support and development activity performance metrics
Provide weekly status report on Support and Development activities
Review, analyze, monitor progress/success of secondary ownership phase based on performance metrics and issues/risks and initiate metrics and issues/risks and initiate corrective/preventive actions
Conduct Phase end review meeting to evaluate success of the secondary ownership transition
Estimation Guidelines
Study the existing estimation process
Prepare estimation guidelines
Other activities
Document initial baseline and track all the changes made to the baseline configurable items.
Finalize the SLA document
Refine Manuals
Update Induction Manual
Review and release Induction Manual
Update Operational Process Document
Review and release Operational Process Document
Sign-Off
Sign off cross over criteria
Update the Transition Project Plan based on the project progress and changes in the scope of the project.