September 11, 2013 Datawarehouse Tools Below are the Data warehousing tools that are available ü Database ü Hardware ü ETL (Extraction, Transformation, and Loading) ü OLAP (online analytical processing, or OLAP is an approach to answering multi-dimensional analytical (MDA) queries swiftly) ü Reporting ü Metadata
September 11, 2013 Datastage Components DataStage Designer: A design interface used to create DataStage applications (known as jobs). Each job specifies the data sources, the transforms required, and the destination of the data. Jobs are compiled to create executables that are scheduled by the Director and run by the Server. DataStage Director: A user interface used to validate, schedule, run, and monitor DataStage jobs. DataStage Manager: A user interface used to view and edit the contents of the Repository. DataStage Administrator: A user interface used to configure DataStage projects and users. Repository: A central store that contains all the information required to build a data mart or data warehouse. DataStage Server: Runs executable jobs that extract, transform, and load data into a data warehouse. DataStage Package Installer: A user interface used to install packaged DataStage jobs and plug-ins
September 11, 2013 Client components of Datstage Datastage consists of client and server components. DataStage has four client components which are installed on any PC running Windows 2000 or Windows XP: ü DataStage Designer ü DataStage Director ü DataStage Manager ü DataStage Administrator
September 11, 2013 Examples of ETL tools Datastage, Informatica, OWB(Oracle Warehouse Builder) Abinitio Talend etc.
September 11, 2013 What is DataWarehouse? A data warehouse is a central integrated database containing data from all the operational sources and archive systems in an organization. It contains a copy of transaction data specifically structured for query analysis. This database can be accessed by all users, ensuring that each group in an organization is accessing valuable, stable data A Data Warehouse Is A Structured Repository of Historic Data. It Is Developed in an Evolutionary Process by Integrating Data from Non-integrated Legacy Systems. It Is Usually: ü Subject Oriented ü Integrated ü Time Variant ü Non-volatile Next: Under stand Why do we need a Data warehouse
September 11, 2013 Datastage Migration checklist Category Checkpoint SERVER INSTALLATION ACTIVTIES: Set up the data stage server and ODBC connections. SERVER INSTALLATION ACTIVTIES: Create login ids and passwords SERVER INSTALLATION ACTIVTIES: Set up the database configuration on the server like tnsnames.ora file in case of oracle. SERVER INSTALLATION ACTIVTIES: In case of a PX version (PX 8.0 now), extra steps need to be taken care like node config file etc CLIENT INSTALLATION ACTIVITY: Install the client components of Datastage on the individual machines and database client has to be set up as well. E.g. tnsnames.ora in case of Oracle. COMPONENTS MIGRATION ACTIVITIES: Get the inventory of the components to be migrated. (All active and functional jobs/processes & Static files on the existing server, if any) COMPONENTS MIGRATION ACTIVITIES: Prepare the mapping spreadsheet to be used in the transformer. It can be also derived from the existing processes. COMPONENTS MIGRATION ACTIVITIES: Convert the business logic from OWB format to Datastage transformation rules. COMPONENTS MIGRATION ACTIVITIES: Construct the Datastage jobs using the transformation rules. COMPONENTS MIGRATION ACTIVITIES: Error/exception handling needs to be taken care. COMPONENTS MIGRATION ACTIVITIES: If there is any in-built scheduling in OWB, then it may need to be shifted as well to Datastage level. Or another scheduling tool like ControlM may be put in place. COMPONENTS MIGRATION ACTIVITIES: If there is any user-defined procedure created in OWB for a specific purpose, then custom defined routines may have to be written in Datastage. COMPONENTS MIGRATION ACTIVITIES: Import the metadata from the database which makes the construction easier. Data modeling tool can be used here for metadata transfer.
September 11, 2013 Industry Verticals § BFSI – Banking, Finance, Securities, Insurance § Telecom § Retail and Distribution § Manufacturing and Process § Transportation § Healthcare § Utilities § Others
September 11, 2013 Business Models for Software companies Fixed price Time & material Dedicated ODC (Offshore Development Center) Preferred Supplier Prime Vendor
September 7, 2013 Project transition from one vendor – Activities for Primary Support Support for Maintenance activities New Vendor team to provide primary support Perform analysis and fix the assigned problems and obtain Client acceptance on the methodology adopted by the maintenance team for completing the fix Client to support and back up New Vendor or guidance, problem resolution, as escalation points if required for critical problems Handle all tickets that are assigned for the supported systems Handle on call support and Helpdesk Review fixes, resolutions provided by New Vendor Support for Development activities Project estimations Review and approval of estimates Own and perform development activities Participate in requirements gathering, Analysis and design activities Ownership of requirements gathering, Analysis and design Facilitate co-ordination of New Vendor activities with support groups Refine Manuals Update Induction Manual Review and release Induction Manual Update Operational Process Document Review and release Operational Process Document Sign-Off Sign off cross over criteria Update the Transition Project Plan based on the project progress and changes in the scope of the project.
September 7, 2013 Project transition from one vendor – Activities for Secondary Support Support for Maintenance activities Client team to provide Primary support Nightly batch monitor and handle calls (New Vendor to observe and monitor) New Vendor to provide Secondary support Handle severity 1 and critical severity 2 problems Assign lower priority severity 2 and severity 3 tickets to New Vendor Provide fixes to assigned problem tickets Provide consultation for problem resolution related to business knowledge Review fixes for problems assigned to New Vendor Support for Development activities Own and perform development activities Assign specific tasks to New Vendor – low level design, specifications coding, unit test Perform assigned tasks – low level design, specifications coding, unit test Collect support and development activity performance metrics Provide weekly status report on Support and Development activities Review, analyze, monitor progress/success of secondary ownership phase based on performance metrics and issues/risks and initiate metrics and issues/risks and initiate corrective/preventive actions Conduct Phase end review meeting to evaluate success of the secondary ownership transition Estimation Guidelines Study the existing estimation process Prepare estimation guidelines Other activities Document initial baseline and track all the changes made to the baseline configurable items. Finalize the SLA document Refine Manuals Update Induction Manual Review and release Induction Manual Update Operational Process Document Review and release Operational Process Document Sign-Off Sign off cross over criteria Update the Transition Project Plan based on the project progress and changes in the scope of the project.