Job Details

Position: Big Data Technical Specialist (Ref: 8291)
Location: Columbus, OH 43215 USA
Duration: 1 Years 4 Months 6 Days - Contract
Openings: 1
Deadline: 02/24/2021
Pay Rate:Login


We are seeking a Technical Specialist who will be responsible for Enterprise data warehouse design, development, implementation, migration, maintenance, and operation activities. Works closely with Data Governance and Analytics team. The candidate will work closely with Data Governance and Analytics team. Will be one of the key technical resource for data warehouse projects for various Enterprise data warehouse projects and building critical data marts, data ingestion to Big Data platform for data analytics and exchange with partners. This position is a member of team and works closely with the Business Intelligence & Data Analytics team.

• Participate in Team activities, Design discussions, stand up meetings and planning Review with team.
• Perform data analysis, data profiling, data quality and data ingestion in various layers using Database queries, Informatica PowerCenter, Informatica Analyst score cards, Pyspark programs and UNIX shell scripts.
• Follow the organization coding standard document, Create mappings, sessions, and workflows as per the mapping specification document.
• Perform Gap and impact analysis of ETL and IOP jobs for the new requirement and enhancements.
• Create jobs in Informatica PowerCenter, Informatica Developer IDQ, Hadoop using SQOOP, PYSPARK and Stream Sets to meet the business user needs.
• Create mockup data, perform Unit testing, and capture the result sets against the jobs developed in lower environment.
• Updating the production support Run book, Control M schedule document as per the production release.
• Create and update design documents, provide detail description about workflows after every production release.
• Continuously monitor the production data loads, fix the issues, update the tracker document with the issues, Identify the performance issues.
• Performance tuning long running ETL jobs by creating partitions, enabling bulk load, increasing commit interval and other standard approaches.
• Perform Quality assurance check, Reconciliation post data loads and communicate to vendor for receiving fixed data.
• Participate in ETL code review and design re-usable frameworks.
• Create Remedy incidents to fix production issues, create Support Requests to deploy Database, UNIX, ETL and SAS code to UAT environment.
• Create Remedy incidents to trigger Control M jobs for FTP and ETL jobs on ADHOC, weekly, Monthly, and quarterly basis as needed.
• Model and create STAGE / ODS / Data warehouse dimension tables as and when needed.
• Create Change requests, workplan, Test results, BCAB checklist documents for the code deployment to production environment and perform the code validation post deployment.
• Work with DBA, ETL and SAS admin teams for code deployments and health checks.
• Create re-usable UNIX shell scripts for file archival, file validations and informatica workflow looping.
• Create re-usable framework for Audit Balance Control to capture Reconciliation, mapping parameters and variables, serves as single point of reference for workflows.
• Create Pyspark programs to ingest historical and incremental data.
• Create SQOOP scripts to ingest historical data from EDW oracle database to Hadoop IOP, created HIVE tables and Impala views creation scripts for Dimension tables.
• Writing Data base Stored procedures for gathering stats on table post data loads, enabling, and disabling constraints and indexes.
• Writing complex SQL queries and performed tuning based on the explain plan results.
• Extract unstructured and semi-structured data using data processor transformation in IDQ.
• Participate in meetings to continuously upgrade the Functional and technical expertise.
• Demonstrate effective leadership, analytical and problem-solving skills.
• Required excellent written and oral communication skills with technical and business teams.
• Ability to work independently, as well as part of a team.
• Stay abreast of current technologies in area of IT assigned.
• Establish facts and draw valid conclusions.
• Recognize patterns and opportunities for improvement throughout the entire organization.
• Ability to discern critical from minor problems and innovate new solutions.
• Utilize ETL standards and practices towards establishing and following centralized metadata repository.
• Computer literacy with Excel, PowerPoint, Word, etc.
• Effective communication, presentation, & organizational skills
• Ability to establish priorities & follow through on projects, paying close attention to detail with minimal supervision.
• Required Education: BS/BA degree or combination of education & experience.
• Familiar with Project Management methodologies like Waterfall and Agile
• Perform other duties as assigned.
• Accountable for ETL design documentation
• Analysis, Design, development, support and Enhancements of ETL/ELT in data warehouse environment with Cloudera Bigdata Technologies (Hadoop, MapReduce, Sqoop, PySpark, Spark, HDFS, Hive, Impala, StreamSets, Kudu, Oozie, Hue, Kafka, Yarn, Python, Flume, Zookeeper, Sentry, Cloudera Navigator) along with Informatica, Oracle SQL/PL-SQL, Unix commands and shell scripting.

Required / Desired Skills

  • Experience with Informatica Power Center on Data Warehousing or Data Integration projects (Required 8 Years)
  • Proven ability to write high quality code (Required 5 Years)
  • Experience with Expertise implementing complex ETL logic (Required 7 Years)
  • Experience Develop and enforce strong reconciliation process (Required 3 Years)
  • Strong SQL experience (prefer Oracle) (Required 5 Years)
  • Good knowledge of relational database, data vault and dimensional model design (Required 5 Years)
  • Basic knowledge of UNIX/LINUX shell scripting (Required 3 Years)
  • Strong development experience in creating Sqoop scripts, PySpark programs, HDFS commands, HDFS file formats (Parquet, Avro, ORC etc.), StreamSets pipeline creation, jobs scheduling, hive/impala queries, Unix commands, scripting and shell scripting etc. (Required 2 Years)

copyright @ All rights reserved
Professional Tech Integration, Inc., Computers  Sys Designers & Consult, Norcross, GA