FASTTEK GLOBALpowered by Fast Switch - Great Lakes
Chennai, Tamil Nadu
Specialty Development Practitioner #1030846Job Description:
- Responsible for gathering business requirements and design ETL systems to meet application needs Work closely with stakeholders to gather requirements and provide technical guidance on the migration process
- Collaborate with the data engineering teams to understand the existing JCL/ETL jobs and their dependencies.
- Design and implement migration strategies for moving ETL to GCP native services.
- Develop, test, and deploy data pipelines and workflows, ensuring scalability, reliability, and performance Troubleshoot and resolve any production issues or challenges that arise during the migration
- Deliver product features as part of roadmap and business needs Lead in Effort estimation for the new developments and enhancements.
- Co-create test plan and lead execution and code deployment.
- Lead performance tuning to optimize the job execution.
- Responsible for validating, run, scheduling and monitoring jobs using data stage director Responsible for debugging Ticketing issues in the production phase
Skills Required:
- Google Cloud Platform - Biq Query, Data Flow, Dataproc, Data Fusion, TERRAFORM, Tekton,Cloud SQL, AIRFLOW, POSTGRES, Airflow PySpark, Python, API
Experience Required:
- 5+ years of complex SQL development experience
- 2+ years of Cloud experience (GCP preferred) with solutions designed and implemented at production scale Strong understanding of key GCP services, especially those related to data processing [Batch/Real Time] leveraging Terraform, BigQuery, BQSQL, Dataflow, DataFusion, DataProc, Cloud Build, Airflow & Pub/Sub Experience in analysis, design, development, implementation using Data Warehousing applications e.g IBM Datastage/DataFlow/DataProc
- Experience in integrating various data sources like Oracle, Teradata, DB2, Big Query & Flat files
- Experience in designing jobs by using various Stages like Look-up, Filter, Sort, copy, remove duplicates, Join, Funnel and Aggregator.
- Experience in debugging ETL jobs to check in for the errors and warnings associated with each Job run Hands on experience in Tuning Jobs process identifying and resolve performance in Parallel Jobs Involving in review meetings and coordinating with the team in job designing and fine-tuning the job performance.
- Good experience in Autosys or Astronomer, Accurev & Github Good experience in production support activities
Education Required:
- Bachelor's Degree