AWS Data Engineer Senior

Roles & Responsibilities

Key Responsibilities

• Data Migration Execution
• Build and optimize ETL/ELT pipelines for bulk data loads, transformations, and Change Data Capture (CDC).
• Assist in schema conversion, SQL optimization, and data validation processes.
• Implement Spark-based jobs for high-volume and high-performance migration workloads.
• Collaboration & Support
• Work under the guidance of the Lead Data Engineer to deliver migration components.
• Partner with cross-functional teams (application engineers, DBAs, QA) to ensure smooth integration with PostgreSQL.
• Provide inputs on tool selection, migration best practices, and automation opportunities.
• Data Quality & Reliability
• Develop data validation and reconciliation frameworks to ensure 100% accuracy.
• Monitor pipeline performance and troubleshoot issues proactively.
• Maintain high availability, compliance, and security of sensitive customer and financial data.
  
  

• 5+ years of experience in Data Engineering, with proven work in database migrations.
• Strong experience in RDBMS to PostgreSQL data migration and SQL performance tuning.
• Hands-on expertise in Apache Spark (PySpark/Scala) for ETL/ELT workloads.
• Familiarity with AWS data services (Glue, EMR, RDS, S3, IAM) or similar cloud platforms.
• Knowledge of data validation frameworks and best practices for reconciliation.
• Solid programming skills in Python.
  
  

• Background in financial/billing systems with mission-critical data flows.
• Exposure to Terraform or Infrastructure-as-Code (IaC).
• Familiarity with DevOps/DataOps practices (CI/CD for data pipelines, monitoring, observability).
• Strong problem-solving, debugging, and optimization skills.
  
  

• 6-8 Years
  
  

• Primary Skill: Data Engineering
• Sub Skill(s): Data Engineering
• Additional Skill(s): Python, SQL, Workflows