Senior Data Engineer - ETL

Key Responsibilities

• Design, develop, and maintain high-performance ETL and real-time data pipelines using Apache Kafka and Apache Flink.
• Build scalable and automated MLOps pipelines for model training, validation, and deployment using AWS SageMaker and related services.
• Implement and manage Infrastructure as Code (IaC) using Terraform for AWS provisioning and maintenance.
• Collaborate with ML, Data Science, and DevOps teams to ensure reliable and efficient model deployment workflows.
• Optimize data storage and retrieval strategies for both structured and unstructured large-scale datasets.
• Integrate and transform data from multiple sources into data lakes and data warehouses.
• Monitor, troubleshoot, and improve performance of cloud-native data systems in a fast-paced production setup.
• Ensure compliance with data governance, privacy, and security standards across all data operations.
• Document data engineering workflows and architectural decisions for transparency and maintainability.
  
  

• 5+ Years of experience as Data Engineer or in similar role.
• Proven experience in building data pipelines and streaming applications using Apache Kafka and Apache Flink.
• Strong ETL development skills, with deep understanding of data modeling and data architecture in large-scale environments.
• Hands-on experience with AWS services, including SageMaker, S3, Glue, Lambda, and CloudFormation or Terraform.
• Proficiency in Python and SQL; knowledge of Java is a plus, especially for streaming use cases.
• Strong grasp of MLOps best practices, including model versioning, monitoring, and CI/CD for ML pipelines.
• Deep knowledge of IaC tools, particularly Terraform, for automating cloud infrastructure.
• Excellent analytical and problem-solving abilities, especially with regard to data processing and deployment issues.
• Agile mindset with experience working in fast-paced, iterative development environments.
• Strong communication and team collaboration skills.