Data Engineer

Role: Data Engineer – ODS Platform

Location: [Onsite / Hybrid / Remote]

Type: [Full-Time / Contract]

Industry: Aviation, Retail, Finance, Healthcare

Job Overview

We are looking for an experienced Data Engineer with strong data modeling expertise, AWS Data Engineering certification, and hands-on experience across key AWS services used in building scalable Operational Data Store (ODS) platforms. The ideal candidate will bring a combination of backend engineering skills in Python or Java, a solid understanding of data pipelines and Lakehouse architectures, and a track record of delivering high-performance, SOX-compliant data systems in the cloud.

Key Responsibilities

Design and implement data ingestion pipelines, ETL/ELT workflows, and data transformation logic for ODS layers (Raw → Curated → Published).

Build scalable data models and schemas to support operational and analytical workloads with historical retention.

Leverage AWS-native services for data processing, storage, cataloging, and orchestration S3, Glue, Athena, Aurora, Redshift, DynamoDB, Step Functions, Lambda, Lake Formation

Ensure compliance with SOX, data governance, and access control standards.

Write reusable, well-tested code in Python and/or Java for custom data transformations and orchestration components.

Implement partitioning, compaction, and optimization techniques for performance and cost efficiency.

Monitor and troubleshoot data jobs using tools such as CloudWatch, Glue job metrics, and custom alerts.

Work closely with architects, analysts, and business stakeholders to understand data needs and deliver quality datasets.

Document technical designs, data flow diagrams, and data dictionaries.

Required Qualifications

5+ years of experience in data engineering with a strong focus on data modeling and cloud-based architecture.

AWS Certified Data Analytics – Specialty or Big Data – Specialty (or actively pursuing).

Solid hands-on experience with AWS Glue, S3, Athena, Aurora, Step Functions, Lambda, API Gateway, Lake Formation

Strong experience building and optimizing data models (3NF, star/snowflake, time-series) for ODS or EDW platforms.

Advanced proficiency in Python and/or Java, including experience building reusable frameworks or libraries.

Experience with data cataloging, schema evolution, and metadata management.

Familiarity with CI/CD, Git, and IaC tools like Terraform or CloudFormation.

Deep understanding of batch vs. streaming patterns, event-driven processing, and system decoupling.

Preferred Qualifications

Experience with Snowflake, Redshift Spectrum, or Databricks is a plus.

Knowledge of Kafka, Kinesis, or other streaming platforms.

Exposure to data quality, lineage, and governance frameworks.

Prior experience implementing SOX-compliant data architectures.

Familiarity with Airflow, dbt, or similar orchestration/transformation tools.