Assistant Manager

Core Technical Skills

• Strong Python programming experience
• Advanced SQL
• DBT development
• Google BigQuery
• PySpark
• Data engineering and ETL/ELT pipeline development
• Data validation and reconciliation
• Data migration experience

Python Libraries & Frameworks

• Pandas
• NumPy
• PyArrow
• Pytest
• Great Expectations or similar data quality framework
• Google Cloud Python SDK

GCP Services

• BigQuery
• Cloud Storage
• Cloud Composer / Airflow
• Dataproc
• Dataflow
• IAM
• Cloud Logging and Monitoring

Responsibilities

Lead smallscale data engineering projects. Ensure stable data processing operations. Mentor junior engineers. Provide project support.

Qualifications

SAS Migration & Code Conversion

• Analyse existing SAS datasets, SAS programs, PROC SQL, Data Steps, and SAS macros.
• Convert SAS business logic into Python, SQL, PySpark, and DBT models.
• Support migration of historical and incremental SAS data into GCP.
• Document SAS-to-GCP transformation logic and technical conversion rules.
• Assist in identifying reusable patterns for SAS macro conversion.

Python Data Engineering

• Develop Python-based data processing, automation, and validation scripts.
• Build reusable Python frameworks for data ingestion, transformation, and reconciliation.
• Use Python libraries such as Pandas, NumPy, PyArrow, and Google Cloud SDK.
• Automate data quality checks, file processing, metadata extraction, and exception reporting.
• Develop utilities to compare SAS outputs with GCP outputs.

DBT & BigQuery Development

• Develop and maintain DBT models for transformation logic.
• Convert SAS PROC SQL logic into DBT SQL models.
• Implement DBT tests for data quality, uniqueness, referential integrity, and business rules.
• Support Bronze, Silver, and Gold layer implementation in BigQuery.
• Optimize SQL queries and BigQuery tables for performance and cost efficiency.

GCP Data Pipeline Development

• Build scalable data pipelines using GCP services such as: 
  • BigQuery
  • Cloud Storage
  • Cloud Composer / Airflow
  • Dataproc
  • Dataflow
  • Cloud Functions
• Develop batch and incremental data processing workflows.
• Support orchestration, monitoring, and production deployment of pipelines.
• Implement secure data movement from legacy SAS platforms to GCP.

Data Quality, Validation & Reconciliation

• Perform source-to-target reconciliation between SAS and GCP.
• Validate record counts, financial totals, business rules, and derived metrics.
• Investigate and resolve data mismatches and transformation defects.
• Build automated reconciliation reports using Python and SQL.
• Support SIT, UAT, regression testing, and production validation.