Data Engineer

Overview

The Role:

The Data Engineer will play a critical role in seamlessly integrating new clients into our performance marketing platform. This individual will be responsible for designing, building, and maintaining robust data pipelines that enable efficient client data ingestion, transformation, and integration across various marketing channels. The ideal candidate will possess strong technical expertise in cloud-based data engineering, exceptional problem-solving abilities, and the capacity to work closely with clients and internal teams to ensure smooth onboarding experiences.

Responsibilities

Key Accountability:

• Design and implement scalable data pipelines using PySpark and Databricks, alongside AWS services (S3, Athena, Glue, Lambda), to support client onboarding across multiple marketing channels.
• Develop and maintain API integrations with marketing platforms including Amazon Marketing Cloud, Amazon Ads API, and third-party systems using OAuth 2.0 authentication.
• Build robust web applications and microservices using Python and Flask to support client onboarding workflows and automated data processing tasks.
• Manage cloud infrastructure and containerized deployments using AWS services, Docker, and CI/CD pipelines to ensure reliable and scalable client integrations.
• Design database schemas and optimize SQL queries for efficient data storage, retrieval, and transformation across PostgreSQL and MySQL, and model data in DynamoDB for scalable NoSQL workloads.
• Monitor data quality and pipeline performance using CloudWatch, implementing automated alerting and proactive issue resolution systems.
• Collaborate with client success and technical teams to translate business requirements into scalable technical solutions for seamless client onboarding.
• Troubleshoot data integration issues and maintain documentation to ensure efficient problem resolution and knowledge transfer across teams.

Qualifications

The Criteria:

• Bachelors degree in Computer Science, Engineering, Data Science, or related technical field.
• 3-5 years of experience in data engineering, software development, or similar technical role.
• Proficiency in Python with hands-on experience in Flask or similar web frameworks for building APIs and web applications.
• Extensive experience with PySpark and Databricks for distributed data processing and large-scale data transformations.

Solid understanding of SQL and practical experience with relational databases including PostgreSQL and MySQL, plus working knowledge of DynamoDB

• Practical expertise with AWS services:
  • Data & Compute: S3, Athena, Glue, Lambda, EC2, ECS, ECR
  • Messaging & Scheduling: EventBridge, SQS, SES
  • Security & Monitoring: Secrets Manager, CloudWatch

• Experience with Docker containerization, CI/CD pipeline implementation using Jenkins or similar tools, and version-control collaboration using GitHub and Bitbucket.
• Strong problem-solving skills with attention to detail and ability to work in fast-paced, client-facing environments.
• Excellent communication skills with ability to explain technical concepts to non-technical stakeholders.