DNEG - Data Engineer - ETL Python

Job Description

As a Data Engineer, youll architect and maintain the pipelines that power our products and services.

Youll work at the intersection of ML, media processing, and infrastructure; owning the data tooling and automation layer that enables scalable, high-quality training and inference.

If youre a developer who loves solving tough problems and building efficient systems, we want you on our team.

Key Responsibilities

• Design and maintain scalable pipelines for ingesting, processing, and validating datasets with main focus visual and voice data.
• Work with other teams to identify workflow optimisation potential, design and develop automation tools, using AI-driven tools and custom model integrations and scripts.
• Write and maintain tests for pipeline reliability.
• Build and maintain observability tooling in collaboration with other engineers to track data pipeline health and system performance.
• Collaborate with data scientists, operators, and product teams to deliver data solutions.
• Debug and resolve complex data issues to ensure system performance.
• Optimise storage, retrieval, and caching strategies for large media assets across environments.
• Deploy scalable data infrastructure using cloud platforms as well as on-premise and containerization.
• Deepen your knowledge of machine learning workflows to support AI projects.
• Stay current with industry trends and integrate modern tools into our Haves :
• 3+ years in data engineering or related backend/infrastructure role.
• Strong programming skills in Python or similar languages.
• Experience with software development lifecycle (SDLC) and CI/CD pipelines.
• Proven experience building and testing data pipelines in production.
• Proficiency in Linux.
• Solid SQL knowledge.
• Experience with Docker or other containerisation technologies.
• Proactive approach to solving complex technical challenges.
• Passion for system optimisation and continuous learning.
• Ability to adapt solutions for multimedia data To Have :
• Experience with Kubernetes (k8s).
• Knowledge of machine learning or AI concepts.
• Familiarity with ETL tools or big data frameworks.
• Familiarity with cloud platforms (e.g., AWS, GCP, Azure).