Azure Data Engineer - A Con- MFT KGS CH

Azure Data Engineer is responsible for designing and developing data solutions/pipelines using Microsoft Azure cloud services. The role involves creating scalable data pipelines, integrating data from multiple sources, and implementing strong data security and privacy controls.

The Azure Data Engineer will also monitor and troubleshoot data flows, optimize data storage and processing for performance and cost efficiency, and support AI and Generative AI (GenAI) initiatives by enabling high‑quality and reliable data pipelines.

Interact with multiple stakeholders and manage the end-to-end data project delivery from project requirements, solutioning and execution and hand off to operations.

Responsibilities

• Own the architecture, design, and delivery of scalable data storage and processing solutions using Azure Data Factory, Azure Data Lake Storage, Azure Synapse Analytics, and Azure Databricks.

• Define and enforce data engineering standards, best practices, and design patterns across teams.

• Provide technical leadership and oversight for complex ETL/ELT pipelines.

• Lead data engineering support for AI and GenAI workloads, including preparation and management of training datasets, embeddings, and inference pipelines.

• Partner with data scientists, AI engineers, and solution architects to meet AI performance and governance requirements.

• Manage, mentor, and grow a team of Azure Data Engineers.

• Plan capacity, allocate work, and ensure timely delivery of initiatives.

• Drive adherence to Agile delivery practices.

• Drive optimization of storage, compute, and pipelines for performance and cost efficiency.

• Review and approve designs with long-term sustainability in mind.

Qualifications

• Bachelor’s degree in computer science, Engineering, Information Systems, or a related technical field, or equivalent practical experience.

• Experience with Azure cloud services, specifically data storage and data processing solutions.

• Strong knowledge of data warehousing and ETL (Extract, Transform, Load) concepts.

• Hands‑on experience with SQL, Python and PySpark.

• Understanding of GenAI and LLM data concepts, including training data, embeddings, and inference pipelines.

• Strong understanding of data security and privacy best practice.

• Experience with monitoring and troubleshooting data pipelines.

• Strong problem‑solving and analytical skills.

• Effective communication skills to collaborate with cross‑functional teams.

• Experience with data governance is a plus.

• Proven experience leading and mentoring teams.

• Strong communication and stakeholder management skills.

#LI-KS1
#KGS