Azure Data Engineer

Key Skills: Customer Journey Analytics(CJA), Pyspark, Azure Synapse, Azure Databricks, Python, Adobe Experience Platform(AEP), Azure SQL, Communications, Interpersonal Skills

Roles and Responsibilities:

• Provide Azure stack technology expertise, specifically in Azure Databricks (ADB) and Azure Data Factory (ADF), for solution design and technical implementation.
• Design, develop, code, unit test, and perform code reviews for iterative platform enhancements and improvements.
• Ensure adherence to sound architecture principles, coding standards, and best engineering practices.
• Build and maintain infrastructure for enterprise data products including data pipelines, storage systems, and processing engines.
• Ensure scalability, reliability, and performance of data infrastructure while optimizing resource utilization and minimizing downtime.
• Collaborate with Data Architects and Domain Data Stewards to design and implement scalable data solutions aligned with business requirements and Data Mesh principles.
• Monitor, troubleshoot, and optimize Azure-based data pipelines and processing frameworks to ensure operational efficiency and platform stability.

Skills Required:

Must-Have

• Minimum 6+ years of experience implementing projects using Azure data engineering stack.
• Strong hands-on experience with Azure Databricks (ADB) and Azure Data Factory (ADF).
• Experience working with Azure data services such as Azure SQL Database, Azure SQL Warehouse, and Azure Synapse Analytics.
• Strong programming experience in Python and PySpark.
• Experience in data ingestion, curation, and transformation using Spark-based frameworks.
• Strong understanding of scalable data engineering architecture and cloud-based data platforms.

Good-to-Have

• Excellent verbal and written communication and interpersonal skills.
• Ability to work independently as well as collaboratively within cross-functional teams.
• Understanding of modern data platform concepts including Data Mesh and distributed data architecture.
• Experience with performance optimization and monitoring of cloud-based data systems.

Education: Bachelor's or Master's degree in Engineering