Senior Data Engineer

About MedInsight Team

The Milliman MedInsight practice has assisted many healthcare organizations in evaluating and developing solutions to complex business problems. Our consultants are experienced in the key issues related to healthcare operations and the use of technology to support those operations. Because of our focus on those unique technology and operations issues facing the healthcare industry, we are uniquely qualified to assist organizations in solving complex business problems. Our Health IT software team has been developing and selling data warehousing solutions for over twelve years.

What Is Expected Of You At This Level

We are seeking an experienced Senior Data Engineer with 5+ years in data engineering to join our team. In this role, you will design, build, and optimize high-performance data pipelines on Azure, delivering scalable, efficient, and reliable data solutions. You will take ownership of developing and maintaining critical data workflows, ensuring they meet performance and quality standards. This position requires a strong focus on performance tuning, code optimization, deployment automation, and advanced debugging, as well as the ability to work independently and collaboratively to solve complex data challenges and drive continuous improvements across our data engineering practices.

What You Must Have

5+ years of data engineering experience, building, and managing large-scale data pipelines

• Proven experience working with Azure Databricks and PySpark/Spark for data engineering tasks, where one has,
• Demonstrated expertise in advanced data engineering techniques and a track record of delivering complex data engineering projects.
• Ability to independently code, troubleshoot, and test complex tasks using appropriate technologies.
• Experience with data modeling concepts related to reporting and analytics, including dimensional modeling and star/snowflake schemas.
• Experience leveraging AI-assisted development tools (such as GitHub Copilot, ChatGPT, Claude or similar) to enhance productivity, code quality, and problem-solving in data engineering tasks.
  
  

• Python, PySpark, Spark, Databricks, Delta Lake, and SQL with hands-on experience using Azure Data Factory, Azure
• Pipelines, Azure DevOps, and Git for version control.
• Solid understanding of data modeling and working with complex data types (arrays, structs, maps).
• In-depth knowledge of performance optimization techniques for large-scale data processing, including code tuning and resource .
• Proficiency in developing, maintaining, and enhancing CI/CD processes, including documentation and monitoring of data pipelines and systems.
  
  

• Confident, energetic self-starter with excellent interpersonal skills.
• Flexibility and adaptability to work with different technologies, platforms, and evolving requirements.
• Experience mentoring junior team members and fostering a collaborative team environment.
• Strong communication skills and ability to communicate technical concepts effectively to both technical and non-technical stakeholders.
• Demonstrates sound judgment, a sense of urgency, and a strong commitment to ethics, regulatory compliance, and business integrity.
  
  

• Understanding of DAX (Data Analysis Expressions), particularly in the context of Power BI or similar business intelligence tools.
• Exposure to business intelligence or analytics projects, bridging data engineering with reporting and visualization solutions.
  
  

• Bachelor’s or master’s degree in computer science, information systems, engineering, or a related field.