Software Engineer - Data Engineer

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.

Primary Responsibilities

• Design and develop ETL/ELT solutions on Azure Databricks, LakeBase and Spark
• Develop, implement, and deploy large scale data pipelines empowering machine learning algorithms, insights generation, business intelligence dashboards, reporting and new data products
• Design, build, optimize, and manage modern large-scale data pipelines ETL/ELT processing to support data integration for analytics, machine learning features and predictive modelling
• Consume data from a variety of sources (RDBMS, APIs, FTPs and other cloud storage) & formats (Excel, CSV, XML, JSON, Parquet, Unstructured)
• Write advanced / complex SQL with performance tuning and optimization
• Build AI based solutions for solving business needs, process automations and improving operational efficiency
• Identify ways to improve data reliability, data integrity, system efficiency and quality
• Participate in architectural evolution of data engineering patterns, frameworks, systems, and platforms including defining best practices and standards for managing data collections and integration
• Mentor other data engineers and provide technical direction by teaching other data engineers how to leverage cloud data platforms
• Comply with all applicable Company policies, procedures, and business directives, changes including those relating to work location, team assignments, work schedules, and flexible work arrangements
  
  

• 3+ years of experience in data engineering, data integration, data modeling, data architecture, and ETL/ELT processes to provide quality data and analytics solutions
• 3+ years of experience in Python
• 3+ years of experience with API design and lifecycle management (GraphQL, REST, etc.)
• 2+ years of hands-on experience with Redis-backed state management and event-driven processing
• 2 + years of experience in Apache Spark (PySpark/Spark SQL)
• 2+ years of experience building and deploying Cloud based solutions using - Azure Databricks with UC, Snowflake, Functions, Service Bus
• 2+ years of experience in SQL with designing complex data schemas and query performance optimization
• Experience building LLM integrations for workflow automation/business needs
• Experience with DevOps automation with Terraform
• Experience with CI/CD process and tools - GitHub Actions, GIT, Artifactory, Sonar