Gen AI And Data Engineering Azure

Summary

We’re seeking a hands-on engineer who blends GenAI expertise with solid data engineering skills to design, build, and operationalize AI-enabled solutions on Azure. You’ll work across the lifecycle—from ingesting and modeling data to orchestrating LLM-powered workflows (incl. RAG)—with strong emphasis on reliability, security, cost efficiency, and measurable outcomes. The ideal candidate is comfortable converting moderately detailed requirements into robust designs and high-quality code. This role is strictly involved in development of product and does not involve direct access to Protected Health Information (PHI), Personally Identifiable Information (PII), or any secured or confidential client data. The work is limited to product development and does not include handling or processing of sensitive health or personal information.

Your role in our mission

Design and implement retrieval-augmented generation (RAG) pipelines: document preprocessing, chunking, embeddings, hybrid retrieval, grounding/citations, prompt templating.

Build and integrate MCP (Model Context Protocol) tools/servers to expose enterprise systems and data sources to LLMs securely.

Develop orchestration layers (tool/function calling, guardrails, prompt contracts) using Azure OpenAI with frameworks like Llamaindex.

Build custom tools that can seamlessly interact with GitHub Copilot to act as force multiplier for the developers.

Build ingestion and transformation pipelines into ADLS Gen2 (bronze/silver/gold), using efficient formats (Parquet, optionally Delta).

Implement ETL/ELT workflows and scheduling (e.g., Data Factory, Functions, Container Apps), with schema design for telemetry, operational metrics, and curated aggregates.

Productionize services on Azure (Functions, App Service/Container Apps, AKS as needed); implement CI/CD (GitHub Actions/Azure DevOps), testing, observability (App Insights/Log Analytics).

What We're Looking For

• Experience (5–7 years) delivering production systems that combine data engineering with LLM applications.
• Azure OpenAI hands-on experience (prompting, model selection, evals, guardrails) and LLM orchestration using Semantic Kernel or LangChain.
• RAG expertise: embeddings, hybrid search, chunking, query rewriting, grounding with citations; Azure AI Search (vector + keyword).
• MCP (Model Context Protocol): building/integrating tools/servers to expose enterprise data to models securely.
• Data Engineering (Azure): ADLS Gen2 (bronze/silver/gold), ETL/ELT pipelines (Data Factory/Functions/Containers), Parquet; solid schema design and partitioning strategies.
• Programming: Strong Python (pandas), solid SQL; working TypeScript for integration or UI layers.
• Analytics: Practical statistics (A/B tests, t‑tests, DiD), time-series analysis; Power BI modeling and dashboards.
• DevOps: CI/CD (GitHub Actions/Azure DevOps), Docker; secrets and configuration management (Key Vault); logging/monitoring (App Insights).
  

• Remote opportunity
• Willingness to report to Hyderabad office whenever required