Lead Data Engineer

Role: Data Engineering Lead

Function: Data Engineering

Location: Not specified

Type: Full-time

Compensation: Competitive compensation and equity

Industry: AI/Technology

About Company

A venture-backed AI company building the intelligence layer for India at massive scale. Backed by partnerships with global tech leaders like Meta and Google.

The team creates AI that serves the entire Indian user base across languages, contexts, and daily needs. Engineered from day one for 100M+ users early and 1B-ready constraints on latency, cost, reliability, and safety.

Position Overview

Youll design and implement the core data infrastructure that powers a multimodal, multilingual consumer AI platform serving 100M+ users. Youll build advanced analytics frameworks, real-time streaming systems, and monitoring infrastructure that operates under 1B-ready constraints on latency, cost, and reliability. This role offers high ownership in architecting data systems that derive insights from massive-scale AI interactions and user behavior patterns.

Role & Responsibilities

• Design and implement advanced analytics frameworks and statistical models to derive insights from user behavior and AI interaction patterns
• Build real-time streaming analytics systems using Apache Spark and Apache Flink for continuous data analysis
• Develop comprehensive business intelligence dashboards and KPI tracking systems for product performance optimization
• Architect complex analytical queries and data models in BigQuery for deep-dive analysis of multimodal AI interactions
• Build product data engineering pipelines to track user journeys, feature adoption, and conversion funnels across the AI platform
• Design and implement data infrastructure for generative AI model training, fine-tuning, and inference monitoring at scale
• Create automated reporting systems and analytical pipelines for business metrics and operational intelligence
• Lead predictive analytics initiatives and A/B testing frameworks to drive data-driven product decisions
• Implement advanced data quality monitoring, anomaly detection, and statistical validation systems for production analytics
  
  

• 10+ years of hands-on data engineering experience with strong data analytics background
• 3+ years of solid prior experience working with Databricks for large-scale data processing
• Proven experience building data systems processing petabyte-scale datasets with billions of events per day
• Expert-level proficiency in Python and SQL for data pipeline development
• Hands-on experience with Apache Airflow for workflow orchestration and pipeline management
• Proven experience with Apache Spark and Apache Flink for batch and stream processing
• Strong experience with Google BigQuery for data warehousing and analytics
  
  

• Prior hands-on experience with GraphQL for data querying and API integration
• Experience with AI/ML data pipelines and model serving infrastructure
• Background in building data systems for conversational AI or chatbot platforms
• Previous work in startup or high-growth environments with rapid iteration cycles
  
  

• Opportunity to build data infrastructure for Indias largest consumer AI platform
• Work with cutting-edge AI technology and massive scale data challenges
• High-ownership environment with direct impact on product outcomes
• Competitive compensation and equity in a high-growth AI company
• Collaborate with world-class AI/ML engineers and product teams