Senior AI SDET

At Reltio®, an SAP Company, we believe data should fuel your success in the enterprise AI era. Our Context Intelligence Platform turns fragmented data into a trusted, connected context so AI agents and systems can act with expert-level judgement at enterprise scale. Reltio’s cloud-native SaaS platform harmonizes, unifies, and governs data across sources and formats—including unstructured data—in real time, turning them into data assets that can be mobilized in milliseconds to any application, user, or AI agent. Trusted by more than 200 of the world’s largest brands across industries such as life sciences, financial services, healthcare, and technology, we fuel frictionless operations and help enterprises accelerate innovation and reduce risk.

At Reltio, our values guide everything we do. With an unyielding commitment to prioritizing our “Customer First”, we strive to ensure their success. We embrace our differences and are “Better Together” as One Reltio. We are always looking to “Simplify and Share” our knowledge when we collaborate to remove obstacles for each other. We hold ourselves accountable for our actions and outcomes and strive for excellence. We “Own It”. Every day, we innovate and evolve, so that today is “Always Better Than Yesterday”. If you share and embody these values, we invite you to join our team at Reltio and contribute to our mission of excellence.

Reltio has earned numerous awards and industry and analyst recognition for our technology, our culture and our people. Reltio was founded on a distributed workforce and offers flexible work arrangements to help our people manage their personal and professional lives. If you’re ready to work on unrivaled technology where your desire to be part of a collaborative team is met with a laser-focused mission to enable digital transformation with connected data, let’s talk!

Senior AI SDET (AI Engineering & Quality)

Location: Bengaluru, India – Hybrid

About Reltio

Reltios AI-powered data unification and management platform—spanning entity resolution, multi-domain master data management (MDM), and data products—transforms siloed data from disparate sources into unified, trusted, and interoperable data. The Reltio Connected Data Platform™ delivers data where and when its needed, empowering data and analytics leaders with unparalleled business responsiveness. Leading enterprise brands across industries rely on our award-winning, cloud-native MDM capabilities to improve efficiency, manage risk, and drive growth.

At Reltio, our values guide everything we do: Customer First, Better Together, Simplify and Share, Own It, and Always Better Than Yesterday. If you share these values, we invite you to join us.

Role Overview

We are looking for a powerhouse engineer who bridges AI development and automated validation. This is a builder-tester role — you will own the full lifecycle of AI features: from writing core Python logic for agentic workflows and tuning prompts, to architecting robust evaluation frameworks that ensure those systems are enterprise-ready.

Core Responsibilities

AI Development & Engineering

• Agentic Implementation: Design and implement autonomous agents using frameworks such as LangGraph, CrewAI, or AutoGen.
• Prompt Engineering: Own the prompt lifecycle — design, version, and tune system prompts to minimize hallucinations and maximize intent recognition.
• RAG Pipeline Development: Build and optimize Retrieval-Augmented Generation (RAG) components, including document ingestion, chunking strategies, and vector database indexing.
• Feature Prototyping: Rapidly prototype AI-driven features in Python to validate feasibility before full-scale integration.
• Data Curation: Build golden datasets and synthetic data generation scripts to train and evaluate models.
  
  

• Automated Evaluation (Eval-as-Code): Build automated pipelines to measure LLM performance across metrics such as faithfulness, relevancy, and toxicity.
• Non-Deterministic Testing: Develop strategies to test "fuzzy" outputs using LLM-assisted evaluation (using one LLM to grade another).
• Hybrid Framework Development: Design and maintain a dual-stack automation framework covering both backend/infrastructure and AI/ML validation.
• End-to-End Orchestration: Integrate AI tests into the MLOps pipeline so that every model deployment or prompt change triggers a full regression of the agents reasoning capabilities.
• Mentorship & Standards: Serve as the teams subject matter expert, defining standards for writing inherently testable AI code.
  
  

• Python: Strong proficiency with Pandas, NumPy, LangChain, and LlamaIndex.
• LLMs: Hands-on experience with OpenAI, Anthropic, or open-source models (Mistral, Llama); solid understanding of temperature, top-p sampling, and tokenization.
• Vector Databases: Experience with Pinecone, Milvus, or Weaviate.
• AI Evals & Observability: Practical experience building LLM-judge evaluation pipelines.
  
  

• Production-Grade Code: Ability to write and review complex automation suites for distributed systems.
• Architecture: Familiarity with microservices, async processing (Kafka/RabbitMQ), and RESTful APIs.
• Performance Testing: Experience with JMeter or Locust to benchmark LLM inference latency and streaming responses.
  
  

• 5+ years of experience as an SDET, QA Automation Engineer, or AI Engineer.
• 2+ years of hands-on experience testing AI/ML products.
• Experience integrating AI tools and agents into quality lifecycle management.
• Proven ability to write and review production-quality code.
• Solid background in test case design, test planning, and issue lifecycle management in a continuous deployment environment.
• Experience working in agile environments and testing highly scalable SaaS applications.