Automation Engineer

Role: Agentic QA & Simulation Engineer

Function: Quality Engineering / AI Systems Testing

Location: Bangalore

Type: Full-time

Industry: Artificial Intelligence, Commerce, Payments, Logistics, Enterprise SaaS

About Company

The company is building a foundational agentic AI platform at India scale. It operates across commerce, payments, and logistics for SMBs, SMEs, and MSMEs.

The platform interprets intent through voice and context, autonomously matching demand with fulfilment. It eliminates operational complexity for millions of small businesses entering the digital economy.

Backed by large-scale infrastructure, the company operates with startup intensity and high ownership. This is a rare, once-in-a-generation platform build with national-scale impact.

Position Overview

As an Agentic QA & Simulation Engineer, you will design and build simulation-driven quality systems that stress-test agentic behaviors across real-world and edge-case scenarios. This is not a traditional QA role — you will architect synthetic environments, validation frameworks, and automated pipelines that ensure agent reliability and correctness at India scale. You will be a foundational member of the quality engineering function, setting the standard for how agentic systems are validated before they touch millions of businesses.

Role & Responsibilities

• Design and build scenario simulators for retail and operational environments that replicate real-world agent interactions at scale
• Create synthetic test environments to validate agent behavior across diverse, multi-step workflows in commerce, payments, and logistics
• Architect and implement agent validation frameworks that assess correctness, robustness, and failure recovery
• Build and maintain fully automated testing pipelines covering functional, behavioral, and chaos testing scenarios
• Define edge-case and failure scenario libraries to ensure agents degrade gracefully under unexpected conditions
• Integrate LLM evaluation harnesses to assess agent output quality, intent accuracy, and decision correctness
• Collaborate with platform and agent engineers to embed quality gates early in the development lifecycle

Must Have Criteria

• 1–3 years of experience in automation-heavy QA, with hands-on involvement in building or significantly contributing to test frameworks
• Hands-on experience with Playwright and REST API automation for end-to-end test coverage
• Strong Python proficiency for test orchestration, framework development, and scripting automation pipelines
• Proven experience automating complex, multi-step business workflows (not just unit or UI tests)
• Experience designing behavioral or scenario-based testing for non-deterministic systems (AI, ML, or rule-based agents)
• Familiarity with chaos testing methodologies and failure injection to validate system resilience
• Experience building or integrating LLM evaluation frameworks or harnesses (e.g., RAGAS, DeepEval, or custom harnesses)

Nice to Have

• Hands-on experience with LangGraph, n8n, or similar agentic workflow orchestration tools
• Prior exposure to distributed systems testing — validating behavior across async, event-driven, or microservices architectures
• Experience in a 0→1 startup or platform build where quality infrastructure was defined from the ground up
• Domain exposure to e-commerce or retail systems — order management, catalogue, inventory, or payments workflows
• Contributions to open-source testing frameworks or published evaluation methodologies for AI/LLM systems

What We Offer

• Opportunity to build the quality foundation of a national-scale agentic AI platform from scratch
• High ownership, low hierarchy — direct influence on how agent reliability is defined and enforced
• Work alongside top-tier AI and platform engineers on genuinely hard, first-principles problems
• Backed by large-scale infrastructure with the operating model of a focused, high-accountability startup
• Meaningful impact: your systems will directly protect outcomes for millions of small businesses across India