Software Engineer Test Infrastructure

About the Role

Tario is hiring a Software Engineer, Test Infrastructure to build the systems that help a fast-moving engineering team ship reliably and confidently. This is an engineering role, not a traditional QA role. The focus is on designing and owning test infrastructure, automation frameworks, evaluation harnesses, and regression systems for modern product workflows, including non-deterministic AI-driven experiences.

You will work closely with product and engineering teams to identify failure modes, design meaningful testing strategies, and create durable quality systems that scale with the product. This role is ideal for someone who enjoys building from scratch, thinking deeply about product behavior, and solving hard testing problems where outputs are not always deterministic.

Key Responsibilities

• Build and own end-to-end automation using Playwright in Python or TypeScript.
• Design evaluation and regression frameworks for agentic AI workflows where outputs may vary across runs.
• Develop API and integration test layers, along with fixtures, seeding, and test data infrastructure for reliability.
• Integrate automated tests into CI pipelines so failures are fast, readable, and trusted.
• Identify and eliminate flaky tests instead of relying on retries.
• Stress test critical product areas such as email deliverability, sequence timing, multi-channel orchestration, and CRM synchronization.
• Investigate production issues end to end and convert them into permanent regression coverage.
• Contribute product and engineering feedback by highlighting risks and failure modes early.
• Help define what should be tested, why it matters, and where engineering effort will have the most impact.

Required Qualifications

• 1 to 2 years of hands-on engineering experience in a product environment.
• Practical experience with Playwright or equivalent depth in a modern automation framework.
• Strong Python skills with comfort in async programming, HTTP, and writing maintainable code.
• Experience working with APIs, databases, CI pipelines, and production logs.
• Ability to build test infrastructure and automation systems from scratch.
• Strong technical curiosity and problem-solving ability, especially around AI-driven workflows.
• Clear communication skills and comfort working in a fast-paced, collaborative environment.
• Ownership mindset with a focus on business impact, reliability, and engineering quality.

Preferred Skills

• Experience building evaluation suites, prompt regression harnesses, or tooling for LLM-based features.
• Exposure to testing non-deterministic systems or agentic AI products.
• Experience shipping as a developer and moving into quality-focused systems engineering.
• Strong writing skills for bug reports, technical notes, and tradeoff documentation.
• Ability to prioritize defects by user impact and business risk.
• Evidence of side projects, scripts, internal tools, or self-initiated technical builds.

Why Join Tario

• Work on AI-native systems where intelligence is core to the product.
• Have direct impact on product reliability and engineering velocity.
• Own meaningful technical problems in a small, high-trust team.
• Grow quickly through autonomy, scope, and deep engineering challenges.