Quality Guardian

Dubai

• Quality assurance: Zero critical/high bugs reach production; catch defects before they impact users
• Testing strategy: Design comprehensive test plans covering happy paths, edge cases, integration scenarios, performance, security
• Automation & efficiency: Build automated test suites (unit, integration, E2E) that run in CI/CD; enable fast, confident releases
• Compliance & audit: Ensure testing evidence for regulatory requirements (audit trails, traceability); support SOC2/compliance reviews
• Continuous improvement: Root-cause analysis on escaped defects; drive testing best practices across POD
  
  

• Conduct requirement review: Understand acceptance criteria, identify ambiguities; propose edge cases (empty data, duplicates, concurrent operations, network failures)
• Create test plans: Test scope (what areas?), test types (unit, integration, E2E, performance, security), timeline, resource allocation
• Design test cases: Step-by-step instructions, expected results; link to requirements (traceability)
• Prioritize testing: Focus on high-risk areas (payment processing, data integrity, security); balance breadth vs. depth based on sprint velocity
• Document testing assumptions: What's mocked? What's real? What's known to be broken (and why)?
  
  

• Execute test cases: Follow steps precisely; log actual results (screenshots/videos if UI)
• Exploratory testing: Beyond test cases; probe for edge cases and unexpected interactions
• Usability testing: Is the UI intuitive? Are error messages clear? Test on different devices/browsers
• Integration testing: Does frontend consume API correctly? Do different POD components work together?
• Regression testing: When code changes, does existing functionality still work? Use checklist of critical paths
  
  

• Build automated test suites: Unit tests (developer-written, but QA validates coverage), integration tests (API contract testing), E2E tests (user workflows)
• Choose tools: Playwright or Cypress for web UI, Postman/Rest Assured for APIs, Jest/Pytest for units
• Implement CI/CD testing: Tests run automatically on pull requests; block merge if coverage drops or tests fail
• Maintain test data: Create databases/APIs in test environment; seed with realistic data; reset after each test run
• Monitor test health: How many tests? How long do they take? Are they flaky (sometimes pass, sometimes fail)?
  
  

• Load testing: Can the system handle 10x peak traffic? Use k6, JMeter, or Locust to simulate concurrent users
• Identify bottlenecks: Where does latency occur? Database, API, frontend rendering? Collaborate with engineers to optimize
• Security testing: Test for injection attacks (SQL, NoSQL), authentication bypasses, PII exposure, insecure dependencies (OWASP top 10)
• Use security tools: Burp Suite (web app security), npm audit (dependency vulnerabilities), SonarQube (code analysis)
• Test compliance: Data residency (UAE), encryption, audit logging, access controls
  
  

• Log defects systematically: Repro steps, environment, attachments (logs, screenshots); clear description so engineers can fix without asking
• Triage severity: Critical (app down, data loss, security breach), High (feature broken, significant impact), Medium (workaround exists), Low (cosmetic)
• Assign to engineers: Clear ownership; track resolution status; re-test fix before closing
• Conduct root-cause analysis: Why did this bug escape? Testing gap? Requirement ambiguity? Design flaw?
• Drive improvements: Update test plans, automate repetitive checks, add CI/CD gates to prevent similar issues
  
  

• Pre-release AI validation: Test AI products before release to clients using curated golden datasets, real-world user scenarios, regression suites, and acceptance thresholds for accuracy, relevance, safety, latency, and cost.
• LLM and agent evaluation: Validate non-deterministic outputs using semantic evaluation, LLM-as-judge scoring, human review for high-risk cases, and repeat testing to detect instability or drift.
• RAG testing: Assess retrieval quality, source relevance, answer faithfulness, citation accuracy, fallback handling, and behaviour when knowledge is missing or ambiguous.
• Safety and red teaming: Run adversarial tests for prompt injection, jailbreaks, harmful content, data leakage, insecure tool use, biased outputs, and policy violations before client deployment.
• Client-release gates: Block release if critical/high AI risks remain open, evaluation scores fall below agreed thresholds, hallucination or safety failures exceed tolerance, or monitoring and rollback plans are incomplete.
• Production monitoring readiness: Ensure observability is in place for prompts, responses, traces, evaluation scores, user feedback, latency, token usage, cost, model drift, and incident escalation.
  
  

• Track metrics: Test coverage (%) , test execution status (passed/failed/blocked), defect escape rate (bugs found in production)
• Report to POD Captain: Weekly status (pass rate, high-risk areas, blockers); monthly trends (improving? declining?)
• Create dashboards: Visible to team; show quality health at a glance
• Establish SLAs: Defect triage within 4 hours, fix verification within 1 day
  
  

• 4+ years QA, testing, or quality engineering
• 2+ years test automation: Selenium, Playwright, Cypress, or similar; writing maintainable test code
• Experience with agile testing: Testing in 2-week sprints, test-driven development, continuous integration
• Full-stack testing: UI, API, database, integration scenarios; understand how components interact
• Analytical mindset: Root-cause analysis, metrics-driven decision-making, ability to design effective test plans
  
  

• Automation frameworks: Building test libraries, Page Object Model, custom utilities to reduce test flakiness
• Performance/load testing: k6, JMeter, understanding of scalability bottlenecks
• Security testing: OWASP knowledge, penetration testing basics, secure coding practices
• AI/ML testing: Evaluating agent outputs, testing LLM behaviors, quality metrics for non-deterministic systems
• UAE compliance: Understanding financial/healthcare regulations relevant to testing
• BDD frameworks: Gherkin syntax, Cucumber; collaborating with non-technical stakeholders on test scenarios
  
  

• Test frameworks: Playwright, Cypress, Selenium, Postman, Jest, Pytest, k6, JMeter
• Languages: JavaScript/TypeScript, Python (for test scripts and CI/CD)
• Defect tracking: Jira, Azure Boards; clear bug reports
• CI/CD platforms: GitHub Actions, Azure DevOps; understand how tests integrate into pipelines
• Performance tools: Chrome DevTools, Lighthouse, Application Insights
• Security tools: Burp Suite, npm audit, OWASP ZAP
• Collaboration: Clear communication with developers, product managers; documentation
• Modern AI testing stack: Promptfoo, DeepEval, RAGAS, LangSmith, Braintrust, Arize Phoenix, Langfuse, Microsoft Foundry AI Red Teaming Agent, PyRIT, OWASP ZAP, GitHub Actions, Azure DevOps, Docker, Kubernetes, Terraform, OpenTelemetry, Application Insights, Azure AI Foundry, Azure OpenAI, vector databases, and feature-flag or canary-release tooling