Quality Assurance Engineer

As part of our flexible scheme, here are just some of the benefits that you’ll enjoy

• Best in class leave policy
• Gender neutral parental leaves
• 100% reimbursement under childcare assistance benefit (gender neutral)
• Sponsorship for Industry relevant certifications and education
• Employee Assistance Program for you and your family members
• Comprehensive Hospitalization Insurance for you and your dependents
• Accident and Term life Insurance
• Complementary Health screening for 35 yrs. and above

Your key responsibilities

• Design, execute, and maintain test strategies and automation frameworks for AI-powered agentic workflows
• Develop and implement specialized testing approaches for LLM-powered applications, validating response accuracy, relevance, and grounding, while identifying hallucinations and edge-case behaviours
• Test and evaluate the behaviour of autonomous and semi-autonomous AI agents, ensuring proper planning, tool selection/sequencing, agent-to-agent interactions, and graceful error/timeout recovery
• Validate Retrieval-Augmented Generation (RAG) performance, including document ingestion, chunking, metadata quality, embedding accuracy, and citation/source attribution
• Perform AI safety and security testing to identify vulnerabilities such as prompt injection (direct/indirect), data leakage, unauthorized tool execution, and privilege escalation
• Establish quantitative AI evaluation metrics and quality gates (e.g., classification accuracy, groundedness, hallucination rate, tool-selection accuracy, and cost/latency) to measure performance across releases
• Build and maintain automated test suites (covering APIs, UI, integration, RAG, and agent trajectories) to progressively transition the testing suite toward automated continuous evaluation
• Analyze production AI interactions and failed agent journeys to identify recurring failure patterns, create regression tests from production incidents, and ensure decision auditability

Your skills and experience

• Bachelor's degree or above in Computer Science, Engineering, Information Technology, or a related discipline
• 5+ years of experience in software quality assurance and testing, with strong knowledge of test planning, functional, integration, regression, and end-to-end testing
• 1–2 years of hands-on experience specifically testing GenAI, LLM, or Agentic AI applications
• Strong experience with test automation and building automation frameworks and test utilities (rather than relying solely on manual tools)
• Strong programming ability in at least one language: Java / Kotlin or Python
• Strong understanding of generative AI technologies, including LLMs, prompt engineering, RAG, embeddings, vector databases, AI agents/agentic workflows, and tool/function calling

• Hands-on experience with LLM and agent evaluation tools such as Langsmith/Langfuse, Ragas, or equivalent frameworks for dataset-based testing, custom evaluation metrics, experiment tracking, tracing, and continuous regression evaluation