AI Quality Assurance Engineer

Location: Bangalore / Remote Experience: 5–6 Years Employment Type: Full-Time About the Role We are looking for an experienced AI Quality Assurance Engineer to help ensure the quality, reliability, and performance of our AI-powered SaaS platform. As part of our engineering team, you will play a critical role in validating traditional application functionality while also testing AI-driven features, intelligent workflows, LLM integrations, and automated decision-making systems. If you're passionate about quality engineering, test automation, and the rapidly evolving world of AI applications, we'd love to hear from you.

Key Responsibilities Design, develop, and execute comprehensive test strategies for AI-powered SaaS applications. Validate AI/ML features, including Large Language Model (LLM) outputs, recommendations, predictions, and intelligent workflows. Create and maintain automated test suites using Playwright, Selenium, or similar frameworks.

Develop test cases covering functionality, accuracy, reliability, performance, security, and user experience of AI systems. Perform prompt testing, response validation, hallucination detection, and regression testing for generative AI features. Collaborate closely with Product, Engineering, Data Science, and AI teams throughout the development lifecycle.

Execute API, database, integration, and end-to-end testing across AI and non-AI components. Design datasets and test scenarios to evaluate AI model behavior, edge cases, bias, consistency, and accuracy. Monitor production AI performance and contribute to continuous quality improvements.

Log, track, and manage defects using JIRA and participate actively in Agile/Scrum ceremonies. Support CI/CD pipelines and ensure quality gates are integrated into automated deployment processes. Required Skills 5–6 years of experience in Quality Assurance, Test Automation, or Quality Engineering.

Strong expertise in manual and automated testing for SaaS applications. Hands-on experience with Playwright, Selenium, Cypress, or equivalent automation frameworks. Experience testing APIs using Postman, REST Assured, or similar tools.

Strong knowledge of SQL and database validation techniques. Understanding of AI/ML concepts, LLMs, Retrieval-Augmented Generation (RAG), prompt engineering, and AI application architectures. Experience validating AI-generated outputs for accuracy, consistency, relevance, and safety.

Familiarity with Agile development methodologies and tools such as JIRA, GitHub, and Jenkins. Strong analytical and problem-solving skills with attention to detail. Preferred Qualifications Experience testing Generative AI applications built on OpenAI, Anthropic, Google Gemini, or similar platforms.

Proficiency in Python for automation, AI testing, and scripting. Knowledge of AI evaluation frameworks, model benchmarking, and prompt-testing methodologies. Experience with cloud platforms such as AWS, Azure, or Google Cloud.

Familiarity with vector databases, embeddings, and RAG-based systems. Exposure to performance, security, and load testing of AI-enabled applications. Experience implementing AI-assisted test automation solutions.

Why Join Us? Work on cutting-edge AI and SaaS technologies that solve real-world business challenges. Be part of a fast-growing, innovation-driven team shaping the future of AI-powered software management.

Collaborate with talented engineers, product leaders, and AI specialists. Gain hands-on experience with Generative AI, LLMs, intelligent automation, and modern QA practices. Enjoy a culture that values learning, experimentation, ownership, and continuous growth.