AI QA Engineer

Job Description

We are looking for a highly motivated AI QA Engineer to join our growing Artificial Intelligence team. In this role, you will be responsible for ensuring the quality, accuracy, reliability, and performance of AI-powered applications, machine learning models, and Generative AI solutions throughout the software development lifecycle.

You will collaborate closely with AI Engineers, Data Scientists, Software Developers, Product Managers, and DevOps teams to develop comprehensive testing strategies for AI systems. The ideal candidate has strong experience in software quality assurance, automation testing, API testing, and validating machine learning or Large Language Model (LLM) outputs.

This position offers an exciting opportunity to work with cutting-edge AI technologies while helping deliver secure, scalable, and high-quality AI products.

Key Responsibilities

• Design, develop, and execute manual and automated test plans for AI-powered applications.
• Validate the accuracy, consistency, and reliability of machine learning model outputs.
• Perform functional, regression, integration, system, and end-to-end testing.
• Develop and maintain automated testing frameworks using Python, Java, or JavaScript.
• Test RESTful APIs, AI microservices, and cloud-based AI applications.
• Validate Large Language Model (LLM) responses for correctness, relevance, bias, and safety.
• Test Retrieval-Augmented Generation (RAG) pipelines and AI agent workflows.
• Verify data quality used for AI model training, validation, and inference.
• Execute performance, load, scalability, and stress testing for AI services.
• Identify, document, and track software defects using Jira or similar tools.
• Collaborate with developers to troubleshoot and resolve defects efficiently.
• Integrate automated tests into CI/CD pipelines for continuous quality assurance.
• Monitor AI model performance and identify model drift after deployment.
• Participate in sprint planning, code reviews, and Agile QA activities.
• Create and maintain detailed QA documentation, test cases, and validation reports.
• Ensure AI applications comply with security, privacy, and responsible AI best practices.

Required Qualifications

• Bachelor’s degree in Computer Science, Software Engineering, Information Technology, or a related field.
• 3+ years of experience in Software QA or Test Automation.
• Experience testing AI, Machine Learning, or Generative AI applications is preferred.
• Strong knowledge of software testing methodologies and SDLC/STLC.
• Experience with automation testing frameworks.
• Strong analytical and problem-solving skills.
• Excellent communication and collaboration abilities.
• Experience working in Agile/Scrum environments.

Required Technical Skills

• Python
• Java
• JavaScript
• Selenium
• Playwright
• Cypress
• PyTest
• Postman
• REST APIs
• SQL
• Git
• Jenkins
• GitHub Actions
• Docker
• Kubernetes
• Azure DevOps
• AWS or Microsoft Azure
• OpenAI API (preferred)
• LangChain (preferred)

Preferred Qualifications

• Experience testing Large Language Models (LLMs) and Generative AI applications.
• Familiarity with Prompt Engineering and AI evaluation techniques.
• Knowledge of cloud-native AI deployments.
• Experience with performance testing tools such as JMeter or Locust.
• Understanding of AI ethics, bias testing, and responsible AI principles.
• Relevant certifications such as ISTQB, AWS Certified Developer, Microsoft Azure AI Engineer Associate, or Certified Selenium Tester are a plus.

Benefits

• Competitive salary and annual performance bonus
• Medical, dental, and vision insurance
• Paid vacation, holidays, and sick leave
• Flexible remote or hybrid work options
• Professional development and certification reimbursement
• Career growth opportunities in AI and Machine Learning
• Collaborative and innovative work environment