Job Description
AI Test Lead
\n
\n
Status: Must be legally eligible to work where you live.
\n
Resume must be in English
\n
Location: Remote, some travel to Atlanta, GA
\n
Engagement: Contract, Long Term, Full Time
\n
\n
About the Role
\n
\n
We are looking for an AI Test Lead with 10+ years of overall testing experience, including 2+ years testing AI/ML systems, to own the quality strategy for our AI-powered applications and models. You will define how we test AI systems end-to-end, lead testing execution using modern AI tools, and set the standard for reliable, trustworthy AI delivery.
\n
\n
Key Responsibilities
\n
- \n
- Define and own the AI testing strategy for applications and models.
- Lead AI application testing across functional, integration, and user-facing behavior.
- Perform model testing and validation (accuracy, robustness, bias, drift, hallucination, and performance).
- Leverage AI-powered testing tools to design, automate, and accelerate test coverage.
- Establish Agentic test frameworks, quality gates, and acceptance criteria for AI features.
- Collaborate with data science, engineering, and product teams to embed quality early.
- Report on quality metrics, risks, and readiness for release.
\n
\n
\n
\n
\n
\n
\n
\n
\n
Required Skills & Experience
\n
- \n
- 10+ years in software QA/testing, with 2+ years hands-on testing AI/ML applications and models.
- Hands-on with AI-based testing tools and test automation.
- Ability to create and drive AI testing strategy from the ground up.
- Strong understanding of model evaluation metrics and AI failure modes.
- Proven experience leading QA teams and mentoring testers.
- Experience testing LLM / GenAI applications (prompt testing, RAG evaluation).
- Familiarity with CI/CD and MLOps pipelines.
\n
\n
\n
\n
\n
\n
\n
\n
\n
Nice to have:
\n
LangFuse, Langsmith, Test Rigor of Functionize experience