Overview
We’re looking for an AI Architect to lead how our testing organization uses AI to improve quality, speed, coverage, and insight across the software delivery lifecycle. This person will define the AI strategy for QA and testing, design AI-enabled testing solutions, guide tool selection and integration, and help teams adopt AI responsibly in day-to-day testing activities.
Key responsibilities:
Define the AI vision and architecture for the testing function.
Identify testing activities that can be improved with AI, including:
test case generation
test data creation
automated test maintenance
defect prediction
root cause analysis
log and failure analysis
requirements-to-test traceability
risk-based test prioritization
intelligent regression selection
Design and implement AI-enabled frameworks for:
functional testing
API testing
UI automation
performance testing
security testing support
exploratory testing augmentation
Partner with QA, developers, DevOps, product managers, and data teams to embed AI into CI/CD and quality engineering workflows.
Evaluate and recommend AI/ML, GenAI, and test automation tools.
Establish standards for AI governance, privacy, security, explainability, and responsible usage in testing.
Create architecture for integrating LLMs, test repositories, defect systems, observability tools, and CI platforms.
Define metrics to measure the effectiveness of AI in testing, such as:
defect leakage
automation coverage
test execution efficiency
flaky test reduction
mean time to diagnose failures
release quality trends
Mentor QA engineers and SDETs on AI-assisted testing practices.
Lead pilots, proofs of concept, and scaled rollout of AI testing capabilities.
Ensure compliance with enterprise policies for data handling and model usage.
Required qualifications:
Bachelor’s or Master’s degree in Computer Science, Engineering, Data Science, or related field.
8+ years in software testing, QA automation, quality engineering, or test architecture.
3+ years designing or implementing AI/ML or GenAI-based solutions in enterprise environments.
Strong understanding of:
software testing methodologies
test automation frameworks
SDLC/STLC
CI/CD pipelines
quality engineering practices
Experience with AI technologies such as:
machine learning models
NLP
LLMs
prompt engineering
retrieval-augmented generation concepts
AI agents/workflow automation
Hands-on experience with tools/languages such as Python, Java, JavaScript, SQL, and test tools like Selenium, Playwright, Cypress, Appium, JUnit, TestNG, PyTest, or similar.
Experience integrating with platforms such as Jira, Azure DevOps, GitHub, Jenkins, GitLab, or cloud test platforms.
Knowledge of test analytics, data pipelines, and observability/log analysis tools.
Strong architecture and design skills for scalable, secure enterprise solutions.
Preferred qualifications:
Experience building AI copilots or assistants for QA teams.
Familiarity with vector databases, model evaluation, and MLOps/LLMOps concepts.
Experience with cloud platforms such as AWS, Azure, or GCP.
Understanding of regulated environments and compliance requirements.
Certifications in test architecture, cloud, AI/ML, or enterprise architecture.
Key skills:
AI strategy for testing
test architecture
quality engineering leadership
GenAI and LLM application design
automation framework design
data-driven decision making
stakeholder communication
governance and risk management
Team enablement
Sample deliverables in this role
AI testing roadmap for 12–24 months
reference architecture for AI-enabled test automation
standards for prompt usage, model access, and test-data protection
POCs for self-healing automation and AI-generated test cases
dashboards for AI testing ROI and quality impact
Pay: $80.00 - $90.00 per hour
Work Location: Hybrid remote in Chicago, IL 60661
