01 — Project Overview
Overview &
Context
As pharmaceutical organizations increasingly adopt Generative AI-powered chatbots to support employees, researchers, medical representatives, and customer service teams, ensuring the quality, safety, and consistency of AI-generated responses becomes a critical business requirement.
AbbVie sought to streamline its chatbot validation process by building an AI-assisted Quality Assurance platform capable of testing conversational agents across multiple scenarios. Traditional manual testing methods were time-consuming, inconsistent, and difficult to scale as new prompts, models, and features were continuously introduced.
"The objective of this project was to design an intuitive dashboard that enabled QA teams to create, execute, monitor, and analyze chatbot test cases from a single interface while leveraging Generative AI to automate repetitive validation tasks."
As the Product Designer, I collaborated closely with product managers, AI engineers, and developers to transform complex testing workflows into an efficient, user-centric experience that reduced manual effort and improved testing visibility.
02 — Problem Statement
The QA Testing
Bottleneck
Existing chatbot testing workflows relied heavily on manual execution and spreadsheet-based documentation, creating several operational challenges:
Manual Execution
Test cases had to be executed individually, making regression testing slow and resource-intensive.
Opaque Test Status
Limited visibility into test execution status, failed prompts, and chatbot response quality.
Model Benchmarking
Comparing outputs across multiple AI model versions required significant manual effort.
Library Maintenance
QA teams struggled to maintain consistency while validating large prompt libraries.
03 — Goals & Success Criteria
Primary Goals
04 — Design Responsibilities
My Product Design Role
As the Product Designer, I was responsible for translating a technically complex QA workflow into an intuitive product experience:
05 — Interactive Prototype
AI Chatbot Test Case
Simulator Console
Configure test parameters in the sidebar and click execute to run automated batch evaluations on AbbVie conversational models.
Medical Rep Assistant — Suite v4.2
STATUS: EVALUATION COMPLETEBatch evaluation completed successfully. 4 prompt deviations detected regarding dosage safety clarifications. Recommendations logged for prompt engineering review.
06 — Design System
Component Architecture
Reusable components were developed to maintain consistency across all QA workflows:
07 — Research & Discovery
Discovery & Stakeholder Alignment
We conducted cross-functional workshops with QA engineers and AI developers to map out friction points in existing chatbot testing procedures.
08 — Design Process
Iterative Methodology
Following a systematic 8-stage product design process from discovery through high-fidelity prototyping and design QA.
09 — User Journey
QA Engineer Journey Flow
10 — Design Decisions
Key Architectural Choices
Dashboard-First Experience
Immediate visibility into test execution status, failed prompts, and overall health.
AI-Assisted Validation
Automated evaluation summaries allow users to focus on meaningful discrepancies.
11 — Key Features
Modules Delivered
Test Case Management
Create, edit, organize, and execute reusable chatbot test scenarios.
Response Comparison
Compare expected outputs with AI-generated responses side-by-side.
12 — Prototype
Interactive Walkthroughs
The interactive prototype simulated complete QA workflows, enabling stakeholders to validate navigation patterns before development.
13 — Technical Challenges
Navigating Constraints
Simplifying highly technical AI evaluation workflows for non-technical users while handling high data volumes.
14 — Final Solution
Unified Testing Platform
The final solution transformed fragmented QA processes into a unified AI-powered testing platform that streamlined chatbot validation through automation, centralized reporting, and actionable analytics.
15 — Business Impact
Measured Outcomes
16 — Team Collaboration
Cross-Functional Synergy
Collaborated closely with product managers, AI engineers, QA engineers, and developers throughout implementation and Design QA.
17 — Final Deliverables
Package & Specifications
Delivered complete UX research findings, wireframes, high-fidelity Figma UIs, interactive prototypes, and design system specs.
18 — Reflections & Summary
Key Learnings
Designing for AI-powered quality assurance reinforced the importance of balancing automation with human oversight and transparent explainability.
19 — Impact Summary
Summary
The AbbVie AI QA Testing Dashboard established a robust, scalable foundation for conversational model evaluation and deployment.