top of page
Pharmacy Interior View

Usability Testing an IVR for a National Pharmacy Chain​
(Client confidential - project anonymized under NDA)

🧭 Project Overview

​

A major U.S. pharmacy chain engaged our team for a second round of testing to evaluate the usability and customer perception of its new voice-driven appointment scheduling system. The goal was to understand whether the conversational IVR supported natural, efficient task completion and whether it aligned with customer expectations during high-stakes tasks like scheduling vaccines.

​

My role: 

  • Lead researcher & moderator

  • Designed the study protocol

  • Facilitated all seven usability sessions

  • Analyzed qualitative + quantitative data

  • Delivered insights and recommendations directly to stakeholders

​

Methods: 

  • Moderated remote usability testing

  • Wizard-of-Oz simulation

  • 7 participants, varied ages and digital comfort levels

​

Tasks tested: 

  1. Schedule a vaccine as a guest

  2. Cancel an existing vaccine appointment

  3. Schedule a COVID + flu vaccine (flu unavailable)

  4. Respond to an upsell offer

Learning to use smartphone

🎯Research Goal & Hypotheses

​

Primary Goals

  • Assess whether users understood prompts and could complete core tasks smoothly

  • Identify friction points, confusion, or breakdowns in the conversational flow

  • Evaluate user trust, perceived naturalness, and alignment with brand voice

  • Understand reactions to upsells and informational messages

  • Gather feedback on clarity, pace, and perceived intelligence of the system

​

Hypotheses

  • Users would complete tasks successfully, but may experience friction around scripted or non-natural prompts

  • Redundant spelling prompts and a lack of contextual options might disrupt conversational flow

  • System transparency (e.g., confirming personal information, explaining limitations) would influence trust and satisfaction 

  • Upsell timing would greatly affect acceptance rates

🔍Research Approach

​

Study Design

To avoid learning effects, I developed a task randomization matrix, ensuring each user experienced a different order. 

​

Moderation

I created a detailed moderator script that: 

  • Established rapport

  • Ensured safety and comfort

  • Captured pre-task perceptions of automated systems

  • Guided users through scenarios 

  • Included structured yet flexible probing questions

​

Data captured

  • Task success

  • Error patterns

  • Time on task

  • Verbatim reactions

  • Behavioral cues (hesitation, backtracking)

  • Customer Satisfaction (CSAT) scoring per task

  • Final brand + persona impressions

👥Participant Profile

​

Participants (n=7) represented: 

  • Ages ranging from young adults to seniors

  • Mixed levels of digital literacy

  • Consistent prior exposure to automated phone systems, mostly negative

  • Regular or occasional pharmacy users

​

Many expressed foundational frustration with automated systems before testing even began, offering valuable baseline sentiment.

Financial Data Charts

🧠Key findings

​​

Finding 1 - The system felt natural and human-like, greatly outperforming traditional IVRs.

 

Many users described it as efficient, natural, human-like, and more advanced than typical automated systems.

​

CSAT scores reflected this:

  • Overall CSAT = 4.7

  • Task completion = 97%

Participants frequently stated it felt like "having a real conversation."

​

Implication: Conversational tone significantly boosts trust and engagement.

Finding 2 - Name-capture flow broke conversational immersion

​

Across tasks, users consistently found the "say and spell your name" prompt unnatural, repetitive, and too long. 

​

Quotes included: 

  • "Why do I need to spell my name if it's already heard right?"

  • "It felt like talking to a person... but then spelling broke that illusion."

​

Participants suggested: 

➡️Say the name once → system attempts recognition → only if needed, request spelling.

Finding 3 - Users wanted clearer pathways to rescheduling

​

In the cancellation task, participants expected a rescheduling option and were disappointed or confused when redirected to the website. 

​

Quotes: 

  • "I wish it asked me if I wanted to reschedule instead."

  • "I didn't expect both doses to cancel at once."

​

Implication: Cancellation flows should acknowledge emotions and logistical needs (e.g., "Would you like to reschedule?")

Finding 4 - Flu-shot unavailability explanations were accepted but incomplete

​

Users trusted the flu-unavailable message but wanted actionable alternatives:

  • Scheduling a future date

  • Checking nearby locations

  • Knowing whether the shortage was local vs. system-wide

  • Receiving reminders

​

A participant described the experience as "dangling" - no clear next step.

Finding 5 - Upsell offer was relevant but poorly timed and lacked context

​

Reactions were mixed: 

  • 6/7 declined the upsell

  • Many lacked information about the vaccine

  • Some wanted to consult their doctor first

  • Others felt that upsells should come after completing the primary task

​

Quotes: 

  • "I'd want more information first."

  • "Let me finish what I called for."

Finding 6 - Small friction points added up (verification, "hold on", age messaging)

​

Examples: 

  • Users disliked being told "If you need a minute..." immediately after a question

  • Under-18 messaging felt unnecessary after providing DOB

  • Users wanted vaccine brand choice 

  • Some expected account recognition to reduce repetitive questions

​

These micro-frictions influenced CSAT for several participants.

📈 Quantitative Results

Task Completion

Across all tasks: 97% success

Only one user failed to complete a scenario (flu shot + COVID scheduling).

​

CSAT Across Tasks

Most users rated a 4 or 5 across dimensions such as clarity, ease of use, and system control.

💡Recommendations

​Based on the usability insights, I delivered a set of experience-level improvements focused on: 

  • Streamlining conversational flow to make interactions feel more natural and reduce user effort

  • Improve clarity and guidance at key decision points

  • Increase user confidence through more transparent, predictable system behavior

  • Ensuring task paths support real user expectations, especially in edge-case scenarios

  • Enhancing the timing and relevance of informational messages during the call

 

These recommendations were framed at a conceptual level for the portfolio, but in practice, they were translated into concrete, prioritized design and product adjustments for the client.​​

💡Impact

While this was an early prototype test, the research: 

  • Validated the conversational IVR approach

  • Identified high-value improvements for product and design teams

  • Revealed key opportunities to enhance trust, flow, and conversion

  • Supported the case of adding rescheduling and richer availability logic

  • Informed strategic refinements that impact millions of callers annually

​

Abstract Network Design

My Role & Contributions

What I owned: 

  • Full study design (protocols, participant flow)

  • Research instruments (moderator script, probes, test data, choice of testing platform)

  • Moderation of all participant sessions

  • Real-time observation capture + structured notes

  • Affinity-based synthesis across 7 users

  • Slide creation and delivery of final insights

​

What I brought to the project:

  • Skillful moderation that built rapport and surfaced deep insights

  • Ability to translate scattered observations into clear patterns

  • Storytelling that balanced user empathy with business needs

  • Experience with Wizard-of-Oz conversational prototypes

  • Strong synthesis across qualitative + quantitative signals

​

What I learned:

  • Even highly natural conversation flows break down with small inconsistencies

  • Users expect voice systems to be as context-aware as human agents

  • Upsell timing must respect cognitive load and task intent

  • Transparency around limitations significantly impacts trust​

bottom of page