AI Customer Support
Automation
End-to-end requirements and process design for an AI-powered support automation project — scoping chatbot boundaries, defining escalation rules, and delivering a 67% ticket automation rate across 10 weeks.
Executive Summary
UrbanCart, a B2C e-commerce platform processing 50,000 orders per month, was struggling with a support operation that couldn't scale. Five agents were handling 2,400 tickets per month — 68% of which were identical, low-complexity queries (order status, returns, password resets). Average first response time had reached 18 hours, and CSAT had dropped to 61%.
As Lead Business Analyst, I was brought in to scope, define, and oversee the delivery of an AI-powered support automation solution using Zendesk Suite and Answer Bot. My work spanned 10 weeks — from discovery and process mapping through to UAT sign-off and go-live — resulting in 67% of Tier-1 tickets being fully resolved by automation, with no agent involvement.
Key Outcomes at a Glance
Project Overview
Context, team, and delivery phases at a glance.
| Attribute | Detail |
|---|---|
| Client | UrbanCart — B2C e-commerce platform (fashion & lifestyle) |
| Company Size | 120 employees · 5 support agents · 50,000 orders/month |
| Project Duration | 10 weeks — January 13 to March 21, 2025 |
| My Role | Lead Business Analyst — discovery, requirements, conversation design, UAT |
| Team | BA, Product Owner, 2 Zendesk developers, 1 QA Analyst, Support Team Lead |
| Platform | Zendesk Suite Professional + Zendesk Answer Bot (AI) |
| Methodology | Agile — 3-week discovery/design + 3 × 2-week delivery sprints + 1-week go-live |
| Key Deliverables | AS-IS process map · Bot conversation flows (18 intents) · BRD · 32 UAT test cases · Training docs |
Delivery Timeline
Agent interviews, ticket data audit, category analysis, AS-IS process mapping
18 bot intents defined, escalation rules, BRD, conversation flow wireframes, stakeholder sign-off
Sprints 1–3: Zendesk bot build, knowledge base articles, routing rules, integration testing
32 UAT test cases executed, agent training (5 sessions), phased bot activation
Business Problem
What was breaking, why it mattered commercially, and what the cost of inaction was.
UrbanCart's support team was receiving 2,400 tickets per month — a volume that had grown 80% over 18 months as the platform scaled, but headcount had only grown from 3 to 5 agents. After auditing 3 months of ticket data, I found that 68% of all tickets fell into just 6 repeatable categories that required no judgment: order status, return initiation, refund tracking, password reset, discount code issues, and delivery address changes.
Despite being simple queries, each took an average of 8 minutes of agent time to resolve — because agents had to manually look up order data, copy-paste responses, and close the ticket. At 2,400 tickets per month and a fully loaded agent cost of £28/hour, the company was spending ~£10,700/month on queries that a bot could resolve in seconds.
Stakeholder Analysis
Who I engaged, their stake in the outcome, and how I managed each relationship.
| Name / Role | Power | Interest | Primary Concern | My Engagement Approach |
|---|---|---|---|---|
| Sarah Yates — Head of Operations | High | High | Cost reduction + scalability | Weekly steering meetings; project sponsor; sign-off authority |
| Dev Patel — Support Team Lead | Med | High | Agents losing their jobs to automation | Involved from Day 1; co-designed escalation rules; UAT champion |
| 3 Support Agents (interviewed) | Low | High | Job security, ease of use post-go-live | Individual interviews; daily stand-ups during UAT |
| Raj Nair — CTO | High | Med | Data security, Zendesk integration quality | Architecture review in Wk 2; monthly update |
| Amit Shah — CFO | High | Low | ROI justification, cost per ticket | ROI model shared in Wk 1; monthly exec briefing |
Requirement Gathering
Techniques I used to understand the problem before proposing a solution.
| Technique | When | Participants | Output | Key Finding |
|---|---|---|---|---|
| Ticket Data Analysis | Wk 1 | BA + Zendesk export | Category breakdown of 3 months of tickets | 68% of volume = 6 repeatable categories |
| Agent Shadowing | Wk 1 | 3 agents (2 hrs each) | Time-per-ticket by category, copy-paste patterns | Agents spent 40% of time on templated responses |
| Structured Interviews | Wk 1–2 | 5 stakeholders (1:1) | Pain points, non-negotiable escalation rules | Agents insisted: billing disputes must ALWAYS reach a human |
| Customer Survey | Wk 2 | 200 recent customers (email) | What customers actually want from support | 73% preferred instant self-service over waiting for a human |
| Competitor Benchmarking | Wk 2 | BA (desk research) | Bot scope and conversation flow patterns | Best-in-class: respond in < 30 sec, escalate cleanly in 1 click |
| Requirements Workshop | Wk 3 | Ops Head + Support Lead + CTO | Agreed bot scope, escalation logic, success KPIs | Defined 18 bot intents and 4 hard escalation triggers |
Current State (AS-IS)
How support actually worked before this project — the reality on the ground, not the assumption.
| Step | Activity | Tool | Owner | Problem |
|---|---|---|---|---|
| 1 | Customer submits query | Email / website contact form | Customer | No acknowledgement; customer doesn't know if query was received |
| 2 | Ticket lands in shared Zendesk inbox | Zendesk (basic) | All agents | No routing rules; agents manually pick tickets from the queue |
| 3 | Agent reads ticket and looks up order | Zendesk + Shopify admin panel | Agent | Avg 3 min of tab-switching before agent can respond |
| 4 | Agent writes manual response | Zendesk compose | Agent | For 68% of queries, the response is nearly identical every time |
| 5 | Customer responds asking a follow-up | Email thread | Customer/Agent | Simple queries generate 1.4 additional replies on average |
| 6 | Ticket closed manually by agent | Zendesk | Agent | Agents have to remember to close; 12% of resolved tickets sit open |
| 7 | Weekly manual CSAT report built by lead | Excel + Zendesk export | Dev Patel | 3 hrs of manual work every Friday; always a week behind |
Root Cause Analysis
5 Whys applied to the core problem — high response times despite a functional support tool.
5 Whys — Why is first response time 18 hours?
Why 1
Why does first response take 18 hours?
Agents are handling 480 tickets per month each — at the limit of human capacity.
Why 2
Why are agents at capacity?
Total ticket volume has grown 80% in 18 months; headcount has not kept pace.
Why 3
Why hasn't headcount kept pace?
Budget was not approved because leadership didn't have visibility on volume trends until Q4 board review.
Why 4
Why wasn't volume growth visible earlier?
No reporting automation — the CSAT/volume report was manually built weekly, giving only lagging indicators.
Why 5
Why is the report manual?
Zendesk was configured as a basic ticketing tool only — routing, automation, and reporting features were never set up.
Root Cause
The root cause was not "not enough agents" — it was that Zendesk was being used as a basic inbox when it had the capability to route, automate, and self-serve most of the volume. Hiring more agents would have treated the symptom. The fix was to unlock automation and AI capabilities the company already owned but had never configured.
Gap Analysis
Current capability vs required capability — used to define the project scope precisely.
| Capability | Current State | Required State | Gap | Priority |
|---|---|---|---|---|
| Tier-1 Query Resolution | Manual agent response | Bot auto-resolves with order data lookup | Critical | Must |
| Ticket Routing | Manual (agents pick from queue) | Rules-based auto-routing by category | High | Must |
| First Response | 18-hour average wait | Bot instant reply < 30 seconds | Critical | Must |
| Escalation to Human | Customer replies repeatedly until seen | One-click escalation from bot to live agent | High | Must |
| Knowledge Base | None — all in agents' heads | 18 structured FAQ articles powering the bot | High | Must |
| CSAT Reporting | Manual weekly export (3 hrs) | Live Zendesk dashboard (automated) | Medium | Should |
| Shopify Order Integration | Manual agent lookup in separate tab | Order data surfaced in Zendesk sidebar | High | Must |
| Out-of-Hours Coverage | 0% — tickets queue until next morning | Bot handles Tier-1 24/7 including weekends | High | Must |
Future State (TO-BE)
How every support query flows through the system post go-live.
TO-BE Support Flow
Automation Scope & Intent Design
Deciding WHAT the bot handles — the most critical BA decision in any automation project.
The most important requirement in this project was not what to automate — it was what not to automate. I led a scoping workshop with the Support Lead, Head of Operations, and CTO to define hard boundaries. Four categories were designated as human-only regardless of volume, based on customer impact risk and complaint escalation history.
| Intent / Category | Bot Handles? | Reasoning | Volume Share |
|---|---|---|---|
| Order status & tracking | ✓ Automated | Fully formulaic — pull from Shopify, reply with status + tracking link | 28% |
| Return initiation | ✓ Automated | Rule-based eligibility check + generate return label | 14% |
| Refund status update | ✓ Automated | Look up refund record, return status + estimated date | 11% |
| Password reset | ✓ Automated | Trigger account reset email via Shopify API | 8% |
| Discount code issues | ✓ Automated | Validate code, identify reason for failure, offer resolution options | 5% |
| Address change (pre-dispatch) | ✓ Automated | Check dispatch status; update if pre-dispatch, escalate if dispatched | 2% |
| Billing disputes | ✗ Human only | Risk of incorrect resolution causing chargebacks — needs human judgment | 14% |
| Complaints / negative sentiment | ✗ Human only | Bot escalation trigger: keywords detected (angry, terrible, complaint etc.) | 10% |
| Complex order issues (multi-item) | ✗ Human only | Too many edge cases; wrong resolution more damaging than a 2-hr wait | 5% |
| Account suspension / fraud flags | ✗ Human only | Legal and security sensitivity; zero tolerance for automated error | 3% |
Business Requirements
Core requirements from the BRD — agreed and signed off by Head of Operations and CTO before build began.
| ID | Priority | Requirement | Acceptance Criterion |
|---|---|---|---|
| BR-001 | Must | Bot shall automatically resolve Tier-1 queries (6 defined intents) without agent involvement | ≥ 65% of all tickets closed by bot; zero agent action required |
| BR-002 | Must | Bot shall respond to any inbound customer message within 30 seconds, 24 hours a day, 7 days a week | Response timestamp < 30 sec from ticket creation across all hours |
| BR-003 | Must | Bot shall surface Shopify order data in responses without the customer needing to repeat order details | Order number, status, and tracking link present in bot reply |
| BR-004 | Must | Bot shall detect negative sentiment and escalate to a human agent within 1 message | Sentiment keywords trigger immediate human routing — tested in UAT |
| BR-005 | Must | Customer shall be able to reach a human agent in 1 click at any point in a bot conversation | Escalation option visible in every bot message; tested in UAT |
| BR-006 | Must | When escalating, bot shall pass full conversation transcript and order context to the agent | Agent receives complete context before sending first reply |
| BR-007 | Should | Zendesk dashboard shall display live automation rate, CSAT, and ticket volume — updated in real time | Dashboard loads in < 3 seconds; CSAT data < 1 hr lag |
| BR-008 | Should | Bot shall support self-service return label generation with eligibility check (within return window) | Return label emailed to customer within 2 minutes of request |
| BR-009 | Could | Bot shall identify and suggest upsell or replacement products where relevant (Phase 2) | Out of scope Phase 1 — documented in backlog for Phase 2 |
User Stories
Sample stories from the 28-story Jira backlog, covering the three primary user types.
EPIC-01: Bot Resolution (Customer)
As a Customer, I want to check my order status instantly via chat so that I don't have to wait 18 hours for a reply when I just need a tracking number.
As a Customer, I want to initiate a return through the chat bot so that I don't have to call or email and wait for manual processing.
As a Customer, I want to be connected to a real person immediately if I'm unhappy with the bot's response so that I don't feel stuck in an automated loop.
EPIC-02: Agent Efficiency (Support Agent)
As a Support Agent, I want to see the full bot conversation and order details before I respond so that I don't have to ask the customer to repeat themselves.
As a Support Agent, I want the bot to handle all password resets and order tracking queries so that I can spend my time on queries that actually need my judgment.
EPIC-03: Visibility (Operations)
As a Head of Operations, I want a live dashboard showing ticket volume, automation rate, and CSAT in real time so that I can monitor support health without waiting for a Friday report.
As a Support Lead, I want a weekly automated CSAT summary emailed to me every Monday so that I don't spend 3 hours building it manually each Friday.
UAT & Testing
How I validated the solution against requirements before go-live.
| Test Case | Scenario | Expected Result | Actual Result | Status |
|---|---|---|---|---|
| TC-001 | Customer types 'where is my order' — valid order number on file | Bot returns order status + tracking link within 30 sec | Resolved in 12 sec | Pass |
| TC-002 | Customer initiates return for 8-day-old order (within 14-day policy) | Bot confirms eligibility, generates return label, emails to customer | Label emailed in 78 sec | Pass |
| TC-003 | Customer types 'I am furious' mid-conversation | Bot detects negative sentiment; routes to human agent immediately | Routed in 1 message | Pass |
| TC-004 | Customer clicks 'Talk to a person' button at any point | Ticket routed to human agent; full transcript attached | Context passed correctly | Pass |
| TC-005 | Customer contacts outside business hours (Sunday 2am) | Bot responds instantly; resolves Tier-1 without agent | Responded in 8 sec | Pass |
| TC-006 | Customer asks about a billing dispute | Bot recognises billing intent; routes to human with context | Routed correctly | Pass |
| TC-007 | Customer submits query with no clear intent | Bot asks clarifying question with category options | Clarification sent | Pass |
| TC-008 | Customer types profanity in first message | Bot flags as escalation; routes to agent with sensitivity note | FAIL — routed but no note | Fail → Fixed |
UAT Summary
Deployment & Go-Live
How I managed the rollout to minimise risk and ensure adoption.
Bot activated on email channel only. Web chat remained human-handled. This allowed us to monitor bot responses in a lower-stakes channel and build confidence before expanding.
Reviewed all 94 bot interactions in the first 48 hours. 81% resolved without agent. Two intents were underperforming — 'order not arrived' and 'wrong item received' — and were retuned.
Bot activated on web chat and email. All 5 agents trained in two 45-minute sessions covering: how to read the bot transcript, how escalations arrive, and how to update the bot's knowledge base.
Reviewed automation rate, escalation rate, and CSAT daily for the first 2 weeks. Flagged any bot responses that generated immediate negative feedback for retraining. Results confirmed stable by end of Week 12.
Business Impact
Measured outcomes against targets, tracked 4 weeks post go-live.
| Metric | Before | After | Change | vs Target |
|---|---|---|---|---|
| Ticket auto-resolution rate | 0% | 67% | +67% | Exceeded (target: 60%) |
| Human ticket volume / month | 2,400 | 860 | −64% | Exceeded (target: −50%) |
| First response time (human) | 18 hours | 2 hours | −89% | Exceeded (target: < 4 hrs) |
| CSAT score | 61% | 82% | +21pts | Exceeded (target: 72%) |
| Cost per ticket | £12.00 | £4.20 | −65% | Exceeded |
| Agent overtime hours | 48 hrs/month | 0 | −100% | Achieved |
| Out-of-hours resolution coverage | 0% | 67% of OOH | New capability | Achieved |
BA Skills Demonstrated
A summary of the core business analysis competencies this project exercised.
Ticket Data Analysis
Audited 3 months of Zendesk data to classify 2,400 tickets into 12 categories — revealing the 68% automatable finding that justified the entire project.
Stakeholder Management
Turned a resistant Support Team Lead into a co-designer and UAT champion by addressing job-security concerns early and making him part of the solution.
Scope Definition
Defined the automation boundary (what the bot handles vs. what it must not) — the most consequential BA decision in any AI project.
Requirements Documentation
Produced BRD with 9 traceable requirements, 28 Jira user stories, and 18 conversation flow specs — all approved before build started.
Process Mapping (AS-IS/TO-BE)
Documented the 7-step current-state support flow and redesigned it with automation touchpoints — giving the dev team a blueprint, not just a wish list.
UAT Design & Execution
Designed 32 test cases covering happy path, edge cases, and failure modes. Found 1 defect in testing (not in production). Go-live with zero critical issues.
Phased Rollout Planning
Recommended a soft launch on email-only before full go-live — a risk mitigation call that let us retune 2 underperforming intents before customers noticed.
ROI Articulation
Quantified the cost of inaction (£10,700/month in automatable agent time) to build the business case, and tracked actual savings post go-live.