AI Chatbot Support Metrics: Containment, Deflection, CSAT & Cost per Resolution

What you'll build

What you will be able to do

Each lab ends with a concrete result that you build yourself and the checker confirms, cell by cell.

• Size Quill Assist's September workload: each intent's share of all conversations, the change against August, conversations per 1,000 accounts and the biggest intent.
• Calculate how many of Quill Assist's conversations stayed with the bot, the containment rate per intent and overall, and how much of it is really just customers giving up.
• Calculate Quill Assist's escalation rate, break the 2,700 handoffs down by reason, measure how many the bot team could fix itself and chart the reasons.
• Compare Quill Assist's deflection rate with its true resolution rate, using re-contacts within 7 days to find the conversations that only looked solved.
• Measure how many customers rate Quill Assist, the thumbs-up rate per intent and overall, and give each intent a verdict that ignores intents with too few ratings.
• Build Quill Assist's monthly cost, the agents' cost, the cost per true resolution of each and the support cost the bot's resolutions avoided.
• Audit 20 escalated Quill Assist chats: the median wait for an agent, the share answered within 60 seconds, how often customers had to repeat themselves and what that costs in agent time.
• Value the planned bot improvements per intent, divide by the effort, rank the intents for the roadmap and chart the value per effort day.