Operations

Analytics, ROI & knowledge gaps

What the bot did, what it was worth, and what it could not answer — and how to turn the third into content.

9 min readUpdated August 2026

Key takeaways

  • Analytics is account-wide; ROI and Knowledge Gaps are per bot
  • The ROI figure is only as honest as the hourly rate and handling time you enter
  • Knowledge Gaps finds questions the bot answered badly and drafts a Q&A pair for each
  • Resolve a gap once you have fixed it, so the list stays a to-do rather than an archive
  • Like and dislike rates are the cheapest quality signal you have — read the dislikes

Three screens answer three different questions, and conflating them is how teams end up with dashboards nobody acts on. Analytics tells you what happened. ROI tells you what it was worth. Knowledge Gaps tells you what to do next — and it is the one that actually changes anything.

1. Three screens, three questions

ScreenWhereAnswers
AnalyticsMain sidebar, account-wideHow much is this being used, by whom, and are they coming back?
ROI & MetricsBot sidebar → AdvancedWhat is this worth in hours and money?
Knowledge GapsBot sidebar → AdvancedWhat did it fail to answer, and what should I write?

2. Analytics

Account-level usage across all your bots — reachable from the main sidebar rather than inside a specific bot.

MetricHow to read it
Unique usersReach. Rising means distribution is working — say nothing yet about quality.
New vs returning usersThe honest engagement signal. People come back to things that worked.
Return rateThe same signal as a ratio, and the one to trend over months rather than days.
Average conversations per userDepth of use. A jump here often means the bot became genuinely useful to a subset.
Average session durationDeliberately ambiguous. Long sessions mean engagement or struggle — read transcripts to tell which.
Like / dislike rateDirect user judgement, and the cheapest quality metric you have.

Usage is also broken down by model, and by which users are consuming the most message credits — useful when an allocation runs down faster than expected and you need to know where it went.

3. ROI & Metrics

Per bot, under Advanced. It converts conversations into hours saved and money saved, over Today, This week, This month, This year, Last 7 days or Last 30 days — each shown against the previous comparable period.

Configure it before you quote it

The model rests on two inputs you set yourself, plus a currency:

InputWhat to put in it
Hourly rateThe fully-loaded hourly cost of whoever would otherwise handle these conversations — not their salary divided by 2,080.
Average human handling timeHow many minutes a person would really spend on one of these, in minutes. Measure a handful rather than guessing.

Conservative inputs also make the trend more useful, because a figure you believe is a figure you will act on when it moves.

4. Knowledge Gaps

The most directly actionable screen in the product. It analyses real conversations and surfaces the questions your bot handled badly — the ones where it had nothing useful to retrieve.

The workflow

  1. 1

    Run the analysis

    Open Knowledge Gaps under Advanced. It works over real conversations, so it needs some traffic before it says anything interesting.

  2. 2

    Read the detected gaps

    Each one is a question a real person asked and did not get a good answer to. Some are genuinely out of scope; most are content you have not written yet.

  3. 3

    Ask for a suggested answer

    Chat.co drafts one from what it does know. Treat it as a starting point — you are the authority on your own business, and the draft is not.

  4. 4

    Create the Q&A pair

    One action turns the gap into a Q&A pair in the knowledge base. The next person who asks gets the answer.

  5. 5

    Mark it resolved

    So the list stays a to-do rather than an archive. You can unresolve later if the answer turns out not to hold.

Knowledge Gaps and handoff attack the same problem from opposite ends. A handoff is a failure a human caught; a gap is a failure nobody caught. Working both is what makes a bot get better over time instead of merely staying live.

5. The weekly loop

Twenty minutes, once a week. This is the whole practice, and it beats any amount of one-off tuning:

  1. 1

    Read the disliked conversations

    Start where users already told you something was wrong.

  2. 2

    Work the Knowledge Gaps list

    Create pairs for the sharp questions; note the topics that need a document.

  3. 3

    Skim the handoff notes

    Anything your team had to answer by hand more than once is content that should exist.

  4. 4

    Re-run your regression questions

    The short list from the knowledge playbook. Confirm the week's additions did not displace something that was already working.

6. Pulling the numbers out

All three surfaces are reachable programmatically with a secret key carrying analytics:read:

  • POST /client/v1/chatbots/:chatbotId/analytics — analytics for one bot
  • GET /client/v1/chatbots/:chatbotId/knowledge-gaps — detected gaps
  • GET /client/v1/message-allocation — remaining message credits
  • GET /client/v1/chatbots/:chatbotId/leads — captured leads

Gaps can also be promoted into Q&A pairs over the API, which makes an automated “draft answers for last week's gaps and open a review queue” job entirely practical. See the API reference.

Still stuck on something here?

Send us the article and the step you're on — we answer with specifics.

Contact support