Skip to main content

Topic Analysis to Enhance Automated Resolution Rates

note

This feature will be released soon. More details will be shared as changes are made. Stay tuned!

In customer service automation, understanding and leveraging conversation data is pivotal. One of the most powerful tools at your disposal is the analysis of topics automatically generated by your bot. This analysis helps identify opportunities to improve your automated resolution rate, leading to a more efficient and satisfying customer experience.

When your bot generates topics, it categorizes conversations based on the themes discussed. These topics are crucial for identifying areas where your bot can improve its performance. The default sorting of topics by Contained Resolution (CR) opportunity highlights the most significant opportunities for enhancing your bot's automated resolution rate.

To access topics page:

  1. Log in to the yellow.ai platform.
  2. Open Analyse > Conversation logs.
  3. Navigate to Topics.

image

Key metrics for topic analysis​

CR opportunity​

Contained resolution(CR) opportunity metric represents the total opportunity a topic has to improve your overall automated resolution rate. It is calculated as:

{CR Opportunity} = {Unresolved conversations under a topic}/{Unresolved conversations across all topics}

This helps identify which topics have the most potential for improvement.

CR opportunity suggestion​

The star next to the Conversation share means that the Topic has knowledge base article suggestion generated by AI to achieve CR. image

Conversation share​

This metric shows the proportion of conversations involving a particular topic compared to all conversations:

{Conversation Share} = {Conversations under a topic}/{Conversations across all topics}

It helps prioritize topics based on their frequency.

For example, in the below screenshot, Out of 4021 conversations taken place in this bot, 144 belong to this topic. image

CR rate​

The contained resolution(CR) rate indicates the percentage of conversations on a topic that were successfully resolved by the bot without human intervention:

{CR Rate} = {Conversations in this topic that were contained AND resolved}/{Conversations in this topic}

A higher CR Rate signifies better bot performance in resolving issues autonomously.

Containment rate​

This metric measures the percentage of conversations on a topic that were not escalated to a human agent:

{Containment Rate} = {Conversations in this topic not handed over to a human agent}/{Conversations in this topic}

A higher containment rate indicates greater efficiency in handling the topic without needing human support.

Sentiment​

This metric assesses the sentiment of users during conversations about a specific topic. It shows the percentage of positive, negative and neutral conversations that have taken place while discussing about this topic. Understanding user sentiment helps in identifying areas where the bot's responses might need improvement to enhance customer satisfaction.

For example, in the below screenshot, out of 144 conversations in this topic, 23 (15.9%) were positive and 100 (69.4%) were negative. image

Utilize topics for bot improvement​

By closely monitoring these metrics, you can gain actionable insights into your bot's performance and identify areas for enhancement. Here are some steps to leverage topic analysis effectively:

  • Prioritize high-opportunity topics: Focus on topics with high CR Opportunity to make the most significant impact on your automated resolution rate. These are the areas where improving the bot's responses can yield the highest returns.
  • Analyze low containment rate topics: Investigate topics with low containment rates to understand why users are being escalated to human agents. This can help in refining the bot's responses or providing better training data.
  • Enhance CR Rate: For topics with lower CR Rates, consider revising the bot’s dialogue scripts, adding more detailed FAQs, or improving the bot’s understanding through advanced natural language processing (NLP) techniques.
  • Monitor sentiment: Keep an eye on user sentiment for each topic. If users consistently express negative sentiments, it’s a signal that the bot’s handling of that topic needs improvement.
  • Iterate and test: Regularly update and test the bot’s responses based on the insights gained from topic analysis. Continuous iteration helps in gradually enhancing the bot’s performance and increasing the automated resolution rate.

Containment rate for 3rd party Inbox apps​

This feature is under development.

Containment rate refers to the number of conversations handled solely by the bot without agent intervention. There are two types of such chat transfers:

  1. Active handover

    • In Yellow Inbox, chat is transferred to an agent with the event agent-transfer.
    • 3rd Party Inbox: Bot is paused, and chat is handed over to an agent with the event configured at the bot-level.

  2. Passive handover

    • Schedule callback: Bot schedules a callback by collecting user details.
    • Email: Bot creates an email ticket.
    • Ticket in 3rd Party System/Trigger Event: Bot creates a support ticket in another system or triggers a custom event.

Current limitation​

Yellow Inbox transfer are tracked with the agent-transfer event. But while using 3rd Party Inbox, the transfer depends on bot-level configuration but that is not currently trackable.

Suggested solution​

With Bot Developer access, you can configure a Third Party Handoff node by entering the same data as in the Raise Ticket node at the end of the conversation flow. This node will serve as an identifier for agent transfers, that enables you to track containment rate.