Skip to main content

Topic Analysis to Enhance Automated Resolution Rates

In customer service automation, understanding and leveraging conversation data is pivotal. One of the most powerful tools at your disposal is the analysis of topics automatically generated by your AI-agent. This analysis helps identify opportunities to improve your automated resolution rate, leading to a more efficient and satisfying customer experience.

When your AI-agent generates topics, it categorizes conversations based on the themes discussed. These topics are crucial for identifying areas where your AI-agent can improve its performance. The default sorting of topics by Contained Resolution (CR) opportunity highlights the most significant opportunities for enhancing your AI-agent's automated resolution rate.

To access topics page:

  1. Log in to the yellow.ai platform.
  2. Open Analyse > Conversation logs.
  3. Navigate to Topics.

Key metrics for topic analysis​

Article suggestion​

Based on the conversations AI has processed, articles are suggested for each topic with Article suggestion icon highlighted. These suggestions can serve as new additions to the existing knowledge base or as training materials.
image

Learn more here.

Automation Opportunity​

Automation Opportunity opportunity metric represents the total opportunity a topic has to improve your overall automated resolution rate. It is calculated as:

{Automation Opportunity} = {Unresolved conversations under a topic}/{Unresolved conversations across all topics}

This helps identify which topics have the most potential for improvement.

Automation Opportunity suggestion​

The star next to the Conversation share means that the Topic has knowledge base article suggestion generated by AI to achieve CR.

Conversation share​

This metric shows the proportion of conversations involving a particular topic compared to all conversations: {Conversation Share} = {Conversations under a topic}/{Conversations across all topics}

It helps prioritize topics based on their frequency.

For example, in the below screenshot, Out of 4021 conversations taken place in this AI-agent, 144 belong to this topic.

AI resolution rate​

The AI resolution rate indicates the percentage of conversations on a topic that were successfully resolved by the AI-agent without human intervention:

{AI resolution rate} = {Conversations in this topic that were contained AND resolved}/{Conversations in this topic}

A higher AI resolution rate signifies better AI-agent performance in resolving issues autonomously.

Automation rate​

This metric measures the percentage of conversations on a topic that were not escalated to a human agent:

{Automation Rate} = {Conversations in this topic not handed over to a human agent}/{Conversations in this topic}

A higher Automation rate indicates greater efficiency in handling the topic without needing human support.

Sentiment​

This metric assesses the sentiment of users during conversations about a specific topic. It shows the percentage of positive, negative and neutral conversations that have taken place while discussing about this topic. Understanding user sentiment helps in identifying areas where the AI-agent's responses might need improvement to enhance customer satisfaction.

For example, in the below screenshot, out of 1045 conversations in this topic, 550 (52.6%) were positive, 364 (34.8%) were negative, and 131 (12.5%) were neutral.

image

Search & Filter Topics​

You can search for a specific topic using the global search feature.
image

To filter data and view a particular topic, click Filter.

The following filters are available:

  1. Article Suggested: Select either "Yes" or "No."
drawing

  1. Timestamp: Filter data for specific dates or a custom time range.

  2. Topics: Filter by the text or subtext of the topics.

drawing

Expanded insights into each topic​

Topic details​

Click on the Topic name to view a detailed analysis of the selected topic.

Topic details

Date filter​

By default, analytics for the selected topic are filtered to the past 30 days. You can adjust the time period by selecting a different date range.

Date filter

Automation Opportunity​

This represents the percentage of conversations that either were not resolved or not contained, calculated as:

Automation Opportunity = (Total unresolved/uncontained conversations in this topic) / (Total unresolved/uncontained conversations across all topics).

Conversation share​

This shows the percentage of total conversations for the selected topic relative to all conversations across topics, calculated as:

Conversation share = (Total conversations in this topic) / (Total conversations across all topics).

You can view all conversations under a specific topic by clicking View conversations.

View conversations

On the Conversations page, you can read each conversation belonging to that topic. Use the filter icon to refine conversations based on your criteria:

  • Contained resolution: Setting this to True filters conversations that are both contained and resolved. Setting it to False filters conversations that are either contained but unresolved, not contained but resolved, or not contained and unresolved.
  • Contained: Filters all contained conversations, regardless of resolution status.
  • Resolved: Filters all resolved conversations, whether contained or not.
  • User sentiment: Filter conversations by sentiment categories such as Positive, Negative, or Neutral.
  • Automation: Select Available to filter conversations whose resolutions are used for automation purposes.

Conversation filters

For each conversation, you will see details such as:

  • Analysis: Whether it was contained/uncontained
  • User query: The specific query
  • Resolution: Whether it was resolved/unresolved
  • User sentiment: Categorized as Positive, Negative, or Neutral

Conversation details

info

This data can be viewed in Data explorer > Contained resolution analysis.

Visualized results​

Key metrics such as CR rate, containment rate, and user sentiment for the selected topic are displayed as graphs for the selected time period. These values, already available on the Topics page, are visualized in graph form to help you understand trends more effectively. You can view conversations pre-filtered by clicking View all.

Visualized results

Automation Opportunity suggestions​

The AI-agent also provides suggestions based on analyzed conversations:

  • Improve Knowledge Base: Click View conversation to see conversations that the AI used to recommend a new knowledge base article.
  • Analyze conversations: Click View conversation to identify opportunities for AI-agent improvement.

CR suggestions

Utilize topics for AI-agent improvement​

By closely monitoring these metrics, you can gain actionable insights into your AI-agent's performance and identify areas for enhancement. Here are some steps to leverage topic analysis effectively:

  • Prioritize high-opportunity topics: Focus on topics with high CR Opportunity to make the most significant impact on your automated resolution rate. These are the areas where improving the AI-agent's responses can yield the highest returns.
  • Analyze low containment rate topics: Investigate topics with low containment rates to understand why users are being escalated to human agents. This can help in refining the AI-agent's responses or providing better training data.
  • Enhance CR Rate: For topics with lower CR Rates, consider revising the AI-agent’s dialogue scripts, adding more detailed FAQs, or improving the AI-agent’s understanding through advanced natural language processing (NLP) techniques.
  • Monitor sentiment: Keep an eye on user sentiment for each topic. If users consistently express negative sentiments, it’s a signal that the AI-agent’s handling of that topic needs improvement.
  • Iterate and test: Regularly update and test the AI-agent’s responses based on the insights gained from topic analysis. Continuous iteration helps in gradually enhancing the AI-agent’s performance and increasing the automated resolution rate.