Intent is the purpose or goal behind someone's action or words. In natural language processing, it specifically relates to determining a user's intention in a conversation, like asking a question, making a request, or giving information.
Let's say that you want to book a flight to Singapore, book a flight is the intent of this case.
On our platform, you can add intents (the purpose of the bot users' response) and utterances (the various responses used by bot users for a specific intent, for example, to book a flight the utterances can be 'flight booking' or 'book flights'). You need to train the bot to identify and understand these sentences.
Yellow.ai DynamicNLP simplifies the process of model training by utilizing advanced technology called Zero-shot Learning. This eliminates the need for tedious and error-prone training processes.
Add intents and utterances
Intents represent the intentions or goals behind user inputs, while utterances are the specific phrases or queries users use to express those intentions. By mapping utterances to corresponding intents, the bot can accurately understand and respond to user queries.
Intents are added to trigger this flow or a identify user goal and trigger this flow accordingly.
Go to Train > Intents > + Add new intent.
- Provide a name to your intent, followed by the utterances that should trigger that flow.
To add multiple utterances, click Add another utterance.
- Click Add intent when you're done.
Add utterances manually
Type in your utterance and click +Add to add the utterance to an intent.
Add suggested utterances
The Yellow.ai platform generates suggestions for utterances. In this section, you will find phrases similar to the initial utterance you provided.
The refresh button will allow you to access a fresh batch of utterances every time you click on it. You may add a few utterances by clicking the + sign next to them or add all of them by selecting the check box next to Suggested utterances.
Add utterances in bulk
To add them in bulk, select the preferred ones and click Add selected on top.
The suggested utterances can also be edited, click the pen icon to do so.
Train your bot with intents
Once you have added the intents and utterances, click Train intents to train your bot on this data.
Train your model
You can train your bot on the following features by setting the right number of epochs.
- Sentence encoder
- Click the arrow.
- Choose the Feature type and set the required number of Epochs.
Sentence Encoder: Converts input sentences or messages into numerical representations, making it easier for the bot to understand and process user inputs.
Epochs: Epoch refers to one complete iteration through the entire dataset during training. It helps the chatbot improve its performance by learning from the data multiple times.
Multilingual: Choose this option if your bot is multilingual. This allows the bot to communicate with users in different languages, catering to a broader and more diverse audience.
Bahasa: Bahasa is an Indonesian language. If you choose this option, the bot can communicate with users in Indonesian, providing more personalized and localized interactions.
- Click Train.
Map intents to flows
To trigger a flow, set this intent to that flow.
- Go to the respective flow and click the Start node.
- Choose Intent and select the intent.
Test your intents
There are two ways by which you can test your intents, you can either test the flows in which they are set or you can test them in the Tools section.
To know about testing specific flows, click here.
To test your bot's confidence and intent identification:
- Go to Tools > Test your bot and enter a phrase. Upon hitting enter, a generated code will display the bot's response to that intent.
In the example code below, the bot confidently recognizes the phrase as part of the order intent, with a confidence score of 0.999.
"text": "place order.",
Resolve conflicts in intents and utterances
A bot is trained with multiple intents and entities to improve its performance. If utterances are not classified correctly during training, it can lead to confusion. To resolve these clashes caused by unclear utterance classification, it is necessary to analyse the report..
Download utterance report
- Go to Studio > Train > Intents , click Generate utterance report.
Two reports will be sent to your registered email address.
- Utterances within intents/faqs
- Utterances across intents/faqs
- Utterances within intents/faqs report identifies similar or highly diverse utterances that require editing within a specific intent/FAQ, including clashes caused by entity features.
- Utterances across intents/FAQs report identifies similar utterances across different flows and suggests modifying one of the similar utterances, taking into account clashes caused by entity features.
Resolve conflicts among utterances
Utterance reports sent to your email address help evaluate the quality of your bot's utterances. They provide insights into the similarity of utterances within an intent and identify any common intents across different flows.
It is recommended to generate an utterance report after initial Train setup and regularly at least once a month.
Conflicts within intents and FAQ's
This report identifies conflicts between two utterances if they have a high level of similarity. It compares utterances within the same intents.
If the similarity exceeds 50%, you need to visit the respective Intents page and either delete one of the similar utterances or rephrase the sentence.
If the similarity is less than 50%, ignore it.
Conflicts across intents and FAQ's
This is a comparison between utterances of different intents.
If the similarity is more than 50%, you must go to any of the Intents page and delete similar utterances or rephrase the sentence.
The following are the recommended best practices to follow while naming intents and adding utteraces to them.
- Best practices to follow while naming intents
- Best practices to follow while adding utterances to the intents
For new bots
Bots created after August 1, 2022.
- Intent names must be at least 3 words long with unique words and no special characters.
- Ensure intent names are descriptive to enhance clarity.
- Avoid using generic names like "intent test one" or "FAQ number one".
- Poor intent names can negatively impact NLP performance, leading to false positives and unnecessary bot issues.
- In the case of cloud, renaming intent names is possible.
- The more descriptive the intent name, the better (add names with more than 3 words).
- Avoid uncommonly and business-specific abbreviations: Example: PO (purchase order ), GMV, etc - use the full forms and add synonyms if necessary. Few common abbreviations like UPI, EMI, and HR are acceptable.
- Phrase the intent name as a verb followed by a noun. Example: get a premium receipt, pay renewal amount, fetch order status.
- Keywords and sentences less than 3 words will fallback to the existing bot model and will work as-is. These types of utterances will not go to the new model.
- This model is applicable and works well for FAQs as well (since FAQs are descriptive and longer sentences)
- Suggestions are automatically enabled for all new bots by default. This default setting is crucial for improving the model and maximizing performance benefits.
For existing bots
The following are a few important pointers for bots created before August 1, 2022.
- Enable suggestions for bots where they may not be enabled. This ensures that the model is used to the fullest.
- Suggestions only show up for intents that are connected to the flows. Verify that unwanted flows are removed (or disconnected from intents).
- If the intent name is camelCase (eg: chatWithAgent) or has underscore/hyphens (eg: chat_with_agent, chat-with-agent), use the edit option to rename these following the guidelines mentioned in the above section(for new bots).
- Ensure that there is no Small Talk in FAQs / Flows. If these are present, delete them - platform small talk is enabled for all cloud bots.
- Enable suggestions for bots.
- To do this, in app.ym ensure that enableDidYouMean is set to true in app options in Function and in Tools > App Options > Prediction > Enable Suggestions.
- If there’s an existing DidYouMean function in default:response, remove it.
- Verify that the flow/journey DESCRIPTION is in line with the guidelines mentioned above.
If these are not in line and are in camelCase or have special characters, change these by going to flow settings for that flow (you need not change the journey name, only the description can be changed).
- Ensure that there is no Small Talk in FAQs/Flows. If these are present, delete them and enable platform small talk in Context Management and enable Small Talk.
Utterance & flows
|DONTs ❌||DOs ✅|
|Do not add utterances in which the only variation is Upper Case/Lower Case||Do add at least 15-20 utterances per flow|
|Do not add utterances in which the only variation is Name, Date, City etc||Do ensure that there are an equal number of utterances in each flow|
|Do not create multiple flows which have a similar purpose||Do merge flows that are subsets of other flows|
|Do not overfit the model while training||Do use the didYouMean (suggestions) feature extensively|
|Do not add utterances if a flow will only be triggered through 'Trigger Journey'||Do minimize False Positives|
|Do add abbreviations/short forms in the “synonyms” section|
|Do not add single words as utterances||Add complete sentences|
Do not add utterances in which the only variation is upper case/lower case
apply for leave
Apply for leave
apply For Leave
APPLY FOR LEAVE
please apply for leave
can you please apply for leave?
how do I apply for leave?
procedure to apply for leave
This will make the model overfit and not learn the underlying sentence structure resulting in bad performance.
Do not add utterances in which the only variation is Name, Date, City etc
apply for leave on 23rd Jan
apply for leave on 24th Jan
apply for leave on 5th Jan
please apply for leave on 23rd Jan
can you please apply for leave tomorrow?
how do I apply for leave?
procedure to apply for leave
schedule meeting with John
Schedule meeting with Adam
Schedule meeting with Ram
schedule meeting with John
please block Adams calendar tomorrow for a meeting
sync up with Ram on 3rd Jan
A few utterances (2-3) like the ones mentioned below are ok but ensure that there are other utterances that show the different variations in sentence structure
- apply for leave tomorrow
- apply for leave on 3rd
Do add at least 15-20 utterances per flow
- The minimum number of utterances in each flow heavily depends on the complexity of the bot (number, type of flows and quality of the utterances)
- More utterances are always better especially when there are less than 10 flows.
Do ensure that there are an equal number of utterances in each flow
Apply Leave flow - 50 Utterances
Leave Balance flow - 2 Utterances
Schedule Meeting flow - 30 Utterances
Apply Leave flow - 50 Utterances
Leave Balance flow - 50 Utterances
Schedule Meeting flow - 50 Utterances
- Try to maintain a balance in the number of utterances per flow
- The NLP model is robust enough to handle small variations in the number of utterances (difference of 3-5 utterances)
- For smaller bots (< 10 flows) maintaining balance is important to ensure good performance.
Do not create multiple flows which have a similar purpose
Single Flow: talk-to-agent
- Having multiple flows which have similar utterances will confuse the model since there is a high amount of overlap.
- Merge all these flows into one single flow.
Do merge flows that are subsets of other flows
Steps: What kind of loan are you interested in?
- Home, Personal, Automobile
Steps: Which loan would you like to know more about?
- Home, Personal, Automobile
In the example above apply-for-home-loan is a subset or part of the apply-for-loan flow. This means that apply-for-home-loan will have utterances that are very similar to apply-for-loan
- Eg: “can you please help me apply for home loan?” , “Can you please apply for loan?”
This will confuse the model during training
There are 2 Steps to fix this:
- Move all utterances to the parent flow in this case apply-for-loan
- Create a step asking the user for additional details (in this case type of loan)
* Another option is to setup entities (eg: type-of-loan - Personal, Home can be a type of entity) within the flow.
Do not add utterances if a flow will only be triggered through the Trigger Journey optionThere may be some flows in the bot that are meant to be triggered only from another flow. (i.e. these won't be triggered by the user’s input)
Eg: Feedback Flow
For these flows do NOT add any user expressions/utterances Adding utterances here will unnecessarily increase the complexity of the NLP Model.
Do minimize false positives
- When an incorrect flow is triggered with high confidence it is considered a False Positive.
- False Positives occur because of overfitting and spoil the customer/user experience
- These are minimized by following the best practices laid out in this document.
- If there are a lot of false positives during training (even after checking for overfitting) try raising the minConfidence threshold.
- The best strategy is to use didYouMean(Suggestions) feature and retrain the bot periodically with the new data.
Do use the didYouMean feature extensively
- After ensuring that the model did not overfit (no False Positives) the next step is to enable the didYouMean feature
- When the user’s input is not recognized by the model the didYouMean feature elegantly handles the case as a fallback.
- This is especially useful in the first few weeks after deploying a bot in production when accuracy may be low.
- The self-learning capability allows the bot to improve the confidence of different types of user expressions.
Do add abbreviations/shortforms in the “synonyms” section
- For short forms and abbreviations add all the possible variations in the Synonyms section located under “entities”
- The NLP pipeline will check for these abbreviations and replace them with the “full form” before passing them into the ML model which will increase the accuracy
Do not overfit the model while training
📌 Training Checklist
- Add 15-20 Utterances in each flow.
- Utterances in flows must be varied and unique.
- Flows are distinct and conflicting flows must be merged.
- Model should not overfit (False Positives have to be handled).
- When didYouMean (Suggestions) feature is enabled, make use of those suggestions.
- Minimum 2 intents are required to train a bot.