Chatbot Optimization
Problem
One of the most important thing to make sure chatbots are performing well is constant optimization by reviewing queries and what responses it match to. As a chatbot provider, we want to make sure our clients - non-technical business users - are able to review and optimize their chatbots easily. We set out to design an optimization feature that is an add-on of our studio product. This feature is also demanded by one of our enterprise clients.
Solution
XAPP AI has a one-page web app that helps our customers - Enterprises and small and medium businesses - to build a chatbot and customize it easily. We provide a "raw queries" page, where customers can review all the past utterances, and marks ones that are incorrect. However, there isn't a way for customer to do anything about the errors. An optimization feature is needed for non-technical business users to correct mismatched utterances to right intent, and tag slots appropriately in an easy and intuitive interface.
My instinct is to start from the Raw Queries page.
First Iteration
Taking inspiration from Prodigy, an online annotation tool, I sketched my first design.
This design takes the full page on the site upon the "review" button is clicked. All the utterances and responses will then be queued for review. On the left there's a panel that shows the progress of review, including information such as total sessions, and how many conversations are accepted versus rejected. Users can immediately change the matched entity and intent from the bottom drop downs of the page. This design mimics Prodigy's method, where each slot is tagged when selected.
Second Iteration
The second iteration utilize the raw queries page as well, but instead of having a whole page for review, once "review" button is clicked, the raw queries interface condense to only two columns important for review - user utterance and matched intent. One the right a wider column serves as a review panel. Selecting each user utterance will show the corresponding review panel on the right.
The + button opens up a pop-up that allow users to either create a new intent, or assign the utterance to an existing intent.
When the "review" button is clicked, a chart appears with the utterance broken down to each word, the user can then assign each word to a particular slot.
Clicking the + button here will open up a pop-up to create a new entity, or add the word to an existing entity.
Third Iteration
Going back to the raw queries page, I had another idea that will help make the interface less overwhelming and more focused on reviewing. Taking advantage of our existing raw queries page, which allows users o make or flag user utterances that are incorrectly matched to an intent, I came up with a pop-up review panel on top of the raw queries page. Once "review all incorrect" is clicked, all and only the incorrect user utterances will be lined up for a simpler experience. This iteration is a significant improvement from the previous two because users will no longer need to click through each utterance whether they are correct or incorrect, saving time and energy.
In this new version, I also added the ability to let users select specific phrases (not just single word) to assign a slot to.