One of the most important thing to make sure chatbots are performing well is constant optimization by reviewing queries and what responses it match to. As a chatbot provider, we want to make sure our clients - non-technical business users - are able to review and optimize their chatbots easily. We set out to design an optimization feature that is an add-on of our studio product. This feature is also demanded by one of our enterprise clients.
XAPP AI has a one-page web app that helps our customers - Enterprises and small and medium businesses - to build a chatbot and customize it easily. We provide a "raw queries" page, where customers can review all the past utterances, and marks ones that are incorrect. However, there isn't a way for customer to do anything about the errors. An optimization feature is needed for non-technical business users to correct mismatched utterances to right intent, and tag slots appropriately in an easy and intuitive interface.
My instinct is to start from the Raw Queries page.
This design takes the full page on the site upon the "review" button is clicked. All the utterances and responses will then be queued for review. On the left there's a panel that shows the progress of review, including information such as total sessions, and how many conversations are accepted versus rejected. Users can immediately change the matched entity and intent from the bottom drop downs of the page. This design mimics Prodigy's method, where each slot is tagged when selected.
The second iteration utilize the raw queries page as well, but instead of having a whole page for review, once "review" button is clicked, the raw queries interface condense to only two columns important for review - user utterance and matched intent. One the right a wider column serves as a review panel. Selecting each user utterance will show the corresponding review panel on the right.
Going back to the raw queries page, I had another idea that will help make the interface less overwhelming and more focused on reviewing. Taking advantage of our existing raw queries page, which allows users o make or flag user utterances that are incorrectly matched to an intent, I came up with a pop-up review panel on top of the raw queries page. Once "review all incorrect" is clicked, all and only the incorrect user utterances will be lined up for a simpler experience. This iteration is a significant improvement from the previous two because users will no longer need to click through each utterance whether they are correct or incorrect, saving time and energy.