During your busy routine managing hotels, reviewing performance data is essential to generate a sense of progress and quickly align actions among the teams involved.
When it comes to the performance of your Asksuite robot, this logic holds true. After all, you must justify the monthly investment with data that proves that its performance really provided benefits to your hotel or inn.
Broadly speaking, data about your robot's performance can involve generated revenue, number of completed bookings, as well as your robot's understanding rate.
With the new platform update, Asksuite now shares this data about your robot's understanding as a percentage, considering the last 30 days or specified periods.
As much as the percentage of understanding is from 0 to 100%, it's easy to think that the maximum is 100%, because our analytical mind is used to having this reference, right?
But this logic does not apply (yet) to the reality of Natural Language (NL), one of the main aspects of Artificial Intelligence, which is also the basis of the technology of several chatbots and the most optimized booking assistants in the world.
In this short article, I intend to briefly present context on:
- The best of the global landscape regarding Natural Language technologies, including Asksuite.
- The limitations so that you have realistic expectations about your robot's improvement action plan.
- Recognition that some companies promote chatbots with false promises that ensure understanding above 90%, and in reality hide 'false positives' in their analyses.
Through this new information, you will have much more confidence and clarity when choosing the best tools available to generate more sales for your hotel or inn.
Let's go!
What are the best technologies available for artificial intelligence's understanding today?
Stanford University recently published a remarkable report called the Artificial Intelligence Index Report 2022 in partnership with leading global organizations, such as Google, Open AI and McKinsey & Company.
In summary, this material offers rich insights into the context of the world linked to the topic of user-centered artificial intelligence, considering Research & Development, Technical Performance, AI Ethics, Economics and Governance.
In this short article, I intend to show you some of the best examples of research with the most advanced technology available today, especially with regard to Natural Language.
This aspect of artificial intelligence is especially important for your context as a hotel manager, as this is the Asksuite product's field of action related to the 'Teach Robot' feature that you are monitoring.
“Natural language inference is the task of determining whether, given a premise, a hypothesis is true (entailment), false (contradiction), or undetermined (neutral)...
Natural language inference necessitates language processing skills, such as named entity recognition (understanding the words you see), as well as being able to use commonsense knowledge to distinguish between reasonable and unreasonable inferences.”
— Artificial Intelligence Index Report 2022, Stanford University
From the moment we learn more about the best performance, it will also be easier to clarify which limitations still exist.
Next, you will learn about a synthesis of the top 2 global studies involving technology applied to Natural Language that were highlighted in the Stanford University report.
Let's get started!
#1 - SuperGLUE
The SuperGLUE initiative is a single-number metric that tracks progress on a diverse set of language tasks involving the English language.
As part of the research, the AI system was tested on 8 different tasks (such as answering yes/no questions, identifying causality in events, and making common sense reading comprehension).
Thus, their performance on these tasks was then averaged into a single score, as shown in the image below:
From the research, it is easier to understand that the maximum understanding ability of SuperGLUE technology is 91%, compared to 89.8% of human understanding for the same tests.
#2 - Stanford Natural Language Inference (SNLI)
The Stanford Natural Language Inference (SNLI) dataset contains about 600,000 sentence pairs (assumptions and associated hypotheses) that have been labeled as related, contradictory, or neutral.
As part of this challenge, several AI systems were asked whether certain pairs of sentences were related or not, as the image below illustrates:
From this, performance on the SNLI was accurately measured based on the percentage of questions answered correctly.
The best performing model in the SNLI is the Facebook AI USA EFL, which recorded a 93.1% accuracy score in April 2021, as shown in the image below:
From these 2 summarized examples, as well as other initiatives exposed in the 2022 Stanford University Artificial Intelligence Report, we realize that it is practically impossible to have a robot with 100% understanding.
This is because it is a challenge for all digital products that involve Artificial Intelligence to be able to extract the context of conversations at scale through a robot.
Even though these robots, with more than 90% understanding, are in highly-controlled environments, the process of offering them to a specific market, such as hotels, is an immense challenge due to the particularities of each business and region.
I remind you that, although all the initiatives mentioned are at very high levels of technology, none of them are geared to the hotel market like Asksuite.
How does Asksuite AI work?
Voted the best chatbot for hospitality in the world by the Hotel Tech Awards for 3 consecutive years, Asksuite is constantly evolving its product.
My goal here is not to give you technical details about the layers of intelligence or machine learning, but rather to give you a didactic context to better understand the dynamics of Asksuite's AI.
So, in summary, the Artificial Intelligence offered to +1,000 customers can be divided into 2 sections, namely:
Questionnaire
We offer a customizable template structure by hotel segments with the most frequently asked questions in chats.
In practice, when talking to the virtual reservation assistant, travelers ask text questions, which are analyzed by our internal natural language processing logic, or they access pre-configured buttons that will lead to the answers programmed by the hotel.
See how it works in the image below:
Teach Robot
It is the dedicated structure for users to improve their robot's understanding with new teachings for the specific reality of their hotel.
In practice, let's say I have an inn in San Francisco and travelers frequently ask about the distance to the Golden Gate Bridge in the chat. As a hotel manager, I could create a teaching for the robot to respond to this question through keywords, synonyms or phrase variations.
With this tool, we enable our customers to customize their robots with the full power of natural language processing through a simplified and accessible interface.
What should I aim for as the ideal maximum limit?
This is a thought-provoking question, and it is very common among our customers.
The best answer is it depends on the engagement of each hotel.
In our customer base, even though our overall average understanding is 76% today, there are hotels with robots whose understanding reaches an incredible 92%!
Let me highlight, this is possible because our tool depends on customer engagement.
Of course, Asksuite designs and updates the products to be intuitive and work in favor of hotels as much as possible.
However, if there are hundreds of travelers asking questions that your robot wasn't taught to answer, chances are your hotel's understanding is less than it could be, right?
Therefore, what we consider an optimal understanding rate is between 80 and 90%.
In this way, it is already possible to take advantage of the best of our technology, enabling responses to the vast majority of your travelers and avoiding the frustration of being far from 100%, since you now know more about the limitations of today's technologies.
In order to help hotels save time by removing the necessity to monitor chats to know which new subjects should be taught to the robot, Asksuite has updated the Teach Robot section with the following features:
- Recommendation of teachings based on major issues not understood
- Filter and list of 'Misunderstood Phrases'
- Count of teaching occurrences already published
- Redesign of the entire user experience to register and monitor the most important teachings for your hotel's potential customers!
[BONUS] Important warning: How can you identify tools that hide false positives?
We know that in the competitive market of technology for hotels, there are several tools that are launched every month with sensational promises of benefits.
However, it is essential for you to be able to understand when they are true or false promises.
In the context of virtual assistants, this is important because there are tools available on the market that, in their sales pitches, claim to offer an understanding rate of approximately 99%. Unbelievable, right?! 😱
But in practice, we know that's not the case...
This is because there is a mistake in calculating the understanding rate, since the count hides a large number of messages that resulted in 'false positives'.
And what are 'false positives'?
Imagine that I, as a hypothetical traveler, want to know the “price of lunch” and ask the robot that.
If by chance it only understands the keyword “price” and responds with a topic related to “price of a room”, it will respond incorrectly, but will count as if it was answering correctly.
We at Asksuite work hard to minimize the false positive count; one of our strategies is to ask if the answer was satisfactory at the end of each robot-traveler chat.
In this way, we can have a more assertive overview of the quality our robot delivers in addition to the other mechanisms created.
Conclusion
Let's take a look at the main takeaways of our article today:
- Remember that not even the best tools in the world deliver an understanding rate above 92%, so create realistic action plans based on the limitations of technologies available on the market today
- Try to gauge your robot's understanding between 80% and 90%, which is already considered excellent!
- Beware of tools that promise assertiveness above that, as they are hiding 'false positives'!
I hope this helped you understand a little more about Natural Language and Artificial Intelligence!
Also, it can be useful to improve your robot's understanding rate and thus generate more bookings and revenue for your hotel or inn!
Until next time!
Related articles
How to use the Teach Robot feature?
9 Tips to Boost Your Robot’s Understanding Rate
Comments
0 comments
Please sign in to leave a comment.