Google says it has developed a new chatbot that it says can talk about anything and conduct more sensible and specific conversations than existing chatbots.
The bot, Meena, is a 2.6 billion parameter end-to-end neural conversational model that Google says learns to respond sensibly to a given conversational context. Meena has a single Evolved Transformer encoder block and 13 Evolved Transformer decoder blocks.
The encoder processes the conversation context to help to help Meena understand what has already been discussed. That information is used to formulate an actual response.
“Through tuning the hyper-parameters, we discovered that a more powerful decoder was the key to higher conversational quality,” Google said in a blog post last week.
According to Google, the Meena model has 2.6 billion parameters and is trained on 341 gigs of text from social media conversations in the public domain.
Compared to OpenAI GPT-2, Meena has 1.7 times greater model capacity and was trained on 8.5 times more data, Google said.
Read Next: Global Chatbot Market Size to Grow 29.7% Between 2019 and 2024, Says Recent Research
To evaluate Meena’s humanlike abilities, Google created the Sensibleness and Specify Average, a new human evaluation metric for open-domain chatbots. It captures basic but important attributes for human conversation, the company said.
Google crowdsourced free-form conversation with chatbots already being tested and other well-known open-domain bots, like Mitsuku, Cleverbot, Xiaolce and DialoGPT
That method involved evaluating whether a response is reasonable in context and if it is specific enough to the given context.
“For example, if A says, “I love tennis,” and B responds, “That’s nice,” then the utterance should be marked, “not specific”. That reply could be used in dozens of different contexts. But if B responds, “Me too, I can’t get enough of Roger Federer!” then it is marked as “specific”, since it relates closely to what is being discussed.”
While this work focused on attributes like sensibleness and specificity, Google next wants to shift to personality and factuality while also tackling safety and bias.
Meena is not yet available for public use, but Google said that time may come within the next few months.
“We are evaluating the risks and benefits associated with externalizing the model checkpoint, however, and may choose to make it available in the coming months to help advance research in this area,” the company said.
If you enjoyed this article and want to receive more valuable industry content like this, click here to sign up for our digital newsletters!
Leave a Reply