It’s been a little over a week since Microsoft began offering access to the new OpenAI-powered Bing search engine, Edge web browser and an integrated chat function similar to ChatGPT, and Microsoft says the Bing chat feature can be confused when chats go longer than 15 questions.
While calling the OpenAI-powered Bing chat an “early success,” the company says long, extended chat sessions of 15 or more questions can cause Bing to become repetitive or be prompted or provoked to give responses that aren’t necessarily helpful or in line with the designed tone.
According to a Microsoft Bing blog post, long chat sessions can confuse the model on what question it is answering, and thus the company says it may need to add a tool to users can more easily refresh the context or start from scratch.
The Bing chatbot will also try to respond or reflect in the tone in which questions are being asked, which can lead to responses in a style that Microsoft didn’t intend.
“This is a non-trivial scenario that requires a lot of prompting so most of you won’t run into it, but we are looking at how to give you more fine-tuned control,” the company says.
Microsoft also says the company has seen increased engagement across traditional search results with new features such as summarized answers, chat and the content creation tools. Feedback on answers generated by the new Bing have been largely positive, as 71% of users have given the AI-powered answers a “thumbs up.”
However, Microsoft says the company has also gotten feedback on how to improve Bing and the new conversational chat model based on ChatGPT.
“This is expected, as we are grounded in the reality that we need to learn from the real world while we maintain safety and trust. The only way to improve a product like this, where the user experience is so much different than anything anyone has seen before, is to have people like you using the product and doing exactly what you all are doing.”
While users are giving good marks on citations and references that underly answers in Bing chat, Microsoft says there are challenges with answers that need timely data, such as live sports scores.
For queries where users are looking for a more direct and factual answers such as numbers from financial reports, Microsoft is planning to 4x increase the grounding data the company sends to the model. In addition, Microsoft is considering adding a toggle that gives users more control over precision versus creativity of the answer.
Microsoft is also looking into new use cases, such as booking flights or ending emails, according to the blog.
If you enjoyed this article and want to receive more valuable industry content like this, click here to sign up for our digital newsletters!