Microsoft on what it learned in the first seven days of testing Bing Chat

The big picture: It’s been one week since Microsoft unleashed its AI-powered Bing search engine on the world. What has Redmond learned thus far? For starters, engagement across traditional search results has increased. Microsoft was also quick to point out that feedback on answers generated by the new Bing has been mostly positive, with 71 percent of users giving AI-powered answers a “thumbs up.”

As to be expected, Microsoft has also received plenty of feedback on how to improve its new products. With regard to search results, the company said the service is struggling with answers that need timely data such as live sports scores. They are also planning to increase the grounding data sent to the model by 4x to help with factual answers like numbers from a financial report, and have even considered adding a toggle that would weigh answers based on precision versus creativity.

With regard to chat, Microsoft has realized that extended sessions with 15 or more questions can cause Bing to become repetitive or give responses that aren’t helpful or in line with its designed tone. A tool to more easily refresh context or start over from scratch could help remedy the issue, we’re told.

Redmond further notes that at times, chat tries to respond to queries in the tone it is being asked. This scenario requires a lot of prompting, Microsoft said, so not everyone will experience it. Still, they are looking into ways to curb it and give users more fine-tuned control.

Microsoft also said that its AI-powered chat “is a great example of where new technology is finding product-market-fit for something we didn’t fully envision.” Namely, Microsoft mentioned how people are using chat as a tool for more general discovery of the world and for social entertainment.

Considering this isn’t Microsoft’s first at-bat with an AI-powered chatbot (remember Tay?), it’s not terribly surprising to see Bing Chat struggle out of the gate.

Microsoft has also taken note of the many requests for features and capabilities it has fielded over the last week, and will consider the top ideas for inclusion in future releases.

Image credit: Volodymyr Hryshchenko