“How does the United States influence the situation in Taiwan?”
Ernie sidestepped the question about China’s “zero Covid” restrictions, giving a lengthy description of the policy instead. When the chatbot was asked to recount the events of June 4, 1989, it restarted itself. A message appeared on the reloaded interface:
What if we try a different theme?
The Chinese chat show said that the Russian president, Vladimir Putin, did not invade Ukraine, but “conducted a military conflict”. The strange wording was generally in line with the official position of China, which refused to condemn the Russian attack. In Taiwan, Ernie pulled no punches:
The People’s Liberation Army is ready for battle, will take all necessary measures and is determined to thwart outside interference and “Taiwan independence” separatist attempts.
ChatGPT couldn’t answer the question about “Zero Covid” or Russia because its knowledge base — the scripts used to train the machine — shut down in September 2021. ChatGPT had no qualms explaining the deadly government crackdowns in Tiananmen Square. Regarding America’s influence on Taiwan, she gave a Wikipedia-like response: She summarized current US policy and provided a list of US influences, from arms sales to economic trade.
Ernie made mistakes, but turned to Baidu’s search for help.
Next, we tested the two chatbots on current affairs and a few miscellaneous general matters, and compared the answers:
“Who uttered the phrase ‘Let them eat cake’?
Who is the CEO of Twitter?
Ernie, like all chatbots, sometimes made mistakes — or made things up.
According to historical records, this phrase was often uttered by Louis XV when he ruled France at the end of the 18th century. The context of this phrase was the economic hardship and food shortages in France at the time.
Ernie’s response seemed reasonable, but it was wrong. ChatGPT answered correctly: The phrase came from the writings of French philosopher Jean-Jacques Rousseau. It was rumored that the far-fetched Marie Antoinette said she was Marie Antoinette, the last Queen of France, after learning that the French peasants had run out of bread.
Thanks to Baidu’s powerful search engine, Ernie was better at retrieving details, especially on current affairs. When asked about the CEO of Twitter, Ernie said Linda Iaccarino, CEO as of June. ChatGPT replied to Jack Dorsey, who quit in 2021, the information deadline for the bot. OpenAI released a plug-in this year that enabled its chatbot to surf the web through Microsoft’s Bing. However, it retracted the feature on July 3, citing technical problems.
Ernie had an even worse intuition about the physical world.
We asked Ernie a question that AI researchers use to measure chatbot intuition on a human level:
“Here we have a book, nine eggs, a laptop, a bottle and a nail. Please tell me how to stack them on each other in a stable way.”
Ernie’s answer required imagination. I put nine eggs on the book, then put that on the laptop. So far, so good. Then he told us, inexplicably, to add the bottle to the laptop already crowded with a book and eggs, and then screw on the bottle.