We are instinctively turned off by stories labeled 'AI-generated' — even if they were secretly written by other people, study finds

News
By published

Even when a creative work is made by humans, we think it’s missing a key element of art if we think it's been written by an AI — that capacity to fully immerse you in the world being created.

A robot android using a typewriter
(Image credit: Getty Images/Moor Studio)

Researchers have found that people react negatively to artistic works generated by artificial intelligence (AI) — even if they were actually created by humans — revealing an innate bias against machine-created art.

In an age of constant worry over whether AI will spell the end of many professions — including in creative fields like writing and design — a new study indicates that we still believe true art can only be made by human hands.

Although Chu said AI has a place in content creation, such as in creating compelling health narratives around diet or vaccination, his study, published Sept. 13 in the Journal of Communication, probed whether people inherently believed or trusted AI-generated text.

The study used two versions of similar stories, one created by ChatGPT and the other by a human author. Participants then read the stories, each of which contained a label identifying the author. The twist was that the labels were sometimes swapped, with participants believing they were reading the machine-generated story when they were reading the human-generated one, and vice versa.

Related: Mathematicians devised novel problems to challenge advanced AIs' reasoning skills — and they failed almost every test

The researchers tested three sets of narrative stories that Chu described as "quite diverse in nature"; a novella of about 1,100 words and two health-related stories around 480 words long.

While the research team didn’t measure the percentage of participants who explicitly liked or disliked the stories, Chu said they observed a clear difference in how they responded depending on whether the story was labeled as AI-generated or human-written.

“Participants who were told the story was AI-generated reported being less engaged with it compared to those who thought it was written by a human," Chu told Live Science. "They were also more critical of the content when it was labeled as AI-generated, meaning they questioned it more or found more reasons to disagree with it.”

The survey focused on two particular elements of storytelling — how persuasive they were and how much they could transport the reader. They found that AI-generated stories were just as persuasive as those written by humans. But human-written stories captured that feeling "of being so engrossed in the narrative you don't feel the sticky seats in the movie theater anymore."

Chu added that when readers are fully engaged in a story, they tend to lower their defenses about persuasive content. When the study's participants were told the story was generated by AI, they reported it was the feeling of being transported that was missing.

The study might also reveal a deeper truth about the place of AI in the arts, where we'll use it to generate text that conveys more prosaic information. It may, however, never quite capture a uniquely human experience with the distinct personality of the creator.

"AI is good at writing something that is consistent, logical and coherent. But it is still weaker at writing engaging stories than people are," Chu said.

Drew Turney
Drew Turney

Drew is a freelance science and technology journalist with 20 years of experience. After growing up knowing he wanted to change the world, he realized it was easier to write about other people changing it instead. As an expert in science and technology for decades, he’s written everything from reviews of the latest smartphones to deep dives into data centers, cloud computing, security, AI, mixed reality and everything in between.