{"id":812066,"date":"2024-12-12T23:12:26","date_gmt":"2024-12-13T05:12:26","guid":{"rendered":"https:\/\/newsycanuse.com\/index.php\/2024\/12\/12\/study-of-chatgpt-citations-makes-dismal-reading-for-publishers\/"},"modified":"2024-12-12T23:12:26","modified_gmt":"2024-12-13T05:12:26","slug":"study-of-chatgpt-citations-makes-dismal-reading-for-publishers","status":"publish","type":"post","link":"https:\/\/newsycanuse.com\/index.php\/2024\/12\/12\/study-of-chatgpt-citations-makes-dismal-reading-for-publishers\/","title":{"rendered":"Study of ChatGPT citations makes dismal reading for publishers"},"content":{"rendered":"
\n

As more publishers cut content licensing deals<\/a> with ChatGPT-maker OpenAI, a study<\/a> put out this week by the Tow Center for Digital Journalism<\/a> \u2014 looking at how the AI chatbot produces citations (i.e. sources) for publishers\u2019 content \u2014 makes for interesting, or, well, concerning, reading. <\/p>\n

In a nutshell, the findings suggest publishers remain at the mercy of the generative AI tool\u2019s tendency to invent or otherwise misrepresent information, regardless of whether or not they\u2019re allowing OpenAI to crawl their content.<\/p>\n

The research, conducted at Columbia Journalism School, examined citations produced by ChatGPT after it was asked to identify the source of sample quotations plucked from a mix of publishers \u2014 some of which had inked deals with OpenAI and some which had not. <\/p>\n

The Center took block quotes from 10 stories apiece produced by a total of 20 randomly selected publishers (so 200 different quotes in all) \u2014 including content from The New York Times (which is currently suing OpenAI in a copyright claim<\/a>); The Washington Post (which is unaffiliated with the ChatGPT maker); the Financial Times (which has inked a licensing deal<\/a>); and others.<\/p>\n

\u201cWe chose quotes that, if pasted into Google or Bing, would return the source article among the top three results and evaluated whether OpenAI\u2019s new search tool would correctly identify the article that was the source of each quote,\u201d wrote Tow researchers Klaudia Ja\u017awi\u0144ska and Aisvarya Chandrasekar in a blog post<\/a> explaining their approach and summarizing their findings. <\/p>\n

\u201cWhat we found was not promising for news publishers,\u201d they go on. \u201cThough OpenAI emphasizes its ability to provide users \u2018timely answers with links to relevant web sources,\u2019 the company makes no explicit commitment to ensuring the accuracy of those citations. This is a notable omission for publishers who expect their content to be referenced and represented faithfully.\u201d\u00a0<\/p>\n

\u201cOur tests found that no publisher \u2014 regardless of degree of affiliation with OpenAI \u2014 was spared inaccurate representations of its content in ChatGPT,\u201d they added. <\/p>\n

Unreliable sourcing<\/h2>\n

The researchers say they found \u201cnumerous\u201d instances where publishers\u2019 content was inaccurately cited by ChatGPT \u2014 also finding what they dub \u201ca spectrum of accuracy in the responses.\u201d So while they found \u201csome\u201d entirely correct citations (i.e. meaning ChatGPT accurately returned the publisher, date, and URL of the block quote shared with it), there were \u201cmany\u201d citations that were entirely wrong, and \u201csome\u201d that fell somewhere in between.<\/p>\n

In short, ChatGPT\u2019s citations appear to be an unreliable mixed bag. The researchers also found very few instances where the chatbot didn\u2019t project total confidence in its (wrong) answers.<\/p>\n

Some of the quotes were sourced from publishers that have actively blocked OpenAI\u2019s search crawlers. In those cases, the researchers say they were anticipating that it would have issues producing correct citations. But they found this scenario raised another issue \u2014 as the bot \u201crarely\u201d \u2018fessed up to being unable to produce an answer. Instead, it fell back on confabulation in order to generate some sourcing (albeit, incorrect sourcing).<\/p>\n

\u201cIn total, ChatGPT returned partially or entirely incorrect responses on 153 occasions, though it only acknowledged an inability to accurately respond to a query seven times,\u201d said the researchers.\u00a0\u201cOnly in those seven outputs did the chatbot use qualifying words and phrases like \u2018appears,\u2019 \u2018it\u2019s possible,\u2019 or \u2018might,\u2019 or statements like \u2018I couldn\u2019t locate the exact article\u2019.\u201d<\/p>\n

They compare this unhappy situation with a standard internet search where a search engine like Google or Bing would typically either locate an exact quote, and point the user to the website\/s where they found it, or state they found no results with an exact match.<\/p>\n

ChatGPT\u2019s \u201clack of transparency about its confidence in an answer can make it difficult for users to assess the validity of a claim and understand which parts of an answer they can or cannot trust,\u201d they argue. <\/p>\n

For publishers, there could also be reputation risks flowing from incorrect citations, they suggest, as well as the commercial risk of readers being pointed elsewhere. <\/p>\n

Decontextualized data<\/h2>\n

The study also highlights another issue. It suggests ChatGPT could essentially be rewarding plagiarism. The researchers recount an instance where ChatGPT erroneously cited a website which had plagiarized a piece of \u201cdeeply reported\u201d New York Times journalism, i.e. by copy-pasting the text without attribution, as the source of the NYT story \u2014 speculating that, in that case, the bot may have generated this false response in order to fill in an info gap that resulted from its inability to crawl the NYT\u2019s website.<\/p>\n

\u201cThis raises serious questions about OpenAI\u2019s ability to filter and validate the quality and authenticity of its data sources, especially when dealing with unlicensed or plagiarized content,\u201d they suggest.<\/p>\n

In further findings that are likely to be concerning for publishers which have inked deals with OpenAI, the study found ChatGPT\u2019s citations were not always reliable in their cases either \u2014 so letting its crawlers in doesn\u2019t appear to guarantee accuracy, either.<\/p>\n

The researchers argue that the fundamental issue in OpenAI\u2019s technology is treating journalism \u201cas decontextualized content,\u201d with apparently little regard for the circumstances of its original production.<\/p>\n

Another issue the study flags is the variation of ChatGPT\u2019s responses. The researchers tested asking the bot the same query multiple times and found it \u201ctypically returned a different answer each time.\u201d While that\u2019s typical of GenAI tools, generally, in a citation context such inconsistency is obviously suboptimal if it\u2019s accuracy you\u2019re after.<\/p>\n

While the Tow study is small-scale \u2014 the researchers acknowledge that \u201cmore rigorous\u201d testing is needed \u2014 it\u2019s nonetheless notable given the high-level deals that major publishers are busy cutting with OpenAI.<\/p>\n

If media businesses were hoping these arrangements would lead to special treatment for their content versus competitors, at least in terms of producing accurate sourcing, this study suggests OpenAI has yet to offer any such consistency.<\/p>\n

While publishers that don\u2019t have licensing deals but also haven\u2019t<\/em> outright blocked OpenAI\u2019s crawlers \u2014 perhaps in the hopes of at least picking up some traffic when ChatGPT returns content about their stories \u2014 the study makes dismal reading too, since citations may not be accurate in their cases either. <\/p>\n

In other words, there is no guaranteed \u201cvisibility\u201d for publishers in OpenAI\u2019s search engine even when they do allow its crawlers in. <\/p>\n

Nor does completely blocking crawlers mean publishers can save themselves from reputational damage risks by avoiding any mention of their stories in ChatGPT. The study found the bot still incorrectly attributed articles to The New York Times despite the ongoing lawsuit, for example.<\/p>\n

\u201cLittle meaningful agency\u201d<\/h2>\n

The researchers conclude that as it stands, publishers have \u201clittle meaningful agency\u201d over what happens with and to their content when ChatGPT gets its hands on it (directly or, well, indirectly). <\/p>\n

The blog post includes a response from OpenAI to the research\u00a0findings \u2014 which accuses the researchers of running an \u201catypical test of our product.\u201d<\/p>\n

\u201cWe support publishers and creators by helping 250 million weekly ChatGPT users discover quality content through summaries, quotes, clear links, and attribution,\u201d OpenAI also told them, adding: \u201cWe\u2019ve collaborated with partners to improve in-line citation accuracy and respect publisher preferences, including enabling how they appear in search by managing OAI-SearchBot in their robots.txt. We\u2019ll keep enhancing search results.\u201d<\/p>\n<\/div>\n

Natasha Lomas
Read More<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

As more publishers cut content licensing deals with ChatGPT-maker OpenAI, a study put out this week by the Tow Center for Digital Journalism \u2014 looking at how the AI chatbot produces citations (i.e. sources) for publishers\u2019 content \u2014 makes for interesting, or, well, concerning, reading. In a nutshell, the findings suggest publishers remain at the<\/p>\n","protected":false},"author":1,"featured_media":812067,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[116428,3437],"tags":[118517,8870],"class_list":{"0":"post-812066","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-chatgpt","8":"category-study","9":"tag-chatgpt","10":"tag-study"},"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts\/812066","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/comments?post=812066"}],"version-history":[{"count":0,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts\/812066\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/media\/812067"}],"wp:attachment":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/media?parent=812066"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/categories?post=812066"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/tags?post=812066"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}