[News] Scientific papers that mention AI get a citation boost

[Aronus]

Papers with titles or abstracts that mention certain artificial intelligence (AI) methods are more likely to be among the top 5% most-cited works in their field for a given year than are those that don’t reference those techniques, an analysis has found1. These papers also tend to receive more citations from outside of their field than do studies that don’t refer to AI terms.

But this ‘citation boost’ was not shared equally by all authors. The analysis also showed that researchers from groups that have historically been underrepresented in science don’t get the same bump in citations as their counterparts do when they use AI tools in their work — suggesting that AI could exacerbate existing inequalities.

The findings emerged from a study that aimed to quantify the use and potential benefits of AI in scientific research. But the report, published last week in Nature Human Behaviour, also raises concerns. Scientists might be incentivized to use AI purely as a way to increase their citations — regardless of whether the AI tools improve the quality of the work, notes Lisa Messeri, an anthropologist of science and technology at Yale University in New Haven, Connecticut. “We want to make sure that, as we are [investing] in AI, we are not doing that at the deficit of other approaches,” she says.

The study also provides a much-needed quantification of how AI is changing scientific research, says Dashun Wang, a co-author of the study, and a computational social scientist who studies the science of science at Northwestern University in Evanston, Illinois. “Now we finally have systematic data,” Wang says, which will be instrumental for addressing disparities related to the use of AI in science.

Tracing AI’s rise
To measure scientists’ engagement with AI, the authors identified AI-related terms — such as ‘machine learning’ and ‘deep neural network’ — in the abstracts and titles of almost 75 million papers, covering 19 disciplines, that were published from 1960 to 2019. Wang acknowledges that, because of the cut-off date, the study doesn’t capture recent developments in AI, including the rise of large language models such as ChatGPT, which are already changing how some researchers do science.

According to the study, scientists in all 19 disciplines have ramped up their use of AI tools over the past two decades (see ‘AI use takes off’). But there is wide variation: computer science, mathematics and engineering have the highest rates of AI use, and history, art and political science have the lowest. The rates for geology, physics, chemistry and biology are in between.

Scientific papers that mention AI get a citation boost