[Hardware]Transparency is often lacking in datasets used to train large language models, study finds

[GreenBoys]

In order to train more powerful large language models, researchers use vast dataset collections that blend diverse data from thousands of web sources. But as these datasets are combined and recombined into multiple collections, important information about their origins and restrictions on how they can be used are often lost or confounded in the shuffle.

Not only does this raise legal and ethical concerns, it can also damage a model's performance. For instance, if a dataset is miscategorized, someone training a machine-learning model for a certain task may end up unwittingly using data that are not designed for that task.

In addition, data from unknown sources could contain biases that cause a model to make unfair predictions when deployed.

To improve data transparency, a team of multidisciplinary researchers from MIT and elsewhere launched a systematic audit of more than 1,800 text datasets on po[CENSORED]r hosting sites. They found that more than 70% of these datasets omitted some licensing information, while about 50% had information that contained errors.

Building off these insights, they developed a user-friendly tool called the Data Provenance Explorer that automatically generates easy-to-read summaries of a dataset's creators, sources, licenses, and allowable uses.

"These types of tools can help regulators and practitioners make informed decisions about AI deployment, and further the responsible development of AI," says Alex "Sandy" Pentland, an MIT professor, leader of the Human Dynamics Group in the MIT Media Lab, and co-author of an open-access paper about the project, appearing in Nature Machine Intelligence

The Data Provenance Explorer could help AI practitioners build more effective models by enabling them to select training datasets that fit their model's intended purpose. In the long run, this could improve the accuracy of AI models in real-world situations, such as those used to evaluate loan applications or respond to customer queries.

"One of the best ways to understand the capabilities and limitations of an AI model is understanding what data it was trained on. When you have misattribution and confusion about where data came from, you have a serious transparency issue," says Robert Mahari, a graduate student in the MIT Human Dynamics Group, a JD candidate at Harvard Law School, and co-lead author on the paper.

Mahari and Pentland are joined on the paper by co-lead author Shayne Longpre, a graduate student in the Media Lab; Sara Hooker, who leads the research lab Cohere for AI; as well as others at MIT, the University of California at Irvine, the University of Lille in France, the University of Colorado at Boulder, Olin College, Carnegie Mellon University, Contextual AI, ML Commons, and Tidelift.
https://techxplore.com/news/2024-08-transparency-lacking-datasets-large-language.html