School of Information Assistant Professor James Howison is collaborating on a new project that aims to transform the way we measure the scholarly impact of software.
Funded by a three-year, $635,000 grant from the Alfred P. Sloan Foundation, Dr. Howison is working with the open-source website Impactstory to create a database of software projects that are mentioned in research papers.
By raising the visibility of software work as a contribution in academic literature, they hope to improve incentives for software work in scholarship, which in turn will support software-intensive, open-science systems to enable future research.
“Software is key for science, but scientists struggle to identify the impact of the software they write,” Dr. Howison said. “I’m very excited to work with Heather Piwowar and Jason Priem from Impactstory to help make software more visible in the literature. We’re going to provide a new resource for those building software to make their case for impact.”
The idea for the project grew out of research that Dr. Howison conducted with iSchool doctoral student Julia Bullard, he said. When they examined how many times research software was mentioned in a random sample of 90 biology articles, they found that 63 percent of those mentions were informal, such as URLs in footnotes, rather than formal citations. Citations are commonly used to measure the impact of research and can influence tenure decisions.
UT Austin will receive a little more than half of the Sloan Foundation’s award, which Dr. Howison said will primarily support student work.
"We're going to be involving students at all levels in research, doctoral, masters, and undergraduates,” Dr. Howison said. “I’ve been thinking for some time about how to provide introductory research experiences for undergrads and masters students in my research, since I don't run a lab or field research program. In this project we're going to have undergrads reading scientific papers and identifying software; their work will then be available as the basis for training computers to identify software in the papers. So the undergrads, masters, and doctoral students will have a chance to see real impact from their research work."
Dr. Howison and Impactstory plan to use the database to build and study three prototype tools:
- CiteSuggest will analyze submitted text or code and make recommendations for normalized citations using the software author’s preferred citation;
- CiteMeAs will help software producers make clear requests for their preferred citations;
- Software Impactstory will help software authors demonstrate the scholarly impact of their software in the literature.
Impactstory is a nonprofit website that helps researchers explore and share the the online impact of their research.