Make a Splash: Notions of Scientific Impact Are Evolving

Of course, all of you toiling away in laboratories this holiday season want the work you are doing to have an impact, to move science forward, or perhaps even society. One obvious way to do that is and has been to publish in journals with a high “impact factor,” a measure that dates back to 1975 and is based on the average number of citations for recent articles. Of course, a publication in Science or Nature is always nice, but in the wired world we are living in, there are plenty of other ways to define and measure scientific impact.

Even if we stick to citation counts for a moment, it’s clear that articles in less-prestigious journals have been gaining ground. A study issued by Google in October entitled “Rise of the Rest: The Growing Impact of Non-Elite Journals” found that the proportion of highly cited articles appearing in lower-prestige journals has risen steadily from 1995 to 2013. Considering citations to all articles, Anurag Acharya and colleagues report, the percentage of citations to articles in non-elite journals went from 27% of all citations in 1995 to 47% in 2013 – yes, almost half.

The effect of online access on scientific impact

Partly, the story has to do with online accessibility. It is now about as easy to find a paper in a lesser-known journal as it is in a top-tier publication. Accessibility is surely also behind the increase in citations for older publications. (As an interesting aside, Nature recently reported that the most cited work in history is a 1951 description of an assay to determine the amount of protein in a solution.)

But are citations and journal publications really the best way to measure impact? As that Nature report also noted, papers detailing Nobel Prize-winning research typically weren’t among the most cited. And, as scientific discourse has increasingly moved online, surely times have changed.

Altmetrics.org calls to expand our view of what impact looks like and what it’s made of. Increasingly, journal publications alone aren’t enough. Scientists are sharing what they call “raw science” – data sets, code, and, I think it’s safe to say all of us here would add materials to that list (plasmids, yay!). Scientists are blogging, talking to the media, and engaging in social media too.

Services like Mendeley are allowing researchers to track and share their own libraries and offering another way to consider and quantify the impact of scientific publications. Ventures like Impact Story seek to find ways to capture scholarly work that exists outside of traditional scientific journals.

Of course, no one measure will ever adequately capture a notion as complex as “scientific impact.” As we look ahead to the end of a year and beginning of another, perhaps it’s worth reflecting on a question of impact that only you can answer: How would you like to measure yours?

---

More Reading

• Fenner M. "Altmetrics and Other Novel Measures for Scientific Impact" from Opening Science - The Book
  
  
• Bollen J, et al. "A Principal Component Analysis of 39 Scientific Impact Measures" (2009) PLOS One 4(6): e6022. doi: 10.1371/journal.pone.0006022
  
  
• Aragon AM. "A measure for the impact of research" (2013) Scientific Reports 3, Article number: 1649 doi:10.1038/srep01649

---