Addgene Depositors Get More Citations

This post was contributed by guest bloggers Samantha Zyontz and Neil Thompson from the MIT Sloan School of Management.

Professor Feng Zhang’s original 2013 gene editing paper on CRISPR/Cas amassed nearly 2,400 citations in its first four years (1). In addition to publishing in Science, Professor Zhang deposited the associated plasmids with Addgene. Since then, Addgene has filled over 6,500 requests for these plasmids. While clearly an outlier, this story had us wondering: is there a larger trend here? Do papers associated with Addgene deposits accumulate more citations than those without Addgene deposits? Even more interestingly, could we tell if depositing a plasmid with Addgene causes a paper to get cited more?

Articles with and without plasmids at Addgene

To find out whether scientific articles associated with Addgene plasmid deposits are more highly cited, we went to one of the largest bibliographic databases available, Web of Science. For each publication, Web of Science provides basic bibliographic information and uses its vast network of articles to track the total number of citations the paper receives. From the database we pulled the journals from 2005 to 2015 in areas most relevant to plasmids: Genetics & Heredity, Cell Biology, Biotechnology & Applied Microbiology, Biochemistry & Molecular Biology, and Biochemical Research Methods. We also pulled the big interdisciplinary journals: Nature, Science, PLOS One, and PNAS. We then broke the publications from these journals into two groups: those containing plasmids that the authors deposited with Addgene and those without plasmids deposited with Addgene. Together these journals gave us data on 1,318,016 total papers, 2,596 of which had plasmids deposited with Addgene.

Articles with Addgene plasmids get more citations

So what do we find? Lots more citations for the papers with plasmids deposited at Addgene – typically about four times as many as papers without plasmids deposited with Addgene (Figure 1).

Even aggregated this much, you can see the 2013 ‘bump’ related to the CRISPR/Cas papers and Addgene deposits, but papers published in other years also get many more citations

This isn’t the whole story, though. It would also be great to know how much of that difference in citations is caused by depositing a plasmid with Addgene, in contrast to other reasons for the correlation. However, determining this exactly is difficult. For example, it is very plausible that science published in more widely-read journals gets more citations and is more likely to lead to an Addgene deposit, since scientists may believe that there will be more demand for the related plasmid. Under such circumstances, we would observe this same correlation, but it wouldn’t be causal.

To control for a number of differences between those papers with Addgene plasmids and those without, we adjust our estimates based on the journal that published the articles.  That is, we re-estimate the citations to the papers not associated with Addgene deposits as if they were published in the same journals as those that did make deposits to Addgene.  Insomuch as different journals represent different research areas, target readers, methodologies, or qualities of research our adjustment controls for them.  

So, for example, in 2010, 10 percent of the papers with Addgene plasmids in our data were published in Nature, but only 3 percent of papers without Addgene plasmids were. Thus, to get our journal-adjusted citation measure we re-weight the 3 percent of Nature papers without Addgene plasmids as if they comprised 10 percent. Based on this new re-weighting, we can compare citations where an equal share of papers with and without Addgene plasmids are in Nature. This is also true for other journals, so our estimates now take into account any differences in citations due to the journal of publication. Table 1 provides an example calculation of the weights.

Table 1: Journal Weighting Example

Article Number (A)	Year Published (B)	Number of Citations (C)	Journal Name (D)	Percent of Papers without Addgene Plasmids in Journal (E)	Percent of Papers with Addgene Plasmids in Journal (F)	Weight (=F/E) (G)	Weighted Citations (=C*G)
1	2010	3	Nature	3.3%	10.5%	3.2	9.6
2	2010	167	Nature	3.3%	10.5%	3.2	534.1
3	2010	3	Applied and Environmental Microbiology	1.6%	1.2%	0.75	2.3
4	2010	167	Applied and Environmental Microbiology	1.6%	1.2%	0.75	125.3

In figure 2, you can see the full impact from using these weighted numbers.

Note: this does drop some journals from our dataset, since not all journals have papers with plasmids deposited at Addgene in them. The adjustment is done within publication years.

What do we find once we do this adjustment? That papers with plasmids deposited at Addgene still accrue on average 2 to 3 times as many citations as other papers in the same journals!

So, is all this remaining difference caused by depositing the related plasmids with Addgene? Probably not all of it. Our journal adjustment is just a first step. There are lots of other factors that could still be controlled for. As an example, plasmid-related areas of science could be growing more quickly which could also generate this type of correlation. But even with that caveat, it is still suggestive that we see such a big difference in citations after controlling for journal. And so, while our quantitative analysis cannot rule out all other explanations, we do think that it lends weight to the argument that Addgene is achieving its goal of making plasmid-based science more accessible…and accruing more citations for depositing scientists along the way.

---

Many thanks to our guest bloggers, Samantha Zyontz and Neil Thompson. Special thanks to Addgenies Pamela McA'Nulty and Zachary Tribbett for helping compile the data used in this work.

Samantha Zyontz is a Ph.D. candidate at the MIT Sloan School of Management. She is particularly interested in exploring how innovative tools influence the direction of scientific research. View her research on her SSRN Author page: https://ssrn.com/author=1232466.

Neil Thompson is an Assistant Professor of Innovation and Strategy at the MIT Sloan School of Management. His research focuses on a range of topics including biotech and research tools, computing performance, and randomized control trials in innovation. Visit his website at: http://www.neil-t.com/ .

References

1. Cong, Le, et al. "Multiplex genome engineering using CRISPR/Cas systems." Science 339.6121 (2013): 819-823. PubMed PMID: 23287718. PubMed PMCID: PMC3795411.