Peer review suffers because of the growth of science

There is serious concern about how the growth of science should continue: more and more people are joining the scientific community as professional researchers. The act of counting scientists is not straightforward, since potentially everyone is a scientist in some way: we all ask questions and answer them by carefully examining data and coming up with conclusions.

Trained scientists with university degrees number in the tens of millions worldwide, and they work for governments, universities, corporations, foundations, schools, hospitals, and practically every other entity imaginable. Anywhere that a question can be asked about the world, and data can be gathered, a scientist will be there to ask the questions, collect the data, and answer the questions. Scientists recognize themselves as peers inside the scientific community.

Using this definition, in the USA alone, the number of scientists increased from 150,000 to 2,685,000 between 1950 and 2001 [1]. Some claim that most of the potential scientists in the US population are already part of the scientific community, but there is no such saturation in the EU, and in recent years, many fast-growing Asian countries have been catching up to the USA in numbers of scientists. The result is a steady growth of the population of competing scientists.

The global scientific community may be the largest self-organized community in the world, apart from religions. Scientists work with a strong sense of belonging, acknowledging the scientific values of their community and seriously considering the opinions of their peers worldwide.

This community, though self-organized, suffers strong pressure from national and international public bodies to produce scientific and technological results. The publication of results is the main, though not sole, measure of scientific output. Since modern scientists are highly professionalized, many of them are very sensitive to this measure that guides their promotions, grants, and even earnings. Consequently, a terrific growth in all types of publications has occurred, as the plot in Figure 1 (first published in [2]) shows, in physics, for the USA only. Although the number of PhDs granted each year has not been increasing recently, the number of scientific papers has increased exponentially over the entire period, so the number of journals created each year must also grow.

Screenshot 2017-06-26 12.13.53

Fig. 1. Growth of scientific journals, US physics PhD theses, and scientific community

The observation that, for over a hundred years, the number of scientists has been growing exponentially means, quite simply, that the rate of production of scientists has always been proportional to the number of existing scientists. We have already seen how that process works at the final stage of education, where each professor in a research university turns out a dozen or so PhDs, most of whom want to become research professors and turn out a dozen or so more PhDs. Hence, there is a lot of competition that may disappear if the growth becomes stagnant. Yet, this is what is already happening in the developed world, as Fig. 1 shows for the United States. We conjecture that, by the year 2050, researchers will struggle to have more than two PhD students during their entire lifetime, similar to what happened in the developed nations when the families there moved from having very many children in the 19th century to having one to three children today. This move motivated the developed societies to change and provide employment, pensions, and activities for the older generations. The scientific community may want to study these implications as it faces similar changes in the near future.

There is an ever-shifting balance between a strong sense of community, driven by a desire to belong to a scientific community and by peer acknowledgement, and the pressure placed by societies and governments on their scientists to rapidly produce more results; this results in an environment in which the competition for resources and promotion creates a sort of global race, which shifts the focus from quality to quantity of results. The crises that science might face, as forecasted in 1994 by David Goodstein [2], are not limited to jobs and research funds. Those are bad enough, but they may be just the beginning. Under pressure from those problems, other parts of the scientific enterprise have started showing signs of distress. One of the most essential signs is the honesty and ethical behavior of scientists. If those attributes slip, the scientific community may lose community bonds, sense of belonging, and long-term thinking. Both the public and scientific communities have been shocked in recent decades by an increasing number of cases of fraud committed by scientists. There is little doubt that the perpetrators in these cases felt themselves to be under intense pressure to compete for scarce resources, compelled to advance, even by cheating if necessary. As the pressure increases, this kind of dishonesty is nearly certain to become more common, unfortunately.

Other kinds of dishonesty may also become more common. For example, peer-review, one of the crucial pillars of the whole edifice, is in critical danger. Scientific journals decide which papers to publish based on peer-review, and granting agencies such as the National Science Foundation in the USA and the European Commission in the European Union use it to decide what research to publish or support. Journals in most cases and agencies in some cases operate by sending manuscripts or research proposals to referees who are recognized experts on the scientific issues in question, but whose identities are not revealed to the authors of the evaluated work. Obviously, good decisions as to what research should be supported and what results should be published are crucial to the proper functioning of science.

Peer review is one of many examples of practices that were well suited to the time of exponential expansion but will become increasingly dysfunctional in the difficult future that we face. There are many reasons for this, not the least being the fact that referees have an obvious conflict of interest, since they are themselves competitors for the same resources. Referees would have to have very high ethical standards to avoid taking (often unconscious) advantage of their privileged anonymity to advance their own interests. As time goes on, more and more referees will have their ethical standards eroded as a consequence of having felt themselves victimized by reviews they perceived as unfair when they were authors. This is often the case for project funding.

For scientific papers, most often the true reviewers are younger scientists trying to learn and advance their careers. If, as forecasted, the ratio of PhD students to senior scientists decreases, the pressure to find qualified peer reviewers will increase still further, even though the reviewers are already overburdened. This overloading of the peer-review process is likely to reduce the quality of the reviews.

Thus, many authors concur with Goodstein’s claim from 1994 [2]: “We must find a radically different social structure to organize research and education in science after The Big Crunch [about the slowdown of the scientific community growth]. That is not meant to be an exhortation. It is meant simply to be a statement of a fact known to be true with mathematical certainty, if science is to survive at all. The new structure will come about by evolution rather than design, because, for one thing, neither I nor anyone else has the faintest idea of what it will turn out to be, and for another, even if we did know where we are going to end up, we scientists have never been very good at guiding our own destiny. Only this much is sure: the era of exponential expansion will be replaced by an era of constraint [or at least of organic growth]”. We share the main claim of Goodstein, adding that the new structure must be the evolution of Internet as a symbiosis with the scientific community. On the other hand, we are not so pessimistic, because we believe very much in the resilience of the scientific community, which will help it stick to its core values: love for science, commitment to the scientific method, support for communication and discussion of scientific results, belief in the fairness of peer-review, the value of bonds to the community, and the desire for appreciation from peers.

References in this post:

[1] Analysis from Current Statistics on Scientists, Engineers and Technical Workers: 2002 Edition, Plastics Engineering, January 1, 2002, DPE [Dept. of Professional Employees, AFL-CIO]: Programs & Publications: DPE Analyses: Scientists, Engineers and Technical Workers and is available at the following web site: http://www.dpeaflcio.org/programs/analyses/2002_sci_eng.htm

[2] David Goodstein, “Scientific Ph.D Problems”, American Scholar, vol. 62, no. 2, spring 1993, and “Scientific Elites and Scientific Illiterates”, Ethics, Values and the Promise of Science, Forum Proceedings, Sigma Xi, The Science Research Society, February 25-26, 1993, pg. 61, and Engineering and Science Spring 1993, 56(3): 22. A talk “The Big Cruch” NCAR 48 Symposium, Portland, OR September 19, 1994 is available at http://www.its.caltech.edu/~dg/crunch_art.html