If any of the above are true, you have been reading too much corporate media and witnessed coverage that they can claim is resulting from peer-reviewed journals.
But what does peer-reviewed mean and why are corporate media consumed with it, which led to some academics fetishizing it? Prior to the 1980s, peer-reviewed publication was a component of academic research but not any gold standard - you don't find a lot of mentions of it before a huge upward curve began in that decade.(1) It was only when President Reagan wanted to fund more basic research and needed a way to show the value for taxpayer money spent that agencies began to give special weighting to scientists who focused more on being in peer-reviewed journals. Peer-reviewed publication became essential in a way it never had been before - by linking it to government money. And the industry exploded from there to where it is now; a wildly profitable industry, all based on work that that public funds.
This year, one open access journal alone, PLOS ONE, will publish 40,000 papers, and they claim they are all peer-reviewed. There are thousands of other journals. By last month, there were already 8,000 preprints for SARS-CoV-2 and, a new paper notes, a firehose of other papers discussing climate change and social issues. All that will be able to claim peer-reviewed status. All competing to appear in the New York Times or The Atlantic. It is a gold rush for journals, because the public truly cares about science in a way most never have, but the pandemic has also created an "infodemic" and the public could emerge from this crisis trusting the process of science less than ever due to so many suspect studies.
How can thousands and thousands of papers be published per month, all claiming that peer-review is the gold standard and their journals have it? No one would ever have real jobs if so many papers are actually peer-reviewed. The answer is that peer-review is the best we've got, but it isn't always great, and of those 8,000 coronavirus papers available now, the only ones likely to get true peer-review are those promoted by President Trump in an election year. That is not a good look for academics who claim to be about public benefit when they are applying for taxpayer money.
Peer-review itself can easily be gamed. If I get three believers in ghosts to peer-review a paper on the science of ghosts, they are going to be more likely to give that paper a pass, and that is why if peer-reviewers at National Institute of Environmental Health Sciences get together and read papers on chemical ghosts - the fancier sounding endocrine disruption that is just rebranded homeopathy - they can easily fill their in-house magazine (Environmental Health Perspectives) with so many scare stories about the world we live in it's a miracle anyone has survived this long.
"Publish or perish" culture to chase government funding has become so problematic that over 33 percent of academic scientists admit to questionable research practices and 72 percent know someone who engages in them. Movies like to sell the narrative that corporations are doing cover-ups but that's very difficult. In order to be sold, a product must first show it works, not that it simply passed peer-review in a journal desperate for content.
Peer review doesn't catch a lot, which is why so many papers that are high-profile, such as when a Republican talks about them while 94 percent of American academic scientists vote Democrat, only get criticized after media buzz. We've seen peer-reviewed work claiming arsenic-based life and faster than light experiments, all that got huge attention before being debunked. Plenty of other papers are just copies with new conclusions and peer-review never noticed. In one examination, over 400 papers shared similar title, graph, and Western blot layouts and no one at any of the journals or their peer review process caught it.
Was the President really wrong for believing a paper that was in the International Journal of Antimicrobial Agents when it correlated the malaria drug hydroxychloroquine to better COVID-19? Should he believe The Lancet? Or listen to the World Health Organisation when they insisted the virus could be spread human to human? I suppose he should. He is not a scientist, each of those instances passed a whole panel of experts signing off on them, but he could use more people on staff who are skeptical. To scientifically literate people, observational findings, from small micron particulate matter (PM2.5) to endocrine disruptors in mice, are placed into an 'exploratory' file, but corporate media have taken epidemiological claims and treated them like they are a Supreme Court over medicine, biology, and chemistry. That makes politicians want to turn statistical correlation into law. Hardly a week goes by when some diet is not linked to longevity or some chemical to harm using surveys. Editors and journalists will say they have no blame because it's all been peer-reviewed.
The issue is not just peer-review, it is that too many fields wrap themselves in the flag of settled science, even if they are just mouse studies or epidemiological correlation. Who is going to tell them their studies should not be hyped, other mice researchers or epidemiologists? That rarely happens. Who reviews the reviewers? Only you and I.
NOTE:
(1) These cultural shifts happen and can become juggernauts without any real reason even if they have an inflection point. Prior to 1979, for example, the NIT college basketball tournament was more prestigious than the NCAA tournament, but in 1979 the Indiana State Sycamores of the Missouri Valley Conference - you read that right - had a finals that was the highest rated basketball game in history. They went into that game undefeated but were going to face USC, who had a mediocre season but had run rampant through their bracket in the tournament. The public were excited about Larry Bird and Magic Johnson playing each other for the first time. They didn't just change basketball as they both went on to Hall of Fame careers, they made the NCAA the place to be over the NIT.
Comments