Interested in being a scientist? Then you had better get used to rejection and failure, because the truth is that most of your experiments will fail, most of your original ideas will be wrong, and most of your grant proposals and papers will be rejected on the first submission (especially if you submit to competitive journals).

This doesn't mean you're a bad scientist - failure and rejection is a normal fact of life for most scientists (that is, all scientists except ones who aren't doing any science!). But this means that peer-review is a good thing, even when it affects you personally as your paper gets rejected (as mine was last week).

How does peer review work in scientific publishing? One of the most important things you do as a scientist is produce publishable results, which generates not just in fame and glory, but hopefully progress in understanding an important question. When you have publishable results, you submit them to a professional journal. At the journal, an editor makes a triage decision (is there any hope at all that this paper will be accepted?), and either immediately rejects your paper, or sends it to 2-3 outside experts, who in all likelihood are people you know, even though they are technically anonymous (scientific communities are generally pretty small). These reviewers pass judgment on the significance of the work, as well as look for flaws in the reasoning or experiments.

A paper is usually rejected, either before or after review, for one of three major reasons:

1. The paper will only get the scientists in your specialty excited, so it doesn't belong in Nature (or Science, or Cell, or PLoS Biology, etc.) - most of these journals reject 80%-95% of the papers they receive.

2. The paper needs more - in Scientific Blogging terms, it's a blog post, not a feature article. Journals want feature articles. (My recent paper fell victim to this particular criticism.)

3. There is some horrendous flaw, completely obvious to the reviewers but missed by you, your labmates, your friends in the department - except for that one curmudgeon who kept badgering you at your talks, telling you that you need to do experiment X, which you thought was ridiculous and too much work anyway.

While the odds may not look good, many of the scientists I work with (and I agree with them, in spite of my recent rejection experience) think that peer-review isn't stringent enough. All of the safeguards notwithstanding, many mediocre papers still get published.

This is my long-winded way of getting to the point of this whole post: when you read about scientific breakthroughs in the media, remember that these breakthroughs have not yet passed the most stringent test of peer-review: the approval of more than just the 4 people who reviewed it before publication; the real test is whether the dozens or hundreds of scientists working in the same field find the results persuasive. When these people start using and citing the results, then you know that the results have some merit - although in the end, they still may be wrong.