Wednesday, April 06, 2005

Peer review problem?

So, I find this article very interesting.

It highlights the problems with peer-reviewed scientific research and some of the possible alternatives. I can see the problem with some sciences, such as medicine, environmental science, or some areas of biology. You get a lot of work that comes down the line that doesn't reach the right conclusion from the data, attempts to draw conclusions too strongly from the data, doesn't have enough data, or is simply shoddy work. But sometimes it gets through. I'm sure there's any number of reasons it happens, but it does.

In my mind, a big problem is also with the sciences that are ideologically charged (such as evolutionary biology or environmental science). There, it is entirely a matter of who you're sending your paper to that will determine who likes it. The problem with such is that often scientists are punished for breaking orthodoxy and thinking outside the box. If a study doesn't mesh with the previous theories or position, then the data must be faulty or the conclusion wrong, but the theory never needs to be altered. It's a bad problem.

Although, I dont' see it as being a problem in my own area, chemistry. There aren't too many ideological issues at stake there (well, talk to Einstein about that . . . apparently quantum physics can get pretty ideological). The problem of quality still exists there, though.

And therein lies the rub. If peer review isn't cutting the mustard, I'd be all for finding an alternative, something that works better. But I haven't heard any good ideas. Getting other scientists to examine research methodology and results seems to be the best method we have at the moment. Perhaps it doesn't have to be universally replaced. In some areas of biology or chemistry, replication of work is easy enough that peer review probably should take place. But with medicine or environmental science? Some things can't be repeated.

What kind of alternative is there?

No comments: