Lisa Wade recently made me aware that a group of psychologists have decided to try and verify the results of every article written in three journals in 2008. I think this is a great idea. And the likelihood that the results can be replicated look pretty unlikely. As the Chronicle piece I’ve linked to points out,
Recently, a scientist named C. Glenn Begley attempted to replicate 53 cancer studies he deemed landmark publications. He could only replicate six. Six! Last December I interviewed Christopher Chabris about his paper titled “Most Reported Genetic Associations with General Intelligence Are Probably False Positives.” Most!”
If only 6 out of 53 cancer studies can be replicated, I shudder what to think what the social science data will say (in part because there is so much more noise in a lot of our research design). It’s more of an open question to me what this means. It could mean that researchers are generally frauds.