Everyone likes the idea of evidence-based policy, but it’s hard to realize it when our most reputable social science journals are still publishing poor quality research.
Interesting article! Good summary of the issues with social science research. But do we have evidence that the situation in social science is worse than in medical science? Many of the same issues here are prevalent in medical science too. There also it's much easier to produce positive than negative results. There also many studies use small sample sizes. Without comparing across fields it's hard to know if social science is truly worse than other fields (which is the main convention of the piece).
I can't speak for Ryan, but I don't think he's arguing that social science is worse than other fields – he's saying that consumers of social science research (eg policy-makers or nonprofits) should moderate their expectations about its validity when they make practical decisions, and also that researchers should try to adopt norms to make their work more reliable. (I'm also interested in commissioning work on similar problems in medicine!)
Interesting article! Good summary of the issues with social science research. But do we have evidence that the situation in social science is worse than in medical science? Many of the same issues here are prevalent in medical science too. There also it's much easier to produce positive than negative results. There also many studies use small sample sizes. Without comparing across fields it's hard to know if social science is truly worse than other fields (which is the main convention of the piece).
I can't speak for Ryan, but I don't think he's arguing that social science is worse than other fields – he's saying that consumers of social science research (eg policy-makers or nonprofits) should moderate their expectations about its validity when they make practical decisions, and also that researchers should try to adopt norms to make their work more reliable. (I'm also interested in commissioning work on similar problems in medicine!)
😂