As a scientist with 40-year’s experience, including being editor of two journals, I'd say your characterization of consensus-followers is wrong as is the suggestion that we can best expect to move forward because of "reformers". At the very least, this gives a false impression.
First, It seems to me anyone who call themselves a reformer has an agenda of change for change’s sake, else there is no need for reform. Charging onto the battlefield under the banner of reform is not being open minded. Understanding the shortcomings of current explanations is not reform. Neither is the recognition of new data for their ability to falsify old notions.
Secondly, folks who are keen on consensus are not slaves to the status quo. They are happy to change when the consensus suggests it would be wise to do so. This sometimes means that change for the good may be slow in coming (progressing sometimes one funeral at a time) but it almost always means that rapid swings from one bad idea to another are avoided.
I don't know a lot about the history of medical science but in my field, earth and space science, "reformers" don't make much of a difference, It's folks that know what the important questions are and how to test them (with properly formulated scientific methods) that move things forward. This is mainstream science. It is not in need of reform.
Thanks for taking the time to write! I suspect we disagree less than it might seem, and our differences are probably due to our professional backgrounds.
While I’m sure there are unsettled questions and ongoing controversies in the earth sciences, they dominate current biomedical science. Much of the reason has to do with the nature of the problems we have to address. The low-hanging fruit was harvested long ago, and we are very good at dealing with acute illnesses and injuries. What remains is largely the management of chronic illnesses, and the signal-to-noise ratio is very high when it comes research in those areas. Yet, we tend to act as if we are more certain than we have cause to believe.
This is clear from the frequency of medical reversals, in which a treatment is initially approved and subsequently withdrawn. Granted, some medical reversals are unforeseeable and therefore understandable, but a majority are the result of accepting an inadequate scientific basis before moving forward to begin with. I’ve seen dozens of these occur in my nearly 37 years as a physician.
I agree that consensus followers, at their best, are not mindless defenders of the status quo. Consensus is often the mechanism by which science protects itself from rapid oscillations between bad ideas. I certainly consider myself a consensus-follower when it comes to foundational ideas in medicine like germ theory.
When I speak of reformers, I don’t mean radical revolutionaries, and agree with you that change for its own sake is not a scientific virtue. What I mean by reformers is those who are advocating for better-quality research and more epistemic humility. In other words, people who are good at asking “how confident should we really be here?” rather than “what should we overthrow next?”
In that sense, I think we’re aligned: progress still comes from people who know which questions matter and how to test them properly. My worry is simply that, in medicine, the environment makes that harder than we often admit, and that acknowledging those constraints openly is a prerequisite for real progress, not an attack on mainstream science.
The following excerpt is from “Reproducibility of Research – Do We Have a Problem Houston?”
The Column, 19 January 2016 Volume 12 Issue 1
Even the most ethical of researchers are susceptible to self-deception; outlined below are a few of the reasons why:
●Hypothesis Myopia — A natural inclination to favor only one hypothesis and look for evidence to
support it, while playing down evidence against it…
●Sharpshooter — Fire off a random series of shots, then draw a target around the bullet holes to ensure the highest number of bullseyes.
●Asymmetric Attention (Disconfirmation Bias) — Giving the expected results smiling approval, while unexpected results are blamed on experimental procedure or error rather than being accepted as a true challenge to your hypothesis.
●Just-So Storytelling — Finding rational explanations to fit the data after the fact. The problem is, we can find a story to fit just about every type of data. Also known as JARKing — “justifying after the results are known” —because it’s really difficult to go back and start again once we are at the end of the process.
●The Ikea Effect — Everyone has a vested interest in loving the furniture they built themselves.
I view your definitions of a Consensus-follower and a Contrarian as two sides of the same coin. They both suffer from Hypothesis Myopia.
Thanks for commenting, Dan! I had not heard of that article before, but found it downloaded it to read later. Your list of reasons why researchers can delude themselves is spot-on.
Hi Chris,
As a scientist with 40-year’s experience, including being editor of two journals, I'd say your characterization of consensus-followers is wrong as is the suggestion that we can best expect to move forward because of "reformers". At the very least, this gives a false impression.
First, It seems to me anyone who call themselves a reformer has an agenda of change for change’s sake, else there is no need for reform. Charging onto the battlefield under the banner of reform is not being open minded. Understanding the shortcomings of current explanations is not reform. Neither is the recognition of new data for their ability to falsify old notions.
Secondly, folks who are keen on consensus are not slaves to the status quo. They are happy to change when the consensus suggests it would be wise to do so. This sometimes means that change for the good may be slow in coming (progressing sometimes one funeral at a time) but it almost always means that rapid swings from one bad idea to another are avoided.
I don't know a lot about the history of medical science but in my field, earth and space science, "reformers" don't make much of a difference, It's folks that know what the important questions are and how to test them (with properly formulated scientific methods) that move things forward. This is mainstream science. It is not in need of reform.
Hey, Pete!
Thanks for taking the time to write! I suspect we disagree less than it might seem, and our differences are probably due to our professional backgrounds.
While I’m sure there are unsettled questions and ongoing controversies in the earth sciences, they dominate current biomedical science. Much of the reason has to do with the nature of the problems we have to address. The low-hanging fruit was harvested long ago, and we are very good at dealing with acute illnesses and injuries. What remains is largely the management of chronic illnesses, and the signal-to-noise ratio is very high when it comes research in those areas. Yet, we tend to act as if we are more certain than we have cause to believe.
This is clear from the frequency of medical reversals, in which a treatment is initially approved and subsequently withdrawn. Granted, some medical reversals are unforeseeable and therefore understandable, but a majority are the result of accepting an inadequate scientific basis before moving forward to begin with. I’ve seen dozens of these occur in my nearly 37 years as a physician.
I agree that consensus followers, at their best, are not mindless defenders of the status quo. Consensus is often the mechanism by which science protects itself from rapid oscillations between bad ideas. I certainly consider myself a consensus-follower when it comes to foundational ideas in medicine like germ theory.
When I speak of reformers, I don’t mean radical revolutionaries, and agree with you that change for its own sake is not a scientific virtue. What I mean by reformers is those who are advocating for better-quality research and more epistemic humility. In other words, people who are good at asking “how confident should we really be here?” rather than “what should we overthrow next?”
In that sense, I think we’re aligned: progress still comes from people who know which questions matter and how to test them properly. My worry is simply that, in medicine, the environment makes that harder than we often admit, and that acknowledging those constraints openly is a prerequisite for real progress, not an attack on mainstream science.
The following excerpt is from “Reproducibility of Research – Do We Have a Problem Houston?”
The Column, 19 January 2016 Volume 12 Issue 1
Even the most ethical of researchers are susceptible to self-deception; outlined below are a few of the reasons why:
●Hypothesis Myopia — A natural inclination to favor only one hypothesis and look for evidence to
support it, while playing down evidence against it…
●Sharpshooter — Fire off a random series of shots, then draw a target around the bullet holes to ensure the highest number of bullseyes.
●Asymmetric Attention (Disconfirmation Bias) — Giving the expected results smiling approval, while unexpected results are blamed on experimental procedure or error rather than being accepted as a true challenge to your hypothesis.
●Just-So Storytelling — Finding rational explanations to fit the data after the fact. The problem is, we can find a story to fit just about every type of data. Also known as JARKing — “justifying after the results are known” —because it’s really difficult to go back and start again once we are at the end of the process.
●The Ikea Effect — Everyone has a vested interest in loving the furniture they built themselves.
I view your definitions of a Consensus-follower and a Contrarian as two sides of the same coin. They both suffer from Hypothesis Myopia.
Thanks for commenting, Dan! I had not heard of that article before, but found it downloaded it to read later. Your list of reasons why researchers can delude themselves is spot-on.