“We are concerned that dubious research methods are becoming increasingly frequent.”
Research funders from several countries, including the Swedish Research Council, expressed that opinion in the journal Nature in the beginning of March this year.
The funders refer to the difficulties of obtaining the same results when published studies are repeated. Nor does there appear to be any difference between publications of Nature’s calibre and smaller journals.
Scientific results in complex research such as computer science, biomedicine and psychology appear to find it increasingly difficult to withstand reviews (read more about this here).
Tough conditions play a role
Brian Martinson, PhD in sociology and demography, is studying the composition of the research community, researchers’ social environment and how it affects scientific quality. He is also a member of the American group of experts which will publish an updated version in the summer of the 1992 report Responsible Science: Ensuring the Integrity of the Research Process.
He stresses that there is little established evidence on what lies behind the problem of unstable research results. However, he sees signs that researchers’ career conditions come into play.
“Those who start researching do it because they want to find true knowledge, to do good. But later, the road to success in a research career turns out to be something far from the scientific ideal,” he says.
Brian Martinson shows data from USA, where it is clear that the number of doctoral students and postdocs has increased, at the same time as it is increasingly difficult for them to find a permanent job. This creates an incentive to take short-cuts, not necessarily by cheating but by doing too few experiments and controls.
In a survey conducted with a thousand participating researchers, he has also shown that more than half knew that they had been careless. Carelessness, such as not documenting work sufficiently, not following protocols for handling lab materials or skipping steps in the process in order to speed up a project.
Sweden is no exception when it comes to conditions of employment. A new report from the Swedish Research Council shows that careers have become increasingly difficult for young researchers.
Hans Wigzell, professor of immunology and former president of Karolinska Institutet, confirms that Sweden follows the same pattern as the international scientific community. Competition for grants and jobs is increasing and a growing proportion of salaries comes from private funds. This leads to a high tempo with insufficient control tests. Read more in Curie about the difficulties involved in repeating research.
Obstacles to the rational search for knowledge
C.K. Gunsalus, manager at the National Center for Professional and Research Ethics (NCPRE) in USA also works with the report Responsible Science. She believes that research is often carried out using good methods, but she has also seen several examples of obstacles to the rational search for knowledge in environments where there is extreme competition.
One example is when a specific research area has been reserved for one professor at a department, or when two doctoral students have had to compete against each other, which can lead to sabotage.
Those who are thorough may also live dangerously at times. A doctoral student came close to being sacked when she could not replicate the studies of the lab superstar. Later it became apparent that the star’s data were fabrications, says C. K. Gunsalus.
“When doctoral students are expected to be constantly productive and never report any bad news, they may be tempted to fudge more and fall into wishful thinking, or interpreting results to fit the hypothesis.
C.K. Gunsalus also points out how researchers often run very complex experiments, possibly without sufficient knowledge of statistics and methods.
Slower pace for good science
Documentation of the work process is often inadequate. It is extremely rare that a researcher actually destroys data when someone wants to examine them closer; more often than not, it is a question of missing details.
C.K. Gunsalus mentions an example in which two laboratories were trying to repeat each other’s results. It took a year, despite the fact that the researcher who had initially done the experiment was present to help.
“That it took such a long time could have been because they stirred the contents of a beaker once out of habit, but forgot to note it down.”
If you weigh in a climate that encourages carelessness, the risk for unstable results increases.
“We used to talk about “bad apples” in research, meaning those who cheated, and the solution was to get rid of them,” says Brian Martinson.
Like many others, he now talks about systems which favour reliable research to varying degrees. The department and the research group can act as a magnifying glass, which reinforces the international pressure on publishing and increases the risk of sloppiness, or as a rainbow prism which provides conditions for creativity and good research.
Hallmark of research threatened
Nils-Eric Sahlin, professor of medical ethics and chairman of the Swedish Research Council’s Expert Group on Ethics, takes a serious view of the current situation.
“The hallmark of research – repeatability – is threatened by the pressure to publish. You have to publish quickly and have an interesting hypothesis. Then you get a gold star in your writing book, but not always stable results.”
Nils-Eric Sahlin believes that more education and a calmer publishing tempo are necessary.
“The system today, with its star researchers and top-lists of publications, has become a kind of Eurovision Song Contest. It does not promote good research.”
Also read in Curie: Perfect research is a myth