Luxembourg National Research Fund

Blood, sweat & tears: This is the story of how a Luxembourg research group discovered the unlikely contamination of a widely used lab kit – compromising their data and setting a question mark over the validity of data in countless journal publications. We speak to group leader FNR ATTRACT Fellow Prof Dr Paul Wilmes about the implications of their discovery, reproducibility, scientific due diligence – and how his group teamed up with the kit manufacturer to find a solution.

The beginning

Rewind to 2011/12. Prof Dr Paul Wilmes and his research group Eco Systems Biology, based at the LCSB of the University of Luxembourg, develop a biomolecular extraction methodology. The group apply it to biomass from a wastewater treatment plant, local river water, and human stool samples. The stool showed apparent enrichment in Small RNA[1][2].

This got the team wondering if this small RNA may play a role in human-microbe interactions. It would make sense – it’s known that in particular microRNAs play essential regulatory roles in mammalian physiology. They have also been found to be dysregulated in e.g. cancer and neurodegenerative diseases, making them a great source for potential disease biomarkers.

“About 50% of the sequence data was not human, but from bacteria, archaea, fungi, food and even mosquito RNA”

Over the years, research has particularly turned to blood to try to find any markers that could reflect the diseased or healthy state. The issue here: there is little or no consistent or common line through many published studies in terms of results.

After Prof Wilmes and his team found Small RNA in human stool, they wondered if this could also be found in blood. In an initial study, teaming up with researchers in the US, the team looked at the blood – this is where the first unexpected results came in:

“We mapped the sequence data against a catalogue of gut microbes and other organisms, and found about 50% of the sequence data was not human, but from bacteria, archaea, fungi, food and even mosquito RNA. At the time, we thought this was an interesting observation”, explains Prof Dr Paul Wilmes.

This finding suggested it may in fact be possible for Small RNA from the microbiota in the gut to travel into the bloodstream. Realising the potentially enormous insights this type of information could provide, Wilmes applied for a grant from the FNR’s PoC (now JUMP) programme to investigate further.

“It was clear that this could be an enormously valuable resource”

“If this was all true, i.e. that a large-scale transfer of microRNA-like molecules, Small RNAs, occurs from the gut into the human blood stream, this might be a marker for whatever might be going on in your gut – not speaking about the potential for discovery of a new form of communication between distantly related organisms. There were also dietary RNAs – it was clear that this could be an enormously valuable source for not only getting an overview of the microbes we have in the gut, but also to see what you have eaten and even for forensic purposes, to e.g. look at which biological agents someone has been exposed to,” Wilmes explains.

In addition to the PoC grant, Prof Wilmes had also applied for a patent for their biomolecular extraction methodology which has since been granted. Successful in getting the FNR grant, Wilmes and team got to work on getting more sequence data.

“We looked at the data using a new, more precise, bioinformatic pipeline we developed especially for the problem of accurately linking the short sequences to the organism of origin and this took 6 – 9 months. When we then looked at the list of identified organisms, it raised many questions.

“For the most part, we were mapping against organisms you would not expect to see in humans, such as algae, plants, water-borne bacteria and fungi. We developed all these hypotheses as to where these sequences might come from and how they end up in human circulation.”

“It’s for example not inconceivable that humans ingest quite a lot of fungi from contaminated food and therefore you might find RNA from these organisms in the blood.”

“Nonetheless, this really boggled our mind and this was beginning to not make sense. The majority of the organisms we were mapping against are just not to be found in/on the human body. I figured there must be something else contributing to this. We very openly discussed this in the lab and at this point none of us were really all too convinced about this anymore.” This was late 2013/ early 2014.

Testing to rule out contamination

At this point, Wilmes explains, they decided to test all sampling material and labware that comes into contact with the samples to rule out contamination:

“If we were going down the road of developing a biomarker, we had to be sure the signals we were seeing were real and not artefacts either from our sampling, the lab environment and/or the reagents used.”

Anna Heintz-Buschart did an enormous amount of “detective” work involving for example mock extractions, running the test with RNA-free water instead of blood, and that’s when it became clear that something was contaminated. But what could it be? The team painstakingly looked at everything and finally discovered that it was the RNA purification columns, specifically the silica in the columns. At this point, the team was nearly two years into the PoC project.

To verify the findings, the team used bleach on the columns, as it removes RNA, and then sequenced the eluate from the column again, confirming that the non-human RNA was already in the column long before they used it – putting an immediate end to their project of looking at RNA making its way from the gut to the blood.

“We had a tough week when we found out about the contamination. We thought ‘how can this possibly be true?’”

Anna Heintz-Buschart, first-author on the paper explains: “Before we had the more accurate bioinformatics pipeline, we did not have a chance to validate the sequencing results on the blood extracts and we also could not search for the sources of the exogenous small RNAs in our environment. With the pipeline’s results, we could design independent, highly specific assays for these sequences – and test blood samples from unrelated individuals and the environment. We were very worried when we found the sequences in question literally in all samples we tested.

“When I set up the experiment to check all the lab equipment used in this study for a potential contamination, I was scared. It’s the worst nightmare of any laboratory researcher to find out that you’ve not been working as carefully as you should have been. But when I saw that the source of our special small RNAs was in the columns, I did not even feel relieved that it wasn’t just our lab’s problem,” says Anna Heintz-Burschart

“In spite of the doubts we all shared, finding that the cornerstone of this project had essentially been missing all the time was a big shock. Fortunately, our crew had a very open, collaborative culture which meant that we discussed the implications of this finding and how we could deal with it in a constructive way,” says Wilmes.

Wilmes continues: “We had a tough week when we found out about the contamination. We thought ‘how can this possibly be true?’ We looked at the neatly sealed column from the kit and were puzzled. Dilmurat Yusuf in my team had worked on this for more than two years. You can imagine the frustration people live through in this situation, it’s not easy. But sometimes science is like that and you have to accept it – it’s about getting to the bottom of a problem and finding out if something is true or not. In this case, our study turned out in a way we would never have been able to conceive. Especially in relation to the PoC project we had submitted.”

“It was important for me that we come up with a solution instead of just leaving it at the stage where we knew there was a problem”

Wilmes contacted the manufacturer – Qiagen – to inform them of the findings and to find out if they would fund a follow-up study so the problem could be addressed.

“It was important for me that we come up with a solution instead of just leaving it at the stage where we knew there was a problem. I have to say Qiagen, the company, was very receptive and responded extremely well. They also wanted to get to the bottom of this and therefore funded the follow-up project. They did benefit in that they were not only informed there was a problem, but also offered a solution.”

With help of Prof Wilmes and his team, Qiagen have now developed a revised version of the kit – Wilmes’ lab has verified contaminants no longer play a role, according their analysis. Wilmes’ team has also come up with a whole suite of recommendations for future work to rule out contaminants playing a role, e.g. to use at least 100 microliters of blood plasma for RNA extraction.

The implications

The implications of the discovery go far beyond the project of Prof Wilmes’ group – this kit is one of the most widely-used RNA extraction kits – there are hundreds of publications, in some more than 99% of the data is from contaminating RNA.

The ongoing issue of reproducibility comes to mind: Could this contaminated kit help explain why there is little or no common line or constant in published studies looking at RNA biomarkers in blood?

“We have noticed that the contamination can vary from batch to batch for this manufacturer. We have focused on one kit, but there are multiple other similar kits. Testing this kit was at least 2 years of work – for one kit. Therefore, our findings also play into the current discussions around reproducibility – if you have inconsistent levels of contaminants across different kits, and then different studies, especially in the area of using microRNA for diagnostic studies – this is a field that has really struggled with the fact that you can’t reproduce findings from one study in another. What is that due to? There are likely multiple reasons but our work highlights at least one important possible explanation,” explains Prof Wilmes.

Scientific due diligence

Asked about how he thinks things would have developed had they not checked for contamination, Wilmes explains that they might have published papers, but that they probably would have discovered the contamination at some point.

“I find it remarkable that innumerable papers have already been published without picking up on the problem of contamination in this kit. This has to do with scientific diligence – we were very critical of the data, we were not buying what we were initially seeing. The fact is that we did all the scientific due diligence that is required. I think this is a reflection of the fact that we are doing very good science here in Luxembourg.”

“The other thing is – when people discover such a problem as the one we found, they often just stop there, judging it to not be in their interest to investigate further. It’s difficult to publish such a “story” and I am really happy we got it published. We may not be making a lot of friends with this type of work, especially not among the people whose publications are based on contaminated data, but I think it’s important we did.”

Contamination findings & recommendations published

A publication detailing the RNA extraction kit contamination was published online (open access) in the journal BMC Biology on Monday, 14 May 2018, with Anna Heintz Burschart as first author. https://doi.org/10.1186/s12915-018-0522-7

You can also read about the publication on the LCSB website.

Biomarker search continues

Wilmes and his group continue their scientific investigations into Small RNA potentially ending up in the bloodstream.

“From our work, we know that bacteria from the gut export small RNA. We have for example published a paper showing that this is the case for E. coli in part through packaging these into outer membrane vesicles. The key now is going to be to look at individuals where the barrier function of the gut is impaired, such as in inflammatory bowel disease and other diseases such as colorectal cancer. If there is this export of Small RNA into the blood due to a lack of barrier function that would be quite interesting from the point of view of blood-borne biomarkers for these diseases.”

[1] RNA = Ribonucleic acid, a nucleic acid present in all living cells. Its principal role is to act as a messenger carrying instructions from DNA for controlling the synthesis of proteins, although in some viruses RNA rather than DNA carries the genetic information. source

[2] usually non-coding RNA molecules, with the most common and well-studied example being RNA interference (RNAi), in which endogenously expressed microRNA (miRNA) induces the degradation of complementary messenger RNA – source