Another ThreadReader unroll of a recent Twitter thread that I did.
I got several questions from other scientists who all are interested in how to detect science misconduct and how to report. So here is a new thread.
First of all, this is a risky business. Reporting misconduct close to home (e.g. in your own lab or by a close collaborator) might damage not only other people’s careers, but also risk your own job, especially if you are early career.
Second, you need to be able to objectively word your concerns. Yelling “misconduct!” is not going to bring you very far. You have to stick to facts.
“These 2 protein bands look unexpectedly similar” is good.
“These 2 protein bands have been copied and pasted” is subjective.
Third, you have to be patient. If you report a paper/set of papers to a journal or institute, the investigation might take years to complete. Of the 1016 papers, I reported to journals in 2014 and 2015, 54 have been retracted, 181 have been corrected. The rest? … crickets.
Let’s assume that after these 3 disclaimers, you are still interested. There are a couple of extreme scenarios. Maybe you are generally interested in how to spot misconduct and how to respond. Or maybe you suspect misconduct in your lab and are not sure what to do.
If you are generally interested in cases of science misconduct, there are a couple of places you can start:
@Pubpeer is the place where people can comment on a paper – anonymously or signed – positive or negative. Check there regularly for the types of comments people leave.
Also, @PubPeer has a great Chrome plugin that will flag papers (based on DOI) on e.g. Pubmed, so you can see that in your literature searches. It does not seem to work with Google Scholar, unfortunately.
Here is how the Chrome extension will work with Pubmed searches. It will show which papers have comments in @PubPeer .
If you found a paper and you have concerns, you can leave a comment yourself, either under your full name (not recommended when you are just starting) or anonymously (will be moderated, so comments don’t appear immediately).
Make sure to remain objective in your comments. Again, stick to wording such as “unexpected similarities” or “sharp transition between two adjacent bands” instead of “cloned” “fabricated” “manipulated” etc. Assume there is a slight chance that it was an honest mistake
Most of the problems that are found in biomedical papers are potential duplications of photographic images. With my coauthors @ACasadevall1 and @FangFerric, we wrote about those types of problems here (apologies for the weird associated photo):
The Prevalence of Inappropriate Image Duplication in Biomedical Research PublicationsInaccurate data in scientific papers can result from honest error or intentional falsification. This study attempted to determine the percentage of published papers that contain inappropriate image d…https://mbio.asm.org/content/7/3/e00809-16
But there are many other potential problems that you might spot in a paper. You can check for plagiarism by taking part of a sentence (5-8 words works well) between quotes, and searching in Google Scholar. See if you get a single result, or re-use of the sentence.
Definitions will of course give many results, so that is not plagiarism. Here is an example of a definition sentence that will give many results in Scholar: “”Probiotics are live organisms that, when administered in adequate amounts”. That is NOT plagiarism.
But multiple hits in Google Scholar with a sentence such as “”Some of these health problems include bone loss, muscle atrophy, cardiac dysrhythmias” could be cause of concern, especially if there are many other sentences in a paper with multiple hits.
Posting on Pubpeer is one thing, but the conventional – albeit much more slow and ineffective – way of reporting papers with concerns is to write to the Editor of the journal in which the paper was published. Or to the institution in case of multiple papers by the same lab.
Most journals have information on their website with their contact information and their Editor in Chief. It is most effective to write to more than one email address (pick a couple of senior editors as well) so that there is more chance that a journal will actually respond.
I always write per email, not paper letters, as to leave a record. Unfortunately, some journals make it very hard to find their contact information. You might have to cyberstalk the editors and search Pubmed publications or faculty pages for their email addresses.
If there are multiple problematic papers, you can also report to the institution / university. There might even be multiple institutions, if a person moves from lab to lab. Search for “Research Compliance” or “Research Integrity” and the university’s name. It might be hidden.
How about if you suspect misconduct close to yourself (e.g. by a co-worker in your lab)? If you trust the PI, you could first raise it with them. If not, you could report it to the Research Integrity office of your university. Write them an anonymous (paper) letter or email.
Unfortunately, it is very risky to write under your own name. Research Integrity officers might promise you anonymity, but might reveal your name to the defendant in a later stage of the investigation. This has happened to me and it sucks.
That is just some general advice that I have about how to spot and report cases of misconduct. Happy to talk more about this in a different thread.
I have decided to take (at least) a year off from paid work to focus on my research integrity work. Since 2013, I have worked on finding plagiarism and image duplication in scientific papers. Every free minute I searched for papers, made reports highlighting the potential problems, and wrote to journals and institutions about these concerns.
Together with Arturo Casadevall, Ferric Feng and other co-authors, we published three papers about our work on the frequency of image duplication in biomedical papers. You can find them here:
As of today, I have reported 85 papers and theses for extensive plagiarism, and over 1200 papers for potential image duplication. But I have many more that I still need to report, and I am getting more and more requests for help, and the work – all done in my spare time – was piling up.
So after working for 2.5 years in microbiome startup companies, I have decided to take at least a year off – maybe longer – so that I will have much more time to work on this project. I will still closely follow the microbiome field, of course, as well.
Thank you all, for your support!
I am taking a year off from paid work to focus more on my science misconduct volunteer work. Science needs more help to detect image duplication, plagiarism, fabricated results, and predatory publishers.
Most of the work detecting these problems in science papers is done by volunteers like me. It takes perseverance and patience. Many journals, authors, and academic institutions will not take action.
Even if they respond, It might take years before papers with serious flaws are corrected. All that time, those papers are not flagged by the journals, and others researchers might cite them or base their research on them.
As of now, we can only flag papers on @PubPeer and install their plugin so you can see which papers have a comment, e.g. when doing literature searches.
And I will still write to journals or institutions about all papers with concerns that I found so far. Even if it takes hours to find their contact info.
I still have 100s of papers that I need to officially report and 100s of reported ones to follow up on. The only way I felt I can catch up on that is to quit my paid job. Which is scary.
It would be nice if journals, institutions, funding agencies, and countries would care more about the quality of their research. If they had more guts to respond to concerns raised by readers – and take action.
The work that volunteers like us do is not very rewarding obviously. No one likes criticism. It can also be dangerous. Authors might start personal attacks on us and sue us for libel.
I am also very aware of the collateral damage, e.g. of coauthors who did not commit misconduct, others workers in those labs, and the effects on family members.
With that in mind, it is important to focus on facts, on the potential problems in the papers and how to address those. The focus should be on the papers, not the authors.
I might make an exception in cases of authors using false affiliations and fake coauthors.
The more you dig into these cases the more other weird stuff you find. I can probably do this full-time for the rest of my life. Maybe I will.
An open peer review of a preprint paper by Hatch A et al. from Viome, posted on OSF Preprints, January 2019.
Full disclosure: I worked for Viome’s competitor uBiome from October 2016 – December 2018. I am currently an independent consultant.
Andrew Hatch, James Horne, Ryan Toma, Brittany L. Twibell, Kalie M. Somerville, Benjamin Pelle, Kinga P. Canfield, Guruduth Banavar, Ally Perlina, Helen Messier, Niels Klitgord and Momchilo Vuyisich*
Viome, Inc, Los Alamos, NM 87544, United States.
In a recent paper authored by Stanford researchers, biotech startups were criticized for not sharing their discoveries through peer-reviewed research studies. So in that light, it is great to see a biotech company such as Viome publish a study about their microbiome consumer product. Viome’s leaders have been very vocal about the superiority of their product – which is based on RNA transcription – over that of other microbiome consumer tests, which are based on DNA amplification and sequencing. But until this preprint came out, no research on the Viome product had been published. So I was excited to hear about this preprint!
It is important to first point out that this paper is not a peer-reviewed paper. It is a preprint, which means it is written as an academic paper, but it has not been peer-reviewed by other scientists. It is a first step, however, to share the work that Viome did to build a metatranscriptomics platform, and show some of their first results. I hope my comments will be useful in the process of getting this study published in a peer-reviewed journal.
The paper describes Viomega, Viome’s automated stool metatranscriptomics method that involves RNA extraction from stool samples, sequencing, and bioinformatics analysis. Let’s go over each of the sections.
The introduction of the paper is mostly stating how metatranscriptomics is superior to other techniques. It has a somewhat oversimplified table showing other methods (bad!) to their own method (good!).
For example, under “Method Biases”, the 16S Gene Sequencing column states “Heavily influenced by amplification method, but also sequencing quality, sample lysis, and bioinformatics“, where “heavily influenced” sounds a bit denigrating, while sequencing quality and bioinformatics are conveniently left out in the Metatranscriptomics column.
The “identifies all living organisms” appears oversimplified as well. First, is a phage or virus alive? Then, how about a bacterium that is in a viable-non-culturable or spore-state? It is alive, but it is probably not transcribing much RNA – can metatranscriptomics detect those?
The statement that Metatranscriptomics “allows assessment of pathway activities that can lead to personalized health insights and recommendations with molecular-level precision” is overly subjective and is not proven in this paper. That last part of the sentence sounds like a Viome commercial, not like something that belongs in a scientific paper.
The introduction text states that 16S gene sequencing misses archaea or eukaryotes, but fails to acknowledge that most of these can be identified with broad-range primers as well.
In short, the Introduction shows a too black- and white comparison between different microbial community methods that is not very objective.
Not unexpectedly from a biotech company, the paper does not provide a lot of technical details on sample extraction, library preparation, or bioinformatics analysis. In order to pass peer review, the authors will likely need to provide more details on their methods – so that others can easily replicated them. The statements about participant consent and IRB approval were also very short; most journals and peer reviewers would like to see something more than “all study procedures were approved by an IRB” from a non-academic institution.
The first part of the Results shows that the Viomega method can detect a range of different microorganisms with relative equal efficiency and precision. Using mock communities, the results are accurate, and reported to contain no false positives or negatives or sample-to-sample crosstalk. However, data given here were very sparse.
Figure 4 is a colorful representation of the mock community sequencing experiment, but is lacking percentages and other details, such as number of reads. Where there really no false positive or false negative reads? Not a single one? There is also no word on how the negative controls did perform over their tests of 10,000 samples. It would have been really valuable to have compared this mock community using all three methods compared in Table 1; 16S, metagenomics, and metatranscriptomics.
This section of the Results also shows that the Viome test is reproducible. Three different experiments were performed in which small numbers of participants (3 to 7 persons) collected stool samples testing the following:
In all three experiments, samples from the same stool specimen or individual were very similar to each other, showing that the Viome test results are reproducible (the dark squares along the diagonal in Figures 5, 6, 7).
Figures 5/6/7 show the test results in two different ways. The A panels in each figure are based on which active microbes are present in the samples, so based on the taxonomy of the RNA reads. The B panels are the results based on gene function composition (which genes are being expressed in that sample).
It was interesting to see that in all cases the microbial composition (taxonomy; A panels) was better able to tell individuals apart than the gene expression (B panels).
Detailed explanation: If you look at the A panels, the comparison of each person to other samples from the same person shows they are very similar (purple) while each individual is very different (light blue) from other persons. In the B panels, the individuals do not differ that much from each other; all individual-to-individual comparisons are darker shades of purple that are more difficult to tell apart.
That is a bit ironical, because the preprint states at the beginning (Abstract and Table 1) that functional gene analysis, not microbial composition based on taxonomy, is needed for personalized health insights. Instead, the paper appears to show that each person has their own, personal gut microbiome, while the functional capacities appear to be pretty similar between individuals.
The paper does not provide an answer how these small person-to-person variations in microbial gene expression will lead to the “goal to develop personalized nutrition algorithms” as stated in the Abstract.
Metatranscriptomics is superior to other techniques, stated the paper in the Introduction and Table 1, but the paper does not do anything to prove that. There was no comparison of 16S sequencing vs metatranscriptomics on e.g., the mock community shown in Figure 4. Such a comparison could add a lot of value to the paper, and could support the bold statements made in the Introduction.
Also, the amount of viruses, archaea, and eukaryotes in the sample set was not very high, suggesting that 16S sequencing does not miss as much diversity as the authors claim in the introduction. For example, strain level analysis (Table S1) shows crAssphage as the most prevalent virus, but it is ranked #144 of all taxa (26%). Similarly, Methanobrevibacter clocks at #269 (14%), while Entamoeba is the most prevalent eukaryote at #336 (10% of samples).
Many of the viruses/phages found in the stool samples appear to be plant associated (Suppl tables). For example, Phaseolus vulgaris endornavirus (beans), Pepper mild mottle virus, Cannabis cryptic virus, and shallot latent virus are among the most prevalent viruses. Could the authors comment on this? Could these be transient microbiome components that were part of a food item, or are they present in the same person over longer periods of time? It would be nice to see some data on that.
Several taxa appear to behave very differently on strain/species/genus level (Tables S1-S3). Examples where genus-level prevalence is much lower than expected based on strain prevalence are given below. In other examples, taxa with high prevalence at genus level seem to disappear at species or strain level; also given below.
Unfortunately, relative abundance data appears to be missing. How abundant are viruses, archaea and eukaryotes ? Which percentage of the transcriptomics reads were assigned to each of these groups? These numbers, now missing, would tell us which groups 16S sequencing would miss and would allow for a better comparison of different microbial community analysis tools.
The paper contained very few functional results on the data from 10,000 Viome samples. Viome has repeatedly claimed that transcriptomics are much more informative than just lists of microbial taxa, and the abstract promises “several small clinical studies to demonstrate the connections between diet and the gut metatranscriptome.”
Therefore, it was disappointing to see that the paper was mainly limited to taxonomic assignments. This dataset, which is one of the largest of its kind, sounded very promising, and I had hoped to see many more functional analyses on these samples.
Table 4 is the only part of the study that analyzes the functional capacity of the data from 10,000 samples. It lists the top 12 KEGG functions (although the legend says “top 10”; see below). Unfortunately, this table is not very informative and might contain some errors (see below). Most importantly, it is just a list of genes without any discussion on their function.
In addition, if the prevalence of these genes is over 99.9% (meaning that almost every person’s microbiome contains those genes), how can one use that for the correlation of microbial taxa or genes with lifestyle or diet? Based on the top 10 or top 100 (Table S4) KEGG functions, all subjects’ microbiomes appear to have the exact same genes. How is this functional data superior to the big inter-individual differences that are found by the more conventional 16S types of microbiome analyses? The paper would be much stronger if this important point would be discussed.
In summary, the strengths of this paper are the experiments showing that the Viomega technique is reproducible. That is a great paper by itself, but the current title and abstract are promising much more than the paper currently delivers. With a transcriptomics dataset of 10,000 stool samples, and repeated claims that transcriptomics will give much more functional insight than just a list of taxa, it is disappointing to see that the results are limited to microbial taxon prevalence and not any functional analysis.
Based on this paper, Viome’s claims that their test can connect the microbiome transcriptome to a subject’s diet – let alone give dietary advice – appear to be very far-fetched.
Examples where genus-level prevalence is much lower than expected based on strain prevalence:
Strain: Eggerthella lenta 1_1_60AFAA, at 97.08% prevalence
Species: Eggerthella lenta at 91.76%
Genus: Eggerthella at 61.12%
Strain: Veillonella dispar A at 92.34
Species: Veillonella dispar at 88.99%
Genus: Veillonella at 78.62%
Strain: Entamoeba nuttalli P19 at 9.85% / Entamoeba dispar SAW760 0.74%
Species: Entamoeba nuttalli: 9.87% / Entamoeba dispar 0.74%
Genus: Entamoeba: 0.79%
In other examples, taxa with high prevalence at genus level seem to disappear at species or strain level:
Strain: Lactococcus piscium MKFS47 at 0.99% / Lactococcus raffinolactis NBRC 100932 at 0.23%
Species: Lactococcus piscium at 0.98%
Genus: Lactococcus at 96.88%
Strain level: Escherichia coli isolate 15 at 37.8%; Escherichia coli strain LS5218 at 9.5%, Escherichia coli M17 at 7.7%.
Species level: Escherichia coli at 27.40%
Genus level: Escherichia: 85.92% prevalence
Strain: Saccharomyces sp. ‘boulardii’ strain unique28 is present at 1.82% of samples, as the most prevalent Saccharomyces strain. Also Saccharomyces cerevisiae S288C at 0.80%.
Species level: S. cerevisiae is present in 3.9% of samples.
Genus level: Saccharomyces it is present in 25.44%.
Strain: Streptococcus sp. 263_SSPC 5.56% / Streptococcus mutans U138 at 1.62%
Species: Streptococcus thermophilus 28.15% / Streptococcus mutans 9.92% / Streptococcus parasanguinis 7.54%
Genus: Streptococcus 90.47%
Strain: Salmonella enterica subsp. enterica strain SE696A 2.30%; Salmonella enterica subsp. enterica strain ADRDL-LA-5-2013 2.28% / Salmonella enterica subsp. enterica serovar Typhimurium strain 1.52%
Species: Salmonella enterica 11.71%
Genus: Salmonella 15.61%
(Posting for our new team member, Zehra Tüzün Güvener.)
Good Morning! Highlights from Monday’s digest include a widely distributed efflux pump in saprophytic soil bacteria, genomic analysis of plant-growth-promoting rhizobacteria, identification of novel mammalian viruses from bats, rectal swabs to study gut microbiomes and TrueBiome’s solution to reduce variability in animal experiments.
Muropeptides Stimulate Growth Resumption from Stationary Phase in Escherichia coli – Joers et al. – BioRxiv
Human oral microbiome
Current Understanding of the Gut Microflora in Subjects with Nutrition-Associated Metabolic Disorder Such as Obesity and/or Diabetes: Is There Any Relevance with Oral Microflora? – Yumoto et al. – Current Oral Health Reports
Human skin microbiome
Scalp bacterial shift in Alopecia areata – Pinto et al. – PLOS ONE
Human gut microbiome
Factors influencing the gut microbiome in children: from infancy to childhood – Kumbhare et al. – Journal of Biosciences
Defining Escherichia coli as a health-promoting microbe against intestinal Pseudomonas aeruginosa – Christofi et al. – BioRxiv
Glyphosate, but not its metabolite AMPA, alters the honeybee gut microbiota – Blot et al. – PLOS ONE
Plant, root and soil microbiome
**MiniReview: Gene mobility in microbiomes of the mycosphere and mycorrhizosphere –role of plasmids and bacteriophages – Pratama et al. – FEMS Microbial Ecology
**Review: Response of microbial communities to biochar-amended soils: a critical review – Palansooriya et al. – Biochar
The effects of soil phosphorous content on microbiota are driven by the plant phosphate starvation response – Finkel et al. – – BioRxiv
Comparative genomic analysis of Bacillus paralicheniformis MDJK30 with its closely related species reveals an evolutionary relationship between B. paralicheniformis and B. licheniformis – Du et al. – BMC Genomics
Are drivers of root-associated fungal community structure context specific? – Alzarhani et al. – The ISME Journal
Water and extremophile microbiome
Metabolic potential of uncultured bacteria and archaea associated with petroleum seepage in deep-sea sediments – Dong et al. – Nature Communications
Probiotics / prebiotics
Screening of single or combined administration of 9 probiotics to reduce ammonia emissions from laying hens – Mi et al. – Poultry Science
Phages and viruses
A viral metagenomic survey identifies known and novel mammalian viruses in bats from Saudi Arabia – Mishra et al. – PLOS ONE
Metaproteomics of Freshwater Microbial Communities – Russo et al. – Mass Spectometry of Proteins (Methods in Molecular Biology)
Functional annotation of orthologs in metagenomes: a case study of genes for the transformation of oceanic dimethylsulfoniopropionate – Gonzales et al. – The ISME Journal
Quantitative and qualitative evaluation of the impact of the G2 enhancer, bead sizes and lysing tubes on the bacterial community composition during DNA extraction from recalcitrant soil core samples based on community sequencing and qPCR – Gobbi et al. — PLOS ONE
Usability of rectal swabs for microbiome sampling in a cohort study of hematological and oncological patients – Biehl et al. – PLOS ONE
Microbes in the news
** Report – North America Human Microbiome Market to 2025 – Regional Analysis and Forecasts by Product, Disease, Application, and Country – Research and Markets
Microbes on the market
** Newsletter – Addressing experimental variability: Taconic launches TruBiome – Outsourcing-Pharma.com
What a nice surprise (insert sarcasm). Naveen Jain, the CEO of Viome and Moon Express, with over 100k followers on Twitter, sent me such a flattering tweet tonight. What else can I do than to make sure the Internet never forgets…. 🙂
My second (and last) day of reporting from the 4th Annual North American Microbiome Congress in Washington DC, organized by Kisaco Research. Here is a roll-out of my tweet storm from today.
Good morning! In 15 minutes I will start live tweeting Day 2 from the 4th Annual Microbiome Congress, in Washington DC, organized by @KisacoRes #MBCongress2019
I am all ready to go in the front row at the plenary session, with Larry Weiss, Amanda Kay, and Ken Blount.
Larry Weiss @LWeissMD, CEO and founder of Persona Biome opens this morning’s session. In this field, we have undergone a transformation. We start thinking more like a microbial community, how everything is connected.
Larry Weiss: As scientists we gather data, and build models. But models are not truth. There is so much we don’t know that we don’t know. This field is still very early.
Larry Weiss: We are intimately connected to our microbiome, but have cut connections with our environment. We are a complex process of processes.
Larry Weiss: Finally, I want to talk about “shit”. We need to treat feces with much more respect. It is not only a waste product, but the product of a bioreactor.
Larry Weiss: This science is going to transform everything. We need to develop partnerships and work together.
The opening presentation of today is by Amanda Kay, VP at Synlogic @synlogic_tx, and Bradford McRae, AbbVie Discovery Immunology @abbvie about “Collaborating with AbbVie to develop a novel class of living medicines”
Bradford McRae starts off: There are several key questions for microbiome-based drug development. Most importantly: Are changes in the microbiome the cause of disease or a secondary consequence of the disease state?
Bradford McRae: Will altering the composition/function of the microbiome change the natural history of disease? Can the composition/function of the microbiome be used to stratify patient populations and/or define future disease course?
Bradford McRae: There are many factors that influence microbiome composition and function, such as diet, exercise, stress, medications, metabolism, immunity.
Bradford McRae: Some challenges for understanding the therapeutic potential of the microbiome: Intra-individual disease and microbiome heterogeneity make it hard to understand patterns of disease.
Bradford McRae: It is also still difficult to link sequence data to function, and to understand what happens in the different compartments of the gut by just looking at what comes out at the far end.
Bradford McRae: Excellent work on altering the function of the microbiome is being done in metabolic disease, such as FMT from lean donors to patients with metabolic syndrome, which improved glucose sensitivity.
Bradford McRae: Evidence that diet may alter the course of IBD through modifying microbiome function from Schwerd et al.
Bradford McRae: Future developments:
1. Moving beyond descriptive analysis to molecular pathways
2. Future deliverables for patients: diagnostics, mechanisms
3. Potential for engineered bacteria to deliver specific therapies
Amanda Kay: At @synlogic_tx, we are designing for life. We do rational design for bacterial based therapy, and collaborate with partners. We do synthetic biotic(TM).
Amanda Kay: Synthetics: genetic circuits, degradation of disease causing metabolites, production of therapeutic molecules
Biotics: bacterial chassis, non-pathogenic, amenable to genetic manipulation
Amanda Kay: In the context of IBD, we @synlogic_tx started a collaboration with AbbVie @abbvie We both brought in our key expertise, in drug development and translation into drug candidates.
Amanda Kay: Goal of our collaboration was to leverage known homeostatic functions to resolve disease and maintain health in patients
The next speaker is Ken Blount, CSO, Rebiotix @Rebiotix, with: “Discovering the potential of Microbiota Restoration Therapy (MRT) drug platforms for the treatment of intestinal diseases”
Ken Blount: Fecal transplant has a long history of working well, in particular in Clostridium difficile infections, but how do you perform it in a controlled and reproducible way?
Ken Blount: In a healthy gut, non-spore forming class Bacteroidia constitutes ~30% of bacteria. In our platform, the Microbiome Restoration Therapy (MRT), we want to restore a healthy gut microbiome in a patient.
Ken Blount: In C. diff infections (CDI), it is challenging to restore the foundation of a healthy microbiome. Our MRT is not a fecal transplant, but a standardized consortium of live spore forming and non-spore forming microbes.
Ken Blount: We designed several trials, phase 2 trials already done, phase 3 enrolling. We got 87% of treatment success in our initial phase 2 trial.
Ken Blount: In our second phase 2 trial, we learned that 2 doses worked as well as 1, but success rate was only around 60%. Placebo had 47% success rate (because of pretreatment with Abx?)
Ken Blount: After treatment, the patients’ microbiomes shifted towards that of a healthy population (HMP dataset). It restores Bacteroidia and Clostridia, decreasing Gammaproteobacteria.
Ken Blount: We developed the Microbiome Health Index (MHI, TM). High MHI means healthy, lower MHI is a deviation from healthy. This single number allows to better differentiate healthy and unhealthy.
Ken Blount: We are now in the middle of a phase 3 trial. We are working on a new stable formulation, in an oral capsule, RBX7455, which had a 90% success rate in a phase 1 trial.
Ken Blount: We are working on many other applications as well (for the enema formulation): ongoing trials in VRE, pediatric UC, UTI, hepatic encephalopathy.
We will now have two talks in the session “Regulation of biotech: Are we Prepared? “
Jim Weston, senior VP at Seres Therapeutics @SeresTX will talk about: “How we navigated the evolving guidelines”
Jim Weston: I will tell you about the process we did to get our products to market and collaborate with the regulatory offices such as the FDA.
Jim Weston: Our strategy at @SeresTX is a focused R&D, (e.g. C. diff infection), and to have a great clinical manufacturing operation. How do you meet the needs of patients as well as regulators?
Jim Weston: Therapeutic Microbiome Products are regulated in US as both biological products (drugs) as well as live biotherapeutic products. The best strategy is to have case-by-case collaborations with the regulators.
Jim Weston: TMPs can vary from stool (FMT), communities of strains, single strains, or genetically modified strains. These all require different regulatory processes.
Jim Weston: In the US, we have one agency for approval, the @US_FDA. It is best to start work with the FDA in an early stage. In Europe there is the EMA @EMA_News and national level agencies.
Jim Weston: It would be great to have some guidance for (fecal) donor screening and safety. Trial design needs guidance too, but end points are well defined.
Jim Weston: How do we ensure good manufacturing, and measurement of results? Regular interaction between industry and regulatory agencies is key.
The second talk in the session about regulatory agencies, will be by Larry Weiss @LWeissMD, CEO of Persona Biome, with “Re-writing the rules for microbiome therapeutics”
Larry Weiss: We are trying to get really solid data, but not by hacking the system or evading the rules. There are existing rules that we can follow, but solid science is important.
Larry Weiss: “The @US_FDA regulates two things: substances and words” – Peter Barton Hutt.
Larry Weiss: We are changing the states of pharmaceutical development, moving from pharmacology (chemistry) towards systems biology (microbiology). This is challenging a lot of our structures, including regulatory.
Larry Weiss: Shows the definitions of “drug”, “cosmetic”, “dietary supplement” by the FDA. There are several situations where it is difficult to distinguish between these categories.
Larry Weiss: The definition of GRAS: Generally Recognized as Safe states “no genuine dispute among qualified experts” – ha! When does that happen?? You can define your own product as GRAS (food ingredient).
Larry Weiss: There are many overlapping categories.
* Drug, cosmetic, medical food, probiotic?
* Bugs as drugs (probiotics, known commensals), drugs from bugs (purified extracts), microbiome manipulation (FMT, phages, prebiotics).
Larry Weiss: Each product that you might develop might have to follow a different path. It is not about evading rules, you have to make ethical decisions.
We will have a coffee break and then I will go to one of the three parallel sessions. Hard to choose! But I will probably go to the “Food and the Microbiome” session with Kristen Beck @theladybeck and Cindy Davis @NIH
Back from coffee, at the “Food and the Microbiome” session with Kristen Beck @theladybeck (we finally meet!) @IBMresearch , and Cindy Davis @NIH
Kristen Beck: I get asked a lot if IBM is involved in life sciences, and the answer is Yes! We have several collaborations in the microbial space, including in the microbiome. 20/3000 researchers at our division are involved in the microbiome.
Kristen Beck: We have partnerships with e.g. UCSD Center for Microbiome Innovation, and with Mars, to look at microbes in the food chain.
Kristen Beck: Food poisoning is very frequent, and companies invest a lot in processes to limit food safety hazards. There are known (e.g. Salmonella) and unknown (never anticipated before) hazards.
Kristen Beck: The microbiome will respond to its environment, much like a canary in a coal mine. In food samples, there can be microbes, which can be detected by sequencing.
Kristen Beck: We analyzed 312 terabytes of data, generating >270,000 data files, building up datasets from food microbiomes. We also build a bioinformatics tool that could work for a wide range of people, including those who cannot run from the command line.
Kristen Beck: Kiwi, an advanced prototype of food microbiome analysis – an easy to use interface for analyzing food microbiomes.
Kristen Beck *shows some Kiwi dashboard screenshots *. On the left, you see a good sample, with no microbial hazards detected (in blue and green) – on the right, hazards detected (in red and orange). These are easy to interpret.
Kristen Beck: For the more advanced users, we offer advanced analysis tools of these samples.
Kristen Beck: We work with deep sequencing of the metatranscriptome (mRNA), around 350 million reads per sample. Samples all retrieved from same factory and sample type.
Kristen Beck: With food, it is not always clear what the matrix (host) is: is it chicken or some other type of meat? This provides unique computational challenges to the bioinformatics analysis.
Kristen Beck: We search for a reference set of 6,000 plant and animals genomes, then use Kraken to filter out these matrix reads. This filtering should only remove the matrix, not the microbial reads.
Kristen Beck: We can also use this process to see the composition of host DNA in a sausage sample (55% sheep, 35% cattle, 7.5% pig and 1% horse!).
Kristen Beck: In screening chicken samples, we found most of them to be >99% chicken, but some had pig and cow DNA in them.
Kristen Beck: The microbiome analysis is then done on the non-matrix nucleic acids. We show the high abundance microbes to signal if something is wrong.
Kristen Beck: Unclassified reads are a missed opportunity, caused by lack of homology. Reference databases are biased towards cultured organisms. We are building better reference sets that include many more genomes to better capture the diversity of strains.
Kristen Beck: Introducing OMXWare: We @IBMResearch assembled 166,000 high quality genomes, annotated genes, proteins, and domains. It has lots of data in a structured and optimized DB2 database.
Kristen Beck: OMXWare can be used e.g. to extract novel CRISPR/Cas sequences and to find higher % of functional annotations.
The next speaker is Cindy Davis, Office of Dietary Supplements, National Institutes of Health @NIH with “Diet, microbiome and health: the influence of diet on the intestinal microbiome”
Cindy Davis: Diet shapes the microbiome in humans: globally distinct populations, food pattern consumption vs enterotypes. High fat vs high fiber diets are associated with different microbiomes.
Cindy Davis: Prevotella is associated with diets with lots of fibers, while Bacteroides is more prevalent in Western diets with more animal fats. Diet can influence the microbiome composition, even in short term experiments.
Cindy Davis: Shows data from the Daphna Rothschild Nature paper.
Environment dominates over host genetics in shaping human gut microbiota
Cindy Davis: Shows WHO definition of probiotics, which now form a >2 billion dollar sales market in the US. They are the 3rd most common dietary supplement. But what do they do on humans?
Cindy Davis: Stool samples do not reflect the whole microbiome in the gut. Some people’s microbiota resists colonization with probiotics.
Shows data from Zmora et al.
Cindy Davis: There have been several studies using the effect of probiotics on BMI. Most of these are short duration, small sample size (underpowered), used variable strains, not preregistered, and not clinically significant.
They don’t prove anything.
Cindy Davis: So where is the science? The FDA has not approved any probiotics for preventing any health problem. There is some preliminary evidence for some effect of probiotics in certain situations.
Cindy Davis: Dietary fiber is associated with a decreased risk of colon cancer. They are fermented in the colon into short chain fatty acids (SCFA), such as butyrate.
Cindy Davis: Butyrate can affect proliferation of cancer cells (which prefer glucose) and increase apoptosis, thus plays role in cancer prevention.
Cindy Davis: Dietary fiber can protect the mucus barrier. In the absence of fiber, the gut bacteria start to eat the mucus layer.
* shows data from Desai et al. 2016
Cindy Davis: Dietary allicin (in garlic) reduces metabolism of L-carnitine into TMAO. See Wu 2015 (link: https://onlinelibrary.wiley.com/doi/full/10.1002/fsn3.199)
We need to think about food interactions!
Cindy Davis: *shows tiny graphs from many more papers by other groups*
Cindy Davis: There is a dynamic relation between microbes, food components, and microbial metabolites. Can your microbiome tell you what to eat?
Zeevi et al. : high interpersonal variability in glucose response in 800 person cohort.
(link: https://www.cell.com/cell/fulltext/S0092-8674(15)01481-6) cell.com/cell/fulltext/
Cindy Davis: When you are going to have lunch now, remember that you are also feeding your microbes. And they might want something different to eat than you. 🙂
This talk gave a good overview of the current state of microbiome research in the setting of diet, but did not present anything new. Would have been a better talk as a keynote/opening lecture, not in a specialized conference track.
We start after lunch with a talk by Scott Jackson from the National Institute of Standards and Technology @usnistgov, with “Standards for Microbiome and Metagenomic Measurements”
Scott Jackson: We are a non-regulatory agency, but often work together with the FDA to develop standards that can be used by industry.
Scott Jackson: The microbiome industry is rapidly growing, and standards are needed for diagnostics and methods. Many biases in metagenomic measurements, e.g. DNA extractions, primer choice, library prep, sequencing technologies, and bioinformatics analyses.
Scott Jackson: 5 years ago, it was “cowboy country” – different methods will give different microbiome community outcomes. We make references and standards to try to guide this.
Scott Jackson: The Mosaic Standards Challenge was launched in May 2018, made possible by Janssen, Biocollective, DNAnexus, and NIST. You can sign up for free. You process samples through your favorite method and upload your data.
(link: http://mosaicbiome.com/) MOSAIC: A Cloud-Based Microbiome Informatics Platform
Scott Jackson: Here are some results from submitted datasets on a set of 10 bacterial standards. Results vary widely between labs, and we are looking at which factors make the most impact.
Scott Jackson: High number of reads often corresponded to more false positives (detection of strains that were not in the tube we sent).
Scott Jackson: None of the companies that do microbial diagnostics do have FDA approval, but the FDA calls on NIST for microbial standards. We have made a mix of 19 different pathogens in human DNA, which can be used to validate tests.
Scott Jackson: We made different serial dilution mixtures and tested expected vs observed abundance using 3 metagenomics analysis tools to infer specificity and sensitivity.
Scott Jackson: International microbiome and metagenomics standards alliance IMMSA provides lots of tools, workshops, videos. We also have a pathogens workshop group. Our next NIST/FDA/NIH workshop is Sept 9-10 in Gaithersburg, MD.
(link: https://microbialstandards.org/) microbialstandards.org
Moved to parallel session 1, where I caught the summary slide of Peter Karp @SRI_Intl talk, in which he presented:
* Multi organism Metabolic Route Search
* BioCyc Databases Combine
* Pathway Tools software
The next talk will be by Curtis Huttenhower @chuttenh from Harvard University @harvard, with “Integrating molecule measurements of the microbiome for translation in population health”
Curtis Huttenhower: Our group works on methods development to analyze the microbiome on different levels, including very large-scale studies.
Curtis Huttenhower: BIOM-Mass: Biobank for Microbiome Research in Massachusetts. Developing a room temp-ship-able kit for stool and oral samples suitable for metagenomics, metatranscriptomics, metabolomics, and culture.
Curtis Huttenhower: Phase 2 of the Human Microbiome Project, HMP2, or integrative HMP (iHMP). Our lab is studying the microbiome of inflammatory bowel disease (IBD) patients over time, including CD, UC. Roughly every 2 weeks + blood draws and biopsies.
Curtis Huttenhower: This allows us to connect host response to microbiome profiles, at roughly the same timepoints. Database and raw data available here: (link: https://www.ibdmdb.org/) ibdmdb.org
Curtis Huttenhower: We can see which microbial metabolites (as measured biochemically) are enriched or depleted in IBD patients vs healthy controls and connect these to variations in microbial taxa.
Curtis Huttenhower: We can also link microbial function to strain specific phenotypes in IBD. *shows heatmap of Ruminococcus gnavus genes and their abundance in patients/controls*. Most gut microbial genes are unfortunately still of unknown function.
Curtis Huttenhower: Type 1 diabetes infant cohorts in Finland, Estonia, Russia / TEDDY study: functional shifts in genes at birth, at year 1, at year 2. Some are linked to specific functions, e.g. HMO utilization in infants in Bifidobacterium longum strains.
Curtis Huttenhower: Nicola Segata: assembled 150,000 genomes from 10,000 metagenomies, 5,000 species-level genomic bins (SGBs). Many of these genome bins are novel, many from uncharacterized, international populations.
Curtis Huttenhower: Even among well characterized clades, such as Bacteroides, we found genomes that were separate from characterized strains, so novel taxa without reference genomes.
Curtis Huttenhower: ASCA and ANCA antibody levels in blood corresponded significantly with microbial dysbiosis. We also found that each patient, and even some of the controls have a very dynamic microbiome, with different stable states, and periodic blooms.
Curtis Huttenhower: Our computational tools are available from the bioBakery website. (link: https://bitbucket.org/biobakery/biobakery/wiki/Home) bitbucket.org/biobakery/biob…
We also teach courses – and we are hiring computational and wet-lab postdocs, so please come and join our lab.
The next speaker is David Zeevi, @DaveZeevi from Rockefeller University @RockefellerUniv with his talk entitled “Sub-genomic variation in the gut microbiome associates with host metabolic health”.
David Zeevi: What are we looking for in microbiome data? Observation? Association? Mechanism? We are moving from the first towards the last, but each step brings bigger challenges.
David Zeevi: Metabolic diseases are on the rise, in particular obesity. We collected blood glucose data and correlated that with microbiome and dietary data. Each person responds differently. See: (link: https://www.cell.com/cell/fulltext/S0092-8674(15)01481-6) cell.com/cell/fulltext/…
David Zeevi: Even small differences, in a few microbial genes, can have a significant phenotypic effect. Think about the presence/absence of a toxin gene. What are the variable regions in microbiome bacteria?
David Zeevi: About 20% of metagenomics read gets assigned to more than one reference microbial genome. Errors caused by differential coverage in reference genomes.
David Zeevi: We have a paper in press for an iterative coverage based algorithm for read-assignment correction. This creates more accurate assignments.
David Zeevi: Sub-genomic variability (SGVs). These regions are very abundant across different datasets (Elies: not sure how they are defined)
David Zeevi: SGVs are person-specific and are shared with habitat. SGVs also correlate with disease risk factors. E.g. Anaerostipes hadrus region: if present, people are leaner and more healthy. It encodes butyrate production and inositol degradation.
David Zeevi: Thus, variable gene clusters facilitate mechanistic insights. This would be not be discovered if you just look at presence of specific strains or species.
After the break, we will have Kara Bortone @kara_bortone, head at jLABS @JLABS, Johnson & Johnson @JNJInnovation with “Catalyzing and supporting the translation of preventative research in the microbiome space”
Kara Bortone: Clinical research is still reactive: we wait until a person has a disease and then try to cure them. Can we study underlying mechanisms and better predict and prevent?
Kara Bortone: Future models of care: a holistic approach to eliminate diseases. Prevent, intercept, and cure disease. We are shifting our effort to a much earlier in the process, before the onset of observable symptoms.
Kara Bortone: The microbiome is a promising target for drug development. @JNJInnovation
has a recognized leading partnership role in the microbiome space. @JLABS
is an incubator space to grow and foster startup companies.
Kara Bortone: Jlabs are life-science incubators. There are now13 JLabs sites all across the globe, 480 portfolio companies in sectors ranging from health tech, medical devices, and pharmaceutical.
Kara Bortone: Small companies get big company benefits at JLabs. It is hard to buy equipment for early stage companies. So we have shared equipment and shared space, with support staff, education, and connections to VCs and other investors.
Kara Bortone: We organize events with investors and foundations to facilitate connections and QuickFire Challenges, with currently over 60 winning companies. And @AstarteMedical is one of them, yay!
Kara Bortone: Of the companies we hosted, 88% are still in business or acquired (that is more than the average restaurant in DC!). 12 companies have gone public, 12 were acquired.
That completes my tweeting from the Kisaco Microbiome Congress in Washington DC. I hope you all enjoyed this report!