Update on Matters Related to Dr. Koren’s Research
In 2014, the Toronto Star began investigating the Motherisk Program at The Hospital for Sick Children (SickKids). The Star raised questions about the accuracy and reliability of hair analyses undertaken by staff working under the direction of Dr. Gideon Koren in the Motherisk Drug Testing Lab (MDTL). An internal Hospital review was launched along with an independent government-commissioned review by Justice Susan Lang. Subsequently, MDTL was closed. In her report, Justice Lang deemed MDTL hair testing to be “inadequate and unreliable” for use in criminal or child protection proceedings. She also identified five questionable studies, which prompted wider concerns about Dr. Koren’s research.
Drawing on Justice Lang’s findings, SickKids implemented various measures to increase oversight of the management of labs, staff interactions with the justice system and internal hospital conflict of interest disclosures. The Hospital reviewed all of Dr. Koren’s active research at the time and investigated research that was identified as concerning.
For a report on the actions SickKids has already taken to investigate concerns regarding Dr. Koren’s research please refer to this Summary.
Despite the actions taken to date, fresh concerns have surfaced in the area of scientific reliability and academic publication conflict of interest disclosures. Here SickKids wishes to acknowledge the investigative work of reporters Rachel Mendelson and Michele Henry of The Star who brought relevant findings to the organization’s attention. They have unearthed publications where, on initial review, it appears that Dr. Koren did not disclose industry support that appears relevant to the primary focus of the publication or otherwise related to the published work.
The Star has identified 422 research papers, where they point to, among other things: concerns with disclosure; uncertain scientific validity of hair testing research; publications in a particular journal in which the journal did not peer review the articles; and the potential for bias in publications relating to nausea and vomiting during pregnancy. We have addressed these issues in the Appendix (below).
Readers familiar with academic publishing will be aware that, in general, standards for disclosure have shifted higher over time. That is true at SickKids as well. However, our initial examination of these publications shows departures from what SickKids currently considers to be best practice.
It is also the case that, when it comes to financial disclosures in academic research publications, most institutions rely primarily on the good faith of scientists working in their employ and/or in their facilities. It is regrettable that the Hospital did not conduct any audits of Dr. Koren’s publications which may have identified disclosure issues sooner.
At this point, SickKids will be pursuing three courses of action.
- First, to protect the integrity of the existing medical literature, the Hospital will be undertaking a systematic examination of Dr. Koren’s published work to identify/confirm instances of incomplete disclosure in academic publications.
- Second, for the same reason and in light of Justice Lang’s findings, we will be undertaking a focused scientific review of subsets of Dr. Koren’s papers that we believe merit further attention.
- Third, to help ensure the integrity of all ongoing published work carrying the SickKids name, we are re-examining our policies, including those updated recently, and adding new measures to strengthen institutional oversight of publication disclosure practices.
These measures are outlined in more detail below.
Publication Disclosure Review
The Hospital recently wrote to Dr. Koren to request that he address issues of disclosure with journals where his work has appeared. We have not heard back from Dr Koren. Thus, we are reviewing the disclosures of funding sources on all of Dr. Koren’s publications since 2001. This will be a lengthy task involving hundreds of articles and commentaries and review of all institutional information related to funding he may have received for any reason from industry. We will be communicating with all involved journals to ensure that the journals have accurate information about Dr. Koren’s sources of industry funding for research as applicable, and to request that they register the relevant information in such a fashion that it is accessible through digital searches of the scientific record.
Focused Scientific Review
The Hospital has already communicated concerns about five of Dr. Koren’s hair analysis publications to the relevant journals. Details can be found here (summary). Dr. Koren contacted the journals at our request but we found his correction to be inadequate, and SickKids followed up to attempt to correct the record.
We will be initiating independent reviews of Dr. Koren’s primary research related to Diclectin and those papers in which hair analysis was the primary focus or analytical method used. We will also be arranging examination of a sample of his highest-cited papers, looking for evidence of inconsistencies and errors. We are aware that our inability to access raw data may limit these reviews, particularly for older papers.
As necessary, relevant findings will be communicated to the journals where the affected studies are published.
SickKids acknowledges that, apart from concerns already addressed in past investigations, it has not received other complaints about Dr. Koren’s work containing allegations of fraud, falsification, plagiarism, or duplicate publication – matters of misconduct in publication that commonly lead journals to post notices of concern or retraction. Furthermore, a large amount of Dr. Koren’s work was published with highly reputable and senior co-authors, and we in no way wish to impugn their work. Last, while it is evident that concerns have already been found with respect to his work, we are not pre-judging this matter, or casting any general aspersions on the extent to which his contributions to the field of maternal and child health should continue to be relied upon in clinical care.
Policy Review and Oversight of Publication Disclosures
Currently, our relationship management process requires scientists, clinicians and clinician-scientists to disclose any relationship that may be a perceived, potential or real conflict of interest. Annually, each faculty member is required to attest to a) the accuracy of information we have on file about ongoing consulting relationships with industry and those in the previous year, b) having made the requisite disclosures for each relationship, and c) their understanding of ongoing disclosure obligations. We are reviewing this system to determine what additional information should be routinely compiled.
The SickKids Foundation also has taken a number of steps to mitigate conflict of interest. This includes processes to manage how the Foundation works with donors that could have a real or perceived conflict of interest and the inclusion of a disclosure clause in all pharmaceutical gift agreements allowing the Hospital and the Foundation to publicly disclose the gift.
Moving forward, SickKids will be implementing a process whereby compliance with publication disclosure requirements is monitored by random audits of disclosures in publications. The focus of audits will be individuals who have research support or personal compensation from industry, pharmaceutical companies or private third-party funders. Disclosures in publications will be tested for accuracy against all available information systems. To the extent possible, the auditing process with be automated.
Last, we note that in 2017, the Government of Ontario enacted the Health Sector Payment Transparency Act, 2017 that would provide another publicly available source of information about industry payments in the health sector. The Act has not yet been proclaimed in force. The Hospital would welcome the government moving forward to enact this legislation.
For well over two decades, the health research community has grappled with the issue of disclosures of potential conflicts of interest, and potential biases in project design, execution, analysis, and reporting of studies arising when sponsors of the project or authors have a financial interest in the outcome. These issues are by no means unique to SickKids as a recent New York Times article indicates; and debate continues in major medical journals (see links below) as to how best to manage these challenges.
- Conflicts of Interests, Authors, and Journals: New Challenges for a Persistent Problem
- Should Failure to Disclose Significant Financial Conflicts of Interest Be Considered Research Misconduct?
As indicated above, until recently, the responsibility for dealing with disclosures in the context of publications has fallen primarily with authors and journals. We are proud of the work done by the scientists at SickKids and we are mindful that this situation is not representative of the practices of our scientists. However, as we examined other jurisdictions, it is clear that more organizational efforts are now being made to regularize and improve financial disclosures. SickKids has concluded that greater institutional facilitation and oversight of financial disclosures will be part of any sustainable solution to these ongoing challenges, and essential to ensure that members of the scientific community have the information necessary to evaluate the work of their colleagues on its merits.
SickKids believes that the responsibility for the quality and integrity of academic publications is shared by authors, journals and the institutions where the scientists work. The steps outlined above signal the Hospital’s commitment to that shared endeavour.
We will be continuing to post updates intermittently as work proceeds along the lines set out above.
Appendix: Research Integrity Concerns Raised by The Toronto Star
Toronto Star Concern
Research Leadership: The article references, in some regard, “The Research Leadership for Better Pharmacotherapy during Pregnancy and Lactation”
This issue arises in approximately 271 of the 422 articles brought forward.
"The Research Leadership for Better Pharmacotherapy during Pregnancy and Lactation" is not an actual fund, but a term Dr. Koren used to describe unrestricted funding he had available at different points in time. He did not include the terminology in his internal hospital conflict of interest disclosures.
When the Hospital became aware of this practice it told Dr. Koren to stop using this term. It is the responsibility of the author to transparently disclose relationships (conflicts of interest) to journals. The Hospital’s expectation would have been that Dr. Koren corrected this issue back in 2014 when the Hospital told him he should not be using the term. Notwithstanding that the Hospital no longer has jurisdiction over Dr. Koren as he is no longer on staff at SickKids, the Hospital recently wrote to Dr. Koren to request that he address this issue with the journals. He has not replied. The Hospital is therefore reviewing all of Dr. Koren’s publications from 2001 forward to check disclosures and will communicate any relevant findings to the journals.
Canadian Family Physician: The article was published in Canadian Family Physician. Last year, the journal acknowledged that it didn’t subject Motherisk articles to a full, double-blind peer review process because of its longstanding relationship with Dr. Koren and Motherisk.
This issue arises in approximately 199 of the 422 articles brought forward.
Motherisk prepared a regular column for the Canadian Family Physician on topics related to pregnant and lactating women. Typically, these columns were not original research but summaries of the published literature on topics relevant to pregnant and lactating women. Canadian Family Physician has recently acknowledged that it published the Motherisk Update without subjecting it to double-blind peer review. Given the unique nature/format of the column, it may have been challenging to have the columns double-blind peer reviewed. Anyone qualified to conduct the review would have been aware that the material had been prepared by Dr. Koren and the Motherisk team.
As Canadian Family Physician has acknowledged, the responsibility for ensuring the quality of the studies it publishes rests primarily with the journal. The Commentary on this subject published in Canadian Family Physician by Nicholas Pimlott, Brent Kvern and Robert Woollard (Canadian Family Physician January 2017, 63 (1) 13-14) is a matter of public record. The Hospital would be pleased to assist Canadian Family Physician in reviewing a sample of these studies to determine whether they accurately reflected the literature available at the time of publication, and is prepared to do so independently as needed.
Hair testing: The article relates to drug and/or alcohol hair-testing research conducted in the former Motherisk Drug Testing Laboratory.
This issue arises in approximately 64 of the 422 articles brought forward.
In her review of the Motherisk Drug Testing Laboratory (MDTL), Justice Susan Lang concluded that the MDTL testing was “inadequate and unreliable” for use in child protection and criminal proceedings. MDTL is no longer in operation and, therefore, there is no risk of further testing being conducted by MDTL for use in child protection or criminal proceedings. The Hospital is in the process of identifying MDTL-related publications that could potentially have therapeutic or diagnostic implications to conduct a review. The journals that have published these studies share responsibility for addressing this issue and to the extent our work results in any findings, our plan is to disclose same to the journals.
Lang: The article was deemed problematic by Justice Lang in her report on Motherisk because it falsely claimed that results were confirmed with GC/MS (gold-standard) testing.
This issue arises in 5 of the 422 articles brought forward.
Justice Lang identified 5 publications that suggested GC/MS confirmation of results at a time when her review demonstrated the MDTL was not conducting GC/MS confirmation. Dr. Koren sent a corrigenda to each of the affected journals. The corrigenda was published in respect of 3 of 5 of the publications
The Hospital’s Research Integrity Adviser reviewed these 5 studies and concluded Dr. Koren failed to comply with the TriAgency Framework on the Responsible Conduct of Research in that he failed to: (a) keep complete and accurate records of data for 7 years after the 2009 publication and; (b) use a high level of rigour in relation to these 5 studies. Subsequent to this review, the Hospital advised Dr. Koren and the Secretariat for Responsible Conduct of Research of the findings from the Research Integrity inquiry. The Chief of Paediatrics wrote to the journals taking issue with Dr. Koren’s suggestion in the corrigenda that the fact that not all positive results had been GC/MS-confirmed had no impact on the results and that the papers’ conclusions remain the same. He indicated that in his opinion, the only way to confirm definitively that there was no impact on the study results would be to retest all of the samples. He advised that because this has not been done, it is the Hospital’s position that it is not possible to state with certainty that the study results were not impacted.
No further action proposed.
Diclectin/morning sickness: The article relates to morning sickness or Diclectin and does not disclose financial support from Duchesnay.
This issue arises in approximately 30 of the 422 articles brought forward.
The Hospital was unaware that Dr. Koren had published on morning sickness and/or Diclectin without disclosing his relationship with Duchesnay. The responsibility for disclosing relationships (conflicts of interest) in a publication rests with the author. Notwithstanding that the Hospital no longer has jurisdiction over Dr. Koren as he is no longer on staff at SickKids, the Hospital recently reached out to Dr. Koren as noted above to request that he address this issue with the journals. The Hospital is reviewing all of Dr. Koren’s publications from 2001 forward to check disclosures and will communicate any relevant findings to the journals. In addition, the Hospital will be undertaking a review of 7 of 34 of Dr. Koren’s publications using primary data to assess the effect of doxylamine-pyridoxine for nausea and vomiting in pregnancy.
Therapeutic Drug Monitoring: The article was deemed problematic in a review conducted by the academic journal Therapeutic Drug Monitoring.
This issue arises in approximately 6 of the 422 articles brought forward.
The Hospital is looking into these articles.
This issue arises in approximately 1 of the 422 articles brought forward.
This issue was addressed many years ago by the University of Toronto and the Hospital has no further comment.