The mission of the Rotman Institute is to bring philosophers into productive engagement with scientists, policy makers and the public. An important component of accomplishing this mission is building strong ties with our alumni and celebrating their accomplishments as they put their training into practice in academic, governmental and private spheres.
Spencer Hey (PhD ’11), a recent graduate from Western’s Rotman Institute, is spearheading cutting edge work on the ethics and epistemology of clinical trials. He is currently a post-doctoral fellow at McGill University, under the direction of Professor Jonathan Kimmelman. Professor Kimmelman recently gave a Rotman Lecture about clinical translation in medical research, which incorporated Dr. Hey’s work. We interviewed Dr. Hey about this research and his broader experience as a student at the Rotman Institute.
Andrew Peterson: Tell us about your current research at McGill’s Biomedical Ethics department.
Spencer Hey: I am involved with Jonathan Kimmelman’s STREAM Research Group at McGill, where I primarily work on two different grants. One grant is a CIHR-funded project investigating how well principle investigators (PIs) of clinical trials predict the outcomes of their studies. Many commentators have argued that PIs are systematically overconfident or biased in various ways that lead to poor study design and outcomes. To assess this, we designed and implemented software that sends out semi-automated queries to the PI of every newly registered clinical trial. We ask them for a handful of predictions at the outset of their trial. When their study is complete, we come back and ask them what happened. If we find that PIs are poor predictors, this would raise some interesting methodological and policy questions about how to intervene and improve the quality of risk/benefit judgment.
The other grant is funded by Genome Canada. Here we are studying the translational trajectory of biomarker diagnostics for personalized cancer medicines. The promise of personalized medicine (PM) hinges on the existence of validated diagnostic tests that determine if a PM drug will benefit a patient. Unfortunately, there is still a lot of uncertainty, skepticism, and criticism surrounding the evidence base and clinical value of these diagnostics.
For this project we are looking across populations of these biomarker studies with the aim of providing a more comprehensive picture of diagnostic development. You’ll recall Professor Kimmelman talked about an “intervention ensemble” and exploring an “ensemble space.” That’s really what we want to study here: How well explored is the ensemble space when a new PM drug is licensed? Do PM research programs provide the health-care system with sufficient information to use these diagnostics and drugs effectively?
AP: At the end of his talk, Professor Kimmelman highlighted an AERO graph you have developed. Could you briefly describe what the diagram represents and how it fits in Kimmelman’s research project?
SH: New medical interventions are put through a series of experiments over the course of their development. The AERO (Accumulated Evidence and Research Organization) graph represents each of these experiments as a node arranged by time, along the x-axis, and modality, along the y-axis. The nodes are then color-coded to reflect trial outcomes: green is a positive result, yellow inconclusive, and red a negative result. The edges (or arrows) between the nodes represent information flow or inferential relationships, usually based on citations in a studies final report.
As an example, this figure (above) is from a publication in Trials, which I co-authored with Charles Weijer, my PhD supervisor, and Chad Heilig of the CDC (who I’ll say much more about below). It gives you an idea of what an AERO diagram looks like. This particular diagram depicts our pilot case. We looked at the research program with the drug, moxifloxacin, as a potential addition to the standard anti-tuberculosis regimen. The additional blue-blue line nodes and blue edges represent next steps in the logical chain of inquiry.
In Jonathan’s talk (at 44:00), he referred to an AERO diagram for the cancer drug, sunitinib. That diagram depicted 55 clinical trials from 2006 to 2012, stratified by the modality of cancer indication (as opposed to the modality of clinical trial phase shown in the figure above). This graph was also color-coded to show either a positive, inconclusive, or negative outcome. The edges represented the explicit justification provided by each trial, based on the citations that appeared in the introduction section of the published report.
The sunitinib AERO graph illuminates a number of interesting issues. First, it tells the story of sunitnib’s development: Early success in a few indications followed by years of failure across many other indications. Second, it shows an asymmetry in citations between the positive and negative studies. The early positive sunitinib trials have a disproportionate influence on the rest of the program. Third, it shows how many trial reports seem to ignore the prior negative studies in the same indication and instead emphasize only positive trials from other indications. Each of these issues raise ethical concerns about efficiency and waste in clinical translation.
Moreover, all of these issues would be completely invisible if one analyzes the ethics and epistemology of research on a trial-by-trial basis. This is where Jonathan is truly pioneering bioethics: He is investigating the balance of risk and benefit across entire research programs, what he calls “research portfolios.” The AERO graph can contribute to this project by providing a principled framework for representing salient properties at the portfolio level.
AP: Some readers may not know that you have collaborated with researchers at the U.S. Centers for Disease Control and Prevention (CDC). Could you tell about this? Has this experience influenced your current work at McGill?
SH: I became involved at the CDC through Chad Heilig, a member of the Rotman Institute, and a long-time collaborator with Charles Weijer. Chad works in the tuberculosis (TB) division of the CDC. During a visit to the Rotman Institute, he described a problem of discordant findings across a series of TB trials. The CDC had produced two studies with the aforementioned anti-TB drug, moxifloxacin, both of which were negative. However, two non-CDC trials with moxifloxacin produced positive findings. Naturally, this created some controversy in the TB research community. Researchers questioned which results were more reliable and whether moxifloxacin should be investigated further.
Given that my thesis project focused on issues of underdetermination and experimental robustness, I was eager to learn more about this case. Chad generously invited me to Atlanta to attend some of the scientific meetings of their TB group. This first-hand experience of a research community negotiating these problems proved incredibly interesting and useful. It also sparked my interest in the kinds of portfolio-level research questions that Jonathan is investigating. Everyone agrees that the goal of clinical testing is to produce robust evidence of clinical effectiveness, but what is the best way to achieve this? How can we optimally coordinate activities across a clinical research program?
Thinking about these questions in the TB trial context ultimately inspired the AERO graphing method. The work I am now doing with Jonathan—that is, investigating the development of many different drugs and diagnostics—allows me to test, extend, and sharpen this method into something both philosophically rich and practically useful.
AP: Do you believe your training at the Rotman Institute has been beneficial to this research? How so?
SH: Absolutely, the Rotman Institute has been beneficial. In fact, I like to think of my thesis as a prototypical-Rotman Institute project, since it emerged from the co-supervision of Charles and Bob Batterman (formerly the Rotman Canada Research Chair in Philosophy of Science). Although Bob departed for Pittsburgh before I defended, he was the one who got me on to Bill Wimsatt’s book, Re-engineering Philosophy for Limited Beings. My thesis really took its shape from discussing that work with Charles and Bob, and working out ways to extend and apply Wimsatt’s ideas about heuristic methods and robustness to clinical trials. If not for their collaboration and support in the early days of the Rotman Institute, my project would never have been what it is. I would never have gone on to do the kind of work that I do now.
Even before “Engaging Science” was the slogan for the Rotman Institute, Charles and Bob encouraged that spirit of engagement. As a philosopher of science or bioethicist, you should be able to communicate with the scientists and physicians whose work you respect and whose domain you study. I’ve found this to be great advice, both intellectually and professionally—I have directly benefited from working with scientists. In my experience, it is only when you talk to scientists and dig through the scientific literature that you come to see how complicated the epistemology and methodology are. That complexity is precisely what makes the story so philosophically interesting.
AP: What advice or reflections do you have for current graduate students at the Rotman Institute, or prospective students interested in joining the Institute?
SH: Big caveat here: I do not have a tenure track position. So, of course, bear that in mind when considering everything I say next. Also the academic job market is as bad and you’ve heard. Prepare yourself for that disturbing, ego-crushing reality.
On a more positive note, I would say that whatever the future job market holds, the Rotman Institute is a great place for getting into new and exciting areas of research. For me, that was key: Finding a project and a field of research that excited me. The dissertation is a long process, but I can honestly say that I had fun writing and defending it. I feel privileged to now be a part of Jonathan’s team, working on these challenging and ambitious projects. I have Charles, Bob, Chad and the opportunities created by the Rotman Institute to thank for getting me here.
AP: Any final thoughts?
SH: A positive and supportive work environment is not always easy to come by. So, again, I feel quite fortunate that I got to experience that at the Rotman Institute. I also have a great respect for, and many fond memories of, my colleagues at the Institute. In particular, I would like to acknowledge James Overton, Valerie Racine, Ariella Binik, Ryan Muldoon, and Andrew McRae. They all provided valuable feedback and support during my time at Western. I learned a lot from reading their work and listening to their scholarly presentations.
Finally, I’d like to thank the current students and staff at Rotman Institute for this interview. It is a wonderful opportunity to share a bit about my work and express my appreciation for those peers and mentors that shaped my thinking and career.
