In this episode, we will be discussing what impactful research means and how you can increase the impact of your own research beyond the publication impact factor. As our guests share their insights, they emphasise the importance of multidisciplinary collaboration, staying connected with policymakers and patient advocates, and the need for effective communication of research to both the scientific community and the general public.
Listen here, scroll down for the transcript and subscribe now via Spotify, Apple Podcasts, Google Podcasts or Amazon Music/Audible so you’ll never miss an episode. You can find the other podcasts and their transcripts here.
Our guests in this episode:
- Elisabete Weiderpass, Director of the International Agency for Research on Cancer, a part of the World Health Organisation. Her research considers the epidemiology and prevention of cancer.
- Javier Carmona, Scientific Strategy Officer at Vall d’Hebron Institute of Oncology, an expert in scientific strategy and management.
- Caroline Dive, Director of the Cancer Research UK Manchester Institute. She is internationally renowned for advancing circulating biomarker research in lung cancer.
And our host: Alexandra Boitor, EACR Scientific Officer.
Episode transcript
Alexandra: Caroline will be joining us slightly later, but until then, let’s get the conversation started. On the third episode of The Cancer Researcher Podcast, “There’s Always More Than Just One Way”, one of our guests, Dr. Elaine Mardis, shared her worries regarding what she calls research in a vacuum: “I always fundamentally worry about what I call research in a vacuum. If you’re pursuing something, whether it’s a computational question or an experimental question, if you don’t take into consideration the reality of that disease or that question in terms of the patients that are involved, and the consequences of their disease, diagnosis, et cetera.”
So today I was hoping to discuss with you how can researchers avoid performing research in a vacuum, if I may borrow the term, and how can they ensure their research is impactful? And I think a good place to start would be defining what impactful research means and why it matters.
Elisabete: This is such an intriguing question. For me, impactful research means research having the greatest public health impact and research that matters most for the beneficiaries. That means individual human beings. So cancer research that matters – that’s what we should be focusing on.
What that means is research that ultimately makes a difference in people’s lives. For example, at the International Agency for Research on Cancer, where I have the privilege to work, the human beings are at the centre of each and every research question we have. We have a pathway from science to policy to impacting people’s lives. Impactful research will be indeed further strengthened by building a better bridge between research and public health priorities related to public health. But I’m sure that Javier has also his views about that.
Javier: Yeah, there are many dimensions for considering impactful research. I think in general, and I completely agree with what Elisabete mentioned, I think it’s to generate new knowledge that advances the field and has far-reaching implications with long-term impact. In terms of having a beneficial effect for patients, which ultimately is what guides cancer research, and that can have different perspectives. And both are equally important from a basic perspective, but also from a clinical perspective. Meaning that you cannot tackle in the clinic what you do not understand at the basic level. And it’s like books of a library that you need to learn from and to build new stories, and to benefit the community beyond the publication metrics. I guess that would be a definition of impactful research in cancer.
Alexandra: This actually brings me to my next question. How is the impact of one’s research being measured? Because traditionally, assessment of impact has focused on academic impact, and many times institutions incentivise researchers to prioritise publication over impact. I was wondering what are the indicators of change outside universities and research institutions? How is impact measured?
Elisabete: Yet another challenging question. So as I mentioned before, we have defined this impact pathway to contribute to a world where fewer people develop cancer, and to reach that we work with several lines of research; etiology, descriptive epidemiology and also implementation research. That means putting on action what we learn from research for public health.
So to monitor that, we developed a monitoring system which is inspired by the theory of change. And this model considers how short-term changes can lead to long-term public health impacts in cancer prevention. It implies that we need to take into consideration health behaviours and assess challenges as well as screening programmes. This monitoring system relies on four categories or key performance indicators: inputs, outputs, outcomes, and impact. This is also known as the IOOI model. It provides a quantitative indicators as well as qualitative indicators. And the impact indicators focus mainly on four dimensions for us: implementation and collaboration with the World Health Organisation for public health action, prevention policies, clinical practices, and economic and societal impacts. The whole objective is to really demonstrate concretely what is the outcome of the research on the ground. Over to Javier, to see how he does it in his institution.
Javier: Thank you, Elisabete. I think measuring impact has been traditionally considered within the journal impact factor or journal metric, which is a very reductionist measure of research impact. I think there are many more dimensions that can be explored. And to complement what Elisabete has mentioned, I think the impact of research beyond the immediate publication can go in terms of benefit to the community, in terms of the resources that research generates, the technology, the experimental systems that enables research from other labs around the world, but also the commercialisation potential, for example, because sometimes that is the way for a research discovery to impact and reach its full potential within entrepreneurial ecosystems or the clinical impact to influence patient management improve patient outcomes, clinical guidelines, influence policy to inform regulators at the level of resource allocation, legislation, or to provide evidence to support public health campaigns that can be scalable to improve the outcomes and the wellbeing of the population.
And one thing I also think is very important is the social impact, which is the ultimate goal of research to focus on homogenising the benefit of scientific productivity so that it doesn’t leave behind anyone and underserved populations are equally served and that research also produces some community engagement to reach its maximum potential.
Alexandra: I read an article earlier this year about the reform of European research that was initiated by the EU. And I quote from that article, their aim was “to make the assessment fairer and more inclusive by reducing reliance on unsuitable use of metrics such as journal impact factors and institutional rankings”, which I think is what you said as well, Javier?
Javier: Yeah, that is also very related to the San Francisco DORA plan which aims also to expand the way research is assessed, and to consider impact of research just beyond the immediate journal metrics, which has been the traditional way of assessing research. And that is very limited because research performed in the lab or in a hospital or epidemiological research has many more implications that go beyond what a number says or the reputation of a journal. And those, I think, need to be considered to fully assess the impact of research.
Elisabete: I agree. But also, this brings us, I think, to the discussion of how do you show potential impact of your research into a grant application? So I think this is really such a fundamental question for all of us. And I think the first thing is to start always your research grant application by highlighting the research gap. And then to start identifying very clearly which hypothesis and which are the anticipated findings that you expect. And these will indirectly address the question of the overall impact of your research. And then, of course, explain the importance of the question or a critical barrier to progress that your research we will address. Also after that, I think it’s important to actively engage with the relevant beneficiaries of the research and the stakeholders to get their perspective and include that in the applications or show that it really goes beyond the scientific community only. And also finally, identify and indicate the potential economical and societal impact of the research.
Javier: Absolutely, I completely agree. Involving all relevant stakeholders is essential in a grant application to really showcase the full potential impact of your research. And there’s a lot of discussion nowadays about patient-centred and patient-driven research, involving patient representatives and patients themselves in the process of co-creation, co-design and evaluation of research. And I think that is a key component. And all of these also need to be within the context when you write or submit research applications. I think that clarity in the definition of objectives is essential. That the proposal is in clear alignment, as Elisabete was mentioning, with the scope of the call, with the unmet needs, and how these will be addressed by the proposed project.
And also feasibility is key, and pragmatism to address the research question, which needs to be supported by the track record of the researcher, all of the collaborations that are established to involve any relevant expertise that should be brought on board, also to advance in terms of innovation and to generate research that advances the field beyond the state. I think those are key elements that can reflect or maximise the innovative potential of a research application.
Alexandra: And realistically speaking, how much impact can research from one group be expected to have in the context we have to work with today? I mean, we have to admit that many of the ways in which research is undertaken, communicated and evaluated have put up barriers between the work itself and those who may benefit from it.
Elisabete: I think that progress is being made and I take one example for the Cancer Grand Challenges that CRUK, NCI, and Spain have launched. So these are large collaborative efforts that put together the best scientific communities to answer a specific research question. And the idea is that the best groups collaborate rather than compete. The grants are well supported and for a sufficient period of time.
Another example is the European Cancer Mission and the European Beating Cancer Plan, two major initiatives in Europe. Actually, we have never seen so much money in Europe to do cancer research, which is absolutely fabulous. And, these two initiatives as well, their aim is to pull together the communities across Europe to answer very specific problems on the ground.
Alexandra: Hi Caroline, and thank you for joining. The conversation is now up to the point where we were discussing how much impact can one research group be expected to actually bring to the overall picture in the context where there are different barriers in communicating research that’s undertaken, and then those that can benefit from it. And Elisabete was explaining what initiatives have been already started to try and address this.
Javier: Yeah, she was mentioning the initiatives from the Cancer Grand Challenge UK, and now she was talking about the European Cancer Mission and the Europe’s Beating Cancer Plan, how research has gone much more global than it traditionally was, and I completely agree. I think research nowadays has become more of a collective enterprise, and for it to have a broader impact it is important that it brings on board different stakeholders, not only in terms of expertise, but as Alexandra was mentioning, in terms of who is the impact intended for, and for that involving patient associations is key. If it’s a project aimed for a practical application, it is important sometimes, I think, to bring on board people from the private sector so that it can be explored in terms of commercialisation and transfer of technology and knowledge to the entrepreneurship.
And I think that all the initiatives that Elisabete was mentioning, the Cancer Grand Challenges and the European Cancer Mission, I think are prime examples of these. Of how collective intellect and creativity can be brought on board to collaborate and to push the boundaries of science.
Elisabete: In fact, I think this sort of idea to be more problem-driven is also being emulated by other countries. I saw yesterday that Canada has similar initiatives now. So it seems that’s a trend. To have calls which are more targeted on problems rather than in more piecemeal research questions.
Caroline: I was just involved recently in a bid for EPSRC (Engineering and Physical Sciences Research Council) here in the UK, which was around pre-invasive disease, which is a really tough challenge. And that was bringing together engineering, radiotherapy, surgery, biology, and those interfaces between the different disciplines. I think Elisabete is right. When you have a particular problem to solve, if you can bring the disciplines together around a particular consortium bid, that is a good way to really understand other people’s worlds because it’s communication that actually makes these things work. So making sure that you can have a conversation with your artificial intelligence specialist, for example, at the same time as talking to your surgeon. It’s these bridging pieces, which I think are the most difficult, but also the most rewarding.
But then wrapped up in that is that we have to think about who the next generation of cancer researchers are going to be. And how many languages. Not Spanish and English, but how many “languages” the new generation of cancer researcher have to speak? So I think there’s a really important piece too, when we think about impact, on how we train the next generation of cancer researchers, because they are going to have to be “multilingual” in the interdisciplinary sense, I think. And I think that’s a really interesting challenge for us all.
Javier: Absolutely.
Elisabete: Yeah, in a way, I think we are now in research where we were in the pedagogies of teaching medicine 30 years ago, when there was a swap between the traditional way, let’s learn pathology, anatomy, physiology, etc. To problem-based learning, where from the very start of medical faculty, you give a problem to the students, a patient comes with jaundice and pains here and there since one month, and then you figure out what this is and what to do. And the students need to deal with the patient from the start. So I think we are doing now more or less the same paradigm shift in science that we are much more focusing on the issues to be sorted out which necessarily are multidisciplinary. And I think that’s a good thing to some extent. So we can see until where we can push that sort of approach. But I think we are starting to do it and I think it’s rather promising.
Caroline: If you’re not careful and you push it too hard, you end up in English, the phrase is a Jack of all trades and a master of none. So we have this balancing act to do to train specialists who are deep and really specialised in their own area, but at the same time be multilingual to speak to someone who’s deep and specialised in another one. And if we try and generate the next generation who can speak a little bit of all the languages, but not fully comprehend any single one of them, then I’m not sure that’s going to solve the problem either. So it’s this balancing act in terms of impact.
If you’re an individual and you’re a young scientist of whichever disciplinary nature, how do I make an impact? I still think the answer is you become expert in your own field. But you also learn some of the language so that you can talk to others. And I think that’s the next generational challenge of being a scientist, because we can’t just have the superficial kind of understanding all of it, but not really understanding any of it.
Elisabete: I agree.
Alexandra: And in terms of some practical advice, how could researchers increase the impact potential of their own research in the context we have now? What can they do today to try and increase the impact of their research?
Elisabete: I think Caroline answered that to some extent because she put it in words so clearly as be the best you, the absolutely best you can in your limited field, but keep an eye and ears open to the big questions around and talk to people. So I think this easy to say, difficult to do combination is where young scientists need to grow.
Caroline: Yeah, I always say it’s the Sunday afternoon reading. So they don’t always like me to say it but I actually think when I was young enough, a long time ago, it was reading a particular article in Nature on a Sunday afternoon that ignited the spark in my head, and it wasn’t on the subject I was studying already, and that’s why some of these broad science journals are the best ones for Sunday afternoons. I think for me, I stick with that advice whenever I talk to young scientists. Take an hour on a Sunday afternoon and read something you don’t know anything about, and I think that’s one of the things you can do.
I think the other thing that you can do, what we do here in Manchester is we try to have a very mixed community in our laboratories and in our clinics. So, we are trying to set up some of the basic science PhD students get to go to clinic. I mean, there’s nothing you can do in clinic apart from make the tea, obviously, but you get to witness. And then we also have an MB PhD program now where we try and get medical students into labs early on.
So I think it’s taking those opportunities when you’re given them to see beyond your own narrow focus of research, not just to do reading on a Sunday afternoon, but also to take the opportunities to go and see how other disciplines function and work. It may only be you need to do it for one hour a year or something, but take the opportunities that are there to do that.
At the EACR, we’re really keen on our Travel Fellowships, right? Going to a country just for a secondment to learn a new technique, just take the opportunities. I think it all comes down to organisation of time. It comes down to your passion. It comes down to making sure you’re putting enough effort into the paper that you know you need to get published by the end of your postdoc or the end of your PhD. But also finding the time and making the time to do some of these extra things that just open your eyes a little bit wider to what’s going on around you in your country, in your city, in your university, or across in a different country. A lot of it comes from within. I think there’s some of this you can’t be taught. I think if you have a passion for science these days, you’ll want to go and do these things. If it’s a chore, maybe you’re not in the right place.
Javier: I completely agree. It needs to come from you, the will to get exposed to different areas, to get out of your comfort zone and be willing to learn from other disciplines, other people, other ways of working, and even if you get a chance to visit other research centres for a short period. That’s super enriching and something that is not in the books or on a specific curriculum. You need to go and look for it and be keen to learn.
Caroline: I do think that COVID was terrible for so many reasons, but one of them, I think, is people forgot how to put their hand up and ask a question. You know, young scientists forgot a little bit, not all obviously, but I think there was a trend. I haven’t done the statistics, so don’t shoot me down too much, but I think everything was on virtual, everything was in the chat, and I think it does take a little bit of courage just to put up your hand and ask an eminent speaker a question in a seminar. And I think people have begun to lose how to do that. And if you want to get noticed, that’s probably one of the best ways as a young scientist. You go to a conference, you get an EACR Travel Grant, you go to a conference, you sit in the front row, you put up your hand, you ask a question, and you go and talk to the speaker at the end of their seminar or their lecture. You need to get noticed.
Alexandra: You started talking about seminars and conferences, and I was wondering whether in this context, if you want to make your research available to as many as possible, is there a benefit or a need in trying to communicate and keep in touch with policymakers and patient advocates? As either a young scientist or a more senior scientist.
Javier: I was thinking, being in a clinical or translational research institute as VHIO, I think we always have in mind the perspective of the patient. What does the patient need? And very often, engaging in a conversation with patient representatives, which we do quite often here, is helpful as it’s super useful not to lose perspective of what we’re doing, why we’re doing it and what are the needs that must be addressed. I think that it’s very important to keep those connections and keep engaging with the final end users or the beneficiaries of the research that we’re doing.
Elisabete: Yeah, I agree. But I think, in particular for the young scientists, it is also important to keep in mind that knowledge is created by repeated evidence, right? So one study never defines the field. So it’s the accumulation of knowledge from different perspectives, different studies that lead to a policy. If we think about public health policies, it’s not only one study, it’s not only two studies, it is a multitude of studies put together, indicating a consistent pattern there. That’s when you start being ready to put that all in a meta-analysis or a pooled analysis and, based on the results of that, eventually inform policymakers. We should never inform policymakers or anybody else, patient organisations or others based on one study only, and think that one study answers any clinical question or other questions.
I think also the important thing for everyone, including early career scientists, is always to communicate, and communicate does not mean telling people what to do, but just be able in simple words to explain what my research is about and what are the results of my research indicating. So that skill I think is quite fundamental to be learned from the beginning of the scientific career already.
Caroline: For me, watching over the years, when I first started out as a basic scientist, I don’t think anyone had really ever used the term “patient advocate” in my presence. I don’t think I ever knew what it was. I don’t think I was ever exposed to it. That’s very different these days, so particularly if you’re doing translational science, all the grant programmes and funding bodies have a section that at first everyone goes, what should I put in there? But what’s happened in recent years is we’ve already learned the value of patient advocates and patient and public engagement.
And so I agree with Elisabete. I think getting young folks to learn how to talk to the patients and public about their science, even if they’re doing the most basic science, which I think of course is fundamental to what eventually comes through for patients. So, I think we have learned in the past years to pay more attention to talking about our science to the lay public and then we could get into the realm of science journalism, and how important it is to get that right. And then I think the other thing that leads us on to is the different types of careers that young scientists can have. And clearly, influencing policy. That’s a very hard question to answer, but I think working through communication of science, so that policy can be influenced properly, I think there’s all sorts of levels that you can go through as you grow as a scientist. As Elisabete said, I think that starts right at the beginning with learning how to talk about your science to Aunty Betty, who’s never done any science at school. I think those are really good skills and we can all practice those because we’ve all got an Aunty Betty that we could tell our science to.
So, again, late in my career, how important the patient and public involvement in design of a translational study can be… You can go down such a rabbit hole and do exactly the wrong kind of design if you ignore the advice of the patients and the public. So, all of this I think is entwined. It’s not answering your question directly, but it is, I think, important as you go through your career to learn some of these additional skills. Lay people, patients and public, how important they are, even if you’re doing the most basic science, actually.
Alexandra: Thank you all for your answers. This has been a very engaging conversation. And especially on, the last question, I’m sure we could elaborate a lot longer. That’s probably a conversation that would take a few hours. And as Caroline mentioned, there’s no concrete answer to it. I just wanted to leave our listeners with some food for thought
for the rest of the day, so that’s why I wanted to end this episode on that note. So everyone can think about how much of patient advocacy or public engagement they want to incorporate into their own research. So thank you again for making the time to join me today for this conversation.
Enjoyed it? Subscribe now via Spotify, Apple Podcasts, Google Podcasts or Amazon Music/Audible so you’ll never miss an episode of The Cancer Researcher Podcast.
