IR Tip #6: Formulating IR questions
What questions should guide the IR?
Formulating the right research question is critical to making sure IR provides the learning you need to take action to address the identified implementation challenge. Because problems implementing health interventions often result from circumstances specific to a local context, engaging diverse stakeholders will help formulate the right question. This TIP provides guidance on how to formulate and prioritize a question for IR, supporting decision makers to find out how and why a health intervention is not achieving what it set out to do, and how to get better results.
For instance, in Malawi, policies leading to increased access to and utilization of health facilities for delivery did not produce equivalent gains in newborn survival. Later analyses concluded that facility quality fell substantially short of global standards of evidence-based care. Higher-quality facilities had substantially few neonatal deaths than other facilities.2,3
IR utilizes real-time data review and reflection to help decision makers to understand why interventions and investments are not achieving their goals in their unique context. Box 6.1 provides some potential questions IR could help to answer.
How to formulate IR questions
While more traditional research questions focus on the impact of an intervention – for instance, does presence of a skilled attendant at birth reduce maternal mortality? – IR questions focus on:
- How an intervention is being implemented (implementation strategies),
- How those implementation strategies affect service outcomes and implementation outcomes (i.e. coverage, feasibility, acceptability), and
- How contextual factors affect the intervention and its outcomes
IR questions can explore, describe, influence, explain or test (see Table 2 of Peters et al, 2013) how implementation is (or is not) working.1 Additionally, IR questions can try to address challenges related to program scale-up, sustainability, replication and robustness, program integration, equitability, and real-life effectiveness, including cost.
Once you have identified some preliminary IR questions, you can refine and finalize those questions with the help of an IR framework (see TIP #7) and input from partners/stakeholders. TIP #5 addresses stakeholder engagement more specifically, but the Robert Wood Johnson Foundation and the Canadian Institutes for Health Research have developed practical guidance on engaging stakeholders in developing research questions.
How to prioritize questions
Health interventions can produce multiple, simultaneous implementation problems and questions. Figure 6.1 illustrates three criteria to utilize when prioritizing IR questions and some questions that will help to establish which IR questions should be prioritized. Box 6.2 presents how Indonesia utilized a collaborative process to establish IR questions to support scale-up of a national insurance policy.TIP#6_Figure_Sept 14
- IR answers why or how an implementation is (or is not) working.
- Collaborating with diverse stakeholders helps to formulate the right IR questions.
- IR can address many challenges you may be facing. For example:
- How can coverage of your intervention be improved?
- How can multiple interventions be effectively packaged and delivered?
- How can the intervention be delivered more equitably?
- Give priority to questions that meet three criteria:
- the problem is urgent;
- the questions can be answered and recommendations can be put in practice;
- the research can provide results that can make the intervention more effective.
- A Practical Guide for Engaging Stakeholders in Developing Evaluation Questions (Robert Wood Johnson Foundation).
- A guide to researcher and knowledge-user collaboration in health research: Section 4 (Canadian Institutes of Health Research, 2009).
- Frame Your Question (University of Washington Resource Hub)
- Morshed, A., Baumann, A., & Proctor, E. D&I Aims Toolkit. St. Louis, MO: Washington University; 2016 November. Eight toolkits related to Dissemination and Implementation. The full set of toolkits is available at https://sites.wustl.edu/wudandi/
- Peters DH, Tran NT, Adam T. Implementation research in health: a practical guide, 2013. Geneva: World Health Organization.
- Godlonton S, Okeke EN. Does a ban on informal health providers save lives? Evidence from Malawi. J of Development Economics. 2016;118:112-32.
- Leslie HH, Fink Gunther, Nsona H, Kruk ME. Obstetric Facility Quality and Newborn Mortality in Malawi: A Cross-Sectional Study. PLoS Med. 2016 Oct; 13(10):e1002151.
- Eichler R, Gigli S, LeRoy L. Implementation Research to Strengthen Health Care Financing Reforms Toward Universal Health Coverage in Indonesia: A Mixed-Methods Approach to Real-World Monitoring. Global Health: Science and Practice. 2018 Vol 6(4): 747-753.
- TDR Implementation Research Toolkit. https://www.who.int/tdr/