Boresha Afya Developmental Evaluation
Director, CIRCLE Project
What is real-time learning and how can we make it happen?
Real-time learning, iterative programming, and closing the “know-do” gap are all terms that have been used over the years to describe the concept of collaborating, learning, and adapting (CLA), and all occur as programs are implemented. The key question is how do we make it happen? Three strategies from the Developmental Evaluation, implemented by the CIRCLE Project in Tanzania, appear to be instrumental in the approach’s success.
In 2017, USAID/Tanzania wanted to better understand how its flagship health services project, Boresha Afya, was helping the Government of Tanzania integrate health services. They commissioned CIRCLE to carry out a Developmental Evaluation focused on health service integration, over the life of the Boresha Afya Project (through 2021). A Developmental Evaluation is a collaborative approach to continually study efforts made by program implementers and to support real-time adaptation among stakeholders. In the case of Boresha Afya, the goal is to improve access to integrated health services like HIV care, family planning, and maternal health services, in the same visit. The stakeholders include staff from the President’s Office – Regional Administration and Local Government (PO-RALG), which manages health services in the regions, USAID, and implementing partners supporting clinics in the regions of Dodoma, Iringa, and Mara.
This process has facilitated changes in Boresha Afya’s work plans and helped the program reallocate resources to areas that need attention. On a facility level, health managers are adjusting and adapting their program priorities. The next step of this process will include sharing learnings with national government stakeholders, so that these better implementation processes can be scaled to other regions, and nationally. So, how is the developmental evaluation helping to facilitate real-time adaptation? The key components are embedded evaluators, ongoing stakeholder engagement, and tracking tools that document change uptake.
Developmental evaluators sit in the regional offices of three implementing partners – they visit facilities with them, attend their meetings and share their ongoing indicator measurement with them, and discuss challenges in implementation. They also participate in monthly meetings with local health management team stakeholders to identify new issues and solve problems quickly. The value of this approach is that since they are not official ‘implementing partners’, they are in a position to provide ongoing, impartial feedback to the group.
Regular stakeholder meetings
In addition to monthly regional meetings with regional health teams, stakeholder meetings are held twice a year with implementing partners, the PO-RALG, the Ministry of Health, and USAID to discuss how services can be better delivered in different service settings. This is not just a session to point out what is wrong. Using an enquiry framework guides discussions so that the group can work through gaps and issues with integrated service delivery, and prioritize actions to address them. This process helps the stakeholders identify the needs for improvement within the project scope. The national meeting also allows them to learn what is and is not working in other regions.
After stakeholders identify and agree on priority actions, developmental evaluators continue to track how actions are taken up by the implementing partners. A tracking dashboard shows these results and is sent to stakeholders quarterly. The dashboard has been an important component to engage stakeholders, who see which changes are taking place each quarter. The “Tracker” helps to mobilize them around issues that are not getting resolved, and to find other solutions.
So far, the Developmental Evaluation team has convened 25 regional meetings and four national meetings to discuss the ongoing observations and emerging information about integrated services in three regions. Based on the observations, Boresha Afya implementing partners have identified 161 actions and processes needed to improve service delivery, of which 72% have been completed. These actions are simple but impactful: ensuring that educational materials are available at the clinic, identifying how to adapt clinic registers to reflect integrated services, communicating to district and regional health teams, the lack of commodities, pre-recording eduation sessions, and using appointment times to maximize the clinic day. These are all proven best practices, but they were not happening. Using the processes described above to create “feedback loops”, the Developmental Evaluation helps health facilities quickly identify, adapt, and resolve problems, so that health services are optimized based on their actual operating context.
Learn more about CIRCLE’s Developmental Evaluation in Tanzania.