Type to search

Quick Read Reading Time: 4 minutes

Using HIP Products to Inform and Strengthen FP Programs, Part 1

Finding “Unknown Unknowns”—Reflections from a Monitoring and Evaluation Officer


High Impact Practices in Family Planning (HIPs) are a set of evidence-based family planning practices vetted by experts against specific criteria and documented in an easy-to-use format. The Evaluation of High Impact Practices in Family Planning Products sought to understand whether and how HIP products were being used among health professionals at the country and global levels. Using key informant interviews (KIIs), a small study team found that various HIP products are used by family planning experts and professionals to inform policy, strategy, and practice.

Cover of Evaluation of High Impact Practices in Family Planning Products

Cover of Evaluation of High Impact Practices in Family Planning Products Summary Report

I recently came across the concept of “unknown unknowns.” While the term generally implies the risk factors that are unexpected and unconsidered in program management and strategic planning, it can broadly cover any critical or surprising things we were not aware of previously. When I thought about whether I have encountered such a situation in my work, I remembered the Evaluation of High Impact Practices in Family Planning Products. I led the project in my role as a monitoring and evaluation officer at the Johns Hopkins Center for Communication Programs in early 2021. 

High Impact Practices in Family Planning (HIPs) are a set of evidence-based family planning practices vetted by experts against specific criteria and documented in an easy-to-use format. The HIP Partnership, which includes co-sponsors, partners, a technical advisory group, a dissemination team, and technical experts, provides the framework for the development of HIP products. This development process is highly participatory to ensure that HIP products are a state-of-the-art synthesis of peer-reviewed, programmatic knowledge of what works in family planning. The evaluation study that I conducted sought to understand whether and how HIP products were being used among health professionals at the country and global levels. It also endeavored to understand how exposure to a HIP product affected knowledge, attitudes, and beliefs related to High Impact Practices in Family Planning. 

I, along with a small study team, used key informant interviews (KIIs) as the main data collection method. Tried and tested, KIIs can be powerful for KM practitioners in global health to uncover whether and how evidence-based knowledge leads to learning and action from people having in-depth knowledge and hands-on experience about a topic of interest.  

We interviewed 35 individuals from 16 countries and gained interesting insights by analyzing their responses. 

We found that various HIP products are used by family planning experts and professionals for decision-making purposes—to inform policy, strategy, and practice. Family planning technical advisors and network coordinators working at the global or regional level generally use HIP products to guide the development of training materials, presentations, and guidance documents.

However, one of the research questions proposed for the evaluation was, “If HIP products are not being used, why not?” We were initially worried about getting sufficient data for this question. We anticipated that most KII participants would be those who were already actively engaging with HIP products. Therefore, we made efforts to identify people who may not have had much experience with HIPs. We used probing questions to get insights into non-use cases.

Before conducting KIIs, I assumed that the products were not effectively reaching the priority audiences and, as a result, were not being used. This assumption was likely rooted in my experience working on global-level projects where there is sometimes a disconnect between the program implementer and the audiences we aim to reach. Therefore, I expected the data to support expanding promotion and dissemination efforts. However, some of those who identified as subject matter experts or technical advisors said they were not using the HIP products because they felt they were already familiar with the subject matter; the content may not address the specific context in which they are working.

Evidently, the suggestion to improve the scalability and contextual nature of HIP products had previously been raised among the family planning community. To address this need, in 2019–2020, the IBP Network hosted a contest. It called for field-based program implementation stories.

An image of the HIP and WHO Guidelines in Family Planning

IBP implementation stories

Each of the 15-winning story submissions includes:

This practice of documenting learnings is a prime example of knowledge sharing and contributes to the ongoing process to scale up and contextualize the HIPs across various settings.

Compared to the many concrete use examples of HIP products that we collected through the KIIs (as you can see in the interactive content above), it was instead a minor finding of non-usage among interview participants. Yet, I was quite fascinated by how the data pointed to a conclusion that I did not expect. Moving forward, I will keep the “unknown unknowns” concept in my checklist so  I don’t miss any critical elements throughout the research process, from the designing stage to the reporting stage. As a knowledge management worker, it is essential to have an open mind to things that “I did not know that I did not know.” 

Read Part 2 in this series to discover more about the findings of the study related to use cases for the HIPs and what the findings suggest for the future of dissemination and exchange—not just of HIP products but of knowledge globally.

Saori Ohkubo

Senior Program Officer, Johns Hopkins Center for Communication Programs

Saori Ohkubo is a senior program officer at the Johns Hopkins Center for Communication Programs. She has significant professional experience in program management, monitoring, evaluation and learning (MEL) and technical expertise in knowledge management (KM), social and behavior change communication (SBCC), and global health and development-related subjects. She has led and contributed to several KM monitoring and evaluation strategies, guides, and tools for the global health community. She holds an MA in International Development Studies from George Washington University, focusing on international education and policy analysis, and an MBA, focusing on marketing and leadership, from the Johns Hopkins Carey Business School.