Impact Evaluation

Objectives and Evaluation Questions

The main objective of the evaluation is to understand if MHs can effectively increase access to safe delivery, particularly for women living farthest from health facilities. Findings from this evaluation will be used to inform program decisions regarding implementation of the Maternity Home Model and, if applicable, to advocate for programmatic improvements as the Maternity Home Model is taken to scale beyond the districts proposed for this project.

Primary impact evaluation question

1. Does the Minimum Core Maternity Home Model increase the number of facility deliveries among mothers living more than 10 km from the facilities compared to facilities with no improved maternity home services?

Secondary evaluation questions

1. To what degree does the Minimum Core Maternity Home Model change maternal and neonatal health outcomes among those living more than 10 km from the facility compared to the standard of care?
2. To determine if the awareness and perceptions of MHs by communities located more than 10 km from the health facility change over the period of this study?
3. To determine if there is an increase in awareness and perceptions of HF associated safe delivery and HF delivery intention among pregnant women living in communities located more than 10 km from the health facility change over time?
4. To determine the degree of financial impact that use of the MH has on the families of women who utilize it compared to women who do not utilize it?
5. How does the perception of quality of care differ between intervention and comparison sites?

Research Design and Methods

Study Design

We propose a quasi-experimental pre-post design wherein one implementing partner (BU/MAHMAZ) will use a cluster-randomized matched-pair design and one implementing partner (University of Michigan/Africare) will utilize a non-randomized matched-pair, comparison design. The framework of the study design is shown below Quasi-experimental Study Design to Evaluate the Impact of MHs.

X = Minimum Core Maternity Home (see above)
O = Observations at baseline (O1, in 2015) and endline (O2, in 2018) at intervention (X) and comparison
(_) sites.

R = cluster randomized; NR = not randomized.
While the partners will manage implementation of the intervention separately, the BU/ZCAHRD team will collect population-level data on the impacts of the MH intervention across all sites. Data will be pooled for analysis but sample sizes provide sufficient power to separately analyze the BU/MAHMAZ and UMich/Africare sites.

Data Sources

We will collect data from two main sources

1. Household Surveys
2. In-depth Interviews

Household Survey (O1, O2)

A quantitative household survey will be conducted at baseline (2015) and endline (2018) among recently delivered women (those who have delivered in the last 12 months) living more than 10 km from the intervention and comparison facilities. Data to be included
in the surveys (details in Annex 2)

• Household and individual demographics
• Health care practices and decisions, particularly around labor and delivery
• Perceived quality of recent delivery care
• Barriers to access facility-based delivery
• Costs of care
• Maternal and neonatal health outcomes

This approach will capture the experiences of those who utilized the facility in their catchment, other facilities, and those did who not access a facility for delivery, allowing us to more accurately estimate the impact of the MH intervention.

In-depth Interviews (IDI) (O1, O2)

IDIs will be conducted among a sample of respondents identified for the household survey to gain a deeper understanding of community perceptions. Content will include perceptions of labor and delivery practices, barriers to accessing care, knowledge and awareness of MHs, perceptions of the quality of maternity homes (guided by the Core Maternity Home Model), perceptions of respectful care at the facility, post-natal care, costs, and perceptions of MH ownership.