OVERVIEW OF STUDY DESIGN AND METHODS

In order to achieve the objectives, we will use a mixed-methods approach and collect longitudinal, cross-sectional data at both the MAHMAZ intervention BEmONC sites (n=10) and a set of matched control sites (n=10) that will not have any interventions implemented, before and during MAHMAZ project implementation. We will also collect data at 5 CEmONC sites, 2 of which will have an MS constructed and 3 of which will have no intervention. This approach has several strengths. First, it allows for triangulating findings from various data sources and methods. Second, the use of a longitudinal time-series study design with a control group allows us to differentiate the challenges or added burden posed by the core model implementation from those that are implementing the standard of care and allows for limiting threats to internal validity of history and maturation. Third, by systematically evaluating the implementation and outcome of each element of the intervention, we can isolate ‘key ingredients’ for effective implementation and relate it to both patient and facility- level outcomes.

Objective 1 will be achieved by analyzing routinely collected project data, including monthly data from the Mothers’ Shelter Experience Surveys (a sample of women (up to 6 per site per month) from each RHC study site, both intervention and control, available on day of data collection who have stayed at an MS or other facility-affiliated lodging/structure at least 3 consecutive nights), and semi- annual in-depth interviews (IDIs) with both governance committee members and management unit members at intervention sites only (see Annex J for interview guide). As part of the project, at each site an MS Register, Others’ Register (for women utilizing the MS after 8 days postpartum or travelers, relatives of patients who are non-maternal cases) and MS Activity Log will be maintained. On a monthly basis, MAHMAZ Monitoring and Evaluation (M&E) study staff will extract data from these registers/log using corresponding data extraction tools (Annexes G, H, and I). The project team will also administer the Core Model Checklist (Annex D) at BEmONC sites and CEmONC intervention sites. The quantitative survey and Core Model Checklist data will be triangulated with the IDIs at each intervention site.

Objective 2 will be achieved by analyzing data collected monthly from the Mothers’ Shelter Experience Surveys (see above), semi-annual IDI with governance committee members and with management unit members, and annual focus group discussions (FGDs) among recently-delivered women (defined as women who have delivered in the past 12 months), men with children under the age of 1, community elders/mothers-in-law of RDW and members of Safe Motherhood Action Groups (SMAGs) at each site (see Annex K for FGD Guide). Data will be analyzed against the community-defined standards of acceptability from the formative evaluation (ERES Protocol#2013- AUG-008/BU IRB Protocol H-32470) and other standards that emerge through ongoing analysis of all qualitative data. Stratified by intervention and control site, data will be triangulated with quantitative data from objective one to understand acceptability by implementation fidelity of the core model, and findings from the impact evaluation (ERES protocol #2015-Dec-012/BU IRB Protocol H-34526) in MAHMAZ sites only.

 Objective 3 will be achieved by using routinely collected health facility data at all 20 BEmONC sites (intervention and control) and at 5 CEmONC sites in the catchment areas beginning six months prior to implementation of the intervention. On a monthly basis, study staff will extract key variables about delivery and neonatal outcomes, referrals and postnatal care from the following BEmONC and CEmONC facility-based data: Delivery Register, Maternity Ward Admission Register, Postnatal Care Register, Facility Transfer Log, Ante-natal care register (where applicable) and other improvised registers as necessary. The data extraction tools can be seen in Annexes A-C. Secondly, using the Safe Motherhood Number (SMN), a unique code that a woman receives when visiting government facilities during her pregnancy, each woman will be tracked through the “care cascade”, meaning throughout mothers’ shelter stay, delivery and post-natal visits, to understand both utilization of MS and health facility services and maternal and neonatal health outcomes. The SMN will also allow tracking during referral from BEmONC to CEmONC sites and to understand patterns of bypassing (i.e. women going directly to CEmONC facilities). We will also compare this information with data captured in Objective 1. Additionally, on a monthly basis we will assess the capacity of the health facility to manage obstetric complications using a health facility assessment tool (Annex I). We will conduct semi-annual IDI with health facility staff and health systems staff (Annex J) and extract MS and health facility financial records (as they relate to MS) to understand the cost-outcomes of the MS.

Objective 4 will be achieved through use of routinely collected monitoring data from MS registers, as well as program data from MS activity logs, training attendance logs, production registers, in-kind donation logs, as well as MS and health facility financial records (as they relate to MS). For each entrepreneurial strategy we will systematically capture the costs of implementation, the change in revenue over time, and the proportion of MS operating costs that each strategy covers using MS costing tools. Additionally, program data will capture decisions made and changes in operating costs. These data will be analyzed to understand what mix of operating costs and revenue streams create a financially sustainable MS. We will also use data from the semi-annual IDIs with governance committee members and management unit members and FGDs among RDW, men with children under the age of 1, elders/community members and SMAGs (see Objectives 1, 2 and 3).

Objective 5 will be achieved by synthesizing findings from this implementation effectiveness evaluation and the intervention effectiveness evaluation, guided by the CFIR and situating it all amongst current peer-reviewed literature.