Implementation and Replication of Evidence Based Sexual Reproductive Health Programs
Embargo until
Date
2016-09-08
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Johns Hopkins University
Abstract
Background: The state-wide scale up and replication of sexual reproductive health evidence based programs (SRH EBPs) in New Jersey provided an opportunity to study the implementation of six of these programs in different replication settings. Implementation science is a developing field with a need for measurement testing of implementation measures. Additionally, implementation research is especially limited within SRH EBPs and with the adolescent population. One of the primary debates in the field as to whether strict adherence to the prescribed curriculum is essential, and whether adaptations negatively affect program outcomes.
Data and Methods: Psychometric testing was conducted on a participant responsiveness measure used on a state-wide survey across all six SRH EBPs, with 2,242 participants. In depth interviews were conducted with program implementers (n=18) of the same six SRH EBPs to further understand the contextual factors that implementation. Thematic analysis was used to identify successes and challenges to implementation of SRH EBPs.
An in-depth study on adherence and adaptations was conducted with one SRH EBP, with 1,608 participants (intervention and comparison). Developer-created fidelity logs were used as the data source for measurement of adherence and adaptations. Frequency calculations were used to describe adherence % and adaptation % by classroom. Thematic analysis was used to categorize types and rationales for adaptations. Subgroups of adaptation levels were created among the intervention group who attended greater than 75% of sessions in order to determine program outcomes by level of adaptation. Statistical analyses utilized propensity scores to increase comparability of intervention adaptation subgroups and comparison participants. Program outcomes for each of these adaptation subgroups were determined using logistic regression analyses and mean differences.
Results: The refined factor structure of the participant responsiveness measure was reliable and valid among an adolescent population of varied age, gender, race, and invariant across multiple SRH EBPs and settings. Program implementers identified relationship building with partner sites and participants as strategies critical for implementation success. Program implementers, however, felt challenged in implementing some of these strategies, which involved adaptations, due to the perceived need to maintain fidelity to the program.
Frequency calculations indicated that adherence and adaptation varied considerably by classroom. Thematic analysis revealed that the adaptations made were related to delivery of content, rather than to the content itself, and were in response to participant needs and setting constraints. Propensity score matching successfully reduced significant differences in key covariates between intervention adaptation subgroups and the comparison group. Program outcomes comparing the intervention condition to the comparison condition for the low, middle, and high adaptation groups, respectively, were as follows: differences in SRH knowledge score intervention vs control [low=+14.3%, middle=+17.4% , high=17.8%], intent to use birth control in next 6 months [low: OR= 2.29 (1.28-4.09), p=.01; middle: OR= 2.36 (1.09-4.13), p=.01; high: OR= 5.67 ( 2.51-12.85), p=.00]; intent to abstain from sex [low: OR=1.63 (.80-3.30), p=.17; middle: OR=1.43 (.79-2.61), p=.23; high: OR=1.34 (.69-2.63), p=.37]; intent to use condoms in the next 6 months [low: OR= 2.04 (1.11-3.76), p=.04; middle: OR= 2.36 (1.09-4.13), p=.04; high: OR= 5.67 (2.51-12.85), p=.04].
Conclusions: Program outcomes did not appear to be reduced for the high adaptation subgroup. Quantitative and qualitative findings support the argument to allow for some flexibility in programs, as well as training for program implementers on how to make adaptations. It is important to include implementation in standard evaluation practice of EBPs in order to continue to understand replication findings, build the evidence base, and test program theory.
Description
Keywords
implementation, evidence based programs, sexual reproductive health, adolescents