One major goal in health behavior research is to develop evidence-based programming (EBP). An EBP is a service that has been demonstrated to elicit an effect on desired behavior among a population. For example, in teen-focused substance misuse prevention an EBP shows that fewer youth who received the program would try out or regularly use a substance (e.g., nicotine, alcohol, illicit drugs) compared to youth who did not receive any programming, or youth who received a program that was not evidence-based.
To create an EBP, researchers might conduct a randomized controlled trial (RCT). That is, they might assign subjects randomly to conditions (i.e., receiving different treatments). In one condition, subjects receive the program and, in another condition, subjects receive equal attention, or standard care, but not the program. Subjects are assigned to conditions through a means of randomization (e.g., the flip of a coin).
People are hired to implement the research project. People who deliver the program are referred to as the “implementers”. The researchers interview staff candidates and hire people who have the proper education and job experience to deliver the program. The implementers of a research program are staff who are probably familiar with program evaluation methods, previous health behavior program delivery, development of the project, believe in the research, and are reasonably well-paid to deliver the project. These carefully trained implementers deliver a program with fidelity: that is, they deliver the program as intended.
The Dilemma: Imagine that you have conducted several randomized controlled trials and found results supporting the existence of an evidence-based program (e.g., substance misuse prevention programming, physical activity programming). The program works under ideal (research) conditions. It is not certain that the program will ever be implemented within a community. If the program is implemented within a community, it is not certain that programming will be sustained; that is, if it continues to be delivered over many years.
There are several hurdles to overcome when trying to implement evidence-based programming in the “real world”. The use of an EBP needs to align with wider institutional goals for the community organization to desire to implement it. The community needs to have the capacity to institutionalize the program. The program would need to be introduced on top of what is already in existence in the community. Higher-level decision makers need to be involved to permit entry of the program into their community (e.g., mayors, school district heads).
Furthermore, the program must pass a rigorous evaluation before being implemented. A proper evaluation of a health behavior research program involves examination of quality of implementation, process, and outcomes to determine if the program is delivered as intended and leads to preventive effects on behavior. Researchers will want to know whether the recipients of the program find the program to be of interest, learn new information, believe in what behaviors the program is trying to instruct (e.g., more time spent in physical activity, keeping a healthier diet, not experimenting with recreational drugs while the brain is rapidly developing), and whether the recipients like how the implementer delivered the material. Finally, researchers want to determine if the program changed variables that might mediate change in behavior (e.g., lowering prevalence estimates, perceived acceptability, and safety of drug use among peers), the recipients’ intentions on how to behave in the future and change future behavior (e.g., last 30 days drug use as assessed at a one-year follow-up).
Implementers who will faithfully implement the program need to be recruited. There may be some problems there. First, staff within a setting have a limited amount of time to accomplish a set number of tasks. Asking them to take on another task may not be realistic. Second, the staff need to view program delivery as part of their role within the setting. If a supervisor mandates that they take on the task, they may not necessarily identify with the role of implementer. A poor implementation may result (e.g., handing out a pamphlet rather than delivering the EBP). A potential implementer may or may not believe that the EBP works regardless of the supporting research.
Third, staff also needs to be trained appropriately to deliver the program with fidelity (as intended). Where do the resources for training come from? Who does the training? If the community organization does not coordinate implementation with the research group that developed the program, or with organizations formally trained by the research group, it is not certain that the program will be delivered appropriately. Fourth, implementers may have a fear of being evaluated as they want to keep their jobs. Yet, continued evaluation of program delivery may be needed to ensure fidelity. That is, it is ideal to measure whether the program was delivered in full, whether any changes were made in the delivery, whether any activities were omitted, whether outside material was added (e.g., a drug misuse video), and whether the population liked the implementer and learned and believed the material.
Fifth, some alteration of programming may be needed due to conflicting demands. For example, maybe a program needs to be implemented twice a week rather than four times a week due to other programming that is being delivered. Implementers need to know how to adapt to the context and yet understand and manipulate the general principles of behavior change asserted by the program. For example, they will not abridge or delete material that would lower drug use prevalence estimates or perceived acceptability and safety of recreational drug use among teens, variables that impact later drug use behavior.
Resources like money and consistent manpower may be needed to keep the programming running. When resources are limited, the ability to sustain implementation becomes limited. Who is going to be willing to donate their time? Who is going to be willing to donate the money? Is this program a priority for the setting currently? There may be alternative programs available. What if decision makers want to try out another non-EBP program? Do supporters have enough political influence to make sure the EBP gets implemented?
Programs change over time, and so does the larger social context in which the program takes place. The program must keep up with changes in the larger social and physical environment. Media-based programming loses its relevance as clothing styles, slang and music change. Written programming will have a much longer shelf-life, but it needs to fly off the page through the skills of the implementer. Staff who have been delivering a program glowingly may move to a new setting, leaving the program short-staffed.
There also may be changes in scientific knowledge or legal policies that impact program content and delivery. For example, marijuana is now essentially legalized in numerous states. Oral nicotine products are now being widely produced, often containing synthetic nicotine. Because these products have become more widely available, teens may view them as relatively popular, acceptable and safe to use. Programming needs to address the changes that have occurred, reasons for the changes (e.g., money and marketing), and separate health from legality. So, what can one do?
Recently, researchers and practitioners have been thinking of different approaches to developing and implementing EBPs that might assist in sustaining EBPs. First, one might engage in community-based partnerships and evaluate a program that has already been utilized by a community to possibly show it is evidence-based. In that case, the program is being implemented, and the public knows that the program positively alters health behavior in practice. Still, concrete relationships with researchers need to be established, and rigorous evaluation is needed to make sure that the program shows the desired changes in the recipients’ behavior. If such a “grass roots” program becomes evidence-based, its delivery still needs to be maintained and it, too, may become less relevant within the community over time.
Second, researchers may try out an online digital program (e.g., programmed learning, through mediums like telephone or text-message-assisted programs). This kind of programming tends to be more passive than in-person programming and possibly may benefit only those recipients most motivated to change their behavior. Third, researchers may attempt an AI program, which is highly interactive. However, AI programming may not be able to handle equivocation, conflict, or complex human emotions needed to be addressed to impact behavior on such topics as substance misuse prevention.
So, what is the answer here? Perhaps the answer lies in what has worked in the past, or in other arenas. For example, people hardly think about brushing their teeth or going to class. There exist fixed locations, equipment, and time to engage in these activities. For EBPs to work consistently, these factors need to be set in stone. That is, it needs to be utilized completely and regularly by the community and become part of what the community considers its responsibility.