Program Evaluation

Typically, organizations work on their missions to identify various overall goals, which must be attained to accomplish the task. Each of these goals represents programs, which must be evaluated to assess if they are working, their effectiveness, and if necessary devise ways to improve them to make them better for the individuals they serve. Program evaluation encourages organizations to examine the operations of programs, the activities involved, people who conduct the activities, and the intended results. Besides, evaluation indicates how faithful the program will adhere to the implementation protocols. In essence, program evaluation allows program administrators to determine if activities were implemented as planned, identify the strengths and weaknesses of a program as well as areas that may need improvement.

Key Considerations when Performing Program Evaluations

Before embarking on a program evaluation, it is important to identify the useful focus of the evaluation and determine whether it is feasible. According to Newcomer, Hatry, & Wholey (2015), a logic model is usually adopted in program evaluation to provide a visual understanding of how a program operates. The model is used to give a complete outlook on the causes, effects, and ensure that the project produces its presumed goals. Normally, not all logic models are the same or designed for the same purpose. For instance, the outcome approach focuses on the strategies and activities and their relationship to the desired results of a program. After determining the type of design appropriate for the program evaluation, the next step involves performing a validity test on both internal and external elements. In this context, the experimental or quasi-experimental designs can be used to determine the functionality of a program. According to McDavid, Huse & Laura (2013), the experimental design takes uses random selection, where members of the service population are given an equal chance to participate in the experiment as either control or treatment participants. Virtually, the control group is adopted as a baseline evolution and is identical to all other items that are being examined except not receiving experimental manipulation subjected to the treatment group. On the other hand, treatment group is the subject being manipulated. In quasi-experimental designs identify a comparison group similar to the treatment group and are used to capture the best possible outcomes if the program was not implemented. Ultimately, at the center of the validity designs, research and program evaluators must be careful of internal and external threats to validity. Langbein (2012) elaborates, internal validity involves causal evaluations and is defended by the accuracy of causal claims. The ultimate threat to internal validity lies in the failure of the evaluator to consider the history of intervening as possible sources of change that are unrelated to the program being evaluated. On the other hand, external validity involves the validity of generalized inferences based on experimental validity. The threat to this type of validity is the explanation of how a program evaluator can be wrong in the generalization process. Ostensibly, generalizability is the result of dependability of a cause on other factors.

Methods used in Program Evaluation

Generally, it is difficult to determine the appropriate methods to use in carrying out successful program evaluation. The task is even more complicated because of the various specific evaluation issues that demand attention, the numerous methods used to gather and analyze information, as well as the need to ensure all relevant matters are addressed. As highlighted above, regardless of the method used in program evaluation, there are reliability and validity concerns and often using more than one method helps solve this kind of problem. Mixed research methodology is based on data collection and analysis through the integration of quantitative methods such as survey and qualitative methods such as interviews. In mixed research methods, a combination of qualitative and quantitative research methods is adopted with the belief that it will yield a comprehensive understanding of the research. By using quantitative data analysis, program evaluators can perform statistical analysis on scores of collected data from questionnaires or checklists and come up with a credible hypothesis. On the other side, qualitative research methods help in the analysis of open-ended data gathered through focus groups and behavior. According to Creswell (2015), combining the two approaches allows the researchers to obtain an in-depth understanding while eliminating the inherent weaknesses that are brought about by use of one method. Practically, mixed methodologies are suitable when a study needs to substantiate results obtained through another method. Program evaluators must be able to determine when it is necessary to combine two methods to achieve a holistic evaluation.

What are Other needs of a Program Evaluation

During program evaluation, terms such as inputs, activities, outputs, outcome, and impacts are commonly used. According to Taylor (2017), these terms are used to measure, determine, and indicate the position of a particular research project, which makes it important for evaluators to be able to differentiate between the terms. In this case, inputs are elements used in the implementation process to ensure the probability of project results delivery. Activities and represent the components of the program and strategies used to generate program outcomes. Outputs as McDavid et al. note, represents medium-term results of a project. On the other hand, outcomes involve the measurable changes in participants’ knowledge, skills, and behaviors. It is important to understand that outcomes are usually linked to the goals of a research and determine the success or failure of a project.

Who is involved in Program Evaluation?

Essentially, program evaluation is a participatory process where various participants play roles to ensure the success of a project analysis and ensure the decision-making process is implemented smoothly. In this context, it is necessary first to identify the stakeholders and hold them accountable to an array of the decisions made during the process. Depending on the focus of evaluation, sometimes it is necessary to mention specific users and their roles in the whole process (Patton, 2008). Ordinarily, stakeholders are involved in the process so that the program evaluator can demonstrate the accountability of the program to them and convince them that the project contributes to making an impact to the proposed concern. Involving stakeholders during the program evaluation ensures that they are determined to ensure proper design and implementation of the proposed project.

Over time, implementation of programs and policies has become more sophisticated. More often than not, problems have in the past been solved amidst significant and challenging behavior alterations on the side of the consumers or providers. Similarly, most of the programs that have succeeded in some areas have dismally failed in others due to factors involving fiscal, demographics, socioeconomic, interpersonal among others. Along with the complexity of programs, the demands for accountability from stakeholders and policymakers have doubled. All these adjustments mean that strong program evaluation is essential to achieve intended outcomes and ensuring accountability is achieved.

References

Creswell, J. W. (2015). A concise introduction to mixed methods research. Los Angeles [u.a.: Sage

Langbein, L. (2012). Public program evaluation: A statistical guide (2nd ed.). Armonk, NY: ME Sharpe

McDavid, J. C., Huse, I., & Hawthorn, L. R. L. (2013). Program evaluation and performance measurement: An introduction to practice. Los Angeles: Sage.

Newcomer, K. E., Hatry, H. P., & Wholey, J. S. (2015). Handbook of practical program evaluation. San Francisco : Jossey-Bass & Pfeiffer Imprints, Wiley.

Patton, M. Q. (2008). Utilization-focused evaluation. Thousand Oaks: SAGE Publications, Inc.

Taylor, R. R. (2017). Kielhofner’s research in occupational therapy: Methods of inquiry for enhancing practice. Philadelphia : F.A. Davis Company.