Programme Evaluation Economic Models.
Any organized action, such as media campaigns, service supply, educational services, public policies, research projects, and so on, is referred to as a “program.” The process of forming value judgments or making decisions regarding events, objects, or their attributes is known as evaluation.
The process of methodically gathering, analyzing, and applying data to assess the efficacy and efficiency of programs is known as program evaluation.
During the implementation of the program, an economic model of program evaluation is utilized to compare the program’s benefits to its expenses. As a result, useful quantitative data is generated, which may be used to assess the program’s effectiveness.
This information is similar to an audit, and it is useful to sponsors and backers who want to know what benefits their money will bring to beneficiaries. Economic model asks the question of when, what, why and how the program can be used effectively.
Importance Of Economic Evaluation
- At the moment, resources are limited, but program expenditures are increasing in tandem with more innovative and technological improvements. As a result, economic examination has become both required and urgent.
- Economic evaluation also aids in the prioritization of programs and the best resource allocation decision.
- Economic evaluations are critical tools for determining the cost-effectiveness and efficacy of health interventions.
Economic Models of Programme Evaluation
These are the economic models of programme evaluation. These are:
- Cost Benefit Analysis Model (CBAM)
- Cost Effectiveness Analysis Model (CEAM)
- Cost Utility Analysis Model (CUAM)
- Cost Minimization Analysis Model (CMAM)
- Cost Efficiency Analysis Model (CEAM)
- Philip’s Model of Evaluation
Cost Benefit Analysis Model (CBAM)
A cost-benefit analysis model is a systematic procedure used by program evaluators to determine which decisions to make and which to forego in terms of the program’s cost and benefit.
The cost benefit analyst counts up all of the anticipated advantages of a condition or activity, then subtracts all of the associated expenses. Some consultants or analysts develop models to put a monetary value on intangibles like the benefits and costs of living in a specific location.
Prudent managers undertake a cost-benefit analysis before embarking on a new project to assess all of the project’s possible costs and earnings. The results of the analysis will indicate if the project is financially viable or whether the company should pursue a different idea.
In many models, the opportunity cost is factored into the decision-making process as part of the cost-benefit analysis.
Alternative advantages that may have been gained if one option had been chosen over another are known as opportunity costs. In other terms, the opportunity cost is the value of an opportunity that is foregone or missed as a result of a choice or decision.
When opportunity costs are taken into account, project managers can examine the benefits of different courses of action rather than just the current path or choice in the cost-benefit analysis.
Cost-Benefit Analysis Process
Compiling a detailed list of all the costs and benefits associated with the project or decision should be the first step in a cost-benefit analysis (CBA).
The costs involved in a CBA might include the following:
- Direct costs include manufacturing labor, inventories, raw materials, and manufacturing charges.
- Electricity, managerial overhead costs, rent, and utilities are examples of indirect costs.
- A decision’s intangible cost, such as the influence on consumers, staff, or delivery schedules.
- Alternative investments or the cost of buying a plant vs building one are examples of opportunity costs.
- Cost of potential risks such as regulatory risks, competition, and environmental impacts,
Benefits might include the following:
- Increased production or the introduction of a new product boosts revenue and sales.
- Intangible benefits include increased staff safety and morale, as well as increased consumer satisfaction as a result of improved product options or quicker delivery.
- Competitive advantage or market share gained as a result of the decision.
The conclusions of the aggregate costs and benefits should be analyzed statistically to determine whether the benefit outweighs the expense. If this is the case, proceeding with the project is the best option.
If not, the company should assess the project to see if there are any adjustments that can be made to increase benefits or reduce expenses in order to make it viable. Otherwise, the firm should generally avoid working on the project.
Cost Effectiveness Analysis Model (CEAM)
The Cost-Effectiveness Analysis Model (CEAM) is a cost-effectiveness analysis model that can be used instead of the Cost-Benefit Analysis Model (CBAM). The technique compares the relative costs of two or more courses of action to their outcomes (effects).
CEAM comes in handy when analysts are faced with constraints that prevent them from doing a cost-benefit analysis. The most common barrier is analysts’ inability to monetize advantages. CEAM is most beneficial prior to the start of a program since it allows for the comparison of two different courses of action.
Consider incorporating it into the program design as well as the evaluation process. CEAM can also be used to create counter-factual scenarios that compare the program’s efficacy to alternative ways that were not used, as well as against other similar programs.
The finest cost-effectiveness studies incorporate a wide view of costs and benefits, including indirect and longer-term consequences, and reflect the interests of all program stakeholders. As a result, make sure your analysis is as thorough as feasible.
Cost Efficiency Analysis Model
Cost-efficiency analysis is the study of a program’s or activity’s cost per output, which allows you to compare cost-per-output for programs that all provide the same output.
“Saving money by enhancing a process or product” is the most basic definition of cost efficiency. Companies evaluate their cost-effectiveness by comparing their operating costs to the output (for a product) or revenue earned (by a process).
It has a lot of promise and longevity built in because there’s always space for growth.
Cost Minimization Analysis Model (CMAM)
When the justification for intervention has been established and the programs or processes under consideration are projected to have the same or similar outcomes, cost minimization analysis is an appropriate evaluation method to utilize.
In these situations, attention may be drawn to the cost side of the equation in order to determine the least expensive choice. This technique of analysis compares the costs of two or more interventions that achieve the same result. After that, the intervention with the lowest cost is picked.
It is important to highlight that the intervention can only be carried out if the outcomes of the contrasting therapies are identical.
The use of CMAM necessitates proof that the interventions being compared have equal outcomes. If this is the case, the only thing that is compared are the costs. This form of analysis is commonly referred to as “comparative cost analysis.”
Given that comparators are unlikely to have completely comparable effects, the assumption of equality in outcomes is a fairly strong one to make. Comparing the cost of providing the same medication in different contexts is an example of a CMAM study.
Cost Utility Analysis Model (CUAM)
The outcomes of alternative processes or programs are stated in terms of a single, “utility-based” unit of assessment in cost-utility analysis. A cost-utility analysis is a sort of cost-effective analysis that analyzes different procedures and results in terms of the quality of life of a person.
The assessment of two or more alternatives based on their cost and usefulness is known as cost-utility analysis. In this context, utility refers to individual satisfaction as a result of one or more outcomes, as well as the perceived value of projected outcomes to a certain constituency.
Cost utility analysis and cost effectiveness are inextricably linked. However, cost effectiveness analysis requires the use of a single measure of effectiveness, whereas cost utility analysis allows researchers to combine multiple measures of effectiveness into a single measure of utility. Following this analysis, the educational decision maker should logically select interventions that provide the greatest utility at the lowest cost.
In the actual world, educators and students may place a higher value on one outcome over another; for example, educators may consider that, while all outcomes are important, one is more significant than the other.
One method to do this is to give various weights to different interventions; for example, in the case above, professional behaviors might be given a higher weight than applied knowledge results.
The fundamental advantage of cost utility techniques is that they allow for the consideration of different outcomes in the evaluation. A secondary benefit is that they compel stakeholders to consider the relative merits of various outcomes, as well as to communicate and record the results of their deliberations.
The downside of cost-utility analysis is that different methods for assigning weightings to different outcomes may be utilized, which might lead to conclusions that are difficult to replicate due to the diverse methodologies used.
Secondly, cost-utility analysis cannot evaluate the value of a particular technique in and of itself; it can only be used to compare a variety of options.
more so, the validity of cost-utility analysis results depends on our capacity to be certain that the observed utility is caused by the intervention and not by other factors.
Phillips’ Evaluation Model
The Philip’s model of evaluation is considered an economic model due to the inclusion of the concept of return on investment as its fifth level.
Though Philip’s model is a step up from Kirkpatrick’s learning evaluation model, its stepping up of Kirkpatrick’s model considers the economic value of learning programs by looking at the monetary returns that investors should expect.
According to Phillip’s methodology, once the business impact/results of a learning program have been determined at Kirkpatrick’s Level 4, the impact may be translated into monetary terms and compared to the entire cost of the program to compute ROI.
These costs include program development and delivery, plus the labour cost of time for learners to complete the training.
Phillips’ Levels Of Evaluation
Level 1: Reaction
The learners’ reaction might be defined as how well they appreciated a certain training program. The less relevant a learning package is to a learner, the more effort that must be placed into its design and presentation.
At this stage of evaluation, the learners’ reactions to the training are assessed using attitude surveys distributed after most training sessions.
This level only evaluates one thing: the learner’s impression (response) to the course; it is not indicative of the training’s performance potential because it does not assess what new skills the learners have gained or how they will use what they have learned in the workplace.
Level 2 : Learning
As a result of participation in the learning process, participants’ attitudes, knowledge, and ability improve to a certain level.
Post-testing is required as part of the learning evaluation to determine what abilities were learned during the course. In order to validate the learning objectives, it is necessary to measure the learning that occurs in a training program. When assessing the learning that has taken place, it is common to ask questions like:
- What knowledge was gained?
- What abilities were honed or improved?
- What mindsets were altered?
Level 3 : Behavior
This assessment assesses a student’s ability to apply what they’ve learned on the job rather than in the classroom.
Level three evaluations might be formal (testing) or informal (observation). It analyzes whether the correct performance is presently taking place by asking, “Do workers apply their newly learned knowledge on the job?”
Because the major goal of training is to enhance results by having students learn new skills and knowledge and then use them on the job, it is critical to track performance. Learning new skills and knowledge is useless to an organization unless the participants put them to use in their daily work.
Level 4 : Results
This assessment measures the training program’s success in terms that executives and managers can understand, such as increased production, increased sales, lower costs, improved quality, lower accident rates.
Also, higher profits or return on investment, positive changes in management style or general behavior, higher levels of direct port engagement, and favorable feedback from peers and subordinates.
This level also aims to assess the training’s practical outcomes, such as cost savings, improved quality and efficiency, greater productivity, staff retention, increased sales, and enhanced morale.
While establishing such benchmarks is not always simple or inexpensive, it is the only method for training organizations to measure the key return on investment (ROI) of their training investments.
Identifying if specific outcomes are genuinely the consequence of the training is a common challenge.
Level 5: Return On Investment (Roi)
According to Deller, the value of training programs is determined at this level using cost-benefit analysis. It enables businesses to determine whether the money spent on training yielded measurable results, and if so, what those returns were.
The Phillips approach requires business data to be collected before, during, and after training to achieve the final aim of determining ROI.
This data, he claims, is analyzed for quantifiable factors such as process improvements, productivity gains, and higher revenues. You can then compare the cost of the training to the monetary benefit received using this information. This provides you an idea of the training’s worth and the influence it had on the company’s bottom line.
Let’s return to the fourth level of the Phillips ROI Model to learn more about how this works. Level 4 – Result – is all about determining a learning program’s business impact on a firm or organization. According to Phillips’ ROI Model, this impact may be quantified and compared to the entire cost of running the program.
These costs could include:
- Program development
- Program delivery
- Labor costs
- Time for participants to complete the training