ECUR 809 - Assignment 1
The evaluation I chose for this assignment was “An Evaluation of the Academic Portion of the Métis Housing Administration Program” submitted by the Gabriel Dumont Institute, to the Canada Mortgage and Housing Corporation (CMHC). This evaluation was completed in 1994 after the program was created in 1992 by the CMHC, Saskatchewan Department of Community Services - Housing Division, and the Provincial Métis Housing Corporation.
The evaluators took a product-based approach to their evaluation. They analyzed the program by describing what the program entailed, then by surveying the clients (students), they came to their conclusion on how to improve the product (the program). The first section of the evaluation gave the background on why the program existed, and who was involved. There were three program objectives identified in section two, which included addressing a shortage of qualified Métis delivering, administrating, and managing housing programs; ensuring that the training that took place addressed the shortcomings of the existing CADRE program; and to assist Métis in entering the job market. Section three described the program design as well as the shortcomings of the CADRE program that this program was meant to overcome. Section four described the recruitment process of the ten students selected, while the fifth section outlined the demographics of the students, including personal information, education, and languages. Section six gave anecdotal evidence of the effectiveness of the preparatory phase of the program by providing student comments. The seventh section analyzes data on the academic phase gathered through student surveys on each class and its instructor. Following the survey section of each class, more information was provided on what the student liked best about the class through anecdotes, and then the evaluator provided a judgement on areas of improvement.
The evaluation contained both strengths and weaknesses which in most cases were intertwined. The background information on the program objectives and design were well written and allowed me to get an excellent snapshot on the purpose and process of the program. The weakness of this area was that the program objectives were not revisited in the conclusion. The evaluator made judgements only on the students’ perceptions of the individual classes and not on the program objectives. The evaluators should have asked the following question at the end of their survey: After taking this program, do you feel qualified in delivering, administrating, and managing a housing program? This would have given more information on if the program was successful in its objective. The evaluators should have also analyzed if the shortcomings of the CADRE program were overcome since they knew exactly what they were. It seemed like such a large portion of the background information to ignore in the judgement. In addition, a description of what the CADRE program is or what CADRE stands for would have helped the report. The third objective of the program could have been answered by following up on the students’ employment following the program. The information gathered through the survey was great, and the comments made by the students really helped evaluate the specific class needs, but the evaluation could have been improved by having the instructors give their evaluations on how the program worked. The evaluators recommended showing the surveys and comments to the instructors to help them improve, but self-reflection could be a part of the process as well.
Overall, the program evaluation did succeed in finding ways the program could improve, therefore improving the product they were evaluating. However, a more thorough examination relating to the specific objectives of the program would have been worthwhile.
Great work David. It seems based on your analysis that this work could have used a second go through. It is important in PE to stick with the plan that you have designed and use as much data as possible to come to conclusions. A final report can be used not just to inform those directly connected to the program but to use for new funds and supporters of the work taking place. If the gaps you identify were corrected this might become a very important tool for supporting an identified need in the Metis community. Was there a particular theoretical model you think might explain the approach taken by the evaluator in this report?
ReplyDeleteThanks for your comments Jay! When I researched a little more into Stufflebeam's CIPP theory for this assignment, it seemed like each step was a separate way of doing an evaluation and so thought the product step made the most sense for what the evaluators were trying to accomplish. Would all four steps be used normally for a PE?
ReplyDelete