5.5. Case study no . 6 Troubleshooting refrigeration systems
The usual long, hot summer. In Townville, Illinois, manager Joseph D Doe of the Buy-It-Here supermarket chain is getting hot under the collar. For the sixth time in three days, he is listening to another complaint from one of his branch managers about 'poor freezer maintenance service'. There is a pattern in the complaints which come from different areas in the state.
A freezer cabinet or cool room malfunctions. A service engineer arrives, looks at it, works on the system, and gets it going again. Some four hours later, the system is out of operation again. The withdrawal of products from freezers and cool rooms where temperatures have risen above safe limits is beginning to bite into branch profits. Joseph D Doe reaches for the phone...Incidents like this once prompted a supplier of refrigerants to create a course on the Systematic troubleshooting of refrigeration systems (E I du Pont de Nemours, 1964). The course was constructed around an algorithm (decision-making tree) via which each subsystem in a refrigeration unit — the condenser, evaporator, thermal expansion valve and compressor electrical system — was subjected to a systematic troubleshooting regime. Through this procedure, the stop-start-stop problems that were known to plague users of refrigeration and air-conditioning installations were minimised.
Imagine you are Joseph D Doe. You have chosen to send your 15 service engineers (in groups of five) on a course that uses the procedure referred to above. The course lasts two days. It covers both theory and practice. The post-test consists of a performance test in which course participants must demonstrate their ability to diagnose and correct three malfunctions using the systematic troubleshooting decision tree taught in the course. Critical theoretical knowledge related to the procedure is assessed through written responses to multiple-choice questions and an oral test administered by an instructor not involved in teaching this course. As
Figure 17 Results from the Buy-IOt Here troubleshooting course
Joseph D Doe you want some guarantee of the efficiency of the course. A pre-test of the service engineers' individual knowledge and skill is requested. The results of the pre-test and post-test for the three groups of engineers are given in Fig. 17. What do you think of these results? Would you be satisfied with them? Under what conditions would you trust the validity of both tests? Would you recommend this course to other supermarket chains based on the engineers' results? Engineers scoring 50 points or more on the combined practice=and-theory tests will have satisfied the course makers' criterion for a participant's success.
Case study no.6 generates a lot of questions as well as a lot of answers when it is given in the Think Tank workshop. Discussions, questions and answers tend to run along the following lines. As Joseph D Doe, most Think Tank workshop participants are satisfied — at first glance — with the end results. One hundred per cent of the Buy-It-Here engineers have scored 70 points or more on the final test: 20% the perfect score, 46% 90 points and 34% 70 points. With one exception (an engineer in group B), there is no indication in the pre-test results that existing knowledge of and skill in the systematic troubleshooting procedure, which the course set out to teach, contributed to these outcomes.
A condition for trusting the validity of the apparent high success rate is that both the pre- and post-tests were comprehensive enough to test everything the course intended to teach. Joseph D Doe would need assurance of this from some test specialist (with appropriate technical knowledge of the troubleshooting system being taught) from outside the course. 20% of the Buy-It-Here service engineers have a perfect score. What should one think about that? One should be suspicious of such results until there is the assurance (as hinted at above) that the test is comprehensive and therefore valid. But one should not be surprised once the result is known to be valid. Short, practical courses which have been developmentally tested (with the help of the jury of eight, for example) before installation should be expected to deliver top-end performance results. Joseph D Doe should have no hesitation in recommending the course when the appropriate test conditions exist.
What Joseph D Doe should be interested in is the apparent difference in performance between group B and groups A and C. The service engineer with the high pre-test score would certainly be worth talking to. What sort of things might account for this group difference? 'Installation problems' and 'motivation problems' are both raised as possible causes during our discussions on this point in the Think Tank workshop. Such might well be the problem. Both motivation and lack of motivation (possibly on the part of the engineer with the high pre-test score) are infectious! Especially so in a course with a sharp end performance goal.
The long, hot summer continues. Joseph D Doe relaxes. From time to time he talks with his chief service engineer about the results of the troubleshooting course. As he listens to the most recent report, a contented smile comes over his face. At that moment the telephone rings.
A complaining voice crackles in his left ear...
Not everything that shines can shine like gold. Even after an effective and efficient training programme, you still have to keep an eye on things!
[By the way, what had the designer of the troubleshooting course used as the REO — the system of stimuli which generated the responses needed for learning? If you have some knowledge of condensers, evaporators, thermal expansion valves and compressor electrical things, what do you think the designer put in the databank (referent 2) when thinking up the plan, structure and strategy of instruction for this course? If you have some technical knowledge, try to fill a referent 2 for yourself.]
In case study no. 6, a performance test, multiple-choice questions and oral questions were mentioned as being used to test the knowledge and skill of the 15 engineers. Essay tests, short answer tests, anecdotal records, rating scales, true—false items and matching items are also used to test what an individual has learned or not learned as a result of the design that has been used. So too are such activities as presentations; solving, carrying out an experiment successfully, writing a poem, making a video product — any activity, in fact, which is appropriate for indicating the individual student's achievement of the intended learning goal.
In practice, it is usually not possible to use a pre-test. You generally have to work only with post-test results, especially when the number of students is large, and it is the teacher who has to prepare, administer, and grade the pre- and post-tests. Instead, you must rely upon the accuracy of your 'Go' decision, needs analysis and needs specification. You must work under the assumption that the students' 'entering profile' does not contribute significantly to the end result. If it does, they don't need to be on the course.
In addition to understanding the didactic effect of your design, you also need to know how each student experienced the design.