summative evaluation

The course in their third year of med school about the Brains and the Sensory system in which psychiatry participates is a course of at least three months. It’s made of different sub courses which each take about 3 to 4 weeks. These courses are evaluated at the end. That’s to say months after the course ended because the med students are preparing for their exams and teachers are already occupied with other teaching obligations. These evaluations are mostly done by the coordinating medical specialist, most of the teacher are not present during the evaluation. In conclusion: to little to late.

An evaluation after each week would be far more effective, will prevent the recency effect of evaluation after consecutive courses. The recency effect is the greater impact of the last course on the evaluation because of a fresher memory. Others have found a primacy effect: the first impression of a course being the strongest and long lasting and influencing the evaluation the most.

In a recent published study they used twitter for evaluation during the course and compared this evaluation with the evaluation after the whole course. The continuing evaluation is officially called formative evaluation and the evaluation at the end of the whole course is called the summative evaluation. They concluded

First, the formative evaluation did not come to the same results as the summative evaluation at the end of term, suggesting that formative evaluations tap into different aspects of course evaluation than summative evaluations do. Second, the results from an offline (i.e., paper-pencil) summative evaluation were identical with those from an online summative evaluation of the same course conducted a week later. Third, the formative evaluation did not influence the ratings of the summative evaluation at the end of the term.

How did they use twitter?
On twitter they had to answer two closed questions on a scale from 1 to 9 (very good to not at all interesting) and three open questions:‘‘How did you like the course today?’’, ‘‘How interesting was the course today?’’ the open questions: ‘‘What was good today?’’ ‘‘What was bad today?’’ ‘‘General comments=just wanted to tell you.’’
The students and teacher created an individual account with an anonymous, non personalized nickname. They were only allowed to use direct messages (DM). This way the students could not read the messages of their colleagues.

Remarkably did the formative evaluation differ from the summative without the authors being able to comprehend this difference other than that both methods measure something else. The finding that offline and online summative evaluation don’t differ has been published in several prior publications. The formative evaluation approach for student evaluations
does not seem to influence the summative evaluation.

Conclusion
Using DM on twitter is not very different from other methods such as email or text messaging but these two alternatives are easier to implement than twitter. Unfortunately this study didn’t measure the effects of formative evaluation on the quality of teaching. Formative evaluation has several advantages such as direct feedback to the teacher who can improve there lesson for the next group based on the evaluation, courses can be modified quickly based on formative evaluations, more active involvement of the med students this could raise their commitment to their study.

Stieger, S., & Burger, C. (2009). Let’s Go Formative: Continuous Student Ratings with Web 2.0 Application Twitter CyberPsychology & Behavior DOI: 10.1089/cpb.2009.0128