evaluation

Learning Objectives

When you have completed this session, you should be able to

• list the criteria for evaluating educational multimedia,
• do an evaluation with respect to your own pedagogical aims.

A good rule of thumb for curriculum design is to aim at being idea-based, not media-based. Every good teacher has found this out. Media can sometimes support the learning of ideas, but often the best solutions are found by thinking about how the ideas could be taught with no supporting media at all. Using what children know, can do, and are often works best. After some good approaches have been found, then there might be some helpful media ideas as well. (Alan Kay, 1996)

It is not easy to devise a list of evaluation criteria to judge the quality of educational multimedia, but it is probably true to suggest that even if a particular solution meets all the technical requirements, it is not certain that it will also satisfy our pedagogical aims .

To determine the pedagogical value of educational multimedia may be as difficult as explaining why do we like, or why we do not like, a painting. Some important points for evaluation are listed below, but this list contains only the "necessary" conditions, or requirements, which may not always be sufficient to help decide whether there is a proper application for multimedia or not. These are also minimum requirements, applicable to all educational multimedia.

An evaluation must take into consideration aspects of general educational content, which can actually be satisfied by textbooks or other more traditional materials used for educational aims. Multimedia goes well beyond this set of content, including user software which has to be examined for conformity and generality. THere also has to be attention paid to the overall presentation, i.e. the consistency of media elements within the overall multimedia show.

Pedagogy, didactics, psychology

At a minimum an evaluation of electronic teaching material must consider the following.

• The structure and the actual content must comply with the set aims, it should be able to be adapted to the learner’s individual learning style, and it should allow the learner to plan the overall learning activity independently (including making it possible to skip certain units).
• The material should attract and maintain the learner’s interest, it should be interactive and should utilize the presentation opportunities of the computer without shifting the emphasis from the content to the way of presentation,
• The material should offer opportunities for practice, through examples, should opportunity for self-assessment, it should motivate by rewarding correct answers, and it should analyze and evaluate the learner’s results at certain times

Assessment criteria

• Does the teaching material meet the set of learning objectives?
• Does it fulfil the target group’s expectations?
• Does it maintain the learner’s interest in the material, i.e. is the principle of maintaining attention achieved?
• Is the principle of patient waiting/expectation achieved?
• Is the principle of confirmation achieved?

Ergonomy

The ergonomics of human behaviour, abilities, limits and other human characteristics should be taken into account when designing tools, machines, systems, work tasks, work environments in order to achieve efficient operation, and to provide a safe and convenient way of application. This is true also of multimedia.

It should be simple to use, should ensure easy navigation, and the layout and the use of icons should be logical. The images and colours used should support and not hinder the handling of the material. The proportion of pictures and texts should be well-balanced and font sizes should be selected to make the text easy to read

Assessment criteria

• Image layout, general impression, originality of image design.
• User friendly (adjusted to the age group) work environment.
• The quality and systematic layout of the navigation elements.
• Occurence of errors.
• The simplicity of the instructions to use (how memory consuming is it to learn).
• Incidents of exhaustion, tension, frustration during usage (appropriate setting of action-reaction time, waiting time).

Media elements

Since media elements affect all the previously set requirements, it is practical to highlight the criteria governing their applications. A basic principle is that media elements and sound effects should be used moderately and only when justifiably used in electronic teaching materials.

• The length of the used video clips should be max. 1-1,5 min, and they should really contain extra information
• Animation should be used only in justified cases, and they should not be too fast
• Sound quality should be appropriate, the narrator’s voice and speed of speech should be comfortable, and any text should be clear

Assessment criteria concerning media elements

Texts

• Simplicity
• Legibility
• Clear structure
• Conciseness
• Eye-friendly image

Symbols-logos

• Simplicity, clarity
• Aesthetic apparence
• Relevance to the symbolized object, phenomenon
• How much do they promote to highlight the main points

Audio materials

• Coherent integration (sg. relevant in the right place)
• Quality, integration of narration
• Originality, appropriate application of background music

Images

• Coherent integration (sg. relevant in the right place)
• Colours, colour combinations
• Quality of images
• Optimalization ( size, quality)
• Quality of figures

Videos

• Coherent integration (sg. relevant in the right place)
• Quality of the video clips
• Optimalization ( size, quality)

Animations

• Coherent integration (sg. relevant in the right place)
• Dynamism ( quick, well-balanced, slow)
• Promoting understanding, drawing learner’s attention
• Graphics

Summary

Creating a textbook requires considerable technical expertise to compile comprehensive, high quality material. Using the right amount of high quality illustrations, selecting the right letter types etc. are all important considerations in the process. However, educational multimedia has to harmonize many more elements. In many cases the material may look really attractive, but the content that the information the creator intended to convey is lost, simply because the multimedia elements are not co-ordinated well and the because emphasis is shifted to unimportant information.

 Exercises

Choose one of the multimedia elements that you have already used in the classroom, or if you have never used any find one on the Internet appropriate to your subject. Evalute it according the criteria listed above and publish the results in your learning diary.

References

Alan Kay: Revealing the Elephant: The Use and Misuse of Computers in Education, Educom review, 1996. (http://net.educause.edu/apps/er/review/reviewArticles/31422.html)