Some Contributions of Cognitive Psychology to Pedagogical Uses of Images

The beneficial effect of images on the understanding of a message – “A picture is worth a thousand words” – is widely accepted and established. Sociologists, educationalists and cognitivists dedicated a vast range of research to this subject since the beginning of the 1980s.

1 – Associating texts and images in teaching materials facilitates memorization and understanding

A number of empirical studies in cognitive sciences support this observation.

Cognitivists first came up with typologies of texts (Brewer, 1980; Adam, 1987) of which they studied how they were memorized and understood (Denhière & Baudet, 1992; Van Dijk & Kintsch, 1983). They carried out similar studies on images: proposal of typologies (Evans, Watson & Willows, 1987), study of the memorization (Paivio, 1986) and of understanding (Denis, 1989).

Together these sets of research paved the way for the study of the comprehension of illustrated texts (Glenberg & Langston, 1992), which showed that adding iconographic information encourages readers to build richer mental representations, thus enhancing comprehension (Johnson-Laird, 1983).

2 – Beneficial effect of images varies with the way images are associated with texts

More recently, the research field has been extended to the understanding of multimedia information (Mayer, 1997).

Positive multimedia effects include modality and contiguity effects.

• The modality effect consists in observing higher levels of memorization and understanding when texts are presented aurally (via speakerphones or headphones) and associated with illustrations or animated images (Mayer & Moreno, 2002; Moreno & Mayer, 2002).
• The contiguity effect consists in better performances in memorization and comprehension when texts and illustrations are presented in close areas on the screen (spatial contiguity) and/or when they are presented simultaneously (temporal contiguity; Moreno & Mayer, 1999).
• The effect of redundancy consists in observing similar gains in memorization and comprehension when texts and illustrations deal with identical notions (Moreno & Mayer, 2002).

However, a negative multimedia effect has been reported: when texts and illustrations are presented in the same (visual) mode, memorization and comprehension are impaired. This impairment is interpreted in terms of a split attention effect as well as a wider cognitive load. Processing the various information displayed on the screen partitions the cognitive resources off, and increases the cognitive effort (Tindall-Ford, Chandler & Sweller, 1997).

3 – The positive effect of images also depends on individual characteristics

The impact of individual differences on multimedia effects has been studied and described extensively. These works highlight the influence of two factors: prior knowledge and specific mental abilities (spatial vs verbal).

It has been broadly reported that images are beneficial for novice learners, while they have little or no impact for learners with high prior knowledge (Mayer & Gallini, 1990).

In addition, it was observed that learners having high levels of intellectual mental abilities are also the ones who benefit the most from text-image associations. For instance, in England, Hegarty & Steinhoff (1997) demonstrated that only learners having a high level of spatial skills take advantage of text-image associations. Presenting illustration did not increase the performances of learners with a lower level of the same skills. More recently, in France, Dubois, Gyselinck & Choplin (2003) indicate that students with good visual and spatial memory abilities take advantage of animated images included in multimedia sequences, while students with low visual and spatial memory abilities do not benefit from animated images (see also Gyselinck, Jamet & Dubois, 2008).

4 – How to leverage this knowledge as a teacher when preparing lectures ?

Ergonomists and communication specialists used the results of empiric research mentioned above. They formulated recommendations aimed at better adapting teaching supports to the characteristics of learners.

From the study of multimedia effects, Dubois (2004) drew ergonomic recommendations to limit the cognitive load imposed to students. Three axes are suggested: to present texts and illustrations simultaneously but in distinct sensorial modes; to gather the illustrations in the same area of the screen to facilitate their analysis; to personalize messages in order to maintain and strengthen the students’ attention and motivation.

Clark & Lyons (2011) and Dirksen (2016) came up with an additional number of recommendations for more efficient teaching supports. Their purpose is to help students focussing and harnessing prior knowledge, to alleviate the cognitive load and to encourage the elaboration of richer mental representations, as well as to facilitate knowledge acquisition and/or transfer combined with new skills acquisition.

Najjar (1996, 1997, 1998) contributed significantly to cognitive ergonomics of multimedia design, via the formulation of principles aiming at aligning the characteristics of learners, the knowledge to acquire and textual and iconographic information.

Unfortunately, these works remain poorly known within teaching communities. A better promotion among high education institutions should however contribute to improve the quality of teaching supports as well as students achievement levels.

Véronique DUBOIS-BOUCHET (Phd), Pedagogical engineer, Active Pedagogy Lab, Studies and Pedagogical Innovation Division, Sciences Po.

Bibliography

• Adam J.M. (1987). Textualité et séquentialité. L’exemple de la description. Langue Française, 74, 51-72.
• Brewer, W. F. (1980).  Literary theory, rhetoric, stylistics: Implications for psychology.  In R.J. Spiro, B.C. Bruce, W.F. Brewer (Eds.), Theoretical issues in reading comprehension (pp. 221-239). Hillsdale NJ: Erlbaum.
• Denhière G., Baudet S. – (1992) Les représentations mentales et le traitement du texte, in G. Denhière, S. Baudet (Eds.),  Lecture, compréhension de textes et science cognitive, Paris: Presses Universitaires de France, 101-158.
• Clark R.C., Lyons C. (2011). Graphics for learning: proven guidelines for planning, designing, and evaluating visuals in training materials. San Francisco: Pfeiffer.
• Denis M. – (1989) Image et cognition, Paris: Presses Universitaires de France.
• Dirksen J. (2016). Design for how people learn. San Francisco: New Riders.
• Dubois V. (2004). Intégration cognitive d’informations multimédia et mémoire de travail. Thèse de doctorat de Psychologie Cognitive, sous la direction de M.-F. Erhlich. Paris: Université René Descartes Paris V, Ecole Pratique des Hautes Etudes. 270 pp.
• Dubois V., Gyselinck V., Choplin H. (2003). Multimodalité et mémoire de travail. In C. Desmoulins, P. Marquet, D. Bouhineau (Eds.), Environnements Informatiques pour l’Apprentissage Humain (pp. 187-197). Paris: ATIEF/INRP.
• Evans M.A., Watson C., Willows D.M. (1987). A naturalistic inquiry into illustrations in intructional textbooks. In H.A. Houghton, D.M. Willows (Eds.), The psychology of illustration, instructional issues (Vol. 2, pp. 86-155). Springer-Verlag.
• Glenberg A.M., Langston W.E. (1992). Comprehension of illustrated text: Pictures help to build mental models. Journal of Memory and Language, 31, 129-151.
• Gyselinck V., Jamet E., Dubois V. (2008). The role of working memory components in multimedia learning. Applied Cognitive Psychology, 22(3), 353-374.
• Hegarty M., Steinhoff K. (1997). Individual differences in use of diagram as external memory in mechanical reasoning. Learning and Individual Differences, 9(1), 19-42.
• Johnson-Laird P.N. (1983). Mental models. Cambridge: Cambridge University Press.
• Mayer R.E. (1997). Multimedia learning: Are we asking the right questions? Educational Psychologist, 32(1), 1-19.
• Mayer R.E., Gallini J.K. (1990). When is an illustration worth ten thousand words? Journal of Educational Psychology, 82, 715-726.
• Mayer R.E., Moreno R. (2002). Aids to computer-based learning. Learning and Instruction, 12, 107-119.
• Moreno R., Mayer R.E. (1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology, 91(2), 358-368.
• Moreno R., Mayer R.E. (2002). Verbal redundancy in multimedia learning: When reading helps listening. Journal of Educational Psychology, 94(1), 156-163.
• Najjar L. (1996). Multimedia information and learning. Journal of Educational Multimedia and Hypermedia, 5, 129-150.
• Najjar L. (1997). A framework for learning from the media: The effects of materials, tasks, and tests on performance. GIT-GVU-97-21 technical report. Atlanta: Georgia Institute of Technology, Graphics, Visualization and Usability Center.
• Najjar L. (1998). Principles of educational multimedia user interface design. Human Factors, 40(2), 311-323.
• Paivio A. (1986). Mental representation: A dual coding approach. Oxford: Oxford University Press.
• Tindall-Ford S., Chandler P., Sweller J. (1997). When two sensory modes are better than one. Journal of Experimental Psychology: Applied, 3(4), 257-287.
• Van Dijk T.A., Kintsch W. (1983). Strategies of discourse comprehension. New York: Academic Press.