Modality Principle

Mayer, R.E. (Ed.) (2005). Cambridge Handbook of Multimedia Learning. New York: Cambridge. [Chapters 9]


Modality effect occurs when information presented in both auditory and visual mode is more effective than when it is presented in one mode only.


Cognitive load theory and modality effect
Instructions creates extraneous load ad become ineffective if they ignore the limitations of working memory.
Modality effect derives from Baddeley's (1986, 1992, 1999) working memory model: central executive with two slave subsystems, phonological loop- that processes auditory information and visual-sketchpad - that processes visual information. 
If there are two channels for information to be processed, then presenting information in two modes is more effective than presenting in one mode only.
- visual and auditory information are independently processed so they do not interfere with each other.


Research on Modality Principle
Many research done that provides evidence of two working memory systems, one for visual information processing and one for auditory information processing.
Penney (1980) suggested if auditory and visual information are processed separately, then one would prefer to recall information by mode of presentation  when presented in mixed modes and proved that processing items in working memory might be organized according to modality.


Split-Attention effect occurs  when the information must be processed simultaneously in time and space. When it occurs, working memory has to deal with integrating the different sources of information by doing extraneous search and that causes extraneous overload.
        - Instructional modality effect occurs when a dual mode presentation is superior to the visual only split attention 
         presentation.
Modality effect occurs only for high element interactivity materials, consisting of elements that have to be processed simultaneously in working memory, creating intrinsic overload.


Expertise reversal effect occurs when the instructions designed for the novice learners become redundant for the advanced learners. Based on this, modality principle may not be true for advanced learners- dual mode instructions may become redundant for advanced learners.