Pioneering English educator and researcher Gunther Kress presents a semiotic theory of literacy to challenge the traditional linguistic theories that have dominated previous scholarship but that fail to address the multimodal realities of today:
One might say the following with some confidence. Language-as-speech will remain the major mode of communication; language-as-writing will increasingly be displaced by image in many domains of public communication, though writing will remain the preferred mode of the political and cultural elites. The combined effects on writing of the dominance of the mode of image and of the medium of screen will produce deep changes in the forms and functions of writing. This in turn will have profound effects on human, cognitive/affective, cultural and bodily engagement with the world, and on the forms and shapes of knowledge. The world told is a different world to the world shown. The effects of the move to the screen as the major medium of communication will produce far-reaching shifts in relations of power, and not just in the sphere of communication. Where significant changes to distribution of power threaten, there will be fierce resistance by those who presently hold power, so that predictions about the democratic potentials and effects of the new information and communication technologies have to be seen in the light of inevitable struggles over power yet to come. It is already clear that the effects of the two changes taken together will have the widest imaginable political, economic, social, cultural, conceptual/cognitive and epistemological consequences. …
The world of communication is not standing still. The communicational world of children now in school is both utterly unremarkable to them and yet it looks entirely different to that which the school still imagines and for which it still, hesitantly and ever more insecurely, attempts to prepare them. All of us already inhabit that new world. Some of us still use the older forms of communication and at the same time have become comfortable enough with many of the possibilities of the newer forms of communicating on paper or on the screen—not fully realising and yet at the same time uncomfortably aware of the profound changes that are taking place around us. We no longer regard it as unusual that we can change fonts in mid-text, that we can embolden the typeface or italicise it, and all with next to no effort.
Of course such changes make only a small difference to the meaning of our ‘written’ texts. Layout, on the other hand, also very readily manipulated now, does change the deeper meanings of the text. It matters whether I put my ideas smoothly flowing along the lines of the page, or whether I present them to you as bullet-points:
- The ‘force’ and
- the ‘feel’ of the text have changed. It has become more insistent,
- more urgent,
- more official. It is now about
- presenting information.
Layout is beginning to change textual structures; that much is clear. With such changes—which may seem superficial—come others, which change not only the deeper meanings of textual forms but also the structures of ideas, of conceptual arrangements, and of the structures of our knowledge. Such seemingly superficial changes are altering the very channels in which we think. Bullet points are, as their name suggests, bullets of information. They are ‘fired’ at us. …
[W]e can no longer treat literacy (or ‘language’) as the sole, the main, let alone the major means for representation and communication. Other modes are there as well, and in many environments where writing occurs these other modes may be more prominent and more significant. …[L]anguage and literacy now have to be seen as partial bearers of meaning only.
There is a consequence for notions of meaning: if the meaning of a message is realised, ‘spread across’, several modes, we need to know on what basis this spreading happens, what principles are at work. Equally, in reading, we need now to gather meaning from all the modes which are co-present in a text, and new principles of reading will be at work. Making meaning in writing and making meaning in reading both have to be newly thought about. ...
Here I will outline some elements of such a theory of literacy; it cannot be complete, but it may provide some useful tools. This theory, as I said, cannot be a linguistic theory. The modes which occur, together with the language-modes of speech and writing, on pages or screens, are constituted on different principles to those of language; their materiality is different; and the work that cultures have done with them has differed also. The theoretical change is from linguistics to semiotics—from a theory that accounted for language alone to a theory that can account equally well for gesture, speech, image, writing, 3D objects, colour, music and no doubt others. …
In the era of the screen and of multimodality some fundamental changes are inevitable as far as forms, functions and uses of writing are concerned. Maybe first and foremost there is the question of how the modes of image and writing appear together, how they are designed to appear together and how they are to be read together. There is the question then—a real question—in what direction writing is likely to move: will it move back towards speech-like forms, and become mere transcription of speech again, or will it move back in the direction of its image origins? And there is the old question of the resources of the mode of writing … .
The current landscape of communication can be characterised by the metaphor of the move from telling the world to showing the world. The metaphor points to a profound change in the act of reading, which can be characterised by the phrases 'reading as interpreting’ and 'reading as ordering’. The metaphor and the two phrases allow us to explore the questions that reading poses—narrowly as 'getting meaning from a written text’, and widely as 'making sense of the world around me’—through a new lens. Both senses of reading rest on the idea of reading as sign-making. The signs that are made by readers in their reading draw on what there is to be read. They draw on the shape of the cultural world of representation, and on the reader’s prior training in how and what to read. New forms of reading, when texts show the world rather than tell the world have consequences for the relations between makers and remakers of meaning (writers and readers, image-makers and viewers). In this it is important to focus on materiality, on the materiality of the bodily senses that are engaged in reading—hearing (as in speech), sight (as in reading and viewing), touch (as in the feel of Braille)—and on the materiality of the means for making the representations that are to be 'read’—graphic stuff such as letters or ideograms, sound as in speech, movement as in gesture.
… [T]he increasingly and insistently more multimodal forms of contemporary texts make it essential to rethink our notions of what reading is.
Kress, Gunther. Literacy in the New Media Age. London: Routledge, 2003, pp. 1, 16, 35, 61, 140-141. || Amazon || WorldCat