Medium

Context (What else is this connected to?)

Medium

0.0 MARY: Media are material raw materials for meaning, the resources of our sound-making bodies that become sound waves in the form of speech, the capacities of bodies to gesture, and the objects, spaces and images that we make or find already-made and put to meaningful use.

The resources for meaning are material artifacts. There could be no representation, communication or interpretation without the mediation of artifacts.

Now, you might think, we come to a strange twist in our argument. We have been saying that speech, sound, body, space, image, and text are forms, on the horizontal axis of our grammatical schema. Now we’ve put them on our vertical, or function axis. The reason we have done this is because these forms have functional aspects as well, kinds of meaning function that are typical of each form.

Psychologist James Gibson coined the word “affordance” to describe the possibilities for action that environments offer to any animal. Like any animal, we can only mean what can be materialized. The very materiality of their media shapes what is doable, sayble, thinkable, and meaningable. In a functional sense, this is somewhat different in each medium.

Reference: Kalantzis, Mary and Bill Cope, 2020, Adding Sense: Context and Interest in a Grammar of Multimodal Meaning, Cambridge UK, Cambridge University Press, pp. 90-94.

1.41 MARY: Read this: “The mountains loomed large.” What springs to mind as your mental image? It can’t quite have been this…

1.59 MARY: These are the mountains of the Peloponnese, and I was born in the house you can see on the first ridge. When I said “the mountains loomed large,” this is what I meant. It’s the same meaning as the words, but it’s different as well.

2.16 MARY: Speech happens in sound, and sound presents across time. Speech is good at representing time, but it only represents space awkwardly. Prepositions like “near” and “behind” are notoriously vague.

2.29 MARY: This is why I want to show you a photo of the place of my birth, and not just talk about it. This picture has an enormous amount of spatial detail that it is nearly impossible to represent so faithfully in speech. Image is good at representing space.

But image is not very good at time. I had to tell you that I was born in this house. This is why we live our lives multimodality, because the functional affordances of different media are different, and to make a fuller meaning we need to supplement one form with another.

Reference: Cope, Bill and Mary Kalantzis, 2020, Making Sense: Reference, Agency and Structure in a Grammar of Multimodal Meaning, Cambridge UK, Cambridge University Press, pp. 12-13.

3.04 MARY: So here, in this table, is a rough map of different media. In a functional sense, media affect meanability. The offer openings for meaning as well constraints peculiar to each medium. This is why we mix media, why we mean multimodally.

Reference: Kalantzis, Mary and Bill Cope, 2020, Adding Sense: Context and Interest in a Grammar of Multimodal Meaning, Cambridge UK, Cambridge University Press, p. 91.

3.23 BILL: Our meanings are made in the world through the material practices of sight, smell and taste, touch and hearing. Perhaps the most remarkable proof the power of multimodality is that we are able to mean in fully human ways without some of these senses. Without hearing, people have developed sign languages that are as comprehensive and powerful as sounded speech. Without sight, Braille represents text in touch. This is further proof of the extraordinary transposability of meanings.

Reference: Kalantzis, Mary and Bill Cope, 2020, Adding Sense: Context and Interest in a Grammar of Multimodal Meaning, Cambridge UK, Cambridge University Press, p. 133.

3.52 BILL: Here is another one of our rough maps of the senses used for each of the forms of meaning. The interesting thing is how radically different text is from speech. Text only uses sight, and is in its form and media very closely related to image. Speech only uses sound – though of course, in reality, these things almost always happen in multimodal transposition.

The very interesting thing is that space, object and body are multimodal by nature. Only text and speech are so very different, another argument we have against the aggregating category, “language.” Of course, we can transcribe speech into text and we can read text aloud, but these are transpositions greater than any others across the forms of human meaning.

Reference: Kalantzis, Mary and Bill Cope, 2020, Adding Sense: Context and Interest in a Grammar of Multimodal Meaning, Cambridge UK, Cambridge University Press, pp. 13-15.