Kress, Gunther R., and Theo Van Leeuwen. Multimodal Discourse: The Modes and Media of Contemporary Communication. London, New York: Arnold; Oxford University Press, 2001.
Kress and Leeuwen’s Multimodal Discourse outlines a “theory of communication” for interactive multimedia. As we move from a mono-modal to multimodal culture lines are blurred between modes and media of communication. The authors claim that what is needed is a theory which describes what happens in sites of practice: for example, when designers freely move between different modes and media. So, the question the authors ask throughout the book is: how do people use communicative modes and media in actual, concrete, interactive instances of communicative practice? Also of note is the authors definition of mode: “a mode is that material resource which is used in recognisably stable ways as a means of articulating discourse” (25); a mode is the abstract organization of specific material drawn into semiosis” (27). In their view modes always have meaning where some media actually contribute no meaning to the text. (!)
While this book doesn’t deal explicitly with the aesthetic, it does devise a theory of discourse in which color plays a role equal to language. The book opens the door for these sometimes overlooked, sometimes considered intangible modes of meaning, such as color and affect. The authors state: “In our view, pleasure (or un-pleasures) are always (though not always to the same extent) attached to meanings, and a vital aspect of communication. Communication never just ‘communicates’, ‘represents,’ and ‘expresses’, it also always and at the same time affects us. The two cannot be separated. Even when communication seeks to do the opposite, the very fact of negating materiality affects us–by failing to engage us affectively” (71). Thus, for my purposes, the most important aspect of this book is how the authors stress that “meaning is made in many different ways, always in the different modes and media which are co-present in a communicational ensemble” (110). The key point here is that meaning is made in a multiplicity of modes and media AND meaning occurs at different places within these. They stress that in every mode of the multimodal, there is communicative “work” being done, with all the available representational forms and such work is always meaningful.