Writing. Media. Social Change.

Posts tagged ‘multimodal’

Forthcoming. Making Space: Writing Instruction, Infrastructure, and Multiliteracies.

This  video project tells the story behind the building of a multimedia production classroom, our second classroom to date. Following the space from its initial proposal stage (Fall 2012) through to completion (Spring 2013), the project provides a detailed calendar view through the months as the project (and our design philosophy) develops. While the video is currently in production; you can view the trailer here:

Design Philosophy for a Multimodal Composition Classroom from Aesthetically on Vimeo.

1 Comment

Kuhn, V., Johnson, D. J., & Lopez, D. (2010). Speaking with Students: Profiles in Digital Pedagogy. Kairos: A Journal of Rhetoric, Technology, and Pedagogy, 14(2).

This webtext features a dozen video-profiles of students in University of Southern California’s Institute for Multimedia Literacy (IML) honors program. All students in the program complete scholarly multimedia thesis projects in their respective major. These snapshots feature students reflecting on their thesis projects in digital media, which demonstrate a range of student work in new media including SecondLife, Flash, and the open source program Sophie. In making this project, Kuhn accounts for the aesthetic sensibilities of each student’s video profile in order to protect the integrity of the work: “In short, the look and feel, or the tone of the project should translate from thesis project to thesis profile.”

Videos show students discussing their projects and the heuristic by which their thesis projects were evaluated:

assessment multimodalThis webtext argues for the importance of creating a lexicon to assess digital work. Not just the domain of the “bean counters,” rigorous new media assessments can help students, academics and administrators understand the “nuances and sophistication” of new media compositions.

Unfortunately, I found this Flash-based webtext somewhat maddening. I accidentally closed it numerous times.
Note: The heuristic above was introduced to digital writing studies as an assessment method in Kuhn’s 2008 Kairos webtext, ‘‘The Components of Scholarly Multimedia at http://kairos.technorhetoric.net/12.3/topoi/gallery/index.html
1 Comment
Madeline Sorapure. “Between Modes: Assessing New Media Compositions,” Kairos: A Journal of Rhetoric, Technology, and Pedagogy 10.2 (2005)
Sorapure’s webtext contributes significantly to discussions of new media assessment. As students are frequently assigned an array of new media projects including websites, blogs, images, videos, audio projects, flash projects, etc. Sorapure argues that we need a broad rhetorical approach to assessment, one that can speak to the multimodal aspects of composition.She emphasizes the need for “new lenses” so that we don’t “lose the chance to see new values emerging in the new medium.” In efforts not to limit new media works, Sorapure draws from the tropes of metaphor and metonymy to understand how meaning “emerges from” multimodal works. Drawing on Roman Jakobson’s (1956) essay “Two Aspects of Language and Two Types of Aphasic Disturbances,” Sorapure claims that metaphor and metonmy name “two different forces at work in the production of meaning.”
Metaphor designates a relation based on substitution; in a multimodal work, one mode can metaphorically represent or stand in for another, as when an animation of a word dynamically represents its meaning. It is a relation based on similarity between elements in different modes.

Metonymy designates a relation based on combination; modes can be metonymically related when they are linked by an association, as when lines from a poem are combined with a melody from a song. It is a relation based on contiguity between elements in different modes. 

Sorapure shares various student examples to illustrate how she uses the tropes in assessing multimodal compositions. She carefully demonstrates how metaphor and metonymy activate strong or weak relations between visual and verbal modes. (For example, if the composition lacks metaphor…it falls flat.)

While “text, sound, and image  each add their own part to the meaning….” it is also crucial to look at the relations between modes -”because metaphor and metonymy designate relations between two or more entities, they can be used to describe the relations between modes.

“Metaphor and metonymy provide a language with which to talk to our students about how the different modes in their projects come together to make meaning.”
I, too, think that the relation between modes is key to meaning making. I wonder, though. Do text, sound, image, each “add their part to the meaning.” Does meaning “emerge” from text, image, sound as Sorapure posits? Or is it made by the audience’s confrontation with it? This may seem like a small semantics thing, but it’s the kind of thing that keeps me up at night.
1 Comment

Cheryl Ball. “Assessing Scholarly Multimedia: A Rhetorical Genre Studies Approach,” Technical Communication Quarterly 21: 61-77, 2012

I appreciate the frank, personable, helpful tone of this article. It sounds very much like Ball’s voice, sitting down and having a chat with the reader about multimodal composition, sharing some interesting work by others, telling stories about her own classroom experience assigning and assessing webtexts, creating rubrics with students, what works, what doesn’t. Her experience comes across at every turn – her ethos is everywhere.  And that makes the article highly readable and engaging.

She draws from her familiarity and expertise with assessing webtexts (scholarly multimedia compositions) in her capacity as editor for the journal Kairos: A Journal of Rhetoric, Technology, and Pedagogy. The purpose of the article is to explore

“how to ask students to compose scholarly multimedia and how to assess their work (63).”

She explores these issues through the lens of her own experience teaching numerous iterations of a course in Multimodal Composition. Students in the course complete a webtext project (scaffolded through a series of assignments [64]), which could be submitted to a digital journal, like Kairos, C&C Online, X= Changes, or The JUMP (Journal for Undergraduate Multimedia Projects). After serious discussion, she describes how her class arrived at appropriate evaluation measures for their projects, which they refer to as Kuhn + 2. These involve:

  • Conceptual Core
  • Research Component
  • Form and Content
  • Creative Realization


  • Audience
  • Timeliness

*See review of Kuhn and pages 65-68 for more details

Ball warns (and rightly so) that this heuristic should not be adopted blindly by teachers of multimodal composition. She explains that this rubric serves her purpose for one particular situation, for one audience, at one point in time (63). Furthermore she emphasizes that rubrics need “to be created fresh” (68), meaning that Kuhn + 2 is not one size fits all. Importantly, Ball emphasizes the importance of making our own, to meet the task at hand, with student input.

In this article Ball also offers some sage advice to webtext authors regarding design concepts, especially how form/content work together: “(a) Your design should enact your argument, and (b) To come up with that design, think of a visual metaphor for your argument (68). *See Susan Delagrange’s 2009 Wunderkammer piece for its exemplarily form and content.

She later notes: “…students should be articulating their design choices (form/content relationship) as rhetorical, aesthetic, technological, and other choices that make sense for the conceptual core of a piece given the medium they have chosen to best present their concept” (70). Although Ball doesn’t explicitly state what an “aesthetic choice” is or could be, I am happy to see it mentioned here. I see is it as part of my own project to explore what we mean when we speak of the aesthetic aspects of multimodal composition.

Leave a comment

Jody Shipka. Negotiating Rhetorical, Material, Methodological, and Technological Difference: Evaluating Multimodal Designs. College Composition and Communication 2005

What do teachers of multimodal composition want? To establish frameworks for responding to student work that take into account the complexities of new media. Or so says Jody Shipka. This is one of many articles I’ve recently read dealing with the creation of heuristics for evaluating multimodal scholarship. Shipka observes:  “That we need to begin articulating and sharing with others our strategies for responding to the “differently shaped products” (Takayoshi 136) students are increasingly invited to produce is evidenced in the dearth of scholarship devoted to the assessment of multimodal and new media texts. (346).

“If we are committed to providing students with opportunities to forge new connections, to work in new ways, to produce new kinds of texts, and to become increasingly cognizant of the ways texts provide shape for and take shape from the contexts in which they are produced, circulated, valued, and responded to, I think it is crucial that we take care not to limit the texts, tools, and composing strategies students might employ and alter in compelling ways.” (348)

In the article, she also describes her own heuristic, which involves asking students to assume responsibility for describing and evaluating the goals and purposes of their work.

Leave a comment

Kress, Gunther R., and Theo Van Leeuwen. Multimodal Discourse: The Modes and Media of Contemporary Communication. London, New York: Arnold; Oxford University Press, 2001.

Kress and Leeuwen’s Multimodal Discourse outlines a “theory of communication” for interactive multimedia. As we move from a mono-modal to multimodal culture lines are blurred between modes and media of communication. The authors claim that what is needed is a theory which describes what happens in sites of practice: for example, when designers freely move between different modes and media. So, the question the authors ask throughout the book is: how do people use communicative modes and media in actual, concrete, interactive instances of communicative practice? Also of note is the authors definition of mode: “a mode is that material resource which is used in recognisably stable ways as a means of articulating discourse” (25); a mode is the abstract organization of specific material drawn into semiosis” (27). In their view modes always have meaning where some media actually contribute no meaning to the text. (!)

While this book doesn’t deal explicitly with the aesthetic, it does devise a theory of discourse in which color plays a role equal to language. The book opens the door for these sometimes overlooked, sometimes considered intangible modes of meaning, such as color and affect.  The authors state: “In our view, pleasure (or un-pleasures) are always (though not always to the same extent) attached to meanings, and a vital aspect of communication. Communication never just ‘communicates’, ‘represents,’ and ‘expresses’, it also always and at the same time affects us. The two cannot be separated. Even when communication seeks to do the opposite, the very fact of negating materiality affects us–by failing to engage us affectively” (71). Thus, for my purposes, the most important aspect of this book is how the authors stress that “meaning is made in many different ways, always in the different modes and media which are co-present in a communicational ensemble” (110). The key point here is that meaning is made in a multiplicity of modes and media AND meaning occurs at different places within these. They stress that in every mode of the multimodal, there is communicative “work” being done, with all the available representational forms and such work is always meaningful.