Communication + Creativity = Multimodality

 I was always an extremely unmotivated student. Since I can remember, tasks like studying for a test using a text book, writing academic papers, reading assigned texts, or other traditional learning methods plagued my primary and secondary education. I was consistently frustrated to tears and often called lazy, but I was the opposite of that. I was a voracious reader, was constantly curious - taking time to learn new and difficult things, and enjoyed creatively expressing myself (even though I wasn’t “good” at it). The rigidity and monotony of expected learning outcomes tailored to WKCE, ACT, and post-secondary targets failed to capture my brainpower (I am sitting here, proctoring the ACT at this very moment. It sucks for everyone in this room). This practice is documented by Jessi Thomsen as she argues that, “[s]tandardized testing, typically focused on alphabetic literacies, as perpetuated the subordination of image to word…in most classrooms, ‘the term literacy commonly refers only to reading and writing’” (p. 54). This, in fact, is not a comprehensive view of literacy and negates the power of multimodal learning composition.

However, I had experiences with multimodal composition that I remember feeling engaged and full of pride in the products I had created. An early memory of this is in elementary school while learning about the agricultural history of Native American tribes in different climates throughout the U.S. In addition to looking up information in the encyclopedia and writing about what we had found, we were also encouraged to find and prepare a traditional recipe utilizing the agricultural products. While I don’t necessarily remember the details of what I was assigned or researched, I have a very strong, fond memory of making misckquatash (succotash) with my grandmother. I have extremely negative memories of utilizing the encyclopedia to research information and of synthesizing that information into an informational text. The addition of a drawing included with the text representing the research, along with the research and execution of a recipe that was 1) culturally relevant, 2) appropriate for the agricultural landscape of the time, and 3) prepared together as multi-generational women, honoring the traditions of the culture we were learning about. The experience was culturally grounded, connected to place, and shared across generations. The written component of the assignment has long faded from memory, but the act of cooking, creating, and sharing knowledge remains vivid. What appeared to be a simple activity from the outside was actually a rich example of multimodal learning.

In my classes, students regularly engage in digital storytelling through video production, programming, and other forms of creative composition. Two forms of multimodal work that inspire me both as an educator and as a writer are student-created digital videos and interactive programming projects. Both allow students to construct meaning using multiple modes such as visual, textual, auditory, and interactive, rather than relying solely on traditional writing.

One example of this multimodal composition can be seen in a student project from my Introduction to Programming course. Before writing any code, students complete a design document where they plan the structure of their program, sketch the visual layout of the game, and explain the logic behind their design decisions. In the example included here, the student designed a simple game where a bee must avoid flowers while the player attempts to achieve a high score. The student begins by describing the concept and audience of the game, then sketches multiple screens including the instructions, gameplay interface, and game-over screen. These drawings include notes about animation, scoring systems, and player interaction.

 

From there, the student translates these ideas into working code using programming concepts such as variables, conditionals, and event-driven functions like onStep() and onKeyPress(). What becomes clear through this process is that programming is itself a form of composition. Students must plan, revise, structure ideas, and think about how their audience will interact with the program. In many ways, this mirrors the writing process as students are drafting ideas, organizing structure, revising functionality, and reflecting on the final product. Through code, visual design, and written explanations, the student is composing an experience rather than simply completing a technical task.

The live version of the project further illustrates how the student’s planning, visual design, and coding choices come together as a multimodal composing experience. Note, this is an early version of the game before the student added the world design. Since they graduated, I can no longer access their personal versions.

Another example of multimodal composition in my classroom comes from my digital video production course, where students develop short documentary-style videos around the theme “Making History.” Unlike the programming assignment, which focuses on interactive design, this project emphasizes storytelling, research, and civic reflection. The planning documents for this assignment demonstrate the extensive composing process students engage in before filming even begins.

In the example included here, the student is developing a short informational video titled The Story of Johnny Chang. The project explores the experiences of a former gang member who overcame trauma and now works to help others facing similar struggles. Through the planning documents, the student defines what “making history” means, brainstorms potential angles for the story, and develops a structured narrative including a hook, background context, key message, and call to action. The student also plans interview questions, possible visuals, audio elements, and research sources.

Even before filming begins, the student is engaging in complex literacy practices. They are synthesizing research, considering audience impact, and thinking carefully about how to communicate a message about trauma, healing, and community. As Chisholm and Trent explain, researchers have documented the ways multimodal literacy practices can “reinvent curricula in diverse classroom contexts” (308). Through projects like this, students draft and revise ideas, work across multiple forms of media, and begin to see themselves as authors whose work can reach authentic audiences (309).

One last example is the PSA project my Cybersecurity students do each year. At this point in the semester, we are deep into the weeds of cyber threats and students are really starting to make the connections between their technology use and all of the ways they are vulnerable. They feel the need to DO SOMETHING so this project fits in perfectly and sets them up to feel not only armed with knowledge, but empowered with it.

Students work along side a real-life Chief Information Officer of a local construction company to understand the intricacies of the human side of cybersecurity in order to better understand how to make their PSA videos impactful for their teenage audience. I am so very grateful to have this individual in classes with me because it makes my content real and tangible beyond the students' experiences. Here is an example of a past PSA projects.

Projects like these also create opportunities for deeper thinking and civic engagement. Students are not simply summarizing information. They are exploring how individual experiences intersect with larger social patterns such as violence, trauma, and community healing. By telling stories that matter to them and their communities, students learn that literacy is not just about producing text, it is about participating in conversations that shape how we understand the world.

Reflecting on my own experiences as both a student and a teacher, I have come to see multimodal composition as essential for meaningful learning. My childhood memory of cooking succotash with my grandmother reminds me that learning becomes powerful when it engages multiple senses, perspectives, and forms of expression. Today, whether my students are designing a video game through programming or planning a documentary-style digital story, they are engaging in similar processes of meaning-making. They are researching, creating, revising, and sharing ideas in ways that extend beyond traditional writing. When classrooms expand their definition of literacy to include these multimodal practices, more students are able to see themselves not just as learners completing assignments, but as creators, storytellers, and authors whose work has the potential to matter.