St&Rt With Alice: A Multimodal Literacy Experience For Science And Technology Learning

Abstract

Inanimate Alice is a multimodal narrative told across six episodes of increasing technological and literary complexity. As Alice grows from age eight to young adulthood, she lives her dream of becoming a game designer. Using her handheld digital device and developing skills, Alice creates a digital best friend, Brad, who is always ready to rush to Alice’s aid as she works through the problems and challenges she encounters in her life. The Inanimate Alice series is a coming-of-age tale that leverages the possibilities of transmedia narratives to engage and immerse readers in the story as they adopt a “first-person-reader” point of view mirroring established expectations from game-play in many popular video games. Readers step into Alice’s shoes, share her experiences as she travels around the globe, play the games she has created, and join her on the pathway towards becoming a game designer. This brief paper discusses the theoretical perspectives and practical implications for integrating the transmedia series,, into STEM curriculum. Public scholarship and a growing body of empirical work point to the benefits to students when is integrated into literacy and STEM classrooms. This evidence is provided against the backdrop of the global health crisis, positioning multimodal storytelling as a tool for remote learning that engages learners in subjects as diverse as physical sciences and computational thinking. This paper forwards a multi-theoretical framework appropriate for planning interdisciplinary instruction in literature and STEM.

Keywords: Curricular integration, digital literacy, multimodality, STEM, transmedia

Introduction

Whenwas first introduced in 2006 (Pullinger, 2015), website statistics indicated profound interest in the series by teachers who understood the benefits of this immersive and engaging reading experience for students. Since that time, teachers around the world have introduced their students to Alice. The European Commission selected for several initiatives, including Intercultural Dialogue and eSkills programs with episodes translated into French, German, Italian, and Spanish. The Arts Council of England invested in the production of 3D storytelling content along with student software guidance and developer journals. Concurrently, Education Services Australia selectedas the first digital text for that country’s national, digitally delivered curriculum, and commissioned translation of the episodes into Indonesian and Japanese for Australian learners of those languages. The series was also chosen as the first digital narrative for Portugal’s National Reading Plan. Translation of the series for Portuguese readers is under way, led by a team at the Faculty of Letters at the University of Coimbra. Portuguese translations are also being adapted to Brazilian Portuguese for educational use in that country. International scholars associated with the project will participate in the STEAM prototype development through their participation in an advisory board that will be formed in the coming months.

Problem Statement

The history of recounts the world-travelling tale of an adventurous girl and her digital companion as it chronicles the development of digital technologies. When the first episode of was launched in 2005, few people were aware of the transformational effect that smartphones, tablets and the widespread use of mobile apps would have on their students. At that point, read-only hypertexts (Pullinger, 2008) were the most common type of digital stories outside of video games, and such stories were emblematic of Web 1.0, a format that would soon change into the more interactive Web 2.0 with blogs, wikis and other forms of digital content creation. A decade or so later, teachers and learners find themselves at another frontier of digital storytelling, the Semantic Web often referred to as Web 3.0, where algorithms, cryptography and the Internet of Things hide as much of the inner workings of the worldwide platform as they reveal (Bridle, 2019).

Coding skills have become an increasingly important aspect of STEM subject education, with an emphasis on the technological T. Alice’s journey includes her understanding of video game programming as she navigates from home-schooled elementary and public high school student into the game designer she hopes to become. Not only is she a girl living in the present-day world, she takes an active role in world building, one of the seven core principles for transmedia education (Teixeira Tárcia, 2019). As Alice acquires the skills that will set her on her pathway to becoming a game designer, each episode includes a gamified structure that represents Alice’s project-based learning. This “born digital” text offers a rebuke to Ryan’s (2017) definition of transmedia storytelling, where successful productions can only be top-down from storyteller to audience. This may work for multimillion dollar series like or the Marvel Cinematic Universe where a dedicated fan-base awaits authorized next additions to the narrative. There is more a bottom up approach with, as evidenced by her Gap-Year stories created by the readers, students and teachers immersed in the storyworld very similar to their own, with plenty of space for learners to contribute their own experiences either real or imaginary.

Research Questions

This work is framed by a conceptual framework, Immersive Literacy, that describes the social and affective aspects of literacy practices that are both contextually and culturally informed. While the term Immersive Literacy has been used in the education field anecdotally to describe literacies related to immersive technologies, the Immersive Literacies framework builds upon the theoretical foundations of the New Literacies Studies (Albers & Harste, 2007), multimodal (Kress, 2010) and new media literacies, embodied literacies (Enriquez et al., 2016), and flow (Siraj-Blatchford & Brock, 2016). It acknowledges the participatory and performative nature of literacy that deeply engages students and enhances learning.

Purpose of the Study

This paper forwards a multi-theoretical framework appropriate for planning interdisciplinary instruction in literature and STEM..

Research Methods

For digital multimodal texts, there are no commonly used metrics for analyzing readability. This is problematic because multimodal reading is recognized as a 21st-century skill, a skill that is foundational to STEM success (Hand et al., 2003; Holsanova, 2020). Transinformation analysis (Weltner, 1973) is a solution for measuring multimodal readability. It is an information theoretic method that uses Shannon entropy to find the difference between the objective information in a digital multimodal text (e.g., image pixel intensity) and the subjective information from its readers (e.g., think-aloud screen recordings, oral retellings). A higher transinformation value reflects greater information complexity and a more difficult level of readability.

To test this solution, transinformation analysis and content analysis were used to measure the multimodal readability of Episode 2 of.

Findings

Middle school students ( = 15) who were identified as advanced readers served as the population sample. Findings revealed that 14 of the 15 readers attended to less than half of the information carried on the multiple modes. With a mean readability score of .57, this indicated higher than average information complexity or a more difficult readability level. Readers attended to and recalled information predominantly from the text mode, largely ignoring the visually rich background imagery in the story. While this may have been a strategy for reducing information load, it may have also reflected their beliefs that reading is a language-based activity. Further, evidence suggests that these strong traditional readers were not strong multimodal readers. These findings highlight the need to create more opportunities for multimodal reading experiences in academic settings. By improving their multimodal reading comprehension, educators can prepare today’s students for tomorrow’s careers that are rich in multimodal information.

The described by Bridle (2019) has people entangled in machinery that is beyond what most of them can comprehend, meanwhile STEM educators promote the importance of computational thinking to make sense of our technology. While coding and game design skills are important for many learners as they are for Alice, there is still a need to embrace what is unknown and emerging above the systematic acquisition of knowledge. 21st century learners have access to the Web 3.0 mass communication tools to read Alice’s story, finding their own place as they learn from and with her. Finally, what exactly are they learning? For Alice, she navigates the storyworld by learning the coding language, she is sharing what she knows as she goes. With so much attention during the global pandemic on remote learning and STEM skills for an “information age” future, Alice becomes a role model for self-sufficiency and project-based solutions as she levels up from one episode to the next. opens up new possibilities for educators to learn from students.Initial investigation of the suite of offerings points to important implications, including close examination of how integration of benefits students. In a project conducted with graduate students to develop the game prototype, benefits of integrating transmedia narratives such as into the STEM curriculum emerged. The products developed in this project, a game prototype and developer’s journal, invite students to engage in computational thinking and technological problem solving through an introduction to coding and inspire students to create their own narrative-based games.

Currently, a pilot project to investigate the use of in after school settings is underway with students enrolled in a club-based, extended learning program in the United States will allow us to investigate the impact of integration on developing digital and traditional literacies among learners (Hovious et al., 2020; Shinas, 2012), the impact on language development in a culturally and linguistically diverse society (Faulkner & Curran, 2016), and the ways that STEM knowledge and skills, critical in a digitally connected world, can be developed through gaming and digital storytelling (Fleming, 2014; Hovious et al., 2020).

Conclusion

The exemplary digital narrative suite, and the episodes together have the potential to advance knowledge by providing a pathway from the age of print-based reading to that of digitally-delivered literature structured for today’s adolescent readers. As they move into the workplace, today’s students will depend entirely on their digital skills for employment and societal inclusion as well as their ability to read and communicate using augmented, virtual, and mixed realities. This includes reading words embedded within three-dimensionally delivered images, sequences, delivered in audio-enriched settings.provides a segue into that new world providing immersive, game-like reading experiences delivered within a literary work worthy of academic investigation.

References

• Albers, P., & Harste, J. C. (2007). The arts, new literacies, and multimodality. English Education, 40(1), 6-20.

• Bridle, J. (2019). New dark age: Technology and the end of the future. Verso.

• Enriquez, G., Johnson, E., Kontovourki, S., & Mallozzi, C. A. (Eds.). (2016). Literacies, learning, and the body: Putting theory and research into pedagogical practice. Routledge.

• Faulkner, J., & Curran, G. (2016). Personal Stories and the Visual Turn: Exploring Digital Stories as Identity Representation. Storytelling and Education in the Digital Age, 131.

• Fleming, L. (2014). Literacy in the Making. Reading Today, 32(2), 28-29.

• Hand, B. M., Alvermann, D. E., Gee, J., Guzzetti, B. J., Norris, S. P., Phillips, L. M., Prain, V., & Yore, L. D. (2003). Message from the “Island Group”: What is literacy in science literacy? Journal of Research in Science Teaching, 40(7), 607-615.

• Holsanova, J. (2020). Uncovering scientific and multimodal literacy through audio description. Journal of Visual Literacy, 39(3-4), 132-148.

• Hovious, A., Shinas, V. H., & Harper, I. (2020). The Compelling Nature of Transmedia Storytelling: Empowering Twenty First-Century Readers and Writers Through Multimodality. Technology, Knowledge and Learning, 1-15.

• Kress, G. R. (2010). Multimodality: A social semiotic approach to contemporary communication. Taylor & Francis.

• Pullinger, K. (2008). Digital fiction: From the page to the screen. In R. Adams, S. Gibson, & S. Müller Arisona (Eds.), Transdisciplinary Digital Art: Sound, Vision and the New Screen, (pp. 120-126). Springer.

• Pullinger, K. (2015). Inanimate Alice: how we accidentally created a digital story for schools. M. Manresa, & N. Real (Eds.), Digital literature for children: Text, readers and educational practices.

• Ryan, M.-L. (2017). Transmedia storytelling as narrative practice. T. Leitch (Ed.), Oxford Handbook of Adaptation Studies, (pp. 527-541). Oxford University Press.

• Shinas, V. H. (2012). Reading path and comprehension: An investigation of eighth-grade skilled readers' engagement with online, multimodal texts. University of Delaware.

• Siraj-Blatchford, J., & Brock, L. (2016). Early childhood digital play and the zone of proximal developmental flow (ZPDF). http://www.schemaplay.com/Docs/CITEI.pdf

• Teixeira Tárcia, L. P. (2019). Transmedia education: Changing the learning landscape. In M. Freeman & R. Rampazzo Gambarato (Eds.), The Routledge Companion to Transmedia Studies (pp. 314-322). Routledge.

• Weltner, K. (1973). The measurement of verbal information in psychology and education. Springer-Verlag.