The irony of writing about extended reality (XR) at this time when so many of us have been thrust into the virtual is not lost on us. The situation is reminiscent of the world depicted in Cline’s Ready Player One (2011), in which the entire population uses virtual reality (VR) to escape the increasingly precarious environment outside their doors. Works such as “The Machine Stops” by E. M. Forster (1909), or He, She, and It by Marge Piercy (1991), provide a dystopic vision of a virtual future, but now, we need—perhaps even crave—optimism. The virtual offers us a way to connect; the possibility of engaging with each other even from a distance. We also need to find ethical and sustainable ways going forward to employ a medium poised for mass adoption. This issue demonstrates the power of XR in pedagogical applications, community partnerships, and future gatherings.
This special issue focused on XR—most often referring to virtual and augmented reality (AR)—emerged from our shared excitement about the potential of immersive media to support innovative pedagogy at all levels of education, but also from our healthy skepticism about the limited circles of people actually empowered to shape this process. The full reality–virtuality continuum, a concept first introduced by Paul Milgram and his colleagues in 1994, encompasses everything from virtual reality theaters and mixed reality headsets to augmented reality experiences on mobile devices. Until very recently, the higher end of XR was limited to specialized labs and researchers. As both the technology itself and the means of creating content have become more accessible, the field has expanded and diversified.
Working in the field at this critical moment, we noticed the lack of scholarship engaged with humanistic concerns regarding XR technologies in pedagogical applications. The literature is dominated by rigorous publications on the technical side describing developments in the field of computer science, and by informative case-study examples from museums, journalistic applications, and popular entertainment. Pedagogical experiments in K–12 education, industry, building technologies, city planning, and medicine, meanwhile, have clearly demonstrated the potential of XR applications for teaching and training. Yet the critical conversations around the medium itself, its affordances, challenges, and opportunities for educational use, still take place within small, often isolated pockets of discipline-specific practitioners. We meet up in local working groups, at conferences and workshops, on Twitter—and now in the pages of JITP. One of our objectives in putting together this issue is to bring these different groups into deeper conversation with one another, promoting critical knowledge construction in the field while building out a body of citable literature in humanistic XR studies. We hope this issue helps expand the field to include a greater diversity of voices and experiences.
In the future, the pressing questions of our current circumstances may find answers in XR. Take for instance the growing number of virtual conferences, virtual tours, and virtual open houses already happening in response to the shift to remote learning and working conditions. Although made more urgent by COVID-19, the creation of virtual labs, virtual workspaces, virtual archives, and virtual art studios has long been the dream of XR researchers. Now is the time for scholars to envision and build this future with or without the collaboration of the big tech companies (most recently Apple and Verizon) that have been quietly buying up XR platforms and start-ups in anticipation of a pivotal moment like this. How will we ensure our values are embedded in the XR systems that emerge, or that the resulting models of pedagogy are immersive, interactive, accessible, and collaborative? Even as we go online with our teaching, we realize how much we’re missing from lived experience in physical proximity. How can we leverage the affordances of the real in pursuit of the digital? How does the digital expand access, opportunity, vision, and community? How might XR facilitate lifelong learning applications and the global communities these interventions make possible? The articles in this issue begin to explore these questions in greater depth, and offer potential avenues for further development, especially in terms of community engagement and social justice. This special issue also makes clear that diversity, in all its many forms, is an essential component for XR-based teaching and research, especially as we consider ways of applying intersectional analysis to applied learning.
Several of the articles in this issue focus on using XR for social justice. In “Immersive Pedagogy: Developing a Decolonial and Collaborative Framework for Teaching and Learning in 3D/VR/AR,” Lorena Gauthereau, Jessica Linker, Emma Slayton, and Alex Wermer-Colan draw from a symposium held at Carnegie Mellon University and conversations held with librarians, technologists, developers, and faculty in attendance there. The authors advocate for continued conversations regarding integration, use, and review of 3D/VR/AR teaching and learning technologies. In “Developing Virtual Reality Modules Aimed to Enhance Social Work Students’ Skills and Reinforce Knowledge,” Nicholas Lanzieri, Henry S. Samelson, and Jonathan Bowen describe how multiple approaches to the use of VR in therapist training—360º video and avatar-based game environments—can be embedded into a social work curriculum. Their work demonstrates how prior exposure to environments and potential conversations can enhance live engagement with a diverse set of clients. One of the more revelatory examples of XR technology applications is evidenced in the essay, “‘Relational Presence’: Designing VR-Based Virtual Learning Environments for Oral History–Based Restorative Pedagogy,” by Jennifer Roberts-Smith, et al. This article describes an approach to designing VR that intentionally makes users aware of their virtual environments in order to situate themselves apart from the oral histories they experience in simulations. In this piece, Roberts-Smith et al. introduce the Digital Oral Histories for Reconciliation (DOHR) project (dohr.ca), which worked in partnership with the Nova Scotia Home for Colored Children Restorative Inquiry (restorativeinquiry.ca) to create a VR experience intended to expose the truth of institutionalized racism and to empower the survivors to “build more just relationships for the future.”
Several of our articles also engage with the potential for partnership building and community engagement through historic sites, landscapes, and the university/college campus. In their article “Representing Indigenous Histories Using XR Technologies in the Classroom,” Amy J. Lueck and Lee M. Panich argue for the wider adoption of XR technologies such as annotated 360º video tours at key locations on campus to help undergraduate students understand and intervene in the continued erasure of Indigenous histories from existing commemorative landscapes there. This emphasis on community partnerships is echoed in the issue’s fifth article. In “Blending Disciplines for a Blended Reality: Virtual Guides for a Living History Museum,” Juilee Decker, Amanda Doherty, Joel Geigel, and Gary D. Jacobs demonstrate how an interdisciplinary partnership between a university and a local museum offered the opportunity for students to develop digital storytelling skills and multimodal literacy. In “Barriers to Supporting Accessible VR in Academic Libraries,” Jasmine Clark and Zach Lischer-Katz address both accessibility and the important role of libraries in VR creation, implementation, and support as we scale up from experimentation to broad-based implementation strategies.
Especially exciting and filled with generative possibilities are the kinds of lessons learned from case studies arising out of various pedagogical contexts and disciplines. A diverse team of David Neville, Vanessa Preast, Sarah Purcell, Damian Kelty-Stephen, Timothy D. Arner, Justin Thomas, and Christopher French, describes a whole-college approach in “Using Virtual Reality to Expand Teaching and Research in the Liberal Arts.” Their approach to infrastructure development highlights how a smaller institution can make XR happen at a thoughtful, systemic scale—in harmony with existing pedagogical values and practices—in a highly selective teaching-focused undergraduate setting. “Truly Immersive Worlds? The Pedagogical Implications of Extended Reality” by Tamara O’Callaghan and Andrea Harbin provides specific examples of the kind of VR and AR applications an instructor might use in a liberal arts context. Specifically, they investigate how 3D models of historic sites and AR overlays on historic documents can serve as virtual tools to enhance the physical space. Another example, from Alison Burke, Elana Blinder, Leah Potter, and David Langendoen in their article, “Mission US TimeSnap: Developing Historical Thinking Skills through Virtual Reality,” shows that VR is a promising and useful tool for K–12 history education. Their TimeSnap game has helped to increase students’ engagement with historical documents, narratives, and terminology. Thinking critically about such XR engagement is not limited to the humanities classroom. In the sciences, there have been several applications supporting laboratory work and surgery, but in “Virtual Chirality: A Constructivist Approach to a Chemical Education Concept in Virtual Reality,” authors Samuel R. Putnam, Michelle M. Nolan, and Ernie Williams-Roby demonstrate how it is important not only to use XR in teaching, but also to bring students into the process of building applications. Together, these articles provide an interdisciplinary view of how XR technologies are shaping education at all levels through a critical engagement with interdisciplinary applications.
This issue celebrates pedagogical innovation and forward thinking, but we would be remiss not to acknowledge that it will be released at a time of profound loss, reflection, and fear. The Journal of Interactive Technology and Pedagogy was founded and remains housed at the Graduate Center, CUNY, in New York City, a city losing multitudes of souls, including brilliant academics, each day. All of us have felt this loss, and it is only because of the incredible dedication and hard work of everyone involved that this issue was published at this time. The greatest thanks go to our managing editor, Patrick DeDauw, who not only ensured that the process of publishing this issue was streamlined and efficient; he also made it enjoyable despite the most incredible of obstacles. Patrick, you are our hero. We would also like to acknowledge the willingness of our authors to work through revisions and copyedits with grace and professionalism amidst a global crisis. And to our reviewers, please know your service is very deeply appreciated.
One final personal note: this issue is dedicated in memory of Dr. David Greetham, a founding member of the Interactive Technology and Pedagogy program at the Graduate Center, CUNY. Without his foresight and inspiration, many digital innovations cited in this issue would not have been possible.
Cline, Ernest. 2011. Ready Player One. New York: Crown Publishers.
Forster, E. M. 1909. “The Machine Stops.” Oxford and Cambridge Review (November).
Milgram, Paul, Haruo Takemura, Akira Utsumi, and Fumio Kishino. 1994. “Augmented Reality: A Class of Displays on the Reality–Virtuality Continuum.” Proceedings of Telemanipulator and Telepresence Technologies 2351: 282–92.
Piercy, Marge. 1991. He, She, and It. New York: Alfred A. Knopf.
About the Editors
Amanda Licastro is the Assistant Professor of Digital Rhetoric at Stevenson University in Maryland, as well serving on the editorial collective of the Journal of Interactive Technology and Pedagogy. Her research explores the intersection of technology and writing, including book history, dystopian literature, and digital humanities. Publications include articles in Kairos, Digital Pedagogy in the Humanities, Hybrid Pedagogy, and Communication Design Quarterly, as well as a recent chapter on social annotation in Digital Reading and Writing in Composition Studies, published by Routledge. Her grant-funded project on virtual reality was awarded the Paul Fortier Prize at the 2017 Digital Humanities conference, and has been featured in the Baltimore Sun and Baltimore Magazine.
Angel David Nieves, BArch, MA, PhD, is Professor of History and Digital Humanities at San Diego State University in the Area of Excellence in Digital Humanities and Global Diversity. He is the author of An Architecture of Education: African American Women Design the New South (University of Rochester Press, 2018). Nieves’s scholarly work and community-based activism critically engage with issues of race and the built environment in cities across the Global South.
Victoria Szabo is a Research Professor of Visual and Media Studies at Duke University. Her teaching and research focus on critical and creative approaches to interactive and computational media in the arts and humanities. She was the founding director of the interdisciplinary PhD program in Computational Media, Arts & Cultures, and currently heads the Digital Humanities Initiative at the John Hope Franklin Humanities Institute at Duke. She is also Chair of the ACM SIGGRAPH Digital Arts Community.
In June 2019, a cohort of CLIR postdoctoral fellows convened Immersive Pedagogy: A Symposium on Teaching and Learning with 3D, Augmented and Virtual Reality at Carnegie Mellon University. The symposium sought to bring together a multidisciplinary group of collaborators to think through pedagogical issues related to using 3D/VR/AR technologies, as well as to produce and disseminate materials for teaching and learning. This essay presents the Immersive Pedagogy symposium as a model for interrogating and developing pedagogical practices and standards for 3D/VR/AR; we offer a decolonial, anti-ableist, and feminist pedagogical framework for collaboratively developing and curating humanities content for this emerging technology by summarizing the symposium’s keynotes, workshops, as well as its goals and outcomes. Workshops, keynotes, and participant conversations engaged with decolonial and feminist methodologies, practiced accessible design for universal learning, offered templates for humanistic teaching, and illustrated the possibilities of using 3D/VR/AR to extend critical thinking. While 3D/VR/AR technologies demonstrate real possibilities for collaborative, multidisciplinary learning, they are also fraught with broader concerns prevalent today about digital technologies, as well as complex issues specific to 3D/VR/AR. There is a clear need to assemble academic practitioners on a regular basis in order to facilitate an ongoing discussion about 3D/VR/AR technology and its responsible, meaningful use in teaching and learning.
As access to three-dimensional (3D) technologies has become increasingly available in academic venues, the desire to teach with these emerging technologies, particularly augmented reality (AR) and virtual reality (VR), has outpaced digital humanists’ abilities to provide meaningful support for immersive projects. There is a growing and ongoing need to produce shared and open pedagogical materials adaptable to the needs of teachers in various professions and disciplines and are accessible to students without significant coding experience. This need is partially driven by the contingencies of relatively new and rapidly updating technologies, as well as the fact that support for commercially-available immersive tools are tailored for industry purposes. Game-driven tutorials, for example, do not always take into consideration the needs of humanities practitioners seeking to integrate critical thinking with technical mastery. Contemporary contexts for emerging technologies can structure our interactions with 3D/VR/AR. Though not always visible to users, these can have the effect of naturalizing problematic historical and political narratives through selective access to resources and functionality.
Nonetheless, game engines that offer free educational licenses have been repurposed for academic inquiry and teaching over the past decade. For example, Unity Technologies’ Unity 3D game engine is utilized by over 4.5 million users and has been at the forefront of historical and archaeological 3D visualizations in scholarly research. First available in 2005, the Unity 3D game engine has been used to make approximately 60% of all AR/VR applications and is used by 90% of AR/VR companies (“Public Relations” 2019, np). Educational licenses are available for students and educators seeking to use the engine for scholarly or creative use. Its main competitor, the Unreal Engine, while initially inaccessible beyond professional and academic institutions with licenses, dropped its paywall for educational use in September 2014. VR headsets, once a hypothetical fantasy or niche short-lived technology, are now commercially viable and relatively inexpensive for institutions to purchase. In a few years, the financial barrier for individuals may diminish; in the meantime, Google Cardboards and other stereoscopic viewers with fewer interactive features currently provide alternatives for students with access to smartphones. However, students are also increasingly able to make use of interactive 3D/VR/AR technology within dedicated spaces in academic libraries, maker spaces, media studios, and community outreach centers. Yet, we would be remiss not to point out that access is still mediated by other social hierarchies; 3D/VR/AR technology is still not accessible in much of the Global South, or in marginalized communities across the world. These aforementioned developments still privilege students at institutions that dedicated staff or faculty to maintain and encourage use of 3D/VR/AR technologies and facilities.
This is all to say that in our current 3D/VR/AR moment, digital humanists have a lot to navigate. Current 3D/VR/AR pedagogy and projects can pose problems related to accessibility and long-term preservation of projects and assets, and often run afoul of minimal computing recommendations. Yet the technology offers rich possibilities for multidisciplinary research and collaboration; many virtual reality projects combine art production, computing, archival research, network theory, and data visualization, among other practices. Given its potential for scholarship and teaching, understanding how to use the technology responsibly necessitates engaging with active practitioners to identify what is now possible and what still needs to be done to facilitate productive use of 3D/VR/AR. As many key problems are likely to persist through subsequent permutations of the technology and its use in educational settings, this conversation needs to be ongoing and open. What humanists within and beyond the academy have to say about 3D/VR/AR will probably not be unique to humanistic inquiry. This dialogue will provide crucial critical approaches to the emerging technologies’ advantages and limitations that will be of use to industry professionals as well as the casual creative user. A vocal contingent of humanists seeking to think and learn with 3D/VR/AR may, in fact, fill a wider sociocultural need by addressing these issues.
This is the context in which a small cohort of 2017–2019 Council on Library and Information Resources (CLIR) Postdoctoral Fellows organized Immersive Pedagogy: A Symposium on Teaching and Learning with 3D, Augmented and Virtual Reality at Carnegie Mellon University on June 26 and 27, 2019. The CLIR cohort included Lorena Gauthereau (University of Houston), Jessica Linker (Bryn Mawr College), Eric Kaltman (Carnegie Mellon University), Emma Slayton (Carnegie Mellon University), Neil Weijer (Johns Hopkins University), Alex Wermer-Colan (Temple University), and Chris Young (University of Toronto). The goal of this symposium was to assemble a wide range of stakeholders to develop teaching materials and strategies that considered problems inherent and specific to immersive technologies, as well as to address problems that affect but are not unique to 3D/VR/AR. It is for this reason the symposium was so attentive to decolonial and feminist methodologies in thinking about appropriate pedagogical applications. Building on the previous work of scholars such as María Cotera, Elizabeth Losh, Tara McPherson, Angel Nieves, Roopika Risam, and Jacqueline Wernimont, we have advocated for an intersectional digital humanities that interrogates a wide range of technologies through the critical methods developed by the fields of ethnic and feminist studies. Such methods, we argue, can highlight the ways that technologies often leave out marginalized people by replicating colonial hierarchical structures including race, ethnicity, class, gender, and disability.
The Immersive Pedagogy symposium offered an early—if not first-of-its-kind—opportunity to have productive conversations about what critical approaches to 3D/VR/AR could look like from a multidisciplinary and multi-professional perspective. Additionally, the symposium sought to seed collaborations within and beyond academic institutions and stand as a model for future conversations on these topics. In recounting our experiences with different applications of 3D/VR/AR technology in pedagogical spaces, the group tackled a number of thorny issues, such as accessibility in hardware and bias in asset stores, while acknowledging that we would need to continue the dialogue by reconvening. We sought to develop teaching materials collaboratively with the long-term plan of sharing these resources through a variety of means, including via open-access publications. In the remainder of this essay, the Immersive Pedagogy organizers describe the symposium’s theoretical foundation and methodological approaches as a model for structuring communities around 3D/VR/AR, summarize some of our group’s findings, and invite digital humanities practitioners to help us to continue this work.
Structuring a Symposium on Decolonial Models of Immersive Pedagogy
Because the initiative was organized by CLIR postdoctoral fellows, the symposium emphasized diverse ways that libraries participate in creating, curating, and preserving 3D/VR/AR pedagogical materials. We considered faculty, staff, and students as equal partners in 3D/VR/AR projects, and aimed to include early career researchers at the table. Overarching goals for the symposium included teaching faculty and librarians how to support and enable learning for students using 3D technologies, and to help students to disseminate skills within their own communities. By bringing together scholars from a wide range of disciplines and professions, we addressed problems while identifying new ones. Participants had the opportunity to share links and descriptions to their projects (current and in progress) with each other prior to the symposium via a Slack channel and Google Docs. They also shared information on their work during a lightning talk round as examples of the kinds of humanistic projects 3D/VR/AR could cultivate. The symposium began and ended with keynotes from experienced practitioners whose work modeled creative and responsible uses of the technologies.
Our opening keynote speaker, Angel Nieves (Associate Professor of History and Digital Humanities at San Diego State University), presented “Developing a Social Justice Framework for Immersive Technologies in Digital Humanities.” Nieves’s talk outlined strategies for achieving social justice through digital-supported inquiry, highlighting his own work on Mapping Soweto, a 3D reconstruction of apartheid South Africa. Nieves emphasized the need to ground digital work in women of color theory and argued that fields such as ethnic studies have developed a foundational structure that would benefit the field of digital humanities as a whole:
If we brought the sorts of methodological and practice-based questions about power, privilege, and access from ethnic studies to our work in immersive technologies, we might begin to see new ways of harnessing these tools–that originated as part of the military industrial complex–to serve our social justice needs. (Nieves 2019)
Mapping Soweto draws from Belinda Robtnett’s (1997) work on social movement theory, revealing the often messy, multilayered narratives of social movements by visualizing a map of spatial liberation. This 3D representation shows what Nieves terms an “intersectional cartography,” or a network of social activists—especially networks of women and young girls—across townships “and how those activist networks were embedded into the physical geography and vernacular architecture of individual houses, streets, and neighborhoods” (Nieves 2019). Attention to intersectionality further reveals the ways multiple identities—township, gender, sexual orientation, class, and race—came together to form a cohesive activist movement, whose complexities are often lost in the official retelling of history. In particular, Nieves identified immersive technologies as one way to “re-establish coalition-building potential” (2019) with local communities and reminded us that the important work of recovering marginalized histories for social justice is often messy.
Our closing keynote speaker, Juliette Levy (Associate Professor of History at the University of California, Riverside), presented “How Not to be a Replicant: Working Towards a Useful VR.” Working with a team of women programmers, Levy has developed VR simulations for teaching abstract concepts related to historical thinking, interpretation, and writing. Levy’s keynote presentation focused on the question of gaming and interactivity; and she traced the origin of her experimentations in VR from teaching large lecture classes numbering in the hundreds in hybrid and online courses. Rather than approach VR in the mode of cultural heritage projects, reproducing a historical location, to deal with pedagogical problems commonly experienced in online learning, Levy’s team built Digital Zombies (see Levy 2017), an abstract simulation meant to introduce students to the hierarchy of library information and assessment of primary and secondary resources through game-based learning. Levy envisioned a VR environment for her historical research methods class that not only encouraged students to follow a written outline of research steps, but to extend their library experience in a more immersive, playful way by completing a series of game-like missions related to research that students would be more likely to remember. Levy argued that the cognitive effect of a VR experience has a lasting impact on users: “What matters about doing something in VR isn’t about what happens in VR, but what happens outside of VR, after the VR experience” (Levy 2019). Yet, despite the advantages of VR, Levy warned that a lack of critical conversation and pedagogy around digital literacy can have dire consequences, as increasingly ubiquitous immersive technologies become exploited to misrepresent historical events. The stakes for fomenting critical conversations between technology creators, consumers, and scholars, therefore, are quite high, as they could have lasting effects on how people choose to build and interpret virtual representations of historical events and people.
The symposium included five workshops that centered on theory, methods, and practices significant to and capable of incubating pedagogy related to US Latinx, Latin American, and Caribbean studies, which we prioritized when considering applicants. The workshop topics were: (a) Decolonial Methodology and Theory; (b) Accessible Immersive Pedagogy; (c) Integrating Immersive Technology in the Classroom; (d) Critical Writing for Immersive Tech; and (e) Collaboratively Designing 3D/VR Experiences. The Immersive Pedagogy organizers, joined by Jasmine Clark (Temple University) and Juliette Levy, led the participants through these interactive workshops (“Program” 2019). Pedagogical content crafted by participants before, during, and after the symposium included a bibliography of 3D/VR/AR-related readings, an archive of workshop slides, video recordings of keynote presentations, adaptable templates for pedagogical activities, and working models of 3D/VR/AR pedagogical applications. For example, Kat Hayes and Samantha Porter submitted a video walkthrough of their IOS app Virtual MISLS that explores historic buildings at Fort Snelling, while Meaghan Moody and Coral Salomón submitted a description of their work with students using a virtual map of historic Paris to better understand life under German occupation during World War II.
Carnegie Mellon University (CMU) Libraries hosts the symposium’s materials on its institutional repository, KiltHub. KiltHub provides stable, long-term global open access storage for 3D/VR/AR assets, and functional applications, as well as pedagogical and technical documentation. Materials in this repository are held for a minimum of ten years, ensuring that what is submitted will remain available past typical terms of software updates. The teaching materials produced during and following the symposium will also be published in the Digital Library Federation’s Pedagogy Working Groups open-access series, the DLF Teach Toolkit. The materials will be revised and tested, including during a pre-conference workshop at DLF’s Annual Forum 2020, pending acceptance.
The following essay sections explore the key components of the symposium, which outlined the theoretical foundations to decolonizing development and curation of 3D/VR/AR tech, before guiding participants through workshops on decolonial critique and accessible design, on integrating immersive technology into the classroom and beyond, and on collaboratively designing 3D/VR projects.
Decolonial Foundations: Critical Approaches to the Development and Curation of 3D/VR/AR Technologies
To practically introduce the decolonial methodologies and theories crucial to our workshops on developing and curating 3D/VR/AR materials, the Immersive Pedagogy symposium opened with a workshop, led by Gauthereau and Young on the “walkthrough method” (Light, et. al. 2018, 881–900), a critical analysis of technology using the Unity Asset Store as an example. This exercise was contextualized through a theory of decolonial pedagogy and a discussion on the critical analysis of the game platforms that curate content for 3D modeling and representation.
The application of decolonial theory and methods to digital pedagogy allows students to interrogate and resist colonial, hierarchical epistemologies, especially the privileging of Western European and Anglocentric knowledge structures. Such an approach is increasingly necessary as 3D/VR/AR technologies become integral to Western education systems and overwhelmingly applied to cultural heritage projects by and for Western consumers. While colonialism refers to the “political and economic relation in which the sovereignty of a nation or people rests on the power of another nation,” making that nation an “empire,” coloniality “refers to long-standing patterns of power that emerged as a result of colonialism, but that define culture, labor, intersubjective relations, and knowledge production well beyond the strict limits of colonial administration” (Maldonado-Torres 2007, 243). Thus, coloniality denotes the ways in which colonial hierarchies of power continue to structure our everyday lives (i.e. racialized class hierarchies, labor hierarchies, gender hierarchies, the gender binary, racism, etc.). Decolonialism urges us to actively de-link from colonial epistemologies and ontologies in order to avoid re-creating colonial worldviews and hierarchies.
Considering the ways that 3D/VR/AR technologies allow users to create immersive worlds and environments, the symposium sought to stress the need to avoid replicating the colonial gaze. Representing marginalized people through this gaze continues to enforce racialized and gendered hierarchies of power. Colonial epistemologies continue to control knowledge production, not only through institutional archives, but also through academic research, digital projects, and 3D/VR/AR environments. Jacqui Alexander and Chandra Talpede Mohanty argue that decoloniality has a “pedagogical dimension” as it obligates us “to understand, to reflect on, and to transform relations of objectification and dehumanization, and to pass this knowledge along to future generations” (1997, xxviii-xxix). For this reason, the symposium’s first workshop exercise involved guiding participants through a decolonial walkthrough of the Unity Assets Store. The walkthrough method requires researchers to directly engage with “an app’s interface to examine its technological mechanisms and embedded cultural references to understand how it guides users and shapes their experiences” (Light, et. al. 2018, 882). We asked participants to browse and search the Unity Assets Store for 2D, 3D, audio, and animation assets and interrogate them using a decolonial approach, as well as to document their walkthrough by taking notes, taking screenshots, and recording audio-visual content.
To guide the decolonial inquiry, we asked participants to consider a set of questions adapted from Roopika Risam’s discussion of the stakes of postcolonial and decolonial digital humanities (2019, 35–46):
What are compulsory activities within the Unity Asset Store?
What are the social hierarchies within the menu system?
To whom and which types of users is this knowledge accessible?
What is considered a “legitimate” asset within the Asset Store?
Whose epistemologies, such as histories, languages and memories, are considered important enough to archive in the Asset Store?
What knowledge or assets are privileged within the Asset Store?
Does the asset avoid the exoticization or fetishization of a people/cultures?
This inquiry resulted in participants recognizing the disproportionate representation of a Eurocentric worldview. For example, they noticed that the search term “Viking” yielded twice as many results as “Native American,” whereas the term “Indigenous” yielded zero. Among results for the search term “Mexican,” participants discovered a Mexican Restaurant Pack that reflected generalized stereotypes of Mexican aesthetics and cuisine, reduced to bottled hot sauce, chips and salsa, and a decorative green parrot. Assets also reduced the multiple and varied cultures, nations, flora, and fauna of the entire African continent to the myopic colonial imaginary of only the Serengeti, populated by wild animals. During group break-out sessions exploring the Asset Store, participants discovered a potential intervention through editing crowdsourced user tags. Like during Wikipedia Edit-a-Thons, users could challenge the authenticity of colonial representations of people, cultures, and nations by tagging or reviewing assets as not authentic, representing stereotypes, reproducing colonial views, etc. Since the symposium, unfortunately, Unity has removed the user tagging option and currently limits metadata generation to the individual uploading the asset.
This workshop stressed that engaging in decolonial work requires a constant questioning of how knowledge (3D/VR/AR environments, research, stories, syllabi, etc.) is being produced, who is producing it, whose stories are being told, and how these stories are being told. Not only should we consider what histories are told in the digital world, but we must also attend to the ways in which they are produced. As a result, the participants learned that generating and interacting with 3D/VR/AR environments they must use decolonial methods to acknowledge their role as world-creators and reflect on the ways that these technologies often replicate colonialism.
In the following workshop, Clark foregrounded the ableism endemic to technological innovation in the West, introducing participants to accessible user design for virtual reality. This involved a tutorial on developing alternate access plans for disabled students in classrooms. Clark’s work with Temple Libraries’ colleagues Jordan Hample and Wermer-Colan has prioritized research into and creation of accessible features for VR during their development of the Virtual Blockson: A Primary Source Teaching Tool for Secondary Education (Clark 2018, np). Clark’s workshop overviewed the standards of the World Wide Web Consortium’s Web Content Accessibility Guidelines in order to showcase the problems with applying standards created for web-based screens to virtual reality environments and experiences. She related an overview of key advancements that can be made to enable universal design for this emerging technology ranging from innovations in haptic feedback to caption legibility. Clark’s talk focused on guiding participants through strategies for accessing resources for disabled students at their universities. She led participants through an exercise with a template she created for developing “alternate access plans” that enable teachers to offer comparable options for students who cannot use the available VR and/or AR hardware and software. This approach to accommodating students with different learning styles provides a realistic way for teachers to work with emerging technologies in academic institutions, most of which still lack sufficient resources to support disabled students in the use of analog technologies.
Virtual Lessons: Integrating Immersive Technology in the Traditional Classroom and Beyond
After the symposium’s opening workshops on decoloniality and anti-ableism in immersive pedagogy, Levy’s workshop put to practice the principles she laid out in her closing keynote address on the idiosyncratic game mechanics for simulating virtually interactive dialogue and exam questions involving classification. VR offers, Levy argued, a unique pedagogical opportunity, functioning as a distraction-free zone where her students were able to recollect experiences at a much higher rate compared to other learning activities. During the workshop, Levy asked symposium participants to select several library books from various library collections and work in groups to think about how to put the texts in conversation with each other based on titles, subject headings, table of contents listing, and a quick skim of their contents. Levy demonstrated how and why she constructed a VR environment that simulated this activity, as her students had to physically place boxes with various titles onto empty shelves in an order that reflected connections. The application of VR to this type of historiographical exercise, Levy maintained, left a lasting impression on the students that they were able to put into practice for essay assignments. Levy’s emphasis throughout her workshop on the pedagogical significance of “what happens before and after” the virtual experience, furthermore, offered a valuable foundation for the subsequent workshop on integrating writing exercises to guide student learning during virtual and augmented reality experiences.
Wermer-Colan’s workshop modeled how to guide undergraduate students across the disciplines through a structured composition exercise for reflection, in particular, by guiding the participants through a reflection on what they hoped to learn and do in the coming school year as they sought to develop their immersive pedagogy projects. To provide a context and model for students before their writing reflections, Wermer-Colan summarized his current projects employing 3D technologies for Temple University Libraries’ Digital Scholarship Center (now the Loretta C. Duckworth Scholars Studio). Temple Libraries has experimented with transforming the purposes of library collections, development, and reference work to enhance its learning and technology outreach, including through its Innovative Teaching with Makerspace Technology Grant and its newly constructed VR Lab in the new Charles Library. Wermer-Colan’s past experiences working in the Medgar Evers College Writing Center in the City University of New York (CUNY) system helped him to think about ways the Digital Scholarship Center can use 3D/AR/VR technology to enhance learning across the disciplines.
As an example of Temple Libraries’ supporting the use of immersive technologies in class-room projects, Wermer-Colan detailed a collaborative project with Ajima Olaghere, Assistant Professor of Criminology working with her ethnography students to do “systematic social observation” of Philadelphia neighborhoods. This project used 360 cameras to record neighborhoods affected by Temple’s gentrification of North Philadelphia. The recordings were later viewed on twenty-dollar Desktek smartphone headsets that allowed students to remotely examine environments to understand what contributes to disorder and crime, while the instructor facilitated ways to maintain a critical understanding of what they were viewing. The accompanying writing exercises guided students to reflect on their mediated experiences of urban space and call into question the “broken windows theory,” common assumptions that visible signs of public disorder exacerbate criminal behaviors. The use of phone-based headsets also invited an opportunity for students to consider the physical processes that enable virtual technology. Instructors were faced with the problem of scaling pedagogical uses of VR; as this project used relatively inexpensive headsets, workshop participants considered how to create immersive experiences similar in quality to those offered by state-of-the-art VR headsets like the HTC VIVE or the Oculus Rift that, as of 2020, cost hundreds of dollars.
To illustrate the role libraries and digital scholarship centers can play in the curation of 3D content for teaching and learning, Wermer-Colan overviewed a complementary use of immersive technologies. His collaboration on the Virtual Blockson project with Digital Scholarship Librarian Jasmine Clark, Academic Technician and Developer, Jordan Hample, and Blockson Archivist Leslie Willis-Lowry aims to recreate Temple’s Charles L. Blockson Afro-American Collection as a virtual reality game for innovating the teaching of primary source literacy in high schools across Philadelphia. The project at its heart allows a small, fixed collection and its reading room to be available to students remotely, lowering the intimidation factor and physical limitations of these spaces, while enabling interactive explorations of historical artefacts. The Virtual Blockson offered an opportunity to discuss how libraries can help curate interactive gaming environments for remediating archival collections and cultural heritage sites to foreground previously marginalized histories. In these contexts, virtual reality offers affordances for lowering the barrier for students to use archival sources and spaces, facilitating access and accessibility, and offering students a novel medium through which to conceptualize analog and digital literacies necessary to navigate the changing new media world today.
Wermer-Colan foregrounded in both these projects the use of writing exercises to help students reflect on their virtual experiences in meaningful ways. The 360 SSO writing exercise encouraged humanistic thinking about the technology by asking students to compare their field work exercises with the virtual experience, as well as writing reflections that asked the students to identify various ways the 360/VR technology mediated said experience. Similarly, humanistic writing exercises were designed to guide students before and after their experience of the Virtual Blockson’s introduction to archival spaces, etiquette, and practices through game-based, interactive experiences. Drawing upon the Society of American Archivists’ Standards for Primary Source Literacy and the Common Core Standards for historical understanding, digital literacy, and critical thinking, these critical writing questions ensure students reflect upon the virtual experience of library collections’ historical artifacts from the African diaspora. After offering these models to the Immersive Pedagogy participants, Wermer-Colan guided the group through a critical writing exercise to reflect on their own plans to implement the 3D/VR/AR technology for various pedagogical purposes. Wermer-Colan encouraged participants to think of resources at their local institutions, pedagogical standards in their disciplines, and affordances in the spatialized medium of VR for enhancing their approaches to teaching. The writing exercise simulated the kind of exercise participants could implement in their own pedagogy, while offering an opportunity for the symposium participants to reflect on what they had learned during the workshops.
The concluding workshop, run by Linker and Young, offered a sustainable model for including students as partners in the creation of 3D/VR pedagogical materials, through an overview of Linker’s time creating the Bryn Mawr Women in Science project with her various undergraduate partners: Elia Anagnostou, Courtney Dalton, Jocelyn Dunkley, Tanjuma Haque, Arianna Li, and Linda Zhu. From 2017 to 2019, Linker taught undergraduate students how to integrate historical inquiry with 3D technology in order to think about women’s invisible scientific labor, the spaces they occupied, the tools they used, and their everyday lived experiences. The project considers Margaret Rossiter’s “The Matthew Matilda Effect in Science,” which articulates a systematic disparity in affording women scientists credit for sophisticated and important discoveries, which in turn necessitates that historians find ways to tell stories in order to make their labor visible. It likewise adapts aspects of Pamela Smith’s Making and Knowing Project by taking seriously the need to consider scientific processes. However, rather than engaging in physical reenactment, students offered up women’s processes in a modern, digital format, contextualized by a recreation of spaces that were no longer intact or available for historical analysis.
Linker and her students recreated two laboratory spaces that had once existed at Bryn Mawr College in the late nineteenth and early twentieth centuries, the Major Chemistry Lab and the Advanced Biology Lab. Students learned a variety of 3D skills, including 3D modeling, photogrammetry, various mechanics of the Unity 3D game engine, and the Oculus Rift. Interactive WebGL versions of the project are available online, and a VR demo of Bryn Mawr’s Advanced Biology Lab was available at the conference. The Advanced Biology Lab was the site of early genetic research and a place once utilized by Nettie M. Stevens, the subject of Stephen Brush’s Nettie M. Stevens and the Discovery of Sex Determination by Chromosomes. Years before Margaret Rossiter coined the phrase “the Matilda Effect,” Brush identified that Stevens’ discoveries had been overshadowed by male collaborators or individuals working concurrently on the same subject. Her contributions had likely been diminished because she was a woman. Students researched each space by spending time in Bryn Mawr’s Special Collections. Through building each laboratory, the students became aware of how to put historical materials in conversation, as no resource could tell them everything they needed to know to build and contextualize the 3D models. Pedagogically, the two-year process of building was designed to seed humanistic deployment of 3D technologies by undergraduate collaborators. Afterward these students participated in professional presentations of the digital and historical work, and served as ambassadors to various communities in order to disseminate the skills the project cultivated to a wider audience.
Linker enabled her students to accomplish a lot in a short period of time; no student was an expert in the technology or in historical research prior to their tenure on the project. This was intentional, as she sought to teach rather than to employ experts. Students represented a diverse range of interests and majors, and all students participated in each phase of production (rather than assigning humanities majors to research and STEM majors to coding) so that afterwards, they could create projects similar to this on their own. Part of what facilitated their success is that she treated them as equal partners in the project, making decisions with them throughout the two years they worked together.
To prepare her students to participate as equals, she devised a plan that would serve as an introduction to using 3D technology to address social and pedagogical problems, and would also serve as a diagnostic tool for assessing student strengths and interests. Essentially, students were asked to propose and implement a 35-hour project (which could be run over the course of days or weeks, depending on individual need) that used an aspect of the Unity 3D game engine’s functionality to teach users about something the students cared about. Students drafted plans that identified what they knew, what they needed to learn, and were prompted to think about modularity, such that students could scale the project if they were running out of time. Students who were not familiar with coding at first could use Unity’s GUI interfaces to produce fully functional scenes, allowing for students with varying levels of proficiency with computer science to produce something useful by the end of the exercise. By the end of the 35-hour period, students not only had a small project they could put in professional portfolios, but had become proficient in a particular aspect of Unity, thought about the technology as a means to serve others, and in implementing their projects, had a better sense of what they would need to do going forward. It also convinced them that they were capable of using the technology in a way the Unity tutorials did not engender. Linker and Young guided the participants in thinking through how symposium participants might adapt this exercise for their own project teams.
Through the symposium and the workshops described above, participants engaged in conversations around designing socially-conscious pedagogy for 3D/VR/AR. Building a framework for teaching and learning with 3D/VR/AR technologies founded in decolonial theory and practices resonated with our participants. This enabled the group to evaluate how projects and assignments fit into an ethical model for cultural heritage pedagogies. The symposium closed with a productive discussion about what the participants learned, with a focus on planning for future steps. Several participants suggested the importance of backward design, which would specifically place the learning outcomes as the first step in creating 3D/VR/AR and related assignments.
Conversations among group members brought up multiple questions, such as: how do we anticipate student use? How do we adjust our use of 3D/VR/AR in response to unexpected circumstances? How do we introduce emerging technologies in the classroom while accommodating individuals unable to take advantage of the intended purposes of ready-made hardware and software? How can these technologies enhance hybrid and online learning? Are students (or faculty) distracted by the freedom of immersive environments? Can we create bilingual metadata in a VR environment? If one could, where would you display subtitles or transcriptions in a virtual or augmented environment?
These conversations confirmed that digital humanists would benefit from future cross-institution discussions of 3D/VR/AR, as well as from shared access to teaching materials, which are often siloed within institutions and departments. Students engage differently with course concepts and each other, depending on the application of the technology within that course. Student learning is dependent on the skills and interests of individual instructors; collaboration is necessary for producing robust materials and responsible projects. Perhaps the most challenging task is creating accessible and sustainable materials applicable to multiple modes of disciplinary learning outcomes at a time of rapid technological and institutional transformation.
In an effort to increase the reach of the conversations that arose out of Immersive Pedagogy, the symposium organizers are working to produce an open-access, peer-reviewed publication containing lesson plans and educational material to facilitate disciplinary and interdisciplinary work that engages 3D/VR/AR technologies. This project aims to extend the work of the Digital Library Federation (DLF) Pedagogy Working Group’s Teach Toolkit that provides lesson plans for digital library instruction. To guide educators to adapt immersive technologies to the needs of diverse disciplines, the Immersive Pedagogy teaching materials will introduce a range of 3D hardware and software, including asset or game repositories. The teaching materials will include diverse lesson plans with tailored learning outcomes, introducing a representative sample of available immersive technologies and resources while addressing humanistic pedagogical goals. Because this project was born out of the CLIR postdoctoral fellowship program, it aims to contribute to the growing field of scholarship on the crucial role that academic libraries or research and teaching centers can play in the integration of immersive technologies across the curriculum.
The Immersive Pedagogy symposium’s prioritization of decolonialism, feminism, and accessibility speak to a radical and critical perspective that can apply to a range of 3D/VR/AR applications and instruction methods. Indeed, in starting conversations on how to promote making immersive experiences accessible and inclusive, there is an opportunity to move beyond operational concerns to lasting pedagogical practices. For decades, contingencies have transformed education and cultural heritage, requiring us to rethink the potential of emerging communication technologies through a critical lens. More evident in the midst of the COVID-19 global pandemic, which has spurred the need for digital ways of teaching and learning, is the critical pedagogical use of virtual surrogates. These include 360° museum spaces and objects, 3D virtual meeting spaces, photogrammetry models, and interactive exhibits. By addressing upfront, rather than through remediation, the issues of social justice, accessibility, and decolonial pedagogies in immersive technology, educators can leverage these tools to respond to a transformative period in the education system.
 For a discussion of problems and considerations specifically related to the construction of historical 3D spaces, see Sullivan, Nieves, and Snyder 2017.
Grayburn, Jennifer, Zach Lischer-Katz, Kristina Golubiewski-Davis, and Veronica Ikeshoji-Orlati, eds. 2019. 3D/VR in the Academic Library: Emerging Practices and Trends. Arlington, VA: Council on Library and Information Resources.
Levy, Juliette. 2017. “‘Digital Zombies’– A Learner-Centered Game: Social Knowledge Creation at the Intersection of Digital Humanites and Digital Pedagogy.” In Social Knowledge Creation in the Humanities: An Open Anthology, edited by Alyssa Arbuckle, Aaron Mauro, and Daniel Powell. ITER Press. https://ntmrs-skc.itercommunity.org/.
———. 2019. “How Not to be a Replicant: Working Towards a Useful VR.” Keynote address, Carnegie Mellon University Symposium, Pittsburgh, PA, June 28.
Light, Ben, Jean Burgess, and Stefanie Duguay. 2018. “The walkthrough method: An approach to the study of apps.” New Media & Society 20, no. 3: 881–900.
Losh, Elizabeth and Jacqueline Wernimont, eds. 2019. Bodies of Information: Intersectional Feminism and Digital Humanities. Minneapolis, MN: University of Minnesota Press.
Maldonado-Torres, Nelson. 2007. “On the Coloniality of Being.” Cultural Studies 21, no. 2: 240–270.
McPherson, Tara. 2012. “Why Are the Digital Humanities So White? or Thinking the Histories of Race and Computation.” In Debates in Digital Humanities, edited by Matthew K. Gold. Minneapolis: University of Minnesota Press.
Nieves, Angel David. 2019. “Developing a Social Justice Framework for Immersive Technologies in Digital Humanities.” Keynote address, Carnegie Mellon University Symposium, Pittsburgh, PA, June 27.
Sullivan, Elaine, Angel David Nieves, and Lisa M. Snyder. 2017. “Making the Model: Scholarship and Rhetoric in 3D Historical Reconstructions.” In Making Things and Drawing Boundaries: Experiments in the Digital Humanities, edited by Jentery Sayers, 301–316. Minneapolis: University of Minnesota Press.
We would like to acknowledge The Andrew W. Mellon Foundation and the Council on Library and Information Resources (CLIR) for the microgrant that funded the Immersive Pedagogy symposium, as well as Carnegie Mellon University Libraries for hosting the event. Thank you to the entire Immersive Pedagogy team, including Eric Kaltman, Neil Weijer, and Chris Young for making the symposium possible. Last, but certainly not least, thank you to all the Immersive Pedagogy participants and keynote speakers, who created a positive, productive community of practice: Andy Anderson, DB Bauer, Katie Chapman, Elena Foulis, Kat Hayes, Juliette Levy, Juan Llamas-Rodriguez, Meaghan Moody, Angel Nieves, Samantha Porter, Coral Salomón, Julia Troche, Jordan Tynes, and Christa Williford.
About the Authors
Lorena Gauthereau is the Digital Programs Manager for the US Latino Digital Humanities program at the University of Houston’s Recovering the US Hispanic Literary Heritage. She received her Ph.D. in English and her M.A. in Hispanic Studies, both from Rice University. Her research interests include US Latinx studies, digital humanities, and decolonial theory. Orcid ID: orcid.org/0000-0002-7185-8982.
Jessica Linker is an Assistant Professor of History at Northeastern University. She was previously a Postdoctoral Fellow and Program Coordinator at the Consortium for History of Science, Technology and Medicine, a Visiting Assistant Professor at Bryn Mawr College, and the Director of Bryn Mawr Women in Science. She researches women’s scientific practices in early America.
Emma Slayton is the Data Curation, Visualization, and GIS specialist at Carnegie Mellon University Libraries. She obtained an MPhil from the University of Oxford in 2013 and completed her Ph.D. at the Faculty of Archaeology, Leiden University in 2018. Her current work centers around improving and supporting digital literacy efforts. Orcid ID: https://orcid.org/0000-0003-2230-3101.
Alex Wermer-Colan is a postdoctoral fellow in Temple University Libraries’ Loretta C. Duckworth Scholars Studio, where he coordinates research and pedagogical projects in cultural analytics and digital media arts. His editorial and scholarly criticism have appeared in PAJ: A Journal of Performance and Art, Twentieth Century Literature, The Yearbook of Comparative Literature, Lost & Found, Indiana University Press, and The Los Angeles Review of Books. Orcid ID: https://orcid.org/0000-0001-7030-6070.
The use of virtual simulations has steadily increased within the social work master’s level curriculum and its benefits have been noted in varied research studies. This paper aims to describe the development of two virtual simulations that are innovative, educational, and responsive to the needs of social work skill development and knowledge reinforcement. The first module is a 360 virtual reality (VR) tool that allows social work students to navigate areas of the Lower East Side of New York City. The second module allows social work students to conduct a home visit and assess a virtual client within an immersive 3D VR environment utilizing a framework built on top of a commercial game engine. Each simulation is individually described and the processes taken to create the scripts, assets, and framework are detailed. Their pedagogical value to social work is noted and next steps in VR development, assessment and research are outlined and discussed.
Social work masters-level education integrates both theoretical and practice based curricula. Traditionally, each facet has been delivered to students within the classroom and Field placement setting. Technology, however, has broadened the systems of delivery and has facilitated and advanced these pedagogical methods.
The use of virtual simulations has steadily increased within social work masters-level education and its benefits have been noted. Generally, simulation based learning has been shown to be effective for adult students because it is problem-focused, self-directed and it has relevant applicability to real settings (Washburn, Bordnick, and Rizzo 2016). Virtual simulations allow for greater diversity and variability in Field settings and in patient demographics that may not be found in “real” Field placement outlets. It also provides students with “immediate” feedback and knowledge from their interactions within an assortment of settings and with diverse clients. Admittedly, the virtual interaction will be limited in some ways, but it will nonetheless provide students the advantage of gaining additional Field/practice perspectives that cannot be feasibly offered in vivo. Virtual simulations may be designed to be accessible throughout the day and will therefore be able to accommodate multiple student connections at one time. Virtual environments grant students “safe spaces” to make mistakes and to hone in on skill sets that they may not ordinarily be processing in “real” settings (Boulos, Hetherington, and Wheeler 2007). Research has indicated that students feel more comfortable and less pressure to perform in virtual settings as opposed to working with face-to-face role plays, a more traditional method of simulation (Fitch, Canada, Cary, and Freese 2016). Within the safety of the virtual simulation, students have the ability to rehearse and reflect upon the methods to use for the presented issues, without jeopardizing the protective standards of practice and client safety (Olson, Lewis, Rappe, and Hatley 2015). To facilitate learning, students may pause a simulation so that they can revisit a scenario multiple times or attempt to answer questions more than once. These systemic techniques will improve reflective skills (Boulos et al. 2007), which ultimately lead to better process and understanding.
The use of simulation training, particularly when incorporating standardized patient information, can in fact be utilized to measure and assess a student’s proficiency levels in the areas of practice and Field learning (Olson et al. 2015; CSWE 2015), both prior to and post the Field placement experience. This is an improvement over current proceedings, where unstandardized assessment modes are often used to evaluate student competencies (Washburn et al. 2016). Social work techniques are not always as clear and as easily defined as those in other disciplines; the delivery and operationalization of certain interpretive and evaluative skill sets are more abstract (Rishel and Majewski 2009) and they encompass various facets of social work subject matters. Therefore, it is important that the virtual simulation address these factors and the intersectionality of the social work curricula so that students may be evaluated on various levels: engagement, assessment, intervention, and critical thinking.
This paper aims to describe the development of two virtual simulations at New York University’s (NYU) Silver School of Social Work (Silver); the content of which is both innovative and educational. The first module is a 360 virtual reality (VR) program that allows social work students to navigate areas of the Lower East Side of New York City. The second, allows social work students to have an immersive experience of conducting a home visit to assess a client. Each of the virtual simulations operationalize critical thinking and attempt to concretize social work competencies in a holistic manner (CSWE 2015). Silver will endeavor to integrate each of the modules within courses to prepare students to enter community environments, and to hone social work practice skills, particularly related to client engagement and assessment.
Module One: Social Environment Immersion
Social work educators, particularly in Practice and Human Behavior related courses emphasize the correlations that exist between the social environment and an individual’s physical, cognitive, and emotional development. Most classes incorporate the biopsychosocial perspective as a tool to understand and process knowledge about a person’s biological, psychological and social interactions. This perspective views the person in the context of the environment and takes into consideration the challenges and stressors that might influence development throughout the life cycle. However, having students apply this perspective with clients as they work with them in their respective neighborhoods and communities can be challenging. Abstract theoretical principles are not easily transferable to real environmental circumstances, and students often struggle to recognize the theoretical frames in social contexts and then apply them to the populations that they are serving. An extensive body of research in the learning sciences supports the efficacy of learning activities situated in authentic practice and scaffolded within a community of practice (Lave and Wenger 1991; Collins 1987; Brown et al. 1989). Research on metacognition and instruction suggests that the incorporation of reflection opportunities into learning experiences improves the potential for knowledge transfer to future practice (Brown 1978; Flavel 1976; Bransford and Schwartz 1999). The 360 VR experience builds on models of cognitive apprenticeship by incorporating opportunities for corrective action based on reflection from students and feedback from instructors.
The Lower East Side (LES) is an area of New York frequented by social work master’s students completing the Field requirements of their curriculum. The LES was chosen as a pilot focus because of its demographic diversity, immigration history, social resources and its connection to the settlement house movement; each are important facets within social work (Citizens’ Committee for Children of New York 2015).
Due to budgetary restrictions on the acquisition and use of virtual simulation (VS) software, the authors adopted WONDA VR, a prosumer-oriented 360 video-editing platform. Wonda enabled our team to incorporate interactive elements such as hotspots, buttons, information panels, branching narratives, and a variety of other multimedia experiences relevant to the learning. In addition to accommodating the learning objectives and scope of the intended project, Wonda VR presented a robust alternative to more costly professional software platforms on the market.
Prior to the start of video production, Silver’s educational technologists designed a storyboard/scripting template that faculty could use for inputting content and visualizing the multiple branches of the narrative and user interactions. The process of designing this template was iterative; faculty and designers worked closely to develop a prototype using a basic spreadsheet that included columns describing each scene, associated competencies, learning objectives, action, and dialogue. Designers and faculty also collaborated on framing the substantive content within the context of the objectives and ensuring that it would succinctly coincide with each video still.
Figure 1. Google sheet storyboard template.
Our educational technology team videotaped each scene of the 360 video using a Nikon KeyMission 360 camera and standard tripod set up. Guided by the storyboard, the camera operator positioned the camera at predetermined locations on the Lower East Side, capturing two minutes of video footage per scene. Each scene of video footage was stitched together using Adobe Premiere software and an actor was subsequently enlisted to record voice-over audio of the script. The VR simulation took approximately nine months to complete.
Our budget included funding for a 360 video production consultant with advanced Wonda expertise who was brought on to complete the video during the final phase of post-production. With a background in fine art and documentary video production, the consultant worked iteratively with faculty and the education technology team to align his aesthetic sensibilities with the learning objectives, and multimedia cognitive design principles for engagement and learning (Mayer and Johnson 2010; Plass 2009).
The 360 VR simulation starts with the disembarkation from the F train, at the Forsythe subway station and ending with a sidewalk view of Orchard Street.
Figure 2. First scene from 360 VR: Forsythe train station with individuals on train platform. “Hotspot” is shown with the map of the Lower East Side.
The VR simulation is approximately 12 minutes in duration, and includes an instructional guide, learning objectives, content chapters, an introduction, and 17 navigable and interactive scenes (environments). Each scene contains verbal and textual dialogues, and a situational map. Reflective exercises and “hot spots” are presented throughout to guide students in thinking critically about the population inhabiting the Lower East Side, the community landscape, and data points, including demography, mental health, and housing. These key components were derived from the storyboard texts, which required collaborative editing by both designers and educators.
Figure 3. Street scene from 360 VR of the Lower East Side. “Hotspot” indicates the number of police stations, fire stations and EMS resources in the area.
The Silver School of Social Work aims to use the 360 VR tool within two courses and their respective curricula: Human Behavior in the Social Environment I (HBI) and Introduction to Social Work Education and Practice in the United States (ISWEPUS). The first, HBI, is a core course that all graduate students are required to take in their first year of study. It examines the different systems (individual, family, and communities) in the environment and the relationship that exists between them. Lessons two and three of the HBI course are specifically centered on the social environment, with topics that include risk and resilience factors within communities that impact human behavior, and social work interventions designed to develop and promote community resources. Faculty would use the VR tool to help students better understand how to recognize the resources that are both present and deficient within the community. Students would then critically think about the implications that their presence and absence poses to community residents. The VR’s verbal and written guide, scripted by faculty, poses reflective questions throughout the virtual journey, prompting students to think about community circumstances and their implications to target populations.
The ISWEPUS course is purposely created to prepare international students for coursework and Field instruction in the graduate social work program and within the greater context of the United States (US). In part, lectures and experiential exercises are constructed to help International students develop an understanding of the background of social work in the US, the core values of social work, and the nuances of the biopsychosocial perspective. Lessons four and five are centered on social work practice in the US. Students are given a foundation of social work history, where they are taught the legacies of the social change movement and they are introduced to perspectives on social reform. Within the context of these lessons, students will have an opportunity to engage with the 360 VR tool. Since the VR setting is the LES, which is historically significant to social work, immigration, and social reform, it will be shown to students via WONDA Spaces. This new platform allows multiple users to enter the virtual space simultaneously, enabling faculty to actively guide the students within the environment. The interactive class exercise will help students proactively apply theoretical ecologically centered models to communities, while also getting students to understand the impact that community environments have on individuals.
Before the school disseminates the 360 VR tool within the HBI and ISWEPUS courses, first-year students enrolled in the Masters of Social Work program at Silver School of Social Work are being asked to participate in a small IRB approved study. The project will assess the VR tool’s pedagogical value and student reactions to VR use, generally. Recruited students are contacted individually to schedule an appointment to view the VR simulation and to complete the pre- and post-test questionnaires. At the scheduled appointment time, students complete the pre-questionnaire via a Qualtrics access link. The questionnaire is based on a measure created and provided by North Carolina State University. The original measure was modified to accommodate the simulation. It contains questions on areas of learning perception, learning experience, technology experience, and general attitudes toward the 360 simulation. Once the questionnaire is completed, students are offered the option to view the simulation via VR goggles or via a designated computer desktop. The research assistant ensures that students are given VR access, and they are present throughout the VR simulation to monitor for any disruption in VR service. They also note any spontaneous feedback that is provided by the participant. Upon completion of the simulation, participants are asked to complete the post-questionnaire via a Qualtrics access link. Data will be downloaded from the Qualtrics website into a Statistical Package for the Social Sciences (SPSS) software.
Initial data comparing pre- and post-tests of students engaging in the 360 VR tool indicate statistically significant learning gains and also that students value and appreciate using 360 VR.
Module Two: Virtual Engagement Utility
By 2030, there are projected to be 80 million older persons, over twice the number living in 2000 (Lowell 2015). Yet many social work students and indeed professionals are reticent to work with this population, which is often based on fear, bias, and ageist beliefs. Research has repeatedly noted that consistent exposure to the population and its nuances is the strongest predictor for future gerontological work (Wang and Chonody 2013). Working toward this objective, a VR tool was created to provide students an opportunity to practice with an avatar so that they might gain some understanding on how it is to engage and assess an older adult in their home.
Silver partnered with NYU’s Teaching and Learning with Technology Group (TLT) to pilot an in-development virtual reality framework known as the Virtual Engagement Utility (VEU). VEU will be provided to NYU as a service to streamline the creation of educational virtual experiences. The client visit scenario built with VEU is multi-layered and contains 11 different stages of interactions that start from a knock at the client’s door initiating the visit and assessment process, to the summarization and end of the in-home visit, where the student can generate an evaluation sheet of how they progressed through the exercise. Such exposure might induce them to work with the population within their Field internships. As will be described, the VEU development was quite detailed and it was one of the first collaborative VR related exercises undertaken. The project took 19 months from its inception to the final deliverable product. The VEU is designed with a reusable framework so it is anticipated that future projects will take shorter periods of time to develop.
The client composite and the main artery of the script guiding the VR were based on the Field notes received from MSW students who were currently interning within agencies that primarily worked with older adults in the context of their homes. Issues cited by students within their notes served as a basis for the challenges that student users would face within the simulation. Once the main script was developed by faculty, it was then placed in storyboard and branching sequences were added to further enrich the interaction between student and avatar. The case dossier or file that was constructed for the client avatar was based on an assessment form used within a social work agency. VEU learning tasks and goals were based on learning objectives stipulated within core Practice courses within the social work graduate curriculum. The final script and storyboard was reviewed by faculty and a content expert.
Concurrent with script authoring, members of the Interactive Development team, part of NYU’s TLT group, created a branching narrative template. Template creation was an iterative process with input from faculty informing its clarity and ease of use. This template was used as a tool to both visualize and organize the complexity of a branching narrative, including interactions with the virtual environment, feedback on performance, and animation and audio cues for voice actors and software developers.
Figure 4. VEU branching narrative template.
VEU was developed in Unity, a cross-platform game engine. Virtual reality experiences created in the VEU framework are deployable to Google Cardboard and Google Daydream compatible devices, both smartphone based VR platforms. These platforms require VR viewers, which are available from both Google and a variety of third party vendors. VEU scenarios can also be deployed as a traditional interactive experience to PC and Mac platforms using a mouse and keyboard for interaction. VEU allows students to interact with virtual characters and their environment, complete tasks, save their progress, view feedback on their performance, and share their progress with faculty.
Under guidance from faculty and their content consultant, members of the TLT media group created both a virtual environment and client avatar. Environmental 3D models were sourced or crafted internally using Autodesk Maya. Textures were created using Adobe Photoshop. The apartment layout and appearance went through several iterations informed by reference images from the content expert and feedback from faculty within the School of Social Work.
The client avatar was created with Autodesk Character Generator and further customized in Maya. The script was recorded by a voice actor as individual responses while the actor’s face was captured using a basic HD webcam. The raw audio and video was processed using Adobe Premiere to remove background noise and normalize audio levels. Faceware Analyzer was used to map the actor’s facial expressions to the face of the avatar. Body animation and generic responses such as frustration, smiling, or nodding yes and no, were manually animated in Maya. These assets were imported into Unity, creating the environment and avatar that students interact with.
Figure 5. Interior Image of VEU apartment.
Figure 6. VEU interior living room area with client.
Framework Customization and Functionality
Once assets were imported into Unity, the VEU framework was used to create an interactive experience. Dialogue between the student and virtual client was inputted into a node based dialogue tree. When a student clicks on the avatar using a pointer in VR, or the mouse on a PC, they are presented with a dialogue bubble and multiple options to respond, guiding the conversation. Most choices provide some form of feedback. Optimal decisions allow the conversation to move forward. Less appropriate responses provide feedback, then return to the previous choice, allowing students to try again. Students can be prompted to observe and interact with their environment, identifying objects relevant to the client visitation. At any time the client file can be accessed so students can refer back to their visitation goals and client background information. Access to controls such as saving or sharing assessment results was also customized to appear as a smartphone. These customizations serve to provide a more immersive experience.
Figure 7. VEU student perspective.
Figure 8. VEU client file that can be accessed by students throughout the VR experience.
Outside of research and professional environments virtual reality is a relatively new technology. Many students (and faculty) who utilize the VEU tool may have no prior experience with VR. For this reason a custom tutorial scenario was developed with the assistance of Silver faculty. The tutorial was designed to cover the types of interactions students might encounter in their virtual visit. It demonstrates basic interactions such as pointing and clicking, interacting with a dialogue tree, and how to access the client file. It utilizes some assets from the client visit scenario so students will be familiar with the look and feel before beginning their assessment.
The School of Social Work aims to incorporate this simulation within the Integrated Social Work Practice Field Instruction I course (Practice I). This core class provides graduate students with generalist knowledge on working with a range of systems that include individuals, couples, families, agencies and communities. It also helps students develop the basic skills of engagement, assessment and goals setting. Lesson nine of this course is centered on teaching students how to conduct a multidimensional biopsychosocial assessment, which needs to be culturally sensitive and attuned to the client’s strengths and risks. The VEU tool would be an assignment that students would need to complete post-lesson. The VEU’s prescribed goals of engagement, assessment, and of identifying needs and resources are in keeping with lesson nine. The VEU also reinforces the prior Practice I lessons that are centered on engagement and the helping process. The internal evaluation that is generated by the VEU for each student will be assessed by faculty and then reviewed with the student. Students can share their VEU progress with faculty at any time. Faculty receive an email with the score for each learning sequence as well as the individual choices made. The value of each question can be weighted for greater control over performance evaluation. This data is anonymized. Students can choose to de-anonymize the data by providing their unique ID assigned when first activating the VEU tool. A reflective exercise post-VEU will also be assigned to better understand the student’s experience and to obtain knowledge on how to further support their learning. Students will have additional opportunities to engage their VEU experience through discussion with classmates in structured classroom activities, as well as a programmed opportunity to revisit the VEU later in the semester with the goal of interleaving their prior experience with later course learning experiences, such as Field work with clients. Research supports the educational value of interleaving multimodal approaches to similar content (Birnbaum et al. 2013), reinforcing prior knowledge, and contextualizing learning experiences within larger conceptual frameworks (Richland et al. 2005).
As was noted in an earlier subsection, the VR 360 tool is currently being assessed for its pedagogical value with students. Although initial results indicate there is overall positive reaction to the tool, further steps will be taken to purposely infuse the VR simulation within HBI and ISWEUS and then obtain student reactions both quantitatively and qualitatively. Similarly to the 360 VR simulation, the VEU simulation will be studied in a pre- and post-test research design in the coming months for its pedagogical value and whether students are comfortable navigating the VR tool. Also, social work faculty with Practice expertise will be asked to review the simulation and respond to questions related to learning objectives, subjectivity, content and overall quality. Results will inform the development of the current VR simulations as they pertain to content, structure, and design.
In addition, several steps will be taken to further improve the VEU simulation’s design and internal evaluative mechanism. Metrics can be reported such as tracking time spent in the experience or on particular questions, and whether the scenario was completed in one session or several. Engagement could be measured by checking if students skip the voiceovers of the avatar or if they completed optional assessments. These metrics could then be reported alongside the multiple choice performance, giving faculty deeper insight into the behaviors and learning styles of their students.
The VEU framework will be adapted to the Oculus Quest headset. In the process it will gain six degrees of freedom hand tracking, so students can reach out and interact with the environment in an intuitive manner. The Quest is also capable of room-scale tracking, which will allow students to walk around the apartment so they may perform closer inspections of the environment. Faculty can then integrate questions related to hazards and safety, while also integrating the assessment within the internal evaluative mechanism. Additional modes of interaction with the virtual client are also being explored, such as a “flow mode” where feedback is limited and the conversation proceeds naturally and with less interruptions. These features will hopefully lead to a more immersive and approachable learning experience for the student.
Birnbaum, Monica S., et al. 2013. “Why interleaving enhances inductive learning: The roles of discrimination and retrieval.” Memory & Cognition 41, no. 3: 392–402.
Boulos, Maged N. Kamel, Lee Hetherington, and Steve Wheeler. 2007. “Second Life: an overview of the potential of 3‐D virtual worlds in medical and health education.” Health Information & Libraries Journal 24, no. 4: 233–45.
Bransford, John D., and Daniel L. Schwartz. 1999. “Chapter 3: Rethinking transfer: A simple proposal with multiple implications.” Review of Research in Education 24, no. 1: 61–100.
Brown, Ann L. 1978. “Knowing when, where and how to remember. A problem of metacognition.” In Advances in Instructional Psychology, Vol. I, edited by R. Glaser, 77–165. Hillsdale, New Jersey: Erlbaum.
Brown, John Seely, Allan Collins, and Paul Duguid. 1989. “Situated cognition and the culture of learning.” Educational Researcher 18, no. 1: 32–42.
Collins, Allan, John Seely Brown, and Susan E. Newman. 1987. “Technical Report No. 403: Cognitive apprenticeship: teaching the craft of reading, writing, and mathematics.” Center for the Study of Reading: U. Illinois at Urbana-Champaign.
Fitch, Dale, Kelli Canada, Suzanne Cary, and Rebekah Freese. 2016. “Facilitating social work role plays in online courses: The use of video conferencing.” Advances in Social Work 17, no. 1: 78–92.
Flavell, J. H. 1976. “Metacognitive aspects of problem solving.” In The Nature of Intelligence, edited by L.B. Resnick, 231–35. Hillsdale, New Jersey: Erlbaum.
Lowell Sun. 2019. “Elder Care–Giving: Will You Be Ready for the Challenge?” Lowell Sun, July 11.
Lave, Jean, and Etienne Wenger. 1991. Situated learning: Legitimate peripheral participation. Cambridge, UK: Cambridge University Press.
Mayer, Richard E., and Cheryl I. Johnson. 2010. “Adding instructional features that promote learning in a game-like environment.” Journal of Educational Computing Research 42, no. 3: 241–65.
Olson, Mark D., Melinda Lewis, Paula Rappe, and Sandra Hartley. 2015. “Innovations in Social Work Training: A Pilot Study of Interprofessional Collaboration Using Standardized Clients.” International Journal of Teaching and Learning in Higher Education 27, no. 1: 14–24.
Plass, Jan L., Bruce D. Homer, and Elizabeth O. Hayward. 2009. “Design factors for educationally effective animations and simulations.” Journal of Computing in Higher Education 21, no. 1: 31–61.
Richland, Lindsey E., Robert A. Bjork, Jason R. Finley, and Marcia C. Linn. 2005. “Linking cognitive science to education: Generation and interleaving effects.” Proceedings of the Twenty-Seventh Annual Conference of the Cognitive Science Society 27: 1850–55.
Rishel, Carrie W., and Virginia Majewski. 2009. “Student gains in self-efficacy in an advanced MSW curriculum: A customized model for outcomes assessment.” Journal of Social Work Education 45, no. 3: 365–83.
Um, Eunjoon, Jan L. Plass, Elizabeth O. Hayward, and Bruce D. Homer. 2012. “Emotional design in multimedia learning.” Journal of Educational Psychology 104, no. 2: 485.
Wang, Donna, and Jill Chonody. 2013. “Social workers’ attitudes toward older adults: A review of the literature.” Journal of Social Work Education 49, no. 1: 150–72.
Washburn, Micki, Patrick Bordnick, and Albert “Skip” Rizzo. 2016. “A pilot feasibility study of virtual patient simulation to enhance social work students’ brief mental health assessment skills.” Social Work in Health Care 55, no. 9: 675–93.
Washburn, Micki, and Shu Zhou. 2018. “Teaching Note—Technology-Enhanced Clinical Simulations: Tools for Practicing Clinical Skills in Online Social Work Programs.” Journal of Social Work Education 54, no. 3: 554–60.
About the Authors
Nicholas Lanzieri is an Associate Clinical Professor at NYU Silver School of Social Work and Faculty Lead for Field Learning & Community Partnerships. He also co-leads Silver’s Faculty Education Technology Board.
Henry Samelson is the Associate Director of Educational Technology at NYU Silver School of Social Work. He received his MA in Digital Media Design for Learning/Games for Learning from NYU Steinhardt School of Media, Culture and Communications.
Jon Bowen received his MS in Biomedical Visualization at the University of Illinois at Chicago and is currently a senior interactive developer at New York University. He creates games and simulations for education and is interested in emerging technologies such as extended reality (XR) and spatial computing.
Tracy Dorrington-Skinner, Victims of Institutional Childhood Exploitation Society (VOICES)
Gerald Morrison, Victims of Institutional Childhood Exploitation Society (VOICES)
Tony Smith, Victims of Institutional Childhood Exploitation Society (VOICES)
The DOHR Team
Relational presence is the core principle of a new approach to designing virtual learning environments (VLEs), which has been developed by the Digital Oral Histories for Reconciliation (DOHR) project (dohr.ca). Presence, normally understood as the sense of being in a virtual environment to the extent that one forgets the environment is virtual, is thought to have significant pedagogical benefits in K–12 experiential learning projects aiming to develop spatial and social competencies that learners can translate into actual-world contexts. DOHR, by contrast, aims to build the understanding needed for learners to address systemic racism in Nova Scotia, through an oral history and restorative justice–based curriculum. To serve this alternative learning goal, relational presence replaces presence. The usual emphasis in VLE design on simulation, interactivity, identity construction, agency, and satisfaction is replaced with new values of impression, witnessing, self-awareness and awareness of difference, interpretation and inquiry, and affective dissonance. This paper introduces relational presence in order to help establish, in the field of VLE design, a productive discourse around issues of justice, representation of marginalized communities, and pedagogy-led design.
This article introduces relational presence, the core principle of a new approach to designing virtual learning environments (VLEs) that has been developed by the Digital Oral Histories for Reconciliation (DOHR) project (dohr.ca). DOHR has worked in partnership with the Nova Scotia Home for Colored Children Restorative Inquiry (restorativeinquiry.ca). The Restorative Inquiry was a four-year, provincially-mandated public inquiry into the history and legacy of the Nova Scotia Home for Colored Children (NSHCC), including the lived experiences of its residents. The Home was a segregated care institution for African Nova Scotian children that operated in Dartmouth, Nova Scotia from 1921 until the early 2000s. Established to meet the care needs of African Nova Scotian children, the Home was a site of significant abuse and harm for many of its residents. Over the decades of its operations, former residents experienced neglect and abuse (Province of Nova Scotia 2019, 153–172). The Restorative Inquiry was established to examine the experience of the Nova Scotia Home for Colored Children in relation to systemic and institutionalized racism, both historic and current, in Nova Scotia. In order to “contribute to the goal of social change to end the harmful legacy of abuse and ensure the conditions, context and causes that contributed to it are not repeated” (Province of Nova Scotia 2015, 4–5), among its goals the Inquiry aimed to:
(a) Empower those involved in, and affected by, the history and legacy of the NSHCC to learn about what happened and the contexts, causes, circumstances and ongoing legacy of the harms related to the NSHCC.
(b) Educate the public about the history and legacy of the NSHCC.
(c) Publicly share the truth and understanding established through the RI and the actions taken, planned, and recommended to address systemic and institutionalized racism and build more just relationships for the future (Province of Nova Scotia 2019, 23).
The DOHR project was an important mechanism through which the Inquiry pursued this part of its mandate (Province of Nova Scotia 2019, 504–505). The DOHR project has brought former residents of the Home, representatives of the Nova Scotia education system, and members of the Inquiry’s Council of Parties together with artists and researchers from seven universities across Canada (Waterloo, Dalhousie, New Brunswick, McGill, Ottawa, Alberta, and British Columbia) to develop a two-week grade eleven Canadian History curriculum unit that supports students in learning about the historical harms experienced by former residents of the Home. In this way, it has served to support the mandate of education and the broader goal of moving toward reconciliation by building the understanding needed to address systemic racism in Nova Scotia.
DOHR is thus a community-driven project. It arises from a need articulated by a community and works to co-create the project with community members. This community mandate is central to the need for designing relational presence in the virtual reality (VR) experience. The Restorative Inquiry, from which the DOHR project was created, pursued a restorative vision of justice that was reflective of a relational worldview focused on connectedness. It sought justice in the form of just relations between individuals, groups, communities, and at the level of institutions and systems (Llewellyn 2011). As a restorative process, the Inquiry was “future focused, yet concerned with getting a comprehensive understanding of the past in order to know how to move forward toward a just future” (Province of Nova Scotia 2019, 26). It focused on learning about past harms in order to build more just relations going forward. This is, in simplistic terms, the impetus for a restorative approach to learning in the DOHR curriculum. A restorative approach, as DOHR members Jennifer Llewellyn and Kristina Llewellyn have articulated, is grounded in relational theory. Relational theory holds that human beings exist in and through relationship with one another (Llewellyn, J. 2011; Llewellyn and Llewellyn 2015). The DOHR project reflects the premise that relationality is at the core of learning about such difficult knowledge as systemic racism in the Home and its legacy. Learning requires attention to the fact that we exist in and through relations, and this fact has implications for justice. Recognizing the relational nature of the historical harms of the Home, requires that learners listen to the lived experiences of former residents.
The DOHR project therefore co-created, as part of its curriculum, a placed-based oral history experience in virtual reality, with three former residents—Gerry Morrison, Tony Smith, and Tracy Dorrington-Skinner—who are recognized leaders and activists in the community. Scholars have demonstrated the many ways that oral history in education, both in conducting interviews and in listening to pre-recorded interviews, builds relational connections that are intergenerational and support reconciliation across divides (Llewellyn and Ng-A-Fook 2017; 2019). Unlike other oral history projects in schools, however, the DOHR project required that learners listen to stories in a contextual way that would connect them to a sense of place and the human experience of it—specifically, to the Home. Yet the DOHR team knew that former residents could not, nor should they be expected to, share their stories in-person with all students. The DOHR team also knew that not all students could visit the site of the Home and, even if it were possible, the site of the Home itself has changed significantly over the decades. While part of the Home’s building still stands, its present structure is considerably different from the structures in which the former residents lived (Morrison from 1954–60, Smith from 1965–68, and Dorrington-Skinner, who lived in the original Home building from 1972–78, and in the newer building now known as the Akoma Family Centre from 1978–84). Indeed, since early 2019, the site of the Old Home has been undergoing yet another phase of major renovations (see Figure 1). Since students cannot interact with the former residents or the site of the Home directly, the DOHR curriculum exemplifies the kind of experiential learning that is consistently identified in the VLE literature as likely to benefit from a virtual learning environment (VLE), and ideally one that is VR-based. DOHR wants to deliver experiential “learning tasks which are expensive” (Dalgarno and Lee 2010, 19) or even “impossible” (Bulu 2012, 153; Kwon 2019, 105) in real life. In order to provoke new, relational understandings of the Home and of systemic racism, which are further supported in the fuller DOHR curriculum, the former residents’ oral histories are shared in DOHR’s VR-based VLE.
Figure 1. At top left, the original Nova Scotia Home for Colored Children (the “Old Home”) on the occasion of its official opening in June 1921. Top right, a large brick extension was added in 1961. Bottom left, the “New Home” building built in 1978 (now the site of the Akoma Family Centre). At bottom right, the Old Home as it appeared during the DOHR research team’s site visit in April 2019. (For further details see Chapter 3 of the Restorative Inquiry Report [Province of Nova Scotia 2019].)
However, DOHR has taken an unusual approach to the design of its VR-based VLE, because the learning outcomes that the DOHR project fosters—grounded in the commitment to restorative justice which seeks to foster just relations (Llewellyn 2011; Llewellyn and Llewellyn 2015)—are unusual in the context of VR-based VLE design. Specifically, we have developed a different approach to presence, which is broadly understood to be the element of VR design that contributes most to student learning in VLEs, and is hence the principal design aim of most VLE projects. Presence is the sense of “being there” (Slater and Wilbur 1997) in a virtual experience or “the psychological state where virtual experiences feel authentic” (McCreery, Schrader, Krach, and Boone 2013, 1635). Outside of the context of VLE design for experiential education, virtual environments have been designed with quite different aims. The digital humanities, for example, has emphasized the use of virtual reconstruction in research contexts as “not a neutral representation of ‘the past,’ but the scholar’s interpretation of specific aspects of a place at a certain time—an interpretation that can be challenged, revised, or rejected” (Sullivan, Nieves, and Snyder 2017, 301; emphasis original). Since the goal of creating a virtual environment in the digital humanities has been to make an argument and provide the locus for future argumentation (Sullivan, Nieves, and Snyder 2017, 301; see Roberts-Smith et al. 2016; Roberts-Smith 2017), attention has been paid to VLE design that encourages critical creativity rather than presence (Roberts-Smith et al. 2013). There is also some very recent, parallel work in VR design for K–12 education exploring the ways students productively fill in the gaps in imperfect historical simulations without compromising their sense of historicity (Papanastasiou et al. 2019). By contrast, in what has come to be known as “immersive journalism” (de la Peña et al. 2010; Reis and Coelho 2018), the use of 360º video to place the viewer in the “center” of a documented event is designed not to help participants learn to do anything specific, but to encourage them to empathize with victims of injustices (see for example Torsei and Philippe 2019). To date, however, neither these alternative approaches to virtual environment design nor their critiques (e.g. Reis and Coelho 2018; Mabrook and Singer 2019) are well integrated into the discourse around VLE design for experiential education. Similarly, 3D graphical approaches to the representation of marginalized communities remain under-interrogated in the VLE design literature, despite some robust work in this area emerging from game studies (e.g. Reis and Coelho 2018; Malkowski, Russwork, and Trea 2017; Taylor and Voorhees 2018).
Our aim for the DOHR project and this paper is to broaden the conversation about VLE design—which has largely followed technology- and psychology-driven lines of thought originating in the early 1990s—to accommodate issues that have arisen more recently in fields outside of VLE design, through a discussion of the DOHR VLE. Since perspectives on how to achieve presence in VLEs, and why such presence is effective, are quite dispersed even in the VLE literature, we begin our discussion here with a synthesis of the most influential concepts. We then provide a description of the DOHR VR experience, and a discussion of how it approaches presence differently, consistent with the relational principles of a restorative approach. In conclusion, we offer some preliminary reflections on the DOHR VR experience’s effectiveness as a learning tool and suggest next steps for future development.
Presence and Learner Engagement in VLE Design for Experiential Learning
VR-based VLEs are most commonly designed to help learners (whether in school contexts or in public education contexts) to develop either spatial or social competencies that are impractical and/or dangerous to teach, especially at introductory levels, in the actual world. Widely-publicized examples include VR-based small motor-skill training for surgeons, in which learners use physical surgical instruments as controllers of avatars of the same instruments, to perform virtual surgeries on digitally-simulated bodies; such systems are increasingly used not just to train new surgeons but also to refresh the skills of practicing surgeons before performing actual-world surgeries (as in Surgical Science’s VR training system for laproscopy and endoscopy). In socially oriented VLEs, students typically learn how to avoid or respond positively to harmful behaviors, such as racial stereotyping, by rehearsing actions in virtually simulated scenarios (as in Kaplana’s 2015 Injustice); or how to develop empathy for the suffering of others by experiencing a simulation of their hardships (as in Kors et al.’s 2016 A Breathtaking Journey). Since the expectation in both kinds of VLE is that students will be able to transfer competencies developed in virtual reality into actual-world situations, these VLEs strive to simulate real-world experiences as vividly and accurately as possible, often incorporating actual-world material objects as well as virtual simulations, such as the surgical instruments used by Surgical Science. Kwon (2019) argues that immersive VLEs are especially relevant to the first stage of Kolb’s (1984) model of the four circulative processes of experiential learning, “concrete experience,” which could be followed by “reflective observation, abstract conceptualization, and active experimentation” in the classroom.
There is general agreement in the literature on VLE design for experiential learning that the closer a simulated experience is to an actual-world experience, the better it functions as a replacement for real-world experience. When a simulation is effective, it produces in the participant the feeling referred to as “telepresence,” which is usually abbreviated to “presence”: “the psychological state where virtual experiences feel authentic” (McCreery, Schrader, Krach, and Boone 2013, 1635). If a technologically mediated experience is effective in generating the sense of presence, the perception of the person experiencing it “fails to accurately acknowledge the role of the technology in the experience” (International Society of Presence Research). In other words: it is generally accepted in the field that if a VR experience is effective in generating presence, the participant forgets they have a VR headset on, and instead feels like they are “there” in the illusion the headset is creating. In education, this experience of forgetting you are in a VLE has been seen as an advantage for experiential learning. Since experiential learning is thought to have been achieved when virtual experience is recognized as similar to an actual experience, “enhanced presence” is an ambition of VR-based VLE design (Kwon 2019, 105).
The sense of presence in VR-based VLE design is often understood to arise from immersive hardware systems (Fowler 2015, 416). Immersion here is understood as the “degree to which a virtual environment submerges the perceptual system of the user in computer-generated stimuli” (Biocca and Delaney 1995). In this understanding, the perceptual system is submerged physically, by the technical hardware employed to create the illusion of the virtual environment. For example, we might think of a VR headset as more immersive than a desktop computer screen, because it excludes the perception of visual stimuli that are not part of the virtual illusion (Dalgarno and Lee 2010, 11). However, as the examples of Surgical Science and Injustice demonstrate, hardware alone is not the greatest determiner of perceptual submersion in immersive systems (Archer and Finger 2018); rather, perceptual submersion is achieved by the degree to which a virtual illusion explicitly mimics the actual world. Although there is no consensus in the literature on the most effective design practices for achieving presence in VR-based VLEs, three design factors are regularly identified as having a significant ability to increase perceptual submersion: representational fidelity (the degree to which a virtual illusion looks or sounds like reality), interactivity (the degree to which the virtual illusion responds realistically to the embodied actions of a spectator), and identity construction (the degree to which spectators can associate themselves with characters represented in the virtual environment). These factors are normally differentiated from one another in the literature, but are also understood to be interdependent in ways that are not yet consistently articulated.
Representational fidelity, for example, is often understood to be achieved by one or more of the following four factors:
(a) The vividness, or “abundance of reenactment … providing information to the senses” (Kwon 2019, 102; see Steuer 1992). On a sliding scale, at the low end of what VR systems can deliver, only the sense of sight is engaged; in more sophisticated systems, hearing is engaged; then at the cutting edge of what is currently possible, touch is manipulated. Taste and smell remain beyond the capacity of existing virtual systems, available only in “actual reality” (see Figure 3).
(b) The realism of the virtual illusion (Bessa, Melo, Sousa, and Vasconcelos-Raposo 2018, 35; Bulu 2012, 156), including its three-dimensionality (Bulu 2012, 154; Dalgarno and Lee 2010, 11); the ways in which it represents users (Fowler 2015, 413); and “the consistency of object behaviour” (Fowler 2015, 413; see Dalgarno and Lee 2010).
(c) The plausibility and dynamics of the virtual environment, including such technical effects as reflecting mirrors or shadows (Sanchez-Vives and Slater 2005).
(d) The “quality of the display, with high-fidelity displays being most realistic or photorealistic” (Fowler 2015, 413; see Dalgarno and Lee 2010). In other words, the sophistication of the equipment used to deliver a VR experience.
Similarly, interactivity is often understood to be achieved by one or more of the following:
(a) The range of embodied actions available to the VR participant (Dalgarno and Lee 2010; Kwon 2019); in other words: the degree to which a participant can use their body in the ways they would in actual reality, by touching, speaking, or moving around, for example.
(b) The degree to which the virtual illusion responds to the participant’s actions (Murray 1997; Dalgarno and Lee 2010; Kwon 2019), when, for example, objects move or other avatars engage in conversation.
(c) The degree to which the participant can create new elements of the virtual illusion (Dalgarno and Lee 2010; Kwon 2019).
(d) The technical ability of the system to respond to action through, for example, head-tracking (Sanches-Vives and Slater 2005) or update rate (Barfield and Hendrix 1995, 3).
However, the first of the four principles thought to contribute to representational fidelity is also sometimes treated independently as a factor that interacts with interactivity to increase a virtual illusion’s ability to simulate reality. Kwon, for example, sees VR-based VLEs that leverage the sense of touch and enable a wide range of bodily gestures as more “authentic” in the sense that they provide a more vivid experience closer to actual reality (see Figure 2). VR that is “authentic” in this way is particularly good at generating a sense of “place illusion” or “place presence” (Bulu 2012), the sensation of being and operating at a remote or virtual place (Slater 2009), or “being there” in the place depicted by the virtual display (Slater and Wilbur 1997). Hence “place presence” is often a design goal of VLEs whose intended learning outcomes include spatial competencies that can be translated to actual-world scenarios (such as, for example, Surgical Science).
Figure 2. “Relationship between virtual reality and actual reality based on the degree of presence” (Kwon 2019, Figure 2).
The third major design factor, identity construction, by comparison, is often understood to be an outcome of the first two, representational fidelity and interactivity (see Figure 3). Identity construction normally refers to the sense of personal “body ownership” (Slater 2009) that learners develop by associating themselves with a manipulable avatar in a virtual environment (Bulu 2012, 154; Fowler 2015, 414). It can also refer to a learner’s ability to construct identities for other player-participants through their respective avatars (Fowler 2015, 414; Bulu 2012, 155; Biocca et al. 2003; Schroeder 2002). Identity construction is often leveraged in educational contexts to help generate a sense of “co-presence”, or “being there together” (Fowler 2015, 414) in a virtual environment. Co-presence has two dimensions: “perceiving others and having a sense or feeling that others [are] actively perceiving us and being part of a group” (Goffman 1963; Slater, Sadagic, Usoh, and Schroeder 2000). Co-presence is normally understood to involve a sense that there is “psychological interaction” among individuals (Nowak 2001; Schroeder 2002; Bulu 2012, 155). As a result, identity construction is often a design goal of VLEs whose intended learning outcomes include social competencies that can be translated to actual-world scenarios (such as Injustice).
Figure 3. Dalgarno and Lee’s elaborated model of learning in a 3D VLE (Dalgarno and Lee 2010, Figure 1).
In general, however, whether designed for spatial or social learning tasks, presence is thought to have three major pedagogical benefits for learners. First, presence helps students focus on the learning tasks they are encountering in a VLE, developing a task-oriented “flow.” When students experience “flow”—the term used in the literature to describe “the state of being absorbed by an activity” (Scoresby and Shelton 2011, 227), which “mediates the relationship between presence and enjoyment” (Weibel, Wissmath, et al. 2008, 2274)—they learn better (Kwon 2019; see Figure 4).
Figure 4. Influence relations among vividness, tactile interactivity, locomotive interactivity, simulator sickness, presence, flow, and learning effect (Kwon 2019, Figure 8). Note that simulator sickness is a counter-indicator of flow here.
Second, presence helps create the sense of agency that learners need to have in order for learning to be experiential. Triberti and Riva, for example, describe presence as “a core neuropsychological phenomenon whose goal is to produce a sense of agency and control: I am present in a real or virtual space if I manage to put my intentions into action (enacting them)” (2016, 2). As Janet Murray puts it, agency is “the satisfying power to take meaningful action” (1997). The third benefit, which arises from the first and second, is that presence is also frequently associated with students’ satisfaction with their own learning (e.g. Bulu 2012). Student satisfaction is a measure frequently used to determine the effectiveness of a virtual learning activity (see Kwon 2019, for example).
If we were to extract the best practices for VR-based VLE design from the literature survey above, we might end up with something like: Make a high-fidelity simulation of the relevant actual-world environment; give learners a way of affecting the virtual learning environment and make the environment respond; and provide representations of learners themselves in the world. Thanks to the resulting sense of agency they will then feel, learners will forget they are in a virtual environment, getting into a flow where they are totally focused on their learning tasks. The outcome will be that they learn effectively and feel satisfied with their learning experience. According to the current state of the VLE literature, then, in an effective VR-based VLE a learner perceives themselves acting in a simulation and perceives the simulation responding; the resulting agency, presence, and flow lead to learning and satisfaction. While the literature describes best practices for VLE design to support the kinds of spatial and social learning outcomes commonly intended in experiential learning curricula, it does not support the DOHR curriculum’s intended learning outcomes.
Designing Presence in the DOHR VR Experience
The DOHR VR experience is a thirteen- to fifteen-minute individual learning activity that is embedded early in a five-lesson curriculum designed to run, typically, over the course of ten history classes. The VR experience was designed to support learning activities outside of the VLE that are constructed based on the principles of historical thinking and oral history education, and on a restorative approach to learning (Llewellyn and Llewellyn 2015; Llewellyn and Ng-A-Fook 2017; 2019; see also Gibson and Case 2019; Epstein and Peck 2019; and the Historical Thinking Project). The first two lessons invite students to join the former residents in their decades-long journey to bring their stories of the Home to light in order to build a better future. Students are introduced to a brief history of the Home and then asked to actively inquire about the historical significance, causes, and consequences of the Home. Their engagement in this inquiry is centered on an examination of oral history as a primary source in itself and in relation to other primary historical evidence (such as social worker reports, newspaper articles, and photographs). The lessons culminate with students developing a “restorative plan” that asks them, among other aims, to share what they have learned in a way that will do justice to the historical experiences of the former residents and contribute to the future-focused goal of reconciliation.
In the third lesson, learners are on-boarded in small groups to the DOHR VR experience in person, by a trained facilitator, at individual stations. The facilitator advises students how to end the experience if they find its content distressing. The facilitator leads a short “sharing circle” (a key activity in a restorative approach to learning) to debrief about the experience afterwards. The VR experience itself begins with a short, documentary-like 360º video segment in which learners see the storytellers, Smith, Morrison, and Dorrington-Skinner, and hear their voices in voiceover. The introduction of the storytellers is followed by a set of oral histories rendered in a multi-modal blend of 3D graphics, 360º and 2D video, 2D images, environmental and spatially-located sound, voiceover narrative, and text. There are 12 stories in total, but each learner can only choose to witness three, one from each storyteller. Following those three stories, all learners witness a short sequence in which the three storytellers reflect together about their memories of one common room in the Home. Finally, learners witness another 360º, documentary-like video for a concluding sequence in which the storytellers describe directly (that is, without the use of voiceover) how they came to be the activists they are today.
From its inception, then, the challenge for the DOHR team in developing the VR experience has been that it is intended for a different kind of use-case than other VR-based VLE projects. DOHR is a project that seeks for students to build a relational understanding of the historical harm of the Home by hearing the oral histories of former residents. The intent of the curriculum is for students to ask: What relationship do they have to, and thus what responsibility do they have to address, the history of harm, based on systemic racism, that is the legacy of the Nova Scotia Home for Colored Children? The aim of the curriculum, including the VR experience, is for students to build a sense of relationship to the lived experience of a place, even though they will never likely be in the actual Home nor meet the former residents in person. This means, in part, that a traditional sense of place presence in the VR experience is not important, since spatial awareness and spatial skills are not primary learning outcomes. Similarly, traditional social presence is not useful, since our aim is not to help students practice social behaviors in a virtual environment so that they can adopt them more confidently or consistently in the real world. What we need students to do is to consider their stance in relation to the stories of the former residents in order to inform their understanding and efforts toward just relations with those whose lived experiences are different from their own. In other words, instead of generating a sense of presence as it is traditionally understood, where a virtual experience feels like an experience that could be replicated in the real world, we need students to remember that they are witnessing a story being told through the perspective of another person, which they could never experience themselves in the real world. The term we are using to describe this form of presence is relational presence.
Relational presence has had several concrete implications for the design of the DOHR VR experience. Since we need to help foster an understanding of what it was like to live in the Home, for different people at different times, we are not invested in representational fidelity in the way that other projects are. Although we have worked extensively with architectural drawings, photographs, and other archival and archaeological evidence of the past structure of its site and buildings, in our renderings, we express the Home in a multi-modal, impressionist aesthetic that reinforces the former residents’ oral histories. Most stories, for example, begin in a line-drawn, white-on-blue rendering of the site of the Home that is intended to evoke a three-dimensional version of the architectural drawings used to structure 3D space at each point in the narrative. The invocation of the documentary record only becomes substantial—opaque 3D graphics, 360º video, light, and environmental sounds helping to establish a more specific impression of place, time, mood, and activities—as the voice of the storyteller (the sound that appears closest to a participant’s ear) begins to recount the story. The representational media are in turn combined in ways that neither attempt to mask the differences among them, nor their individual differences from the actual-world phenomena they evoke. Our aim is to privilege lived experience over the fragmentary documentary record, making it clear that the world learners are encountering is not an attempt to reconstruct the past through simulation. Instead, it is an attempt to construct a present encounter with oral histories about past experiences in the Home and the long-term impacts of those experiences. In contrast to the traditionally sought sense of “place presence”, then, the DOHR VR experience seeks to foster a sense of what we are calling “relational place.” Relational place is an invocation of what a place means—in the case of the DOHR VR experience, what it means for storytellers and learners—rather than a simulation of how a place looked or was configured at any given point time (see Figure 5).
Figure 5. Screen captures showing three different approaches to multi-modal impressionist rendering in excerpts from Morrison’s story, “Swamp Water” (top left); Dorrington-Skinner’s story, “Mrs. Johnson’s Helper” (top right); and Smith’s story, “The Switch” (bottom). See Roberts-Smith et al. 2019.
Since learners need to maintain a sense of the difference between their own perspectives and the perspectives they are learning about, we are, similarly, not invested in identity construction in the sense of the identification of self or others with avatars to enhance social presence as it is understood in the VLE literature. Rather, we seek to support in students the development of a sense of relation to the stories rendered in our VR experiences and the storytellers from their own position and perspective. The VR experience does not create the illusion that the storytellers are really “there” with learners in the virtual environment. This means there are no anthropomorphic avatars in our VLE, and we make no attempt to create roles or characters for the storytellers or for learners to “play.” Learners are characterized by means of an avatar that is a literal representation of the story-selection controller held in the learner’s hand—the only avatar in the entire build—only as the force that uses the controller to select a story. Instead of creating virtual representations of either storyteller or student, we make space for each to occupy their own, actual-world perspectives. For storytellers, that means that their oral histories are told in recordings of the adult storytellers themselves, and for learners, it means witnessing those stories as grade eleven Canadian History students themselves, and not in the kind of role-play scenario that is common in social competency VLEs.
The emphasis on witnessing oral histories of the Home, then, means that there is also very little traditional interactivity in the DOHR VR experience. Since the world of our VLE represents the lived experiences of the storytellers, students do not need agency in the sense of being able to take action that initiates a response from objects or characters in the virtual world. If they were able to do that, the world would no longer represent the storytellers’ perspectives, and would not help learners understand the difference between their own perspectives and those of the storytellers. It would also give students the illusion of having power to change the stories, which, since justice in the DOHR project depends on hearing stories that have not been heard before, would subvert the project’s aim of encouraging an active listening that may provoke new understandings of the past. It would undermine the students’ ability to consider how lessons from the past can contribute to future just relations, as required of them in the restorative plan they develop as part of the curriculum. In the DOHR VLE, interactivity is hence extremely limited in traditional VLE design terms. To the extent that it is available at all, it is designed to characterize learners as witnesses to the stories. Students are able to choose one story from each of the three former residents whose stories are represented, and then listen to it. However, learners are not inactive, because listening to the stories is itself an important cognitive activity that has been recognized, for example, in Indigenous studies as an active “inhabiting” of representational worlds (Ridington 1998), and in performance studies as an active self-reflection on one’s “role and experience as a spectator” (Rokem 2002). Bronwen Low, a member of the DOHR research team, has written about “the pedagogy of listening” in similar terms (Low 2015). Drawing upon the work of Jean-Luc Nancy, Low describes listening as a learning process that extends the ear towards the other. This is not a silent nor passive process, but rather, one that builds relations of deep listening between storyteller and listener (Low 2015, 270–75; Llewellyn and Cook 2017). We differentiate this form of “witnessing” from “co-presence” as it is understood in the field of VLE design: namely, as the perception of others in a virtual environment, and the perception that others perceive us, giving us the sense that we are “part of a group” (Bulu 2012, 155; see Goffman 1963; Slater, Sadagic, Usoh, and Schroeder 2000). In VLE design, “co-presence” normally refers to the presence of other users in the virtual environment, and it is achieved through synchronous user-manipulated avatar interactions. By contrast, we are interested in giving students a sense of relation not to other VR participants, but to the lived experience of the non-player characters of the storytellers, Gerry, Tony, and Tracy, who are not simultaneously present, but represented through pre-recorded media and pre-fabricated digital assets. Relational interactivity, then—the invitation to witness—places agency in the context of relationship. The learner’s power is to relate across differences in perspective.
In these different approaches to representational fidelity, identity construction, and interactivity, DOHR takes an approach that is also different from precedents in fields outside VLE design. For example, despite a shared lack of interest in in-world interactivity, our conception of “witnessing” also differs significantly from the concept of non-interactive “immersive witness” (Nash 2018) that has been taken up in 360º video-based journalism inspired by the work of Nonny de la Peña (2010). In this context, 360º video is understood as a means of simulating a distant event, and consequently as offering the experience, rather than a representation, of that event; this experience is “immersive witnessing” (Nash 2018). While immersive witnessing has been critiqued for its relative lack of interest in the distance or disinterestedness normally expected in journalistic reporting (Nash 2018; Reis and Coelho 2018), it has been taken up by activist and humanitarian organizations because it was thought to instill a sense of responsibility for others. However, immersive witness makes different assumptions than the DOHR project does about the nature and aims of witnessing. Immersive witness is interested in “providing the audience with something of an experience that is linked in various ways to the experiences of others” (Nash 2018) through passive reception of a photo-realistic simulation. DOHR, by contrast, avoids simulation to encourage the active exploration of differences of perspective arising from differences in lived experience.
Finally, another important difference between the DOHR VLE and other VLEs is that witnessing the different perspectives of former residents of the Home can be an uncomfortable experience. The stories are about the harms that former residents suffered there, and the resilience that they and other children drew upon to survive those harms. So, another difference between relational presence and traditional presence is that, although relational presence can be absorbing and lead to the kind of “flow” where students are fully engaged in their learning task, it does not necessarily offer the pleasant kind of self-satisfaction related to taking action within the VLE intended by Janet Horowitz Murray (1997) and others. Instead, relational presence can lead to affective dissonance, which is the discomfort we feel when we experience difficult knowledge (Zembylas 2015; Zembylas and Bekerman 2008; Simon 2015). That discomfort prompts thought-provoking questions for learners, providing a different opportunity and experience of agency, which learners explore in the fuller classroom curriculum. These questions include: How could this have happened? Why didn’t I know about this before? What is my responsibility now that I know these stories? That kind of questioning is a learning agency—the agency to inquire and to reconsider how we act in and through relationship with others in the world.
Designing the DOHR VR experience has suggested to us that presence need not necessarily be understood as a simulation-based forgetting that we are witnessing an illusion, nor as an erasure of our awareness of the technology that delivers it. The project does not use a model of presence that requires the reconstruction of spaces for us to “be” in, identifying with representations of ourselves or others, and feeling satisfying agency by interacting with place and social context. Instead, we think of presence as the unsettling agency to witness a different perspective on meaning, which offers an opportunity to consider, and possibly change, our actual-world understanding and behavior as a result. If presence is thought of relationally, an alternative model of effective VLE design emerges. Instead of acting in a simulation, a learner occupies a relational place to witness a story. The resulting sense of relational presence fosters forms of agency and affect that are critical for learners to inquire further and seek restorative actions for justice in their actual-world contexts. For DOHR to achieve relational presence, our VLE needed to offer the opportunity to witness a past world described by those who lived it and provoke questions, based on the opportunity to witness, that would otherwise be impossible to formulate. In the case of the DOHR project, relational presence was achieved by means of a mixed-mode, impressionistic representation of the lived experience of a real-world environment, which avoided avatars, and limited interactivity to opportunities to witness (see Table 1).
simulation (representational fidelity)
impression (representation of meaning)
identity construction (recognizing self)
self-awareness, awareness of difference
Table 1. A comparison of traditional and relational approaches to presence in VR-based VLE design.
In the DOHR VR experience, we have developed a theory of relational presence, and one approach to achieving it, which have yet to be validated through empirical study of student learning, or in other VR-based VLE design projects. At the time of this publication, the DOHR team has conducted a study of the DOHR curriculum, including the VR experience, and is analyzing the data. Preliminary results from the data indicate positive learning outcomes. Students reported sensations that indicated they did experience a strong sense of flow, and acquired important new knowledge, despite our unconventional approach to designing the VLE. Future study of the delivery of the curriculum will help the team to understand how diverse social and geographical factors affect learning outcomes, and the need to address the accessibility limitations of our current design, beyond physical and auditory enhancements. An analysis of the data from classroom implementation will provide us with the evidence required to determine how the VR experience with a focus on relational presence, embedded within the curriculum as a whole, may lead to learners’ increased relational competency; that is, to an increase in students’ ability to engage in the work of building more just relations in their worlds.
Two avenues for further research have already suggested themselves to the DOHR team. First, there may be productive research to be done on the role of aesthetics as a support for learning in VLEs. The concept of representational fidelity has so far been limited to a very narrow subsection of what might be more fully understood as the representational aesthetics of a virtual learning experience (by which we mean the intentional manipulations of the media of expression to both represent and generate experiential phenomena). This may be a result of the strong influence, to date, of STEM disciplines on the development of existing VLEs; STEM-based work typically thinks of aesthetics as a means of enhancing user experience and usability (e.g. Tuch et al. 2012). In the DOHR VR experience, we found that a significant investment in representational aesthetics was essential to the pedagogical goals of the project—both in terms of the theatre-based design experts gathered to work with former residents of the Home and other members of the DOHR steering committee on the VR development team, and also in terms of financial and material resources. What the DOHR build lacks in traditional presence-inducing features, it perhaps makes up for in aesthetic features. This may also explain why the DOHR team found some preliminary precedents for some components of relational presence in Indigenous studies and performance studies, two fields that are deeply invested in what we might think of as a twenty-first century development of what Nicholas Bourriaud first termed “relational aesthetics.” For Bourriaud, “the possibility of a relational art (an art taking as its theoretical horizon the realm of human interactions and its social context, rather than the assertion of an independent and private symbolic space), points to a radical upheaval of the aesthetic, cultural and political goals introduced by [its predecessor] modern art” (2002, 14; emphases original). The DOHR VR experience’s stylistic gesture of intermedial, impressionist representation to achieve relational presence may be one way VLEs can begin to explore social systems (or “formations,” as Bourriaud calls them) in much richer ways than the field of VLE design has yet done. Similarly, DOHR’s engagement of a relational approach may encourage others to explore the ways in which Bourriaud’s articulation of “relational aesthetics” might be altered or expanded to better serve the aims of projects fostering restorative justice.
In addition, there is certainly more work to be done on the contextualization of VR-based VLEs within classroom-based curricula and with reference to in-class teaching strategies. Work in this area currently consists of a decade or more of advocacy in pedagogical game studies (see, most recently, Hébert and Jenson 2019 for both context and evidence of best practices). Although VLE designers have devoted a great deal of energy to the technical design of stand-alone VLEs, the field has not yet taken full advantage of the opportunity to apply best practices in pedagogical design to VLEs (Fowler 2015). This could be done in stand-alone VLEs but could also be approached by situating a VLE as one in a series of classroom learning activities, as the DOHR project has done. A significant advantage of considering VLEs in the context of an overall blended (in-person and virtual) curriculum design is that it avoids “technological determinism” (Reis and Coelho 2018, 1093) whereby virtual experiences are “considered both a product and an outcome of technology” (Reis and Coelho 2018, 1093), rather than an outcome of the ways designers have manipulated the technologies in question. Understanding a VLE as one learning activity in the context of a larger curriculum necessarily makes its technology secondary and emphasizes the agency of educators to design and use the VLE in the ways that best serve their students. Relational presence is one offering that DOHR can make to the larger project of reconsidering the role of VLEs in K–12 and public education, with a view to addressing issues of pedagogy, representation, and justice that are not yet well accounted-for in the field.
 The DOHR VR experience was designed using a process that aligns generally with the principles of “co-design” articulated in the seminal work of Steen (2013), whereby parties characterized as “stakeholders” are actively involved throughout the design process and afterwards (Steen 2013); as distinct from “participatory design,” in which stakeholders are consulted only at key points (Schuler and Namoika 1993; Björgvinsson, Pelle, and Hillgren 2010). However, DOHR’s process differs from this and other activist, participatory artistic practices leveraging digital media (e.g. Gubraim, Harper, and Otañez 2015) in its centering of a relational approach to all project activities. The full citation for the DOHR VR experience, acknowledging specific roles of individual co-design participants, can be found in our reference list under Roberts-Smith et al. 2019.
 See Steuer 1992 (also cited below) for an example of an influential early technology-focused work; Chittaro 2013 for an example of work using psychological concepts to better understand human-computer action; and Riva 2018 for a compelling example of the integration of philosophy, human-computer interaction, and psychology in current work. Lombard and Ditton 1997 offer a survey of early 1990s trends; Fowler 2015 and Reis and Coelho 2018 critique the outcomes of the emphasis on technology in particular.
 We note, however, that the classical conditions under which virtual experience is advantageous (i.e. where embodied experience is “expensive, dangerous, or impossible” [Dalgarno and Lee 2010]) may be as likely to occur at the active experimentation stage as at the concrete experience stage.
 An important additional factor, beyond the scope of our discussion here, is immersive tendency, which operates outside of the context of the VLE itself. Immersive tendency refers both to the pre-disposition of some participants “to involve and focus on the [sic] common activities in real life” (Bulu 2012, 159), and also to participants’ desire to immerse because they “have specific expectations about what the outcome should be” (Shin 2017, 71; citing Weibel et al. 2010; Burns & Fairclough 2015; Hou, Nam, Peng, and Lee 2012).
 Kwon’s scale addresses only the five most familiar senses. There is also a great deal of work being done on proprioception in research related to motion sickness in VR, which Kwon acknowledges as a counter-indication of presence. For a substantial review of the relevant literature, see Weech, Kenny, and Barnett-Cowan 2019.
Archer, Dan and Katharina Finger. 2018. “Walking in Another’s Virtual Shoes: Do 360-degree Video News Stories Generate Empathy in Viewers?” TOW Reports. Columbia Journalism Review. Tow Center for Journalism, Columbia University Graduate School of Journalism. March 18.
Barfield, Woodrow, and Claudia Hendrix. 1995. “The Effect of Update Rate on the Sense of Presence within Virtual Environments.” Virtual Reality 1, no. 1: 3–15. https://doi.org/10.1007/BF02009709.
Bessa, Maximino, Miguel Melo, A. Augusto de Sousa, and José Vasconcelos-Raposo. 2018. “The Effects of Body Position on Reflexive Motor Acts and the Sense of Presence in Virtual Environments.” Computers & Graphics 71 (April): 35–41. https://doi.org/10.1016/j.cag.2017.11.003.
Biocca, Frank, and Ben Delaney. 1995. “Immersive Virtual Reality Technology.” In LEA’s Communication Series: Communication in the Age of Virtual Reality, 57–124. Hillsdale, New Jersey: Lawrence Erlbaum Associates, Inc.
Björgvinsson, Erling, Pelle Ehn, and Per-Anders Hillgren. 2010. “Participatory Design and ‘Democratizing Innovation.’” PDC ’10: Proceedings of the 11th Biennial Participatory Design Conference (November): 41–50. https://doi.org/10.1145/1900441.1900448.
Kwon, C. 2019. “Verification of the possibility and effectiveness of experiential learning using HMD-based immersive VR technologies.” Virtual Reality 23: 101–118.
Llewellyn, Jennifer J. 2011. “Restorative Justice: Thinking Relationally about Justice.” In Being Relational: Reflections on Relational Theory & Health Law, edited by Jocelyn Downie and Jennifer J. Llewellyn, 89–108. Vancouver: UBC Press.
Llewellyn, Kristina R. and Sharon Cook. 2017. “Oral History as Peacebuilding Pedagogy.” In Oral History and Education: Theories, Dilemmas, and Practices, edited by Kristina Llewellyn and Nicholas Ng-A-Fook, 17–41. New York: Palgrave Macmillan.
Llewellyn, Kristina R. and Jennifer J. Llewellyn. 2015. “A Restorative Approach to Learning: Relational Theory as Feminist Pedagogy in Universities.” In Feminist Pedagogy in Higher Education, edited by Tracy Penny Light, Jane Nicholas, and Renee Bondy, 11–31. Waterloo, Ontario: Wilfrid Laurier UP.
Llewellyn, Kristina R. and Nicholas Ng-A-Fook, eds. 2017. Oral History and Education: Theories, Dilemmas, and Practices. New York: Palgrave Macmillan.
Llewellyn, Kristina R. and Nicholas Ng-A-Fook, eds. 2019. Oral History, Education, and Justice: Limitations and Possibilities for Redress and Reconciliation. New York: Routledge.
Low, Bronwen E., and Emmanuelle Sonntag. 2015. “Towards a Pedagogy of Listening: Teaching and Learning from Life Stories of Human Rights Violations.” In The Canadian Oral History Reader, edited by Kristina Llewellyn, Alexander Freund, and Nolan Reilly, 266–281. Montreal and Kingston: McGill-Queen’s University Press.
Mabrook, Radwa, and Jane B. Singer. 2019. “Virtual Reality, 360° Video, and Journalism Studies: Conceptual Approaches to Immersive Technologies.” Journalism Studies 20, no. 14: 2096–112. https://doi.org/10.1080/1461670x.2019.1568203.
Malkowski, Jennifer, and Treaandrea M. Russworm. 2017. Gaming Representation: Race, Gender, and Sexuality in Video Games. Bloomington: Indiana University Press.
McCreery, Michael P., P. G. Schrader, S. Kathleen Krach, and Randy Boone. 2013. “A Sense of Self: The Role of Presence in Virtual Environments.” Computers in Human Behavior 29, no. 4: 1635–40. https://doi.org/10.1016/j.chb.2013.02.002.
Murray, Janet Horowitz. 1997. Hamlet on the Holodeck: The Future of Narrative in Cyberspace. Cambridge, Massachusetts: MIT Press.
Nowak, Kristine. 2001. “Defining and Differentiating Copresence, Social Presence, and Presence as Transportation.” A Comparison of Four New Communication Technologies, Proceedings of HCI International Conference on Human-Computer Interaction, Lawrence Erlbaum Associates.
Papanastasiou, George, Athanasios Drigas, Charalabos Skianis, Miltiadis Lytras, and Effrosyni Papanastasiou. 2018. “Virtual and Augmented Reality Effects on K-12, Higher and Tertiary Education Students’ Twenty-First Century Skills.” Virtual Reality 23, no. 4: 425–36. https://doi.org/10.1007/s10055-018-0363-2.
Reis, António Baía, and António Fernando Vasconcelos Cunha Castro Coelho. 2018. “Virtual Reality and Journalism: A Gateway to Conceptualizing Immersive Journalism.” Digital Journalism 6, no. 8: 1090–100. https://doi.org/10.1080/21670811.2018.1502046.
Roberts-Smith, Jennifer, Paul Cegys, William Chesney, Arda Kizilkay, Colin Labadie, Robert Plowman, Tracy Dorrington-Skinner, Gerald Morrison, Tony Smith, Kristina Llewellyn, Jennifer Llewellyn, and the DOHR Team. The Home: The Digital Oral Histories for Reconciliation Virtual Reality Experience. Pilot Release. Auburn Drive High School, Dartmouth, and Hants North Rural High School, Kennetcook, Nova Scotia, Canada, Oct 28–Nov 8, 2019. Beta Release. Nova Scotia Home for Colored Children Restorative Inquiry and Nova Scotia Black Cultural Centre, Dartmouth, Nova Scotia, Canada, Apr 11–12, 2019. Alpha Release. Halifax, Nova Scotia, Canada, July 2018.
Roberts-Smith, Jennifer. 2017. “Théâtre et humanités numériques. Du développement des outils au design expérimental. [Theatre and the Digital Humanities: From Tool Development to Experience Design].” Trans. Ioana Galleron. Revue d’Historiographie du Théâtre 4. Special Issue: Études théâtrales et humanités numériques.
Roberts-Smith, Jennifer, Shawn DeSouza-Coelho, Paul Stoesser, and the SET Research Team. 2016. “Cambridge Revisited? Methodological and Phenomenological Theatricalities in Simulated Environments for Theatre History Research.” In Early Modern Studies and the Digital Turn: New Technologies in Medieval and Renaissance Studies, edited by Laura Estill, Diane Jakacki, and Michael Ullyot, 318–44. Toronto: Iter Press.
Roberts-Smith, Jennifer, and the SET Research Team. 2013. “Visualizing Theatrical Text: From Watching the Script to the Simulated Environment for Theatre (SET).” Digital Humanities Quarterly 7, no. 3.
Sanchez-Vives, Maria V., and Mel Slater. 2005. “From Presence to Consciousness through Virtual Reality.” Nature Reviews Neuroscience 6, no. 4: 332–39. https://doi.org/10.1038/nrn1651.
Schroeder, Ralph. 2002. The Social Life of Avatars: Presence and Interaction in Shared Virtual Environments. Berlin: Springer Verlag.
Schuler, Douglas, and Aki Namioka. 1993. Participatory Design: Principles and Practices. Hillsdale, NJ: L. Erlbaum Associates Inc.
Scoresby, Jon, and Brett E. Shelton. 2011. “Visual Perspectives within Educational Computer Games: Effects on Presence and Flow within Virtual Immersive Learning Environments.” Instructional Science 39, no. 3 (May): 227–54. https://doi.org/10.1007/s11251-010-9126-5.
Simon, Roger I. 2015. A Pedagogy of Witnessing: Curatorial Practice and the Pursuit of Social Justice. Albany: SUNY Press.
Slater, Mel. 2009. “Place Illusion and Plausibility Can Lead to Realistic Behaviour in Immersive Virtual Environments.” Philosophical Transactions of the Royal Society B: Biological Sciences 364 (1535): 3549–57. https://doi.org/10.1098/rstb.2009.0138.
Slater, Mel, and Sylvia Wilbur. 1997. “A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments.” Presence: Teleoperators and Virtual Environments 6, no. 6: 603–16. https://doi.org/10.1162/pres.1922.214.171.1243.
Slater, Mel, Amela Sadagic, Martin Usoh, and Ralph Schroeder. 2000. “Small-Group Behavior in a Virtual and Real Environment: A Comparative Study.” Presence: Teleoperators and Virtual Environments 9, no. 1: 37–51. https://doi.org/10.1162/105474600566600.
Sullivan, Elaine, Angel David Nieves, and Lisa M. Snyder. 2017. “Making the Model: Scholarship and Rhetoric in 3-D Historical Reconstructions.” In Making Things and Drawing Boundaries: Experiments in the Digital Humanities, edited by Jentery Sayers. Minneapolis: University of Minnesota Press.
Taylor, Nicholas, and Gerald Voorhees, eds. 2018. Masculinities in Play. Cham, Switzerland: Palgrave Macmillan.
Triberti, Stefano, and Giuseppe Riva. 2016. “Being Present in Action: A Theoretical Model About the ‘Interlocking’ Between Intentions and Environmental Affordances.” Frontiers in Psychology 6. https://doi.org/10.3389/fpsyg.2015.02052.
Tuch, Alexandre N., Sandra P. Roth, Kasper Hornbæk, Klaus Opwis, and Javier A. Bargas-Avila. 2012. “Is Beautiful Really Usable? Toward Understanding the Relation between Usability, Aesthetics, and Affect in HCI.” Computers in Human Behavior 28, no. 5: 1596–607. https://doi.org/10.1016/j.chb.2012.03.024.
Weech, Séamas, Sophie Kenny, and Michael Barnett-Cowan. 2019. “Presence and Cybersickness in Virtual Reality Are Negatively Related: A Review.” Frontiers in Psychology 10 (April). https://doi.org/10.3389/fpsyg.2019.00158.
Weibel, David, Bartholomäus Wissmath, Stephan Habegger, Yves Steiner, and Rudolf Groner. 2008. “Playing Online Games against Computer- vs. Human-Controlled Opponents: Effects on Presence, Flow, and Enjoyment.” Computers in Human Behavior 24, no. 5: 2274–91. https://doi.org/10.1016/j.chb.2007.11.002.
Zembylas, Michalinos. 2015. Emotion and Traumatic Conflict: Reclaiming Healing in Education. New York: Oxford University Press.
Zembylas, Michalinos and Zvi Bekerman. 2008. “Education and the Dangerous Memories of Historical Trauma: Narratives of Pain, Narratives of Hope,” Curriculum Inquiry 38, no. 2: 125–54.
About the Authors
Jennifer Roberts-Smith (Associate Professor, Theatre Performance, University of Waterloo) is an award-winning artist-researcher, whose transdisciplinary work in performance, digital media, design, education, and social justice has appeared in theatres, exhibitions, and scholarly publications internationally. She is currently director of the qCollaborative (the intersectional feminist design research lab housed in the University of Waterloo’s Games Institute), and of the Social Sciences and Humanities Research Council-funded Theatre for Relationality and Design for Peace projects. Since 2017, JRS has served as creative director and virtual reality cluster lead for the Digital Oral Histories for Reconciliation project.
Justin Carpenter is a PhD Candidate in English Language and Literature at the University of Waterloo. His current research traces the use of the term “generative” from literary to computational contexts, arguing that an understanding of this term opens up a variety of arguments around concepts such as authorship, agency, and emergence. He argues that such a genealogy can help situate game studies scholarship in dialogue with modernist and postmodernist literary studies, as well as cinema and other media. His other research interests include poetry, philosophy of technology, and aesthetics.
Kristina R. Llewellyn is Associate Professor of Social Development Studies at Renison University College, University of Waterloo. She is an expert in oral history, history education, history of education, and women’s history. Llewellyn has numerous award-winning publications, including The Canadian Oral History Reader (MQUP, 2015), Oral History and Education: Theories, Dilemmas, and Practices (Palgrave, 2017), and Oral History, Education, and Justice: Possibilities and Limitations for Redress and Reconciliation (Routledge, 2019). Llewellyn is a co-investigator on the SSHRC-funded project Thinking Historically for Canada’s Future, which is working to revise history education across Canada. She is the Principal Investigator and Director for Digital Oral Histories for Reconciliation: The Nova Scotia Home for Colored Children History Education Initiative project.
Jennifer J. Llewellyn is a Professor of Law and the Yogis and Keddy Chair in Human Rights Law at the Schulich School of Law, Dalhousie University. She is an expert in relational theory and a restorative approach. She served as a Commissioner on the Restorative Inquiry for the Nova Scotia Home for Colored Children. She directs the Restorative Approach International Learning Community and the Restorative Research, Innovation and Education Lab at Dalhousie University. She is a member of the Steering Committee for the Digital Oral Histories for Reconciliation project.
Tracy Dorrington-Skinner is a member and former co-chair of Victims of Institutional Childhood Exploitation Society (VOICES). She was a resident of the Nova Scotia Home for Colored Children. A member of the DOHR Team she was one of the three storytellers. Tracy was a member of the UJIMA Design Team for the Restorative Inquiry and a member of the Advisor Group for the Restorative Inquiry.
Gerald “Gerry” Morrison is a co-chair of Victims of Institutional Childhood Exploitation Society (VOICES). He was a resident of the Nova Scotia Home for Colored Children. A member of the DOHR Team, he was one of the three storytellers. Gerry was also a member of the UJIMA Design Team for the Restorative Inquiry and a Commissioner on the Restorative Inquiry.
Tony Smith is a co-chair of Victims of Institutional Childhood Exploitation Society (VOICES). He was a resident of the Nova Scotia Home for Colored Children. A member of the DOHR Team, he was one of the three storytellers. Tony was also a member of the UJIMA Design Team for the Restorative Inquiry and a Commissioner on the Restorative Inquiry. He served as the co-chair of the Council of Parties (Commissioners) for the Restorative Inquiry.