This article details a student-centered assignment that integrates primary source analysis and the immersive medium of virtual reality. The goal of the activity was to increase student spatial familiarity and geographic knowledge of historic spaces, as well as expand their interest in primary sources.
Virtual Reality Design has been co-taught annually at Vanderbilt University since 2017 by professors Bobby Bodenheimer (Computer Science) and Ole Molvig (History, Communications of Science and Technology). This paper discusses the pedagogical and logistical strategies employed during the creation, execution, and subsequent reorganization of this course through multiple offerings. This paper also demonstrates the methods and challenges of designing a team-based project course that is fundamentally structured around interdisciplinarity and group work.
Introduction
What is virtual reality? What can it do? What can’t it do? What is it good/bad for? These are some of the many questions we ask on the first day of our course, Virtual Reality Design (Virtual Reality for Interdisciplinary Applications from 2017–2018). Since 2017, professors Ole Molvig of the History Department and Bobby Bodenheimer of Computer Science and Electrical Engineering have been co-teaching this course annually to roughly 50 students at a time. With each offering of the course, we have significantly revamped our underlying pedagogical goals and strategies based upon student feedback, the learning literature, and our own experiences. What began as a course about virtual reality has become a course about interdisciplinary teamwork.
Both of those terms, interdisciplinarity and teamwork, have become deeply woven into our effort. While a computer scientist and a historian teach the course, up to ten faculty mentors from across the university participate as “clients.” The course counts toward the computer science major’s project-class requirement, but nearly half the enrolled students are not CS majors. Agile design and group mechanics require organizational and communication skills above all else. And the projects themselves, as shown below, vary widely in the topic and demands, requiring flexibility, creativity, programming, artistry, and most significantly, collaboration.
This focus on interdisciplinary teamwork, and not just in the classroom, has led to a significant, if unexpected, outcome: the crystallization of a substantial community of faculty and students engaging in virtual reality related research from a wealth of disciplinary viewpoints. Equipment purchased for the course remain active and available throughout campus. Teaching projects have grown into research questions and collaborations. A significant research cluster in digital cultural heritage was formed not as a result of, but in synergy with, the community of class mentors, instructors, and students.
Evolution of the Course
Prior to offering the joint course, both Bodenheimer (CS) and Molvig (History) had previously offered single-discipline VR based courses.
From the Computer Science side, Bodenheimer had taught a full three-credit course on virtual reality to computer science students. In lecture and pedagogy this course covered a fairly standard approach to the material for a one semester course, as laid out by the Burea and Coiffet textbook or the more recent (and applicable) Lavalle textbook (Lavalle 2017). Topically, the course covered such material as virtual reality hardware, displays, sensors, geometric modeling, three-dimensional transformations, stereoscopic viewing, visual perception, tracking, and the evaluation of virtual reality experiences. The goal of the course was to teach the computer science students to analyze, design, and develop a complex software system in response to a set of computing requirements and project specifications that included usability and networking. The course was also project-based with teams of students completing the projects. Thus it focused on collaborative learning, and teamwork skills were taught as part of the curriculum, since there is significant work that shows these skills are best taught and do not emerge spontaneously (Kozlowski and Ilgen 2006). This practice allowed a project of significant complexity to be designed and implemented over the course of the semester, giving a practical focus to most of the topics covered in the lectures.
From History, Molvig offered an additional one credit “lab course” option for students attached to a survey of The Scientific Revolution. This lab option offered students the opportunity to explore the creation of and meaning behind historically informed re-constructions or simulations. The lab gave students their first exposure to a nascent technology alongside a narrative context in which to guide their explorations. Simultaneous to this course offering, Vanderbilt was increasing its commitment to the digital humanities, and this course allowed both its instructor and students to study the contours of this discipline as well. While this first offering of a digital lab experience lacked the firm technical grounding and prior coding experience of the computer science offering, the shared topical focus (the scientific revolution) made for boldly creative and ambitious projects within a given conceptual space.
Centering Interdisciplinarity
Unlike Bodenheimer, Molvig did not have a career-long commitment to the study of virtual reality. Molvig’s interest in VR comes rather from a science studies approach to emergent technology. And in 2016, VR was one of the highest profile and most accessible emergent technologies (alongside others such as artificial intelligence, machine learning, CRISPR, blockchain, etc). For Molvig, emergent technologies can be pithily described as those technologies that are about to go mainstream, that many people think are likely to be of great significance, but no one is completely certain when, for whom, how, or really even if, this will happen.
For VR then, in an academic setting, these questions look like this: Which fields is VR best suited for? Up to that point, it was reasonably common in computer science and psychology, and relatively rare elsewhere. How might VR be integrated into the teaching and research of other fields? How similar or dissimilar are the needs and challenges of these different disciplines pedagogical and research contexts?
Perhaps most importantly, how do we answer these questions? Our primary pedagogical approach crystallized around two fundamental questions:
How can virtual reality inform the teaching and research of discipline X?
How can discipline X inform the development of virtual reality experiences?
Our efforts to answer these questions led to the core feature that has defined our Virtual Reality Design course since its inception: interdisciplinarity. Rather than decide for whom VR is most relevant, we attempted to test it out as broadly as possible, in collaboration with as many scholars as possible.
Our course is co-taught by a computer scientist and a humanist. Furthermore, we invite faculty from across campus to serve as “clients,” each with a real-world, disciplinary specific problem toward which virtual reality may be applicable. While Molvig and Bodenheimer focused on both questions, our faculty mentors focused on question 1: is VR surgery simulation an effective tool? Can interactive, immersive 3D museums provide users new forms of engagement with cultural artifacts? How can VR and photogrammetry impact the availability of remote archeological sites? We will discuss select projects below, but as of our third offering of this course, we have had twenty-one different faculty serve as clients representing twelve different departments or schools, ranging from art history to pediatrics and chemistry to education. A full list of the twenty-four unique projects may be found in Appendix 1.
At the time of course planning, Vanderbilt began a program of University Courses, encouraging co-taught, cross disciplinary teaching experiments, incentivizing each with a small budget, which allowed us to purchase the hardware necessary to offer the course. One of our stated outcomes was to increase access to VR hardware, and we have intentionally housed the equipment purchased throughout campus. Currently, most available VR hardware available for campus use is the product of this course. Over time, purchases from our course have established 10 VR workstations across three different campus locations (Digital Humanities Center, The Wond’ry Innovation Center, and the School of Engineering Computer Lab). Our standard set up has been the Oculus Rift S paired with desktop PCs with minimum specs of 16GB RAM and 1080GTX GPUs.
As the design of the joint, team-taught and highly interdisciplinary course was envisioned, several course design questions presented themselves. In our first iteration of the course, a condensed and more accessible version of the computer science virtual reality class was lectured on. Thus Bodenheimer, the computer science instructor, lectured on most of the same topics he had lectured on but at a more general level, and focused on how the concepts were implemented in Unity, rather than from a more theoretical perspective that was present in the prior offering. Likewise, Molvig brought with him several tools of his discipline, a set of shared readings (such as the novel Ready Player One (Cline 2012)) and a response essay to the moral and social implications of VR. The class was even separated for two lectures, allowing Bodenheimer to lecture in more detail on C#, and Molvig to offer strategies on how to avoid C# entirely within Unity.
Subsequent offerings of the course, however, allowed us to abandon most of this structure, and to significantly revise the format. Our experience with how the projects and student teams worked and struggled led us to re-evaluate the format of the course. Best practices in teaching and learning recommend active, collaborative learning where students learn from their peers (Kuh et al. 2006). Thus, we adopted a structured format more conducive to teamwork, based on Agile (Pope-Ruark 2017). Agile is a framework and set of practices originally created for software development but which has much wider applicability today. It can be implemented as a structure in the classroom with a set of openly available tools that allow students to articulate, manage, and visualize a set of goals for a particular purpose, in our case, the creation of a virtual experience tailored to their clients specific research. The challenge for us, as instructors, was to develop methods to instrument properly the Agile methods so that the groups in our class can be evaluated on their use of them, and get feedback on them so that they can improve their practices. This challenge is ongoing. Agile methods are thus used in our class to help teams accomplish their collaborative goals and teach them teamwork practices.
Course Structure
We presume no prior experience with VR, the Unity3D engine, or C# for either the CS or non-CS students. Therefore the first third of the course is mainly focused on introducing those topics, primarily through lecture, demonstration, and a series of cumulative “daily challenges.” By the end of this first section of the course, all students are familiar with the common tools and practices, and capable of creating VR environments upon which they can act directly through the physics engine as well as in a predetermined, or scripted, manner. During the second third of the course, students begin working together on their group projects in earnest, while continuing to develop their skills through continued individual challenges, which culminate in an individual project due at the section’s end. For the second and third sections of the course, all group work incorporates aspects of the Agile method described above, with weekly in-class group standups, and a graded, bi-weekly sprint review, conducted before the entire class. The final section of the course is devoted entirely to the completion of the final group project, which culminates in an open “demo day” held during final examinations, which has proven quite popular.
Three-fifths of our students are upper level computer science students fulfilling a “project course” major requirement, while two-fifths of our students can be from any major except computer science. Each project team is composed of roughly five students with a similar overall ratio, and we tend to have about 50 students per offering. This distribution and size are enforced at registration because of the popularity of the CS major and demand for project courses in it. The typical CS student’s experience will involve at least three semesters of programming in Java and C++, but usually no knowledge of computer graphics or C#, the programming language used by Unity, our virtual reality platform. The non-CS students’ experience is more varied, but currently does not typically involve any coding experience. To construct the teams, we solicit bids from the students for their “top three” projects and “who they would like to work with.” The instructors then attempt to match students and teams so that everyone gets something that they want.
It is a fundamental assertion of this course that all members of a team so constructed can contribute meaningfully and substantially to the project. As it is perhaps obvious what the CS students contribute, it is important to understand what the non-CS students contribute. First, Unity is a sophisticated development platform that is quite usable, and, as mentioned, we spend significant course time teaching the class to use it. There is nothing to prevent someone from learning to code in C# using Unity. However, not everyone taking our class wants to be a coder, but they are interested in technology and using technical tools. Everyone can build models and design scenes in Unity. Also, these projects must be robust. Testing that incremental progress works and is integrated well into the whole project is key not only to the project’s success as a product, but also to the team’s grade. We also require that the teams produce documentation about their progress, and interact with their faculty mentor about design goals. These outward-facing aspects of the project are key to the project’s success and often done by the non-CS students. Each project also typically requires unique coding, and in our experience the best projects are one in which the students specialize into roles, as each project typically requires a significant amount of work. The Agile framework is key here, as it provides a structure for the roles and a way of tracking progress in each of them.
Since each project is varied, setting appropriate targets and evaluating progress at each review is one of the most significant ongoing challenges faced by the instructors.
Projects
A full list of the twenty-four projects may be found in Appendix 1.
Below are short descriptions and video walkthroughs of four distinctive projects that capture the depth, breadth, and originality fostered by our emphasis on interdisciplinarity in all aspects of the course design and teaching.
Example Project: Protein Modeling
The motivation for this project, mentored by Chemistry Professor Jens Meiler, came from a problem common to structural chemistry: the inherent difficulty of visualizing 3D objects. For this prototype, we aimed to model how simple proteins and molecules composed of a few tens of atoms interact and “fit” together. In drug design and discovery, this issue is of critical importance and can require significant amounts of computation (Allison et al. 2014). These interactions are often dominated by short-range van der Waals forces, although determining the correct configuration for the proteins to bind is challenging. This project illustrated that difficulty by letting people explore binding proteins together. Two proteins were given in an immersive environment that were graspable, and users attempted to fit them together. As they fit together, a score showing how well they fit was displayed. This score was computed based on an energy function incorporating Van der Waals attractive and repulsive potentials. The goal was to get the minimum score possible. The proteins and the energy equation were provided by the project mentor, although the students implemented a Van der Waals simulator within Unity for this project. Figures 1 and 2 show examples from the immersive virtual environment. The critical features of this project worth noting are that the molecules are three-dimensional structures that are asymmetric. Viewing them with proper depth perception is necessary to get an idea of their true shape. It would be difficult to recreate this simulation with the same effectiveness using desktop displays and interactions.
While issues of efficiency and effectiveness in chemical pedagogy drove our mentor’s interest, the student creators and demo day users were drawn to this project for its elements of science communication and gamification. By providing a running “high score” and providing a timed element, users were motivated to interact with the objects and experience far longer than with a 2D or static 3D visualization. One student member of this group did possess subject matter familiarity which helped incorporate the energy function into the experience.
Figure 1. Two proteins shown within the simulation. The larger protein on the left is the target protein to which the smaller protein (right) should be properly fit. A menu containing the score is shown past the proteins. Proteins may be grabbed, moved, and rotated using the virtual reality controllers. Embedded video: Figure 1. Two proteins shown within the simulation. The larger protein on the left is the target protein to which the smaller protein (right) should be properly fit. A menu containing the score is shown past the proteins. Proteins may be grabbed, moved, and rotated using the virtual reality controllers.
Example Project: Vectors of Textual Movement in Medieval Cypress
Professor of French Lynn Ramey served as the mentor for this project. Unlike most other mentors, Prof. Ramey had a long history of using Unity3D and game technologies in both her research and teaching. Her goal in working with us was to recreate an existing prototype in virtual reality, and determine the added values of visual immersion and hand tracked interactivity. This project created a game that simulates how stories might change during transmission and retelling (Amer et al. 2018; Ramey et al. 2019). The crusader Kingdom of Cyprus served as a waypoint between East and West during the years 1192 to 1489. This game focuses on the early period and looks at how elements of stories from The Thousand and One Nights might have morphed and changed to please sensibilities and tastes of different audiences. In the game, the user tells stories to agents within the game, ideally gaining storytelling experience and learning the individual preferences of the agents. After gaining enough experience, the user can gain entry to the King’s palace and tell a story to the King, with the goal of impressing the King. During the game play, the user must journey through the Kingdom of Cyrus to find agents to tell stories to.
This project was very successful at showcasing the advantages of an interdisciplinary approach. Perhaps the project closest to a traditional video game, faculty and students both were constantly reminded of the interplay between technical and creative decisions. However, this was not simply an “adaption” of a finished cultural work into a new medium, but rather an active exploration of an open humanities research project asking how, why, when, and for whom are stories told. No student member of this group majored in the mentor’s discipline.
A video walkthrough of the game can be seen below.
Figure 2. Video walk-through of gameplay. Embedded video: Fig 2. Video walk-through of medieval storytelling project gameplay. Video shows gameplay in main screen, with small inset filming user in VR headset. Gameplay shows the goal and user interface by which players tell stories and explore medieval village. Scenes include a market, a castle, and a village environment.
Example Project: Interactive Geometry for K–8 Mathematical Visualization
In this project, Corey Brady, Professor of Education, challenged our students to take full advantage of the physical presence offered by virtual environments, and build an interactive space where children can directly experience “mathematical dimensionality.” Inspired by recent research (Kobiela et al. 2019; Brady et al. 2019) examining physical geometrical creation in two dimensions (think paint, brushes and squeegees), the students created a brightly lit and colored virtual room, where the user is initially presented with a single point in space. Via user input, the point can be stretched into a line, the line into a plane, and the plane into a solid (rectangles, cylinders, and prisms). While doing so, bar graph visualizations of length, width, height, surface area, and volume are updated in real-time while the user increases or decreases the object along its various axes.
Virtual Reality as an education tool has proven very popular, both amongst our students and in industry. No student member of this group specialized in education, but all members had of course first hand experience learning these concepts themselves as children. The opportunity to reimagine a nearly universal learning process was a significant draw for this project. After this course offering, Brady and Molvig have begun a collaboration to expand its utility.
A video demonstration of the project can be seen below.
Figure 3. User manipulates the x, y, and z axes of a rectangle. Real-time calculations of surface area and volume are shown in the background. Embedded video: Figure 3. Video demonstration of geometry visualization project gameplay. User manipulates the x, y, and z axes of a various shapes, including regular polygons and conic sections. Real-time calculations of surface area and volume are shown in the background.
Example Project: Re-digitizing Stereograms
For this project, Molvig led a team to bring nineteenth-century stereographic images into 21st century technology. Invented by Charles Wheatstone in 1838 and later improved by David Brewster, stereograms are nearly identical paired photographs that when viewed through a binocular display, a single “3D image” [1] was perceived by the viewer, often with an effect of striking realism. For this reason, stereoscopy is often referred to as “Victorian VR.” Hundreds of thousands of scanned digitized stereo-pair photos exist in archives and online collections, however it is currently extremely difficult to view these as intended in stereoscopic 3D. Molvig’s goal was to create a generalizable stereogram viewer: capable of bringing stereopair images from remote archives for viewing within a modern VR headset.
Student interest quickly coalesced around two sets of remarkable stereoscopic anatomical atlases, the Edinburgh Stereoscopic Atlas of Anatomy (1905) and Bassett Collection of Stereoscopic Images of Human Anatomy from the Stanford Medical Library. Driven by student interest, the 2019 project branched into a VR alternative to wetlab or flat 2D medical anatomy imagery. This project remains ongoing, as is Molvig’s original generalized stereo viewer, which now includes a machine learning based algorithm to automated the import and segmentation of any stereopair photograph.
Two demonstrations of the stereoview player are below, the first for medical anatomy images, the second are stereophotos taken during the American Civil War. All images appear in stereoscopic depth when viewed in the headset.
Figure 4. Demonstration of anatomy stereoscopic viewer. Images from the Bassett Collection of Stereoscopic Images of Human Anatomy, Stanford Medical Library. Embedded video: Figure 4. Video demonstration of medical anatomy stereoscopic viewer project gameplay. User selects and relocates various stereoscopic images of cranial anatomy. Images from the Bassett Collection of Stereoscopic Images of Human Anatomy, Stanford Medical Library.Figure 5. Demonstration of Civil War stereoviews. Images from the Robert N. Dennis collection of stereoscopic views, New York Public Library Digital Collection. Embedded video: Figure 5. Video demonstration of Civil War stereoview project gameplay. User selects and and relocated various stereoscopic images taken during the American Civil War. Images depict scenes from battlefields, army encampments, and war material preparations. Images from the Robert N. Dennis collection of stereoscopic views, New York Public Library Digital Collection.
Challenges
This course has numerous challenges, both inside and outside of the classroom, and we have by no means solved them all.
Institutional
Securing support for co-teaching is not always easy. We began offering this course under a Provost level initiative to encourage ambitious teaching collaborations across disciplines. This initiative made it straightforward to count co-teaching efforts with our Deans, and provided some financial support for the needed hardware purchases. However, that initiative was for three course offerings, which we have now completed. Moving forward, we will need to negotiate our course with our Deans.
We rely heavily on invested Faculty Mentors to provide the best subject matter expertise. So far we have had no trouble finding volunteers, and the growing community of VR engaged faculty has been one of the greatest personal benefits, but as VR becomes less novel, we may experience a falloff in interest.
Interdisciplinarity
This is both the most rewarding and most challenging aspect of this course. Securing student buy-in on the value of interdisciplinary teamwork is our most consistent struggle. In particular, these issues arise around the uneven distribution of C# experience, and perceived notions of what type of work is “real” or “hard.” To mitigate these issues, we devote significant time during the first month of the course exposing everyone to all aspects of VR project development (technical and non-technical), and require the adoption of “roles” within each project to make responsibilities clear and workload distributed.
Cost
Virtual reality is a rapidly evolving field, with frequent hardware updates and changing requirements. We will need to secure new funding to significantly expand or update our current equipment.
Conclusions and Lessons Learned
Virtual reality technology is more accessible than ever, but it is not as accessible as one might wish in a pedagogical setting. It is difficult to create even moderately rich and sophisticated environments, without the development expertise gleaned through exposure to the computer science curriculum. A problem thus arises on two fronts. First, exposure to the computer science curriculum at the depth currently required to develop compelling virtual reality applications should ideally not be required of everyone. Unfortunately, the state of the art of our tools currently makes this necessary. Second, those who study computer science and virtual reality focus on building the tools and technology of virtual reality, the theories and algorithms integral to virtual reality, and the integration of these into effective virtual reality systems. Our class represents a compromise solution to the accessibility problem by changing the focus away from development of tools and technology toward collaboration and teamwork in service of building an application.
Our class is an introduction to virtual reality in the sense that students see the capability of modern commodity-level virtual reality equipment, software, and these limitations. They leave the class understanding what types of virtual worlds are easy to create, and what types of worlds are difficult to create. From the perspective of digital humanities, our course is a leveraged introduction to technology at the forefront of application to the humanities. Students are exposed to a humanities-centered approach to this technology through interaction with their project mentors.
In terms of the material that we, the instructors, focus most on in class, our class is about teamwork and problem-solving with people one has not chosen to work with. We present this latter skill as one essential to a college education, whether it comes from practical reasons, e.g., that is what students will be faced with in the workforce (Lingard & Barkataki 2013), or from theoretical perspectives on best ways to learn (Vygotsky 1978). The interdisciplinarity that is a core feature of the course is presented as a fact of the modern workforce. Successful interdisciplinary teams are able to communicate and coordinate effectively with one another, and we emphasize frameworks that allow these things to happen.
Within the broader Vanderbilt curriculum, the course satisfies different curricular requirements. For CS students, the course satisfies a requirement that they participate in a group design experience as part of their major requirements. The interdisciplinary nature of the group is not a major requirement, but is viewed as an advantage, since it is likely that most CS majors will be part of interdisciplinary teams during their future careers. For non-CS students, the course currently satisfies the requirements of the Communication of Science and Technology major and minor.[2]
Over the three iterations of this course, we have learned that team teaching an interdisciplinary project course is not trivial. In particular, it requires more effort than each professor lecturing on their own specialty, and expecting effective learning to emerge from the two different streams. That expectation was closer to what we did in the first offering of this course, where we quickly perceived that this practice was not the most engaging format for the students, nor was it the most effective pedagogy for what we wanted to accomplish. The essence of the course is on creating teams to use mostly accessible technology to create engaging virtual worlds. We have reorganized our lecture and pedagogical practices to support this core. In doing this, each of us brings to the class our own knowledge and expertise on how best to accomplish that goal, and thus the students experience something closer to two views on the same problem. While we are iteratively refining this approach, we believe it is more successful.
Agile methods (Pope-Ruark 2017) have become an essential part of our course. They allow us to better judge the progress of the projects and determine where bottlenecks are occurring more quickly. They incentivize students to work consistently on the project over the course of the semester rather than trying to build everything at the end in a mad rush of effort. By requiring students to mark their progress on burn down charts, the students have a better visualization of the task remaining to be accomplished. Project boards associated with Agile can provide insight into the relative distribution of work that is occurring in the group, ideally allowing us to influence group dynamics before serious tensions arise.
This latter effort is a work in progress, however. A limitation of the course as it currently exists is that we need to do a better job evaluating teams (Hughes & Jones 2011). Currently our student evaluations rely too heavily on the final outcome of the project and not enough on the effectiveness of the teamwork within the team. Evaluating teamwork, however, has seemed cumbersome, and the best way to give meaningful feedback to improve teamwork practices is something we are still exploring. If we improved this practice, we could give students more refined feedback throughout the semester on their individual and group performance, and use that as a springboard to teach better team practices. Better team practices would likely result in increased quality of the final projects.
Notes
[1] These images are not truly three dimensional, as they cannot be rotated or peered behind. Rather two images are created precisely to fool the brain into adding a perception of depth into a single combined image. [2]https://as.vanderbilt.edu/cst/. There is currently no digital humanities major or minor at Vanderbilt.
References
Allison, Brittany, Steven Combs, Sam DeLuca, Gordon Lemmon, Laura Mizoue, and Jens Meiler. 2014. “Computational Design of Protein–Small Molecule Interfaces.” Journal of Structural Biology 185, no. 2: 193–202.
Amer, Sahar, and Lynn Ramey. 2018. “Teaching the Global Middle Ages with Technology.” Parergon: Journal of the Australian and New Zealand Association for Medieval and Early Modern Studies 35: 179–91.
Cline, Ernest. 2012. Ready Player One. New York: Broadway Books.
Hughes, Richard L., and Steven K. Jones. 2011. “Developing and assessing college student teamwork skills.“ New Directions for Institutional Research 149: 53–64.
Kobiela, Marta, and Richard Lehrer. 2019. “Supporting Dynamic Conceptions of Area and its Measure.” Mathematical Thinking and Learning: 1–29.
Kozlowski, Steve W.J., and Daniel R. Ilgen. 2006. “Enhancing the Effectiveness of Work Groups and Teams.” Psychological Science in the Public Interest 7, no.3: 77–124.
Kuh, George D., Jillian Kinzie, Jennifer A. Buckley, Brian K. Bridges, and John C. Hayek. 2006. What Matters to Student Success: A Review of the Literature. Vol. 8. Washington, DC: National Postsecondary Education Cooperative.
LaValle, Steve 2017. Virtual Reality. Cambridge, UK: Cambridge University Press.
Lingard, Robert, and Shan Barkataki 2011. “Teaching Teamwork in Engineering and Computer Science.” 2011 Frontiers in Education Conference. Institute of Electrical and Electronics Engineers.
Pope-Ruark, Rebecca. 2017. Agile Faculty: Practical Strategies for Managing Research, Service, and Teaching. Chicago: University of Chicago Press.
Ramey, Lynn, David Neville, Sahar Amer, et al. 2019. “Revisioning the Global Middle Ages: Immersive Environments for Teaching Medieval Languages and Culture.” Digital Philology 8: 86–104.
Takala, Tuukka M., Lauri Malmi, Roberto Pugliese, and Tapio Takala. 2016. “Empowering students to create better virtual reality applications: A longitudinal study of a VR capstone course.” Informatics in Education 15, no. 2: 287–317.
Zimmerman, Guy W., and Dena E. Eber. 2001. “When worlds collide!: an interdisciplinary course in virtual-reality art.” ACM SIGCSE Bulletin 33, no. 1.
Appendix 1: Complete Project List
Project Title (Mentor, Field, Year(s))
Aristotelian Physics Simulation (Molvig, History of Science, 2017, 2018).
Peripersonal Space (Bodenheimer, Computer Science, 2019).
Solar System Simulation (Weintraub, Astronomy, 2019).
Accessing Stereograms (Molvig, History, 2019).
About the Authors
Ole Molvig is an assistant professor in the Department of History and the Program in Communication of Science and Technology. He explores the interactions among science, technology, and culture from 16th-century cosmology to modern emergent technologies like virtual reality or artificial intelligence. He received his Ph.D. in the History of Science from Princeton University.
Bobby Bodenheimer is a professor in the Department of Electrical Engineering and Computer Science at Vanderbilt University. He also holds an appointment in the Department of Psychology and Human Development. His research examines virtual and augmented reality, specifically how people act, perceive, locomote, and navigate in virtual and augmented environments. He is the recipient of an NSF CAREER award and received his Ph.D. from the California Institute of Technology.
In June 2019, a cohort of CLIR postdoctoral fellows convened Immersive Pedagogy: A Symposium on Teaching and Learning with 3D, Augmented and Virtual Reality at Carnegie Mellon University. The symposium sought to bring together a multidisciplinary group of collaborators to think through pedagogical issues related to using 3D/VR/AR technologies, as well as to produce and disseminate materials for teaching and learning. This essay presents the Immersive Pedagogy symposium as a model for interrogating and developing pedagogical practices and standards for 3D/VR/AR; we offer a decolonial, anti-ableist, and feminist pedagogical framework for collaboratively developing and curating humanities content for this emerging technology by summarizing the symposium’s keynotes, workshops, as well as its goals and outcomes. Workshops, keynotes, and participant conversations engaged with decolonial and feminist methodologies, practiced accessible design for universal learning, offered templates for humanistic teaching, and illustrated the possibilities of using 3D/VR/AR to extend critical thinking. While 3D/VR/AR technologies demonstrate real possibilities for collaborative, multidisciplinary learning, they are also fraught with broader concerns prevalent today about digital technologies, as well as complex issues specific to 3D/VR/AR. There is a clear need to assemble academic practitioners on a regular basis in order to facilitate an ongoing discussion about 3D/VR/AR technology and its responsible, meaningful use in teaching and learning.
Introduction
As access to three-dimensional (3D) technologies has become increasingly available in academic venues, the desire to teach with these emerging technologies, particularly augmented reality (AR) and virtual reality (VR), has outpaced digital humanists’ abilities to provide meaningful support for immersive projects. There is a growing and ongoing need to produce shared and open pedagogical materials adaptable to the needs of teachers in various professions and disciplines and are accessible to students without significant coding experience. This need is partially driven by the contingencies of relatively new and rapidly updating technologies, as well as the fact that support for commercially-available immersive tools are tailored for industry purposes. Game-driven tutorials, for example, do not always take into consideration the needs of humanities practitioners seeking to integrate critical thinking with technical mastery. Contemporary contexts for emerging technologies can structure our interactions with 3D/VR/AR. Though not always visible to users, these can have the effect of naturalizing problematic historical and political narratives through selective access to resources and functionality.
Nonetheless, game engines that offer free educational licenses have been repurposed for academic inquiry and teaching over the past decade. For example, Unity Technologies’ Unity 3D game engine is utilized by over 4.5 million users and has been at the forefront of historical and archaeological 3D visualizations in scholarly research. First available in 2005, the Unity 3D game engine has been used to make approximately 60% of all AR/VR applications and is used by 90% of AR/VR companies (“Public Relations” 2019, np). Educational licenses are available for students and educators seeking to use the engine for scholarly or creative use. Its main competitor, the Unreal Engine, while initially inaccessible beyond professional and academic institutions with licenses, dropped its paywall for educational use in September 2014. VR headsets, once a hypothetical fantasy or niche short-lived technology, are now commercially viable and relatively inexpensive for institutions to purchase. In a few years, the financial barrier for individuals may diminish; in the meantime, Google Cardboards and other stereoscopic viewers with fewer interactive features currently provide alternatives for students with access to smartphones. However, students are also increasingly able to make use of interactive 3D/VR/AR technology within dedicated spaces in academic libraries, maker spaces, media studios, and community outreach centers. Yet, we would be remiss not to point out that access is still mediated by other social hierarchies; 3D/VR/AR technology is still not accessible in much of the Global South, or in marginalized communities across the world. These aforementioned developments still privilege students at institutions that dedicated staff or faculty to maintain and encourage use of 3D/VR/AR technologies and facilities.
This is all to say that in our current 3D/VR/AR moment, digital humanists have a lot to navigate. Current 3D/VR/AR pedagogy and projects can pose problems related to accessibility and long-term preservation of projects and assets, and often run afoul of minimal computing recommendations. Yet the technology offers rich possibilities for multidisciplinary research and collaboration; many virtual reality projects combine art production, computing, archival research, network theory, and data visualization, among other practices. Given its potential for scholarship and teaching, understanding how to use the technology responsibly necessitates engaging with active practitioners to identify what is now possible and what still needs to be done to facilitate productive use of 3D/VR/AR. As many key problems are likely to persist through subsequent permutations of the technology and its use in educational settings, this conversation needs to be ongoing and open. What humanists within and beyond the academy have to say about 3D/VR/AR will probably not be unique to humanistic inquiry. This dialogue will provide crucial critical approaches to the emerging technologies’ advantages and limitations that will be of use to industry professionals as well as the casual creative user. A vocal contingent of humanists seeking to think and learn with 3D/VR/AR may, in fact, fill a wider sociocultural need by addressing these issues.
This is the context in which a small cohort of 2017–2019 Council on Library and Information Resources (CLIR) Postdoctoral Fellows organized Immersive Pedagogy: A Symposium on Teaching and Learning with 3D, Augmented and Virtual Reality at Carnegie Mellon University on June 26 and 27, 2019. The CLIR cohort included Lorena Gauthereau (University of Houston), Jessica Linker (Bryn Mawr College), Eric Kaltman (Carnegie Mellon University), Emma Slayton (Carnegie Mellon University), Neil Weijer (Johns Hopkins University), Alex Wermer-Colan (Temple University), and Chris Young (University of Toronto). The goal of this symposium was to assemble a wide range of stakeholders to develop teaching materials and strategies that considered problems inherent and specific to immersive technologies, as well as to address problems that affect but are not unique to 3D/VR/AR. It is for this reason the symposium was so attentive to decolonial and feminist methodologies in thinking about appropriate pedagogical applications. Building on the previous work of scholars such as María Cotera, Elizabeth Losh, Tara McPherson, Angel Nieves, Roopika Risam, and Jacqueline Wernimont, we have advocated for an intersectional digital humanities that interrogates a wide range of technologies through the critical methods developed by the fields of ethnic and feminist studies. Such methods, we argue, can highlight the ways that technologies often leave out marginalized people by replicating colonial hierarchical structures including race, ethnicity, class, gender, and disability.
The Immersive Pedagogy symposium offered an early—if not first-of-its-kind—opportunity to have productive conversations about what critical approaches to 3D/VR/AR could look like from a multidisciplinary and multi-professional perspective. Additionally, the symposium sought to seed collaborations within and beyond academic institutions and stand as a model for future conversations on these topics. In recounting our experiences with different applications of 3D/VR/AR technology in pedagogical spaces, the group tackled a number of thorny issues, such as accessibility in hardware and bias in asset stores, while acknowledging that we would need to continue the dialogue by reconvening. We sought to develop teaching materials collaboratively with the long-term plan of sharing these resources through a variety of means, including via open-access publications. In the remainder of this essay, the Immersive Pedagogy organizers describe the symposium’s theoretical foundation and methodological approaches as a model for structuring communities around 3D/VR/AR, summarize some of our group’s findings, and invite digital humanities practitioners to help us to continue this work.
Structuring a Symposium on Decolonial Models of Immersive Pedagogy
Because the initiative was organized by CLIR postdoctoral fellows, the symposium emphasized diverse ways that libraries participate in creating, curating, and preserving 3D/VR/AR pedagogical materials. We considered faculty, staff, and students as equal partners in 3D/VR/AR projects, and aimed to include early career researchers at the table. Overarching goals for the symposium included teaching faculty and librarians how to support and enable learning for students using 3D technologies, and to help students to disseminate skills within their own communities. By bringing together scholars from a wide range of disciplines and professions, we addressed problems while identifying new ones. Participants had the opportunity to share links and descriptions to their projects (current and in progress) with each other prior to the symposium via a Slack channel and Google Docs. They also shared information on their work during a lightning talk round as examples of the kinds of humanistic projects 3D/VR/AR could cultivate. The symposium began and ended with keynotes from experienced practitioners whose work modeled creative and responsible uses of the technologies.
Our opening keynote speaker, Angel Nieves (Associate Professor of History and Digital Humanities at San Diego State University), presented “Developing a Social Justice Framework for Immersive Technologies in Digital Humanities.” Nieves’s talk outlined strategies for achieving social justice through digital-supported inquiry, highlighting his own work on Mapping Soweto, a 3D reconstruction of apartheid South Africa. Nieves emphasized the need to ground digital work in women of color theory and argued that fields such as ethnic studies have developed a foundational structure that would benefit the field of digital humanities as a whole:
If we brought the sorts of methodological and practice-based questions about power, privilege, and access from ethnic studies to our work in immersive technologies, we might begin to see new ways of harnessing these tools–that originated as part of the military industrial complex–to serve our social justice needs. (Nieves 2019)
Mapping Soweto draws from Belinda Robtnett’s (1997) work on social movement theory, revealing the often messy, multilayered narratives of social movements by visualizing a map of spatial liberation. This 3D representation shows what Nieves terms an “intersectional cartography,” or a network of social activists—especially networks of women and young girls—across townships “and how those activist networks were embedded into the physical geography and vernacular architecture of individual houses, streets, and neighborhoods” (Nieves 2019). Attention to intersectionality further reveals the ways multiple identities—township, gender, sexual orientation, class, and race—came together to form a cohesive activist movement, whose complexities are often lost in the official retelling of history. In particular, Nieves identified immersive technologies as one way to “re-establish coalition-building potential” (2019) with local communities and reminded us that the important work of recovering marginalized histories for social justice is often messy.
Our closing keynote speaker, Juliette Levy (Associate Professor of History at the University of California, Riverside), presented “How Not to be a Replicant: Working Towards a Useful VR.” Working with a team of women programmers, Levy has developed VR simulations for teaching abstract concepts related to historical thinking, interpretation, and writing. Levy’s keynote presentation focused on the question of gaming and interactivity; and she traced the origin of her experimentations in VR from teaching large lecture classes numbering in the hundreds in hybrid and online courses. Rather than approach VR in the mode of cultural heritage projects, reproducing a historical location, to deal with pedagogical problems commonly experienced in online learning, Levy’s team built Digital Zombies (see Levy 2017), an abstract simulation meant to introduce students to the hierarchy of library information and assessment of primary and secondary resources through game-based learning. Levy envisioned a VR environment for her historical research methods class that not only encouraged students to follow a written outline of research steps, but to extend their library experience in a more immersive, playful way by completing a series of game-like missions related to research that students would be more likely to remember. Levy argued that the cognitive effect of a VR experience has a lasting impact on users: “What matters about doing something in VR isn’t about what happens in VR, but what happens outside of VR, after the VR experience” (Levy 2019). Yet, despite the advantages of VR, Levy warned that a lack of critical conversation and pedagogy around digital literacy can have dire consequences, as increasingly ubiquitous immersive technologies become exploited to misrepresent historical events. The stakes for fomenting critical conversations between technology creators, consumers, and scholars, therefore, are quite high, as they could have lasting effects on how people choose to build and interpret virtual representations of historical events and people.
The symposium included five workshops that centered on theory, methods, and practices significant to and capable of incubating pedagogy related to US Latinx, Latin American, and Caribbean studies, which we prioritized when considering applicants. The workshop topics were: (a) Decolonial Methodology and Theory; (b) Accessible Immersive Pedagogy; (c) Integrating Immersive Technology in the Classroom; (d) Critical Writing for Immersive Tech; and (e) Collaboratively Designing 3D/VR Experiences. The Immersive Pedagogy organizers, joined by Jasmine Clark (Temple University) and Juliette Levy, led the participants through these interactive workshops (“Program” 2019). Pedagogical content crafted by participants before, during, and after the symposium included a bibliography of 3D/VR/AR-related readings, an archive of workshop slides, video recordings of keynote presentations, adaptable templates for pedagogical activities, and working models of 3D/VR/AR pedagogical applications. For example, Kat Hayes and Samantha Porter submitted a video walkthrough of their IOS app Virtual MISLS that explores historic buildings at Fort Snelling, while Meaghan Moody and Coral Salomón submitted a description of their work with students using a virtual map of historic Paris to better understand life under German occupation during World War II.
Carnegie Mellon University (CMU) Libraries hosts the symposium’s materials on its institutional repository, KiltHub. KiltHub provides stable, long-term global open access storage for 3D/VR/AR assets, and functional applications, as well as pedagogical and technical documentation. Materials in this repository are held for a minimum of ten years, ensuring that what is submitted will remain available past typical terms of software updates. The teaching materials produced during and following the symposium will also be published in the Digital Library Federation’s Pedagogy Working Groups open-access series, the DLF Teach Toolkit. The materials will be revised and tested, including during a pre-conference workshop at DLF’s Annual Forum 2020, pending acceptance.
Figure 3. Immersive Pedagogy symposium participants in discussion.
The following essay sections explore the key components of the symposium, which outlined the theoretical foundations to decolonizing development and curation of 3D/VR/AR tech, before guiding participants through workshops on decolonial critique and accessible design, on integrating immersive technology into the classroom and beyond, and on collaboratively designing 3D/VR projects.
Decolonial Foundations: Critical Approaches to the Development and Curation of 3D/VR/AR Technologies
To practically introduce the decolonial methodologies and theories crucial to our workshops on developing and curating 3D/VR/AR materials, the Immersive Pedagogy symposium opened with a workshop, led by Gauthereau and Young on the “walkthrough method” (Light, et. al. 2018, 881–900), a critical analysis of technology using the Unity Asset Store as an example. This exercise was contextualized through a theory of decolonial pedagogy and a discussion on the critical analysis of the game platforms that curate content for 3D modeling and representation.
The application of decolonial theory and methods to digital pedagogy allows students to interrogate and resist colonial, hierarchical epistemologies, especially the privileging of Western European and Anglocentric knowledge structures. Such an approach is increasingly necessary as 3D/VR/AR technologies become integral to Western education systems and overwhelmingly applied to cultural heritage projects by and for Western consumers. While colonialism refers to the “political and economic relation in which the sovereignty of a nation or people rests on the power of another nation,” making that nation an “empire,” coloniality “refers to long-standing patterns of power that emerged as a result of colonialism, but that define culture, labor, intersubjective relations, and knowledge production well beyond the strict limits of colonial administration” (Maldonado-Torres 2007, 243). Thus, coloniality denotes the ways in which colonial hierarchies of power continue to structure our everyday lives (i.e. racialized class hierarchies, labor hierarchies, gender hierarchies, the gender binary, racism, etc.). Decolonialism urges us to actively de-link from colonial epistemologies and ontologies in order to avoid re-creating colonial worldviews and hierarchies.
Considering the ways that 3D/VR/AR technologies allow users to create immersive worlds and environments, the symposium sought to stress the need to avoid replicating the colonial gaze. Representing marginalized people through this gaze continues to enforce racialized and gendered hierarchies of power. Colonial epistemologies continue to control knowledge production, not only through institutional archives, but also through academic research, digital projects, and 3D/VR/AR environments. Jacqui Alexander and Chandra Talpede Mohanty argue that decoloniality has a “pedagogical dimension” as it obligates us “to understand, to reflect on, and to transform relations of objectification and dehumanization, and to pass this knowledge along to future generations” (1997, xxviii-xxix). For this reason, the symposium’s first workshop exercise involved guiding participants through a decolonial walkthrough of the Unity Assets Store. The walkthrough method requires researchers to directly engage with “an app’s interface to examine its technological mechanisms and embedded cultural references to understand how it guides users and shapes their experiences” (Light, et. al. 2018, 882). We asked participants to browse and search the Unity Assets Store for 2D, 3D, audio, and animation assets and interrogate them using a decolonial approach, as well as to document their walkthrough by taking notes, taking screenshots, and recording audio-visual content.
To guide the decolonial inquiry, we asked participants to consider a set of questions adapted from Roopika Risam’s discussion of the stakes of postcolonial and decolonial digital humanities (2019, 35–46):
What are compulsory activities within the Unity Asset Store?
What are the social hierarchies within the menu system?
To whom and which types of users is this knowledge accessible?
What is considered a “legitimate” asset within the Asset Store?
Whose epistemologies, such as histories, languages and memories, are considered important enough to archive in the Asset Store?
What knowledge or assets are privileged within the Asset Store?
Does the asset avoid the exoticization or fetishization of a people/cultures?
This inquiry resulted in participants recognizing the disproportionate representation of a Eurocentric worldview. For example, they noticed that the search term “Viking” yielded twice as many results as “Native American,” whereas the term “Indigenous” yielded zero. Among results for the search term “Mexican,” participants discovered a Mexican Restaurant Pack that reflected generalized stereotypes of Mexican aesthetics and cuisine, reduced to bottled hot sauce, chips and salsa, and a decorative green parrot. Assets also reduced the multiple and varied cultures, nations, flora, and fauna of the entire African continent to the myopic colonial imaginary of only the Serengeti, populated by wild animals. During group break-out sessions exploring the Asset Store, participants discovered a potential intervention through editing crowdsourced user tags. Like during Wikipedia Edit-a-Thons, users could challenge the authenticity of colonial representations of people, cultures, and nations by tagging or reviewing assets as not authentic, representing stereotypes, reproducing colonial views, etc. Since the symposium, unfortunately, Unity has removed the user tagging option and currently limits metadata generation to the individual uploading the asset.
This workshop stressed that engaging in decolonial work requires a constant questioning of how knowledge (3D/VR/AR environments, research, stories, syllabi, etc.) is being produced, who is producing it, whose stories are being told, and how these stories are being told. Not only should we consider what histories are told in the digital world, but we must also attend to the ways in which they are produced. As a result, the participants learned that generating and interacting with 3D/VR/AR environments they must use decolonial methods to acknowledge their role as world-creators and reflect on the ways that these technologies often replicate colonialism.
In the following workshop, Clark foregrounded the ableism endemic to technological innovation in the West, introducing participants to accessible user design for virtual reality. This involved a tutorial on developing alternate access plans for disabled students in classrooms. Clark’s work with Temple Libraries’ colleagues Jordan Hample and Wermer-Colan has prioritized research into and creation of accessible features for VR during their development of the Virtual Blockson: A Primary Source Teaching Tool for Secondary Education (Clark 2018, np). Clark’s workshop overviewed the standards of the World Wide Web Consortium’s Web Content Accessibility Guidelines in order to showcase the problems with applying standards created for web-based screens to virtual reality environments and experiences. She related an overview of key advancements that can be made to enable universal design for this emerging technology ranging from innovations in haptic feedback to caption legibility. Clark’s talk focused on guiding participants through strategies for accessing resources for disabled students at their universities. She led participants through an exercise with a template she created for developing “alternate access plans” that enable teachers to offer comparable options for students who cannot use the available VR and/or AR hardware and software. This approach to accommodating students with different learning styles provides a realistic way for teachers to work with emerging technologies in academic institutions, most of which still lack sufficient resources to support disabled students in the use of analog technologies.
Virtual Lessons: Integrating Immersive Technology in the Traditional Classroom and Beyond
After the symposium’s opening workshops on decoloniality and anti-ableism in immersive pedagogy, Levy’s workshop put to practice the principles she laid out in her closing keynote address on the idiosyncratic game mechanics for simulating virtually interactive dialogue and exam questions involving classification. VR offers, Levy argued, a unique pedagogical opportunity, functioning as a distraction-free zone where her students were able to recollect experiences at a much higher rate compared to other learning activities. During the workshop, Levy asked symposium participants to select several library books from various library collections and work in groups to think about how to put the texts in conversation with each other based on titles, subject headings, table of contents listing, and a quick skim of their contents. Levy demonstrated how and why she constructed a VR environment that simulated this activity, as her students had to physically place boxes with various titles onto empty shelves in an order that reflected connections. The application of VR to this type of historiographical exercise, Levy maintained, left a lasting impression on the students that they were able to put into practice for essay assignments. Levy’s emphasis throughout her workshop on the pedagogical significance of “what happens before and after” the virtual experience, furthermore, offered a valuable foundation for the subsequent workshop on integrating writing exercises to guide student learning during virtual and augmented reality experiences.
Wermer-Colan’s workshop modeled how to guide undergraduate students across the disciplines through a structured composition exercise for reflection, in particular, by guiding the participants through a reflection on what they hoped to learn and do in the coming school year as they sought to develop their immersive pedagogy projects. To provide a context and model for students before their writing reflections, Wermer-Colan summarized his current projects employing 3D technologies for Temple University Libraries’ Digital Scholarship Center (now the Loretta C. Duckworth Scholars Studio). Temple Libraries has experimented with transforming the purposes of library collections, development, and reference work to enhance its learning and technology outreach, including through its Innovative Teaching with Makerspace Technology Grant and its newly constructed VR Lab in the new Charles Library. Wermer-Colan’s past experiences working in the Medgar Evers College Writing Center in the City University of New York (CUNY) system helped him to think about ways the Digital Scholarship Center can use 3D/AR/VR technology to enhance learning across the disciplines.
As an example of Temple Libraries’ supporting the use of immersive technologies in class-room projects, Wermer-Colan detailed a collaborative project with Ajima Olaghere, Assistant Professor of Criminology working with her ethnography students to do “systematic social observation” of Philadelphia neighborhoods. This project used 360 cameras to record neighborhoods affected by Temple’s gentrification of North Philadelphia. The recordings were later viewed on twenty-dollar Desktek smartphone headsets that allowed students to remotely examine environments to understand what contributes to disorder and crime, while the instructor facilitated ways to maintain a critical understanding of what they were viewing. The accompanying writing exercises guided students to reflect on their mediated experiences of urban space and call into question the “broken windows theory,” common assumptions that visible signs of public disorder exacerbate criminal behaviors. The use of phone-based headsets also invited an opportunity for students to consider the physical processes that enable virtual technology. Instructors were faced with the problem of scaling pedagogical uses of VR; as this project used relatively inexpensive headsets, workshop participants considered how to create immersive experiences similar in quality to those offered by state-of-the-art VR headsets like the HTC VIVE or the Oculus Rift that, as of 2020, cost hundreds of dollars.
To illustrate the role libraries and digital scholarship centers can play in the curation of 3D content for teaching and learning, Wermer-Colan overviewed a complementary use of immersive technologies. His collaboration on the Virtual Blockson project with Digital Scholarship Librarian Jasmine Clark, Academic Technician and Developer, Jordan Hample, and Blockson Archivist Leslie Willis-Lowry aims to recreate Temple’s Charles L. Blockson Afro-American Collection as a virtual reality game for innovating the teaching of primary source literacy in high schools across Philadelphia. The project at its heart allows a small, fixed collection and its reading room to be available to students remotely, lowering the intimidation factor and physical limitations of these spaces, while enabling interactive explorations of historical artefacts. The Virtual Blockson offered an opportunity to discuss how libraries can help curate interactive gaming environments for remediating archival collections and cultural heritage sites to foreground previously marginalized histories. In these contexts, virtual reality offers affordances for lowering the barrier for students to use archival sources and spaces, facilitating access and accessibility, and offering students a novel medium through which to conceptualize analog and digital literacies necessary to navigate the changing new media world today.
Wermer-Colan foregrounded in both these projects the use of writing exercises to help students reflect on their virtual experiences in meaningful ways. The 360 SSO writing exercise encouraged humanistic thinking about the technology by asking students to compare their field work exercises with the virtual experience, as well as writing reflections that asked the students to identify various ways the 360/VR technology mediated said experience. Similarly, humanistic writing exercises were designed to guide students before and after their experience of the Virtual Blockson’s introduction to archival spaces, etiquette, and practices through game-based, interactive experiences. Drawing upon the Society of American Archivists’ Standards for Primary Source Literacy and the Common Core Standards for historical understanding, digital literacy, and critical thinking, these critical writing questions ensure students reflect upon the virtual experience of library collections’ historical artifacts from the African diaspora. After offering these models to the Immersive Pedagogy participants, Wermer-Colan guided the group through a critical writing exercise to reflect on their own plans to implement the 3D/VR/AR technology for various pedagogical purposes. Wermer-Colan encouraged participants to think of resources at their local institutions, pedagogical standards in their disciplines, and affordances in the spatialized medium of VR for enhancing their approaches to teaching. The writing exercise simulated the kind of exercise participants could implement in their own pedagogy, while offering an opportunity for the symposium participants to reflect on what they had learned during the workshops.
The concluding workshop, run by Linker and Young, offered a sustainable model for including students as partners in the creation of 3D/VR pedagogical materials, through an overview of Linker’s time creating the Bryn Mawr Women in Science project with her various undergraduate partners: Elia Anagnostou, Courtney Dalton, Jocelyn Dunkley, Tanjuma Haque, Arianna Li, and Linda Zhu. From 2017 to 2019, Linker taught undergraduate students how to integrate historical inquiry with 3D technology in order to think about women’s invisible scientific labor, the spaces they occupied, the tools they used, and their everyday lived experiences. The project considers Margaret Rossiter’s “The Matthew Matilda Effect in Science,” which articulates a systematic disparity in affording women scientists credit for sophisticated and important discoveries, which in turn necessitates that historians find ways to tell stories in order to make their labor visible. It likewise adapts aspects of Pamela Smith’s Making and Knowing Project by taking seriously the need to consider scientific processes. However, rather than engaging in physical reenactment, students offered up women’s processes in a modern, digital format, contextualized by a recreation of spaces that were no longer intact or available for historical analysis.
Figure 5. Screenshot of the 3D-rendered Advanced Biology Lab c. 1900, from Bryn Mawr Women in Science.
Linker and her students recreated two laboratory spaces that had once existed at Bryn Mawr College in the late nineteenth and early twentieth centuries, the Major Chemistry Lab and the Advanced Biology Lab.[1] Students learned a variety of 3D skills, including 3D modeling, photogrammetry, various mechanics of the Unity 3D game engine, and the Oculus Rift. Interactive WebGL versions of the project are available online, and a VR demo of Bryn Mawr’s Advanced Biology Lab was available at the conference. The Advanced Biology Lab was the site of early genetic research and a place once utilized by Nettie M. Stevens, the subject of Stephen Brush’s Nettie M. Stevens and the Discovery of Sex Determination by Chromosomes. Years before Margaret Rossiter coined the phrase “the Matilda Effect,” Brush identified that Stevens’ discoveries had been overshadowed by male collaborators or individuals working concurrently on the same subject. Her contributions had likely been diminished because she was a woman. Students researched each space by spending time in Bryn Mawr’s Special Collections. Through building each laboratory, the students became aware of how to put historical materials in conversation, as no resource could tell them everything they needed to know to build and contextualize the 3D models. Pedagogically, the two-year process of building was designed to seed humanistic deployment of 3D technologies by undergraduate collaborators. Afterward these students participated in professional presentations of the digital and historical work, and served as ambassadors to various communities in order to disseminate the skills the project cultivated to a wider audience.
Linker enabled her students to accomplish a lot in a short period of time; no student was an expert in the technology or in historical research prior to their tenure on the project. This was intentional, as she sought to teach rather than to employ experts. Students represented a diverse range of interests and majors, and all students participated in each phase of production (rather than assigning humanities majors to research and STEM majors to coding) so that afterwards, they could create projects similar to this on their own. Part of what facilitated their success is that she treated them as equal partners in the project, making decisions with them throughout the two years they worked together.
To prepare her students to participate as equals, she devised a plan that would serve as an introduction to using 3D technology to address social and pedagogical problems, and would also serve as a diagnostic tool for assessing student strengths and interests. Essentially, students were asked to propose and implement a 35-hour project (which could be run over the course of days or weeks, depending on individual need) that used an aspect of the Unity 3D game engine’s functionality to teach users about something the students cared about. Students drafted plans that identified what they knew, what they needed to learn, and were prompted to think about modularity, such that students could scale the project if they were running out of time. Students who were not familiar with coding at first could use Unity’s GUI interfaces to produce fully functional scenes, allowing for students with varying levels of proficiency with computer science to produce something useful by the end of the exercise. By the end of the 35-hour period, students not only had a small project they could put in professional portfolios, but had become proficient in a particular aspect of Unity, thought about the technology as a means to serve others, and in implementing their projects, had a better sense of what they would need to do going forward. It also convinced them that they were capable of using the technology in a way the Unity tutorials did not engender. Linker and Young guided the participants in thinking through how symposium participants might adapt this exercise for their own project teams.
Conclusion
Through the symposium and the workshops described above, participants engaged in conversations around designing socially-conscious pedagogy for 3D/VR/AR. Building a framework for teaching and learning with 3D/VR/AR technologies founded in decolonial theory and practices resonated with our participants. This enabled the group to evaluate how projects and assignments fit into an ethical model for cultural heritage pedagogies. The symposium closed with a productive discussion about what the participants learned, with a focus on planning for future steps.[2] Several participants suggested the importance of backward design, which would specifically place the learning outcomes as the first step in creating 3D/VR/AR and related assignments.
Conversations among group members brought up multiple questions, such as: how do we anticipate student use? How do we adjust our use of 3D/VR/AR in response to unexpected circumstances? How do we introduce emerging technologies in the classroom while accommodating individuals unable to take advantage of the intended purposes of ready-made hardware and software? How can these technologies enhance hybrid and online learning? Are students (or faculty) distracted by the freedom of immersive environments? Can we create bilingual metadata in a VR environment? If one could, where would you display subtitles or transcriptions in a virtual or augmented environment?
These conversations confirmed that digital humanists would benefit from future cross-institution discussions of 3D/VR/AR, as well as from shared access to teaching materials, which are often siloed within institutions and departments. Students engage differently with course concepts and each other, depending on the application of the technology within that course. Student learning is dependent on the skills and interests of individual instructors; collaboration is necessary for producing robust materials and responsible projects. Perhaps the most challenging task is creating accessible and sustainable materials applicable to multiple modes of disciplinary learning outcomes at a time of rapid technological and institutional transformation.
In an effort to increase the reach of the conversations that arose out of Immersive Pedagogy, the symposium organizers are working to produce an open-access, peer-reviewed publication containing lesson plans and educational material to facilitate disciplinary and interdisciplinary work that engages 3D/VR/AR technologies. This project aims to extend the work of the Digital Library Federation (DLF) Pedagogy Working Group’s Teach Toolkit that provides lesson plans for digital library instruction.[3] To guide educators to adapt immersive technologies to the needs of diverse disciplines, the Immersive Pedagogy teaching materials will introduce a range of 3D hardware and software, including asset or game repositories. The teaching materials will include diverse lesson plans with tailored learning outcomes, introducing a representative sample of available immersive technologies and resources while addressing humanistic pedagogical goals. Because this project was born out of the CLIR postdoctoral fellowship program, it aims to contribute to the growing field of scholarship on the crucial role that academic libraries or research and teaching centers can play in the integration of immersive technologies across the curriculum.
The Immersive Pedagogy symposium’s prioritization of decolonialism, feminism, and accessibility speak to a radical and critical perspective that can apply to a range of 3D/VR/AR applications and instruction methods. Indeed, in starting conversations on how to promote making immersive experiences accessible and inclusive, there is an opportunity to move beyond operational concerns to lasting pedagogical practices. For decades, contingencies have transformed education and cultural heritage, requiring us to rethink the potential of emerging communication technologies through a critical lens. More evident in the midst of the COVID-19 global pandemic, which has spurred the need for digital ways of teaching and learning, is the critical pedagogical use of virtual surrogates. These include 360° museum spaces and objects, 3D virtual meeting spaces, photogrammetry models, and interactive exhibits. By addressing upfront, rather than through remediation, the issues of social justice, accessibility, and decolonial pedagogies in immersive technology, educators can leverage these tools to respond to a transformative period in the education system.
Notes
[1] For a discussion of problems and considerations specifically related to the construction of historical 3D spaces, see Sullivan, Nieves, and Snyder 2017.
Alexander, Jacqui M. and Chandra Talpede Mohanty, eds. 1997. “Introduction.” In Feminist Geneologies, Colonial Legacies, Democratic Futures: xxvii-xxiv. New York and London: Routledge.
Brush, Stephen G. 1978. “Nettie M. Stevens and the Discovery of Sex Determination by Chromosomes.” Isis 69, no. 2: 163–172. https://www.jstor.org/stable/230427.
Cotera, María. 2018. “Nuestra Autohistoria: Toward a Chicana Digital Praxis.” American Quarterly 70, no. 3: 483–504. https://muse.jhu.edu/article/704334.
Grayburn, Jennifer, Zach Lischer-Katz, Kristina Golubiewski-Davis, and Veronica Ikeshoji-Orlati, eds. 2019. 3D/VR in the Academic Library: Emerging Practices and Trends. Arlington, VA: Council on Library and Information Resources.
Levy, Juliette. 2017. “‘Digital Zombies’– A Learner-Centered Game: Social Knowledge Creation at the Intersection of Digital Humanites and Digital Pedagogy.” In Social Knowledge Creation in the Humanities: An Open Anthology, edited by Alyssa Arbuckle, Aaron Mauro, and Daniel Powell. ITER Press. https://ntmrs-skc.itercommunity.org/.
———. 2019. “How Not to be a Replicant: Working Towards a Useful VR.” Keynote address, Carnegie Mellon University Symposium, Pittsburgh, PA, June 28.
Light, Ben, Jean Burgess, and Stefanie Duguay. 2018. “The walkthrough method: An approach to the study of apps.” New Media & Society 20, no. 3: 881–900.
Linker, Jessica, Elia Anagnostou, Courtney Dalton, Jocelyn Dunkley, Stella Fritzell, Tanjuma Haque, Arianna Li, Rachel Starry, and Linda Zhu. 2019. Bryn Mawr Women in Science. https://digitalscholarship.brynmawr.edu/howis/.
Losh, Elizabeth and Jacqueline Wernimont, eds. 2019. Bodies of Information: Intersectional Feminism and Digital Humanities. Minneapolis, MN: University of Minnesota Press.
Maldonado-Torres, Nelson. 2007. “On the Coloniality of Being.” Cultural Studies 21, no. 2: 240–270.
McPherson, Tara. 2012. “Why Are the Digital Humanities So White? or Thinking the Histories of Race and Computation.” In Debates in Digital Humanities, edited by Matthew K. Gold. Minneapolis: University of Minnesota Press.
Nieves, Angel David. 2019. “Developing a Social Justice Framework for Immersive Technologies in Digital Humanities.” Keynote address, Carnegie Mellon University Symposium, Pittsburgh, PA, June 27.
Risam, Roopika. 2019. New Digital Worlds: Postcolonial Digital Humanities in Theory, Praxis, and Pedagogy. Evanston, IL: Northwestern University Press.
Robnett, Belinda. 1997. How Long? How Long?: African American Women in the Struggle for Civil Rights. Oxford: Oxford University Press.
Rossiter, Margaret. 1993. “The Matthew Matilda Effect in Science.” Social Studies of Science, 23, no. 2: 325–341. http://www.jstor.org/stable/285482.
Sayers, Jentery, ed. 2017. Making Things and Drawing Boundaries: Experiments in the Digital Humanities. Minneapolis: University of Minnesota Press.
Smith, Pamela. n.d. The Making and Knowing Project: Intersections of Craft Making and Scientific Knowing. https://www.makingandknowing.org/. Accessed Dec. 13, 2019.
Sullivan, Elaine, Angel David Nieves, and Lisa M. Snyder. 2017. “Making the Model: Scholarship and Rhetoric in 3D Historical Reconstructions.” In Making Things and Drawing Boundaries: Experiments in the Digital Humanities, edited by Jentery Sayers, 301–316. Minneapolis: University of Minnesota Press.
We would like to acknowledge The Andrew W. Mellon Foundation and the Council on Library and Information Resources (CLIR) for the microgrant that funded the Immersive Pedagogy symposium, as well as Carnegie Mellon University Libraries for hosting the event. Thank you to the entire Immersive Pedagogy team, including Eric Kaltman, Neil Weijer, and Chris Young for making the symposium possible. Last, but certainly not least, thank you to all the Immersive Pedagogy participants and keynote speakers, who created a positive, productive community of practice: Andy Anderson, DB Bauer, Katie Chapman, Elena Foulis, Kat Hayes, Juliette Levy, Juan Llamas-Rodriguez, Meaghan Moody, Angel Nieves, Samantha Porter, Coral Salomón, Julia Troche, Jordan Tynes, and Christa Williford.
About the Authors
Lorena Gauthereau is the Digital Programs Manager for the US Latino Digital Humanities program at the University of Houston’s Recovering the US Hispanic Literary Heritage. She received her Ph.D. in English and her M.A. in Hispanic Studies, both from Rice University. Her research interests include US Latinx studies, digital humanities, and decolonial theory. Orcid ID: orcid.org/0000-0002-7185-8982.
Jessica Linker is an Assistant Professor of History at Northeastern University. She was previously a Postdoctoral Fellow and Program Coordinator at the Consortium for History of Science, Technology and Medicine, a Visiting Assistant Professor at Bryn Mawr College, and the Director of Bryn Mawr Women in Science. She researches women’s scientific practices in early America.
Emma Slayton is the Data Curation, Visualization, and GIS specialist at Carnegie Mellon University Libraries. She obtained an MPhil from the University of Oxford in 2013 and completed her Ph.D. at the Faculty of Archaeology, Leiden University in 2018. Her current work centers around improving and supporting digital literacy efforts. Orcid ID: https://orcid.org/0000-0003-2230-3101.
Alex Wermer-Colan is a postdoctoral fellow in Temple University Libraries’ Loretta C. Duckworth Scholars Studio, where he coordinates research and pedagogical projects in cultural analytics and digital media arts. His editorial and scholarly criticism have appeared in PAJ: A Journal of Performance and Art, Twentieth Century Literature, The Yearbook of Comparative Literature, Lost & Found, Indiana University Press, and The Los Angeles Review of Books. Orcid ID: https://orcid.org/0000-0001-7030-6070.
The use of virtual simulations has steadily increased within the social work master’s level curriculum and its benefits have been noted in varied research studies. This paper aims to describe the development of two virtual simulations that are innovative, educational, and responsive to the needs of social work skill development and knowledge reinforcement. The first module is a 360 virtual reality (VR) tool that allows social work students to navigate areas of the Lower East Side of New York City. The second module allows social work students to conduct a home visit and assess a virtual client within an immersive 3D VR environment utilizing a framework built on top of a commercial game engine. Each simulation is individually described and the processes taken to create the scripts, assets, and framework are detailed. Their pedagogical value to social work is noted and next steps in VR development, assessment and research are outlined and discussed.
Introduction
Social work masters-level education integrates both theoretical and practice based curricula. Traditionally, each facet has been delivered to students within the classroom and Field placement setting. Technology, however, has broadened the systems of delivery and has facilitated and advanced these pedagogical methods.
The use of virtual simulations has steadily increased within social work masters-level education and its benefits have been noted. Generally, simulation based learning has been shown to be effective for adult students because it is problem-focused, self-directed and it has relevant applicability to real settings (Washburn, Bordnick, and Rizzo 2016). Virtual simulations allow for greater diversity and variability in Field settings and in patient demographics that may not be found in “real” Field placement outlets. It also provides students with “immediate” feedback and knowledge from their interactions within an assortment of settings and with diverse clients. Admittedly, the virtual interaction will be limited in some ways, but it will nonetheless provide students the advantage of gaining additional Field/practice perspectives that cannot be feasibly offered in vivo. Virtual simulations may be designed to be accessible throughout the day and will therefore be able to accommodate multiple student connections at one time. Virtual environments grant students “safe spaces” to make mistakes and to hone in on skill sets that they may not ordinarily be processing in “real” settings (Boulos, Hetherington, and Wheeler 2007). Research has indicated that students feel more comfortable and less pressure to perform in virtual settings as opposed to working with face-to-face role plays, a more traditional method of simulation (Fitch, Canada, Cary, and Freese 2016). Within the safety of the virtual simulation, students have the ability to rehearse and reflect upon the methods to use for the presented issues, without jeopardizing the protective standards of practice and client safety (Olson, Lewis, Rappe, and Hatley 2015). To facilitate learning, students may pause a simulation so that they can revisit a scenario multiple times or attempt to answer questions more than once. These systemic techniques will improve reflective skills (Boulos et al. 2007), which ultimately lead to better process and understanding.
The use of simulation training, particularly when incorporating standardized patient information, can in fact be utilized to measure and assess a student’s proficiency levels in the areas of practice and Field learning (Olson et al. 2015; CSWE 2015), both prior to and post the Field placement experience. This is an improvement over current proceedings, where unstandardized assessment modes are often used to evaluate student competencies (Washburn et al. 2016). Social work techniques are not always as clear and as easily defined as those in other disciplines; the delivery and operationalization of certain interpretive and evaluative skill sets are more abstract (Rishel and Majewski 2009) and they encompass various facets of social work subject matters. Therefore, it is important that the virtual simulation address these factors and the intersectionality of the social work curricula so that students may be evaluated on various levels: engagement, assessment, intervention, and critical thinking.
This paper aims to describe the development of two virtual simulations at New York University’s (NYU) Silver School of Social Work (Silver); the content of which is both innovative and educational. The first module is a 360 virtual reality (VR) program that allows social work students to navigate areas of the Lower East Side of New York City. The second, allows social work students to have an immersive experience of conducting a home visit to assess a client. Each of the virtual simulations operationalize critical thinking and attempt to concretize social work competencies in a holistic manner (CSWE 2015). Silver will endeavor to integrate each of the modules within courses to prepare students to enter community environments, and to hone social work practice skills, particularly related to client engagement and assessment.
Module One: Social Environment Immersion
Social work educators, particularly in Practice and Human Behavior related courses emphasize the correlations that exist between the social environment and an individual’s physical, cognitive, and emotional development. Most classes incorporate the biopsychosocial perspective as a tool to understand and process knowledge about a person’s biological, psychological and social interactions. This perspective views the person in the context of the environment and takes into consideration the challenges and stressors that might influence development throughout the life cycle. However, having students apply this perspective with clients as they work with them in their respective neighborhoods and communities can be challenging. Abstract theoretical principles are not easily transferable to real environmental circumstances, and students often struggle to recognize the theoretical frames in social contexts and then apply them to the populations that they are serving. An extensive body of research in the learning sciences supports the efficacy of learning activities situated in authentic practice and scaffolded within a community of practice (Lave and Wenger 1991; Collins 1987; Brown et al. 1989). Research on metacognition and instruction suggests that the incorporation of reflection opportunities into learning experiences improves the potential for knowledge transfer to future practice (Brown 1978; Flavel 1976; Bransford and Schwartz 1999). The 360 VR experience builds on models of cognitive apprenticeship by incorporating opportunities for corrective action based on reflection from students and feedback from instructors.
The Lower East Side (LES) is an area of New York frequented by social work master’s students completing the Field requirements of their curriculum. The LES was chosen as a pilot focus because of its demographic diversity, immigration history, social resources and its connection to the settlement house movement; each are important facets within social work (Citizens’ Committee for Children of New York 2015).
Due to budgetary restrictions on the acquisition and use of virtual simulation (VS) software, the authors adopted WONDA VR, a prosumer-oriented 360 video-editing platform. Wonda enabled our team to incorporate interactive elements such as hotspots, buttons, information panels, branching narratives, and a variety of other multimedia experiences relevant to the learning. In addition to accommodating the learning objectives and scope of the intended project, Wonda VR presented a robust alternative to more costly professional software platforms on the market.
Storyboarding
Prior to the start of video production, Silver’s educational technologists designed a storyboard/scripting template that faculty could use for inputting content and visualizing the multiple branches of the narrative and user interactions. The process of designing this template was iterative; faculty and designers worked closely to develop a prototype using a basic spreadsheet that included columns describing each scene, associated competencies, learning objectives, action, and dialogue. Designers and faculty also collaborated on framing the substantive content within the context of the objectives and ensuring that it would succinctly coincide with each video still.
Figure 1. Google sheet storyboard template.
Video Production
Our educational technology team videotaped each scene of the 360 video using a Nikon KeyMission 360 camera and standard tripod set up. Guided by the storyboard, the camera operator positioned the camera at predetermined locations on the Lower East Side, capturing two minutes of video footage per scene. Each scene of video footage was stitched together using Adobe Premiere software and an actor was subsequently enlisted to record voice-over audio of the script. The VR simulation took approximately nine months to complete.
Our budget included funding for a 360 video production consultant with advanced Wonda expertise who was brought on to complete the video during the final phase of post-production. With a background in fine art and documentary video production, the consultant worked iteratively with faculty and the education technology team to align his aesthetic sensibilities with the learning objectives, and multimedia cognitive design principles for engagement and learning (Mayer and Johnson 2010; Plass 2009).
Walkthrough
The 360 VR simulation starts with the disembarkation from the F train, at the Forsythe subway station and ending with a sidewalk view of Orchard Street.
Figure 2. First scene from 360 VR: Forsythe train station with individuals on train platform. “Hotspot” is shown with the map of the Lower East Side.
The VR simulation is approximately 12 minutes in duration, and includes an instructional guide, learning objectives, content chapters, an introduction, and 17 navigable and interactive scenes (environments). Each scene contains verbal and textual dialogues, and a situational map. Reflective exercises and “hot spots” are presented throughout to guide students in thinking critically about the population inhabiting the Lower East Side, the community landscape, and data points, including demography, mental health, and housing. These key components were derived from the storyboard texts, which required collaborative editing by both designers and educators.
Figure 3. Street scene from 360 VR of the Lower East Side. “Hotspot” indicates the number of police stations, fire stations and EMS resources in the area.
Pedagogical Integration
The Silver School of Social Work aims to use the 360 VR tool within two courses and their respective curricula: Human Behavior in the Social Environment I (HBI) and Introduction to Social Work Education and Practice in the United States (ISWEPUS). The first, HBI, is a core course that all graduate students are required to take in their first year of study. It examines the different systems (individual, family, and communities) in the environment and the relationship that exists between them. Lessons two and three of the HBI course are specifically centered on the social environment, with topics that include risk and resilience factors within communities that impact human behavior, and social work interventions designed to develop and promote community resources. Faculty would use the VR tool to help students better understand how to recognize the resources that are both present and deficient within the community. Students would then critically think about the implications that their presence and absence poses to community residents. The VR’s verbal and written guide, scripted by faculty, poses reflective questions throughout the virtual journey, prompting students to think about community circumstances and their implications to target populations.
The ISWEPUS course is purposely created to prepare international students for coursework and Field instruction in the graduate social work program and within the greater context of the United States (US). In part, lectures and experiential exercises are constructed to help International students develop an understanding of the background of social work in the US, the core values of social work, and the nuances of the biopsychosocial perspective. Lessons four and five are centered on social work practice in the US. Students are given a foundation of social work history, where they are taught the legacies of the social change movement and they are introduced to perspectives on social reform. Within the context of these lessons, students will have an opportunity to engage with the 360 VR tool. Since the VR setting is the LES, which is historically significant to social work, immigration, and social reform, it will be shown to students via WONDA Spaces. This new platform allows multiple users to enter the virtual space simultaneously, enabling faculty to actively guide the students within the environment. The interactive class exercise will help students proactively apply theoretical ecologically centered models to communities, while also getting students to understand the impact that community environments have on individuals.
Before the school disseminates the 360 VR tool within the HBI and ISWEPUS courses, first-year students enrolled in the Masters of Social Work program at Silver School of Social Work are being asked to participate in a small IRB approved study. The project will assess the VR tool’s pedagogical value and student reactions to VR use, generally. Recruited students are contacted individually to schedule an appointment to view the VR simulation and to complete the pre- and post-test questionnaires. At the scheduled appointment time, students complete the pre-questionnaire via a Qualtrics access link. The questionnaire is based on a measure created and provided by North Carolina State University. The original measure was modified to accommodate the simulation. It contains questions on areas of learning perception, learning experience, technology experience, and general attitudes toward the 360 simulation. Once the questionnaire is completed, students are offered the option to view the simulation via VR goggles or via a designated computer desktop. The research assistant ensures that students are given VR access, and they are present throughout the VR simulation to monitor for any disruption in VR service. They also note any spontaneous feedback that is provided by the participant. Upon completion of the simulation, participants are asked to complete the post-questionnaire via a Qualtrics access link. Data will be downloaded from the Qualtrics website into a Statistical Package for the Social Sciences (SPSS) software.
Initial data comparing pre- and post-tests of students engaging in the 360 VR tool indicate statistically significant learning gains and also that students value and appreciate using 360 VR.
Module Two: Virtual Engagement Utility
By 2030, there are projected to be 80 million older persons, over twice the number living in 2000 (Lowell 2015). Yet many social work students and indeed professionals are reticent to work with this population, which is often based on fear, bias, and ageist beliefs. Research has repeatedly noted that consistent exposure to the population and its nuances is the strongest predictor for future gerontological work (Wang and Chonody 2013). Working toward this objective, a VR tool was created to provide students an opportunity to practice with an avatar so that they might gain some understanding on how it is to engage and assess an older adult in their home.
Silver partnered with NYU’s Teaching and Learning with Technology Group (TLT) to pilot an in-development virtual reality framework known as the Virtual Engagement Utility (VEU). VEU will be provided to NYU as a service to streamline the creation of educational virtual experiences. The client visit scenario built with VEU is multi-layered and contains 11 different stages of interactions that start from a knock at the client’s door initiating the visit and assessment process, to the summarization and end of the in-home visit, where the student can generate an evaluation sheet of how they progressed through the exercise. Such exposure might induce them to work with the population within their Field internships. As will be described, the VEU development was quite detailed and it was one of the first collaborative VR related exercises undertaken. The project took 19 months from its inception to the final deliverable product. The VEU is designed with a reusable framework so it is anticipated that future projects will take shorter periods of time to develop.
Script Development
The client composite and the main artery of the script guiding the VR were based on the Field notes received from MSW students who were currently interning within agencies that primarily worked with older adults in the context of their homes. Issues cited by students within their notes served as a basis for the challenges that student users would face within the simulation. Once the main script was developed by faculty, it was then placed in storyboard and branching sequences were added to further enrich the interaction between student and avatar. The case dossier or file that was constructed for the client avatar was based on an assessment form used within a social work agency. VEU learning tasks and goals were based on learning objectives stipulated within core Practice courses within the social work graduate curriculum. The final script and storyboard was reviewed by faculty and a content expert.
Concurrent with script authoring, members of the Interactive Development team, part of NYU’s TLT group, created a branching narrative template. Template creation was an iterative process with input from faculty informing its clarity and ease of use. This template was used as a tool to both visualize and organize the complexity of a branching narrative, including interactions with the virtual environment, feedback on performance, and animation and audio cues for voice actors and software developers.
Figure 4. VEU branching narrative template.
Framework Development
VEU was developed in Unity, a cross-platform game engine. Virtual reality experiences created in the VEU framework are deployable to Google Cardboard and Google Daydream compatible devices, both smartphone based VR platforms. These platforms require VR viewers, which are available from both Google and a variety of third party vendors. VEU scenarios can also be deployed as a traditional interactive experience to PC and Mac platforms using a mouse and keyboard for interaction. VEU allows students to interact with virtual characters and their environment, complete tasks, save their progress, view feedback on their performance, and share their progress with faculty.
Asset Creation
Under guidance from faculty and their content consultant, members of the TLT media group created both a virtual environment and client avatar. Environmental 3D models were sourced or crafted internally using Autodesk Maya. Textures were created using Adobe Photoshop. The apartment layout and appearance went through several iterations informed by reference images from the content expert and feedback from faculty within the School of Social Work.
The client avatar was created with Autodesk Character Generator and further customized in Maya. The script was recorded by a voice actor as individual responses while the actor’s face was captured using a basic HD webcam. The raw audio and video was processed using Adobe Premiere to remove background noise and normalize audio levels. Faceware Analyzer was used to map the actor’s facial expressions to the face of the avatar. Body animation and generic responses such as frustration, smiling, or nodding yes and no, were manually animated in Maya. These assets were imported into Unity, creating the environment and avatar that students interact with.
Figure 5. Interior Image of VEU apartment.
Figure 6. VEU interior living room area with client.
Framework Customization and Functionality
Once assets were imported into Unity, the VEU framework was used to create an interactive experience. Dialogue between the student and virtual client was inputted into a node based dialogue tree. When a student clicks on the avatar using a pointer in VR, or the mouse on a PC, they are presented with a dialogue bubble and multiple options to respond, guiding the conversation. Most choices provide some form of feedback. Optimal decisions allow the conversation to move forward. Less appropriate responses provide feedback, then return to the previous choice, allowing students to try again. Students can be prompted to observe and interact with their environment, identifying objects relevant to the client visitation. At any time the client file can be accessed so students can refer back to their visitation goals and client background information. Access to controls such as saving or sharing assessment results was also customized to appear as a smartphone. These customizations serve to provide a more immersive experience.
Figure 7. VEU student perspective.
Figure 8. VEU client file that can be accessed by students throughout the VR experience.
Tutorial Development
Outside of research and professional environments virtual reality is a relatively new technology. Many students (and faculty) who utilize the VEU tool may have no prior experience with VR. For this reason a custom tutorial scenario was developed with the assistance of Silver faculty. The tutorial was designed to cover the types of interactions students might encounter in their virtual visit. It demonstrates basic interactions such as pointing and clicking, interacting with a dialogue tree, and how to access the client file. It utilizes some assets from the client visit scenario so students will be familiar with the look and feel before beginning their assessment.
Pedagogical Integration
The School of Social Work aims to incorporate this simulation within the Integrated Social Work Practice Field Instruction I course (Practice I). This core class provides graduate students with generalist knowledge on working with a range of systems that include individuals, couples, families, agencies and communities. It also helps students develop the basic skills of engagement, assessment and goals setting. Lesson nine of this course is centered on teaching students how to conduct a multidimensional biopsychosocial assessment, which needs to be culturally sensitive and attuned to the client’s strengths and risks. The VEU tool would be an assignment that students would need to complete post-lesson. The VEU’s prescribed goals of engagement, assessment, and of identifying needs and resources are in keeping with lesson nine. The VEU also reinforces the prior Practice I lessons that are centered on engagement and the helping process. The internal evaluation that is generated by the VEU for each student will be assessed by faculty and then reviewed with the student. Students can share their VEU progress with faculty at any time. Faculty receive an email with the score for each learning sequence as well as the individual choices made. The value of each question can be weighted for greater control over performance evaluation. This data is anonymized. Students can choose to de-anonymize the data by providing their unique ID assigned when first activating the VEU tool. A reflective exercise post-VEU will also be assigned to better understand the student’s experience and to obtain knowledge on how to further support their learning. Students will have additional opportunities to engage their VEU experience through discussion with classmates in structured classroom activities, as well as a programmed opportunity to revisit the VEU later in the semester with the goal of interleaving their prior experience with later course learning experiences, such as Field work with clients. Research supports the educational value of interleaving multimodal approaches to similar content (Birnbaum et al. 2013), reinforcing prior knowledge, and contextualizing learning experiences within larger conceptual frameworks (Richland et al. 2005).
Future Developments
As was noted in an earlier subsection, the VR 360 tool is currently being assessed for its pedagogical value with students. Although initial results indicate there is overall positive reaction to the tool, further steps will be taken to purposely infuse the VR simulation within HBI and ISWEUS and then obtain student reactions both quantitatively and qualitatively. Similarly to the 360 VR simulation, the VEU simulation will be studied in a pre- and post-test research design in the coming months for its pedagogical value and whether students are comfortable navigating the VR tool. Also, social work faculty with Practice expertise will be asked to review the simulation and respond to questions related to learning objectives, subjectivity, content and overall quality. Results will inform the development of the current VR simulations as they pertain to content, structure, and design.
In addition, several steps will be taken to further improve the VEU simulation’s design and internal evaluative mechanism. Metrics can be reported such as tracking time spent in the experience or on particular questions, and whether the scenario was completed in one session or several. Engagement could be measured by checking if students skip the voiceovers of the avatar or if they completed optional assessments. These metrics could then be reported alongside the multiple choice performance, giving faculty deeper insight into the behaviors and learning styles of their students.
The VEU framework will be adapted to the Oculus Quest headset. In the process it will gain six degrees of freedom hand tracking, so students can reach out and interact with the environment in an intuitive manner. The Quest is also capable of room-scale tracking, which will allow students to walk around the apartment so they may perform closer inspections of the environment. Faculty can then integrate questions related to hazards and safety, while also integrating the assessment within the internal evaluative mechanism. Additional modes of interaction with the virtual client are also being explored, such as a “flow mode” where feedback is limited and the conversation proceeds naturally and with less interruptions. These features will hopefully lead to a more immersive and approachable learning experience for the student.
Birnbaum, Monica S., et al. 2013. “Why interleaving enhances inductive learning: The roles of discrimination and retrieval.” Memory & Cognition 41, no. 3: 392–402.
Boulos, Maged N. Kamel, Lee Hetherington, and Steve Wheeler. 2007. “Second Life: an overview of the potential of 3‐D virtual worlds in medical and health education.” Health Information & Libraries Journal 24, no. 4: 233–45.
Bransford, John D., and Daniel L. Schwartz. 1999. “Chapter 3: Rethinking transfer: A simple proposal with multiple implications.” Review of Research in Education 24, no. 1: 61–100.
Brown, Ann L. 1978. “Knowing when, where and how to remember. A problem of metacognition.” In Advances in Instructional Psychology, Vol. I, edited by R. Glaser, 77–165. Hillsdale, New Jersey: Erlbaum.
Brown, John Seely, Allan Collins, and Paul Duguid. 1989. “Situated cognition and the culture of learning.” Educational Researcher 18, no. 1: 32–42.
Collins, Allan, John Seely Brown, and Susan E. Newman. 1987. “Technical Report No. 403: Cognitive apprenticeship: teaching the craft of reading, writing, and mathematics.” Center for the Study of Reading: U. Illinois at Urbana-Champaign.
Fitch, Dale, Kelli Canada, Suzanne Cary, and Rebekah Freese. 2016. “Facilitating social work role plays in online courses: The use of video conferencing.” Advances in Social Work 17, no. 1: 78–92.
Flavell, J. H. 1976. “Metacognitive aspects of problem solving.” In The Nature of Intelligence, edited by L.B. Resnick, 231–35. Hillsdale, New Jersey: Erlbaum.
Lowell Sun. 2019. “Elder Care–Giving: Will You Be Ready for the Challenge?” Lowell Sun, July 11.
Lave, Jean, and Etienne Wenger. 1991. Situated learning: Legitimate peripheral participation. Cambridge, UK: Cambridge University Press.
Mayer, Richard E., and Cheryl I. Johnson. 2010. “Adding instructional features that promote learning in a game-like environment.” Journal of Educational Computing Research 42, no. 3: 241–65.
Olson, Mark D., Melinda Lewis, Paula Rappe, and Sandra Hartley. 2015. “Innovations in Social Work Training: A Pilot Study of Interprofessional Collaboration Using Standardized Clients.” International Journal of Teaching and Learning in Higher Education 27, no. 1: 14–24.
Plass, Jan L., Bruce D. Homer, and Elizabeth O. Hayward. 2009. “Design factors for educationally effective animations and simulations.” Journal of Computing in Higher Education 21, no. 1: 31–61.
Richland, Lindsey E., Robert A. Bjork, Jason R. Finley, and Marcia C. Linn. 2005. “Linking cognitive science to education: Generation and interleaving effects.” Proceedings of the Twenty-Seventh Annual Conference of the Cognitive Science Society 27: 1850–55.
Rishel, Carrie W., and Virginia Majewski. 2009. “Student gains in self-efficacy in an advanced MSW curriculum: A customized model for outcomes assessment.” Journal of Social Work Education 45, no. 3: 365–83.
Um, Eunjoon, Jan L. Plass, Elizabeth O. Hayward, and Bruce D. Homer. 2012. “Emotional design in multimedia learning.” Journal of Educational Psychology 104, no. 2: 485.
Wang, Donna, and Jill Chonody. 2013. “Social workers’ attitudes toward older adults: A review of the literature.” Journal of Social Work Education 49, no. 1: 150–72.
Washburn, Micki, Patrick Bordnick, and Albert “Skip” Rizzo. 2016. “A pilot feasibility study of virtual patient simulation to enhance social work students’ brief mental health assessment skills.” Social Work in Health Care 55, no. 9: 675–93.
Washburn, Micki, and Shu Zhou. 2018. “Teaching Note—Technology-Enhanced Clinical Simulations: Tools for Practicing Clinical Skills in Online Social Work Programs.” Journal of Social Work Education 54, no. 3: 554–60.
About the Authors
Nicholas Lanzieri is an Associate Clinical Professor at NYU Silver School of Social Work and Faculty Lead for Field Learning & Community Partnerships. He also co-leads Silver’s Faculty Education Technology Board.
Henry Samelson is the Associate Director of Educational Technology at NYU Silver School of Social Work. He received his MA in Digital Media Design for Learning/Games for Learning from NYU Steinhardt School of Media, Culture and Communications.
Jon Bowen received his MS in Biomedical Visualization at the University of Illinois at Chicago and is currently a senior interactive developer at New York University. He creates games and simulations for education and is interested in emerging technologies such as extended reality (XR) and spatial computing.
Tracy Dorrington-Skinner, Victims of Institutional Childhood Exploitation Society (VOICES)
Gerald Morrison, Victims of Institutional Childhood Exploitation Society (VOICES)
Tony Smith, Victims of Institutional Childhood Exploitation Society (VOICES)
and
The DOHR Team
Abstract
Relational presence is the core principle of a new approach to designing virtual learning environments (VLEs), which has been developed by the Digital Oral Histories for Reconciliation (DOHR) project (dohr.ca). Presence, normally understood as the sense of being in a virtual environment to the extent that one forgets the environment is virtual, is thought to have significant pedagogical benefits in K–12 experiential learning projects aiming to develop spatial and social competencies that learners can translate into actual-world contexts. DOHR, by contrast, aims to build the understanding needed for learners to address systemic racism in Nova Scotia, through an oral history and restorative justice–based curriculum. To serve this alternative learning goal, relational presence replaces presence. The usual emphasis in VLE design on simulation, interactivity, identity construction, agency, and satisfaction is replaced with new values of impression, witnessing, self-awareness and awareness of difference, interpretation and inquiry, and affective dissonance. This paper introduces relational presence in order to help establish, in the field of VLE design, a productive discourse around issues of justice, representation of marginalized communities, and pedagogy-led design.
Introduction
This article introduces relational presence, the core principle of a new approach to designing virtual learning environments (VLEs) that has been developed by the Digital Oral Histories for Reconciliation (DOHR) project (dohr.ca). DOHR has worked in partnership with the Nova Scotia Home for Colored Children Restorative Inquiry (restorativeinquiry.ca). The Restorative Inquiry was a four-year, provincially-mandated public inquiry into the history and legacy of the Nova Scotia Home for Colored Children (NSHCC), including the lived experiences of its residents. The Home was a segregated care institution for African Nova Scotian children that operated in Dartmouth, Nova Scotia from 1921 until the early 2000s. Established to meet the care needs of African Nova Scotian children, the Home was a site of significant abuse and harm for many of its residents. Over the decades of its operations, former residents experienced neglect and abuse (Province of Nova Scotia 2019, 153–172). The Restorative Inquiry was established to examine the experience of the Nova Scotia Home for Colored Children in relation to systemic and institutionalized racism, both historic and current, in Nova Scotia. In order to “contribute to the goal of social change to end the harmful legacy of abuse and ensure the conditions, context and causes that contributed to it are not repeated” (Province of Nova Scotia 2015, 4–5), among its goals the Inquiry aimed to:
(a) Empower those involved in, and affected by, the history and legacy of the NSHCC to learn about what happened and the contexts, causes, circumstances and ongoing legacy of the harms related to the NSHCC.
(b) Educate the public about the history and legacy of the NSHCC.
(c) Publicly share the truth and understanding established through the RI and the actions taken, planned, and recommended to address systemic and institutionalized racism and build more just relationships for the future (Province of Nova Scotia 2019, 23).
The DOHR project was an important mechanism through which the Inquiry pursued this part of its mandate (Province of Nova Scotia 2019, 504–505). The DOHR project has brought former residents of the Home, representatives of the Nova Scotia education system, and members of the Inquiry’s Council of Parties together with artists and researchers from seven universities across Canada (Waterloo, Dalhousie, New Brunswick, McGill, Ottawa, Alberta, and British Columbia) to develop a two-week grade eleven Canadian History curriculum unit that supports students in learning about the historical harms experienced by former residents of the Home. In this way, it has served to support the mandate of education and the broader goal of moving toward reconciliation by building the understanding needed to address systemic racism in Nova Scotia.
DOHR is thus a community-driven project. It arises from a need articulated by a community and works to co-create the project with community members. This community mandate is central to the need for designing relational presence in the virtual reality (VR) experience. The Restorative Inquiry, from which the DOHR project was created, pursued a restorative vision of justice that was reflective of a relational worldview focused on connectedness. It sought justice in the form of just relations between individuals, groups, communities, and at the level of institutions and systems (Llewellyn 2011). As a restorative process, the Inquiry was “future focused, yet concerned with getting a comprehensive understanding of the past in order to know how to move forward toward a just future” (Province of Nova Scotia 2019, 26). It focused on learning about past harms in order to build more just relations going forward. This is, in simplistic terms, the impetus for a restorative approach to learning in the DOHR curriculum. A restorative approach, as DOHR members Jennifer Llewellyn and Kristina Llewellyn have articulated, is grounded in relational theory. Relational theory holds that human beings exist in and through relationship with one another (Llewellyn, J. 2011; Llewellyn and Llewellyn 2015). The DOHR project reflects the premise that relationality is at the core of learning about such difficult knowledge as systemic racism in the Home and its legacy. Learning requires attention to the fact that we exist in and through relations, and this fact has implications for justice. Recognizing the relational nature of the historical harms of the Home, requires that learners listen to the lived experiences of former residents.
The DOHR project therefore co-created, as part of its curriculum, a placed-based oral history experience in virtual reality, with three former residents—Gerry Morrison, Tony Smith, and Tracy Dorrington-Skinner—who are recognized leaders and activists in the community.[1] Scholars have demonstrated the many ways that oral history in education, both in conducting interviews and in listening to pre-recorded interviews, builds relational connections that are intergenerational and support reconciliation across divides (Llewellyn and Ng-A-Fook 2017; 2019). Unlike other oral history projects in schools, however, the DOHR project required that learners listen to stories in a contextual way that would connect them to a sense of place and the human experience of it—specifically, to the Home. Yet the DOHR team knew that former residents could not, nor should they be expected to, share their stories in-person with all students. The DOHR team also knew that not all students could visit the site of the Home and, even if it were possible, the site of the Home itself has changed significantly over the decades. While part of the Home’s building still stands, its present structure is considerably different from the structures in which the former residents lived (Morrison from 1954–60, Smith from 1965–68, and Dorrington-Skinner, who lived in the original Home building from 1972–78, and in the newer building now known as the Akoma Family Centre from 1978–84). Indeed, since early 2019, the site of the Old Home has been undergoing yet another phase of major renovations (see Figure 1). Since students cannot interact with the former residents or the site of the Home directly, the DOHR curriculum exemplifies the kind of experiential learning that is consistently identified in the VLE literature as likely to benefit from a virtual learning environment (VLE), and ideally one that is VR-based. DOHR wants to deliver experiential “learning tasks which are expensive” (Dalgarno and Lee 2010, 19) or even “impossible” (Bulu 2012, 153; Kwon 2019, 105) in real life. In order to provoke new, relational understandings of the Home and of systemic racism, which are further supported in the fuller DOHR curriculum, the former residents’ oral histories are shared in DOHR’s VR-based VLE.
Figure 1. At top left, the original Nova Scotia Home for Colored Children (the “Old Home”) on the occasion of its official opening in June 1921. Top right, a large brick extension was added in 1961. Bottom left, the “New Home” building built in 1978 (now the site of the Akoma Family Centre). At bottom right, the Old Home as it appeared during the DOHR research team’s site visit in April 2019. (For further details see Chapter 3 of the Restorative Inquiry Report [Province of Nova Scotia 2019].)
However, DOHR has taken an unusual approach to the design of its VR-based VLE, because the learning outcomes that the DOHR project fosters—grounded in the commitment to restorative justice which seeks to foster just relations (Llewellyn 2011; Llewellyn and Llewellyn 2015)—are unusual in the context of VR-based VLE design. Specifically, we have developed a different approach to presence, which is broadly understood to be the element of VR design that contributes most to student learning in VLEs, and is hence the principal design aim of most VLE projects. Presence is the sense of “being there” (Slater and Wilbur 1997) in a virtual experience or “the psychological state where virtual experiences feel authentic” (McCreery, Schrader, Krach, and Boone 2013, 1635). Outside of the context of VLE design for experiential education, virtual environments have been designed with quite different aims. The digital humanities, for example, has emphasized the use of virtual reconstruction in research contexts as “not a neutral representation of ‘the past,’ but the scholar’s interpretation of specific aspects of a place at a certain time—an interpretation that can be challenged, revised, or rejected” (Sullivan, Nieves, and Snyder 2017, 301; emphasis original). Since the goal of creating a virtual environment in the digital humanities has been to make an argument and provide the locus for future argumentation (Sullivan, Nieves, and Snyder 2017, 301; see Roberts-Smith et al. 2016; Roberts-Smith 2017), attention has been paid to VLE design that encourages critical creativity rather than presence (Roberts-Smith et al. 2013). There is also some very recent, parallel work in VR design for K–12 education exploring the ways students productively fill in the gaps in imperfect historical simulations without compromising their sense of historicity (Papanastasiou et al. 2019). By contrast, in what has come to be known as “immersive journalism” (de la Peña et al. 2010; Reis and Coelho 2018), the use of 360º video to place the viewer in the “center” of a documented event is designed not to help participants learn to do anything specific, but to encourage them to empathize with victims of injustices (see for example Torsei and Philippe 2019). To date, however, neither these alternative approaches to virtual environment design nor their critiques (e.g. Reis and Coelho 2018; Mabrook and Singer 2019) are well integrated into the discourse around VLE design for experiential education. Similarly, 3D graphical approaches to the representation of marginalized communities remain under-interrogated in the VLE design literature, despite some robust work in this area emerging from game studies (e.g. Reis and Coelho 2018; Malkowski, Russwork, and Trea 2017; Taylor and Voorhees 2018).
Our aim for the DOHR project and this paper is to broaden the conversation about VLE design—which has largely followed technology- and psychology-driven lines of thought originating in the early 1990s[2]—to accommodate issues that have arisen more recently in fields outside of VLE design, through a discussion of the DOHR VLE. Since perspectives on how to achieve presence in VLEs, and why such presence is effective, are quite dispersed even in the VLE literature, we begin our discussion here with a synthesis of the most influential concepts. We then provide a description of the DOHR VR experience, and a discussion of how it approaches presence differently, consistent with the relational principles of a restorative approach. In conclusion, we offer some preliminary reflections on the DOHR VR experience’s effectiveness as a learning tool and suggest next steps for future development.
Presence and Learner Engagement in VLE Design for Experiential Learning
VR-based VLEs are most commonly designed to help learners (whether in school contexts or in public education contexts) to develop either spatial or social competencies that are impractical and/or dangerous to teach, especially at introductory levels, in the actual world. Widely-publicized examples include VR-based small motor-skill training for surgeons, in which learners use physical surgical instruments as controllers of avatars of the same instruments, to perform virtual surgeries on digitally-simulated bodies; such systems are increasingly used not just to train new surgeons but also to refresh the skills of practicing surgeons before performing actual-world surgeries (as in Surgical Science’s VR training system for laproscopy and endoscopy). In socially oriented VLEs, students typically learn how to avoid or respond positively to harmful behaviors, such as racial stereotyping, by rehearsing actions in virtually simulated scenarios (as in Kaplana’s 2015 Injustice); or how to develop empathy for the suffering of others by experiencing a simulation of their hardships (as in Kors et al.’s 2016 A Breathtaking Journey). Since the expectation in both kinds of VLE is that students will be able to transfer competencies developed in virtual reality into actual-world situations, these VLEs strive to simulate real-world experiences as vividly and accurately as possible, often incorporating actual-world material objects as well as virtual simulations, such as the surgical instruments used by Surgical Science. Kwon (2019) argues that immersive VLEs are especially relevant to the first stage of Kolb’s (1984) model of the four circulative processes of experiential learning, “concrete experience,” which could be followed by “reflective observation, abstract conceptualization, and active experimentation” in the classroom.[3]
There is general agreement in the literature on VLE design for experiential learning that the closer a simulated experience is to an actual-world experience, the better it functions as a replacement for real-world experience. When a simulation is effective, it produces in the participant the feeling referred to as “telepresence,” which is usually abbreviated to “presence”: “the psychological state where virtual experiences feel authentic” (McCreery, Schrader, Krach, and Boone 2013, 1635). If a technologically mediated experience is effective in generating the sense of presence, the perception of the person experiencing it “fails to accurately acknowledge the role of the technology in the experience” (International Society of Presence Research). In other words: it is generally accepted in the field that if a VR experience is effective in generating presence, the participant forgets they have a VR headset on, and instead feels like they are “there” in the illusion the headset is creating. In education, this experience of forgetting you are in a VLE has been seen as an advantage for experiential learning. Since experiential learning is thought to have been achieved when virtual experience is recognized as similar to an actual experience, “enhanced presence” is an ambition of VR-based VLE design (Kwon 2019, 105).
The sense of presence in VR-based VLE design is often understood to arise from immersive hardware systems (Fowler 2015, 416). Immersion here is understood as the “degree to which a virtual environment submerges the perceptual system of the user in computer-generated stimuli” (Biocca and Delaney 1995). In this understanding, the perceptual system is submerged physically, by the technical hardware employed to create the illusion of the virtual environment. For example, we might think of a VR headset as more immersive than a desktop computer screen, because it excludes the perception of visual stimuli that are not part of the virtual illusion (Dalgarno and Lee 2010, 11). However, as the examples of Surgical Science and Injustice demonstrate, hardware alone is not the greatest determiner of perceptual submersion in immersive systems (Archer and Finger 2018); rather, perceptual submersion is achieved by the degree to which a virtual illusion explicitly mimics the actual world. Although there is no consensus in the literature on the most effective design practices for achieving presence in VR-based VLEs, three design factors are regularly identified as having a significant ability to increase perceptual submersion: representational fidelity (the degree to which a virtual illusion looks or sounds like reality), interactivity (the degree to which the virtual illusion responds realistically to the embodied actions of a spectator), and identity construction (the degree to which spectators can associate themselves with characters represented in the virtual environment).[4] These factors are normally differentiated from one another in the literature, but are also understood to be interdependent in ways that are not yet consistently articulated.
Representational fidelity, for example, is often understood to be achieved by one or more of the following four factors:
(a) The vividness, or “abundance of reenactment … providing information to the senses” (Kwon 2019, 102; see Steuer 1992). On a sliding scale, at the low end of what VR systems can deliver, only the sense of sight is engaged; in more sophisticated systems, hearing is engaged; then at the cutting edge of what is currently possible, touch is manipulated. Taste and smell remain beyond the capacity of existing virtual systems, available only in “actual reality” (see Figure 3).
(b) The realism of the virtual illusion (Bessa, Melo, Sousa, and Vasconcelos-Raposo 2018, 35; Bulu 2012, 156), including its three-dimensionality (Bulu 2012, 154; Dalgarno and Lee 2010, 11); the ways in which it represents users (Fowler 2015, 413); and “the consistency of object behaviour” (Fowler 2015, 413; see Dalgarno and Lee 2010).
(c) The plausibility and dynamics of the virtual environment, including such technical effects as reflecting mirrors or shadows (Sanchez-Vives and Slater 2005).
(d) The “quality of the display, with high-fidelity displays being most realistic or photorealistic” (Fowler 2015, 413; see Dalgarno and Lee 2010). In other words, the sophistication of the equipment used to deliver a VR experience.
Similarly, interactivity is often understood to be achieved by one or more of the following:
(a) The range of embodied actions available to the VR participant (Dalgarno and Lee 2010; Kwon 2019); in other words: the degree to which a participant can use their body in the ways they would in actual reality, by touching, speaking, or moving around, for example.
(b) The degree to which the virtual illusion responds to the participant’s actions (Murray 1997; Dalgarno and Lee 2010; Kwon 2019), when, for example, objects move or other avatars engage in conversation.
(c) The degree to which the participant can create new elements of the virtual illusion (Dalgarno and Lee 2010; Kwon 2019).
(d) The technical ability of the system to respond to action through, for example, head-tracking (Sanches-Vives and Slater 2005) or update rate (Barfield and Hendrix 1995, 3).
However, the first of the four principles thought to contribute to representational fidelity is also sometimes treated independently as a factor that interacts with interactivity to increase a virtual illusion’s ability to simulate reality. Kwon, for example, sees VR-based VLEs that leverage the sense of touch and enable a wide range of bodily gestures as more “authentic” in the sense that they provide a more vivid experience closer to actual reality (see Figure 2). VR that is “authentic” in this way is particularly good at generating a sense of “place illusion” or “place presence” (Bulu 2012), the sensation of being and operating at a remote or virtual place (Slater 2009), or “being there” in the place depicted by the virtual display (Slater and Wilbur 1997). Hence “place presence” is often a design goal of VLEs whose intended learning outcomes include spatial competencies that can be translated to actual-world scenarios (such as, for example, Surgical Science).
Figure 2. “Relationship between virtual reality and actual reality based on the degree of presence” (Kwon 2019, Figure 2).[5]
The third major design factor, identity construction, by comparison, is often understood to be an outcome of the first two, representational fidelity and interactivity (see Figure 3). Identity construction normally refers to the sense of personal “body ownership” (Slater 2009) that learners develop by associating themselves with a manipulable avatar in a virtual environment (Bulu 2012, 154; Fowler 2015, 414). It can also refer to a learner’s ability to construct identities for other player-participants through their respective avatars (Fowler 2015, 414; Bulu 2012, 155; Biocca et al. 2003; Schroeder 2002). Identity construction is often leveraged in educational contexts to help generate a sense of “co-presence”, or “being there together” (Fowler 2015, 414) in a virtual environment. Co-presence has two dimensions: “perceiving others and having a sense or feeling that others [are] actively perceiving us and being part of a group” (Goffman 1963; Slater, Sadagic, Usoh, and Schroeder 2000). Co-presence is normally understood to involve a sense that there is “psychological interaction” among individuals (Nowak 2001; Schroeder 2002; Bulu 2012, 155). As a result, identity construction is often a design goal of VLEs whose intended learning outcomes include social competencies that can be translated to actual-world scenarios (such as Injustice).
Figure 3. Dalgarno and Lee’s elaborated model of learning in a 3D VLE (Dalgarno and Lee 2010, Figure 1).
In general, however, whether designed for spatial or social learning tasks, presence is thought to have three major pedagogical benefits for learners. First, presence helps students focus on the learning tasks they are encountering in a VLE, developing a task-oriented “flow.” When students experience “flow”—the term used in the literature to describe “the state of being absorbed by an activity” (Scoresby and Shelton 2011, 227), which “mediates the relationship between presence and enjoyment” (Weibel, Wissmath, et al. 2008, 2274)—they learn better (Kwon 2019; see Figure 4).
Figure 4. Influence relations among vividness, tactile interactivity, locomotive interactivity, simulator sickness, presence, flow, and learning effect (Kwon 2019, Figure 8). Note that simulator sickness is a counter-indicator of flow here.
Second, presence helps create the sense of agency that learners need to have in order for learning to be experiential. Triberti and Riva, for example, describe presence as “a core neuropsychological phenomenon whose goal is to produce a sense of agency and control: I am present in a real or virtual space if I manage to put my intentions into action (enacting them)” (2016, 2). As Janet Murray puts it, agency is “the satisfying power to take meaningful action” (1997). The third benefit, which arises from the first and second, is that presence is also frequently associated with students’ satisfaction with their own learning (e.g. Bulu 2012). Student satisfaction is a measure frequently used to determine the effectiveness of a virtual learning activity (see Kwon 2019, for example).
If we were to extract the best practices for VR-based VLE design from the literature survey above, we might end up with something like: Make a high-fidelity simulation of the relevant actual-world environment; give learners a way of affecting the virtual learning environment and make the environment respond; and provide representations of learners themselves in the world. Thanks to the resulting sense of agency they will then feel, learners will forget they are in a virtual environment, getting into a flow where they are totally focused on their learning tasks. The outcome will be that they learn effectively and feel satisfied with their learning experience. According to the current state of the VLE literature, then, in an effective VR-based VLE a learner perceives themselves acting in a simulation and perceives the simulation responding; the resulting agency, presence, and flow lead to learning and satisfaction. While the literature describes best practices for VLE design to support the kinds of spatial and social learning outcomes commonly intended in experiential learning curricula, it does not support the DOHR curriculum’s intended learning outcomes.
Designing Presence in the DOHR VR Experience
The DOHR VR experience is a thirteen- to fifteen-minute individual learning activity that is embedded early in a five-lesson curriculum designed to run, typically, over the course of ten history classes. The VR experience was designed to support learning activities outside of the VLE that are constructed based on the principles of historical thinking and oral history education, and on a restorative approach to learning (Llewellyn and Llewellyn 2015; Llewellyn and Ng-A-Fook 2017; 2019; see also Gibson and Case 2019; Epstein and Peck 2019; and the Historical Thinking Project). The first two lessons invite students to join the former residents in their decades-long journey to bring their stories of the Home to light in order to build a better future. Students are introduced to a brief history of the Home and then asked to actively inquire about the historical significance, causes, and consequences of the Home. Their engagement in this inquiry is centered on an examination of oral history as a primary source in itself and in relation to other primary historical evidence (such as social worker reports, newspaper articles, and photographs). The lessons culminate with students developing a “restorative plan” that asks them, among other aims, to share what they have learned in a way that will do justice to the historical experiences of the former residents and contribute to the future-focused goal of reconciliation.
In the third lesson, learners are on-boarded in small groups to the DOHR VR experience in person, by a trained facilitator, at individual stations. The facilitator advises students how to end the experience if they find its content distressing. The facilitator leads a short “sharing circle” (a key activity in a restorative approach to learning) to debrief about the experience afterwards. The VR experience itself begins with a short, documentary-like 360º video segment in which learners see the storytellers, Smith, Morrison, and Dorrington-Skinner, and hear their voices in voiceover. The introduction of the storytellers is followed by a set of oral histories rendered in a multi-modal blend of 3D graphics, 360º and 2D video, 2D images, environmental and spatially-located sound, voiceover narrative, and text. There are 12 stories in total, but each learner can only choose to witness three, one from each storyteller. Following those three stories, all learners witness a short sequence in which the three storytellers reflect together about their memories of one common room in the Home. Finally, learners witness another 360º, documentary-like video for a concluding sequence in which the storytellers describe directly (that is, without the use of voiceover) how they came to be the activists they are today.
From its inception, then, the challenge for the DOHR team in developing the VR experience has been that it is intended for a different kind of use-case than other VR-based VLE projects. DOHR is a project that seeks for students to build a relational understanding of the historical harm of the Home by hearing the oral histories of former residents. The intent of the curriculum is for students to ask: What relationship do they have to, and thus what responsibility do they have to address, the history of harm, based on systemic racism, that is the legacy of the Nova Scotia Home for Colored Children? The aim of the curriculum, including the VR experience, is for students to build a sense of relationship to the lived experience of a place, even though they will never likely be in the actual Home nor meet the former residents in person. This means, in part, that a traditional sense of place presence in the VR experience is not important, since spatial awareness and spatial skills are not primary learning outcomes. Similarly, traditional social presence is not useful, since our aim is not to help students practice social behaviors in a virtual environment so that they can adopt them more confidently or consistently in the real world. What we need students to do is to consider their stance in relation to the stories of the former residents in order to inform their understanding and efforts toward just relations with those whose lived experiences are different from their own. In other words, instead of generating a sense of presence as it is traditionally understood, where a virtual experience feels like an experience that could be replicated in the real world, we need students to remember that they are witnessing a story being told through the perspective of another person, which they could never experience themselves in the real world. The term we are using to describe this form of presence is relational presence.
Relational presence has had several concrete implications for the design of the DOHR VR experience. Since we need to help foster an understanding of what it was like to live in the Home, for different people at different times, we are not invested in representational fidelity in the way that other projects are. Although we have worked extensively with architectural drawings, photographs, and other archival and archaeological evidence of the past structure of its site and buildings, in our renderings, we express the Home in a multi-modal, impressionist aesthetic that reinforces the former residents’ oral histories. Most stories, for example, begin in a line-drawn, white-on-blue rendering of the site of the Home that is intended to evoke a three-dimensional version of the architectural drawings used to structure 3D space at each point in the narrative. The invocation of the documentary record only becomes substantial—opaque 3D graphics, 360º video, light, and environmental sounds helping to establish a more specific impression of place, time, mood, and activities—as the voice of the storyteller (the sound that appears closest to a participant’s ear) begins to recount the story. The representational media are in turn combined in ways that neither attempt to mask the differences among them, nor their individual differences from the actual-world phenomena they evoke. Our aim is to privilege lived experience over the fragmentary documentary record, making it clear that the world learners are encountering is not an attempt to reconstruct the past through simulation. Instead, it is an attempt to construct a present encounter with oral histories about past experiences in the Home and the long-term impacts of those experiences. In contrast to the traditionally sought sense of “place presence”, then, the DOHR VR experience seeks to foster a sense of what we are calling “relational place.” Relational place is an invocation of what a place means—in the case of the DOHR VR experience, what it means for storytellers and learners—rather than a simulation of how a place looked or was configured at any given point time (see Figure 5).
Figure 5. Screen captures showing three different approaches to multi-modal impressionist rendering in excerpts from Morrison’s story, “Swamp Water” (top left); Dorrington-Skinner’s story, “Mrs. Johnson’s Helper” (top right); and Smith’s story, “The Switch” (bottom). See Roberts-Smith et al. 2019.
Since learners need to maintain a sense of the difference between their own perspectives and the perspectives they are learning about, we are, similarly, not invested in identity construction in the sense of the identification of self or others with avatars to enhance social presence as it is understood in the VLE literature. Rather, we seek to support in students the development of a sense of relation to the stories rendered in our VR experiences and the storytellers from their own position and perspective. The VR experience does not create the illusion that the storytellers are really “there” with learners in the virtual environment. This means there are no anthropomorphic avatars in our VLE, and we make no attempt to create roles or characters for the storytellers or for learners to “play.” Learners are characterized by means of an avatar that is a literal representation of the story-selection controller held in the learner’s hand—the only avatar in the entire build—only as the force that uses the controller to select a story. Instead of creating virtual representations of either storyteller or student, we make space for each to occupy their own, actual-world perspectives. For storytellers, that means that their oral histories are told in recordings of the adult storytellers themselves, and for learners, it means witnessing those stories as grade eleven Canadian History students themselves, and not in the kind of role-play scenario that is common in social competency VLEs.
The emphasis on witnessing oral histories of the Home, then, means that there is also very little traditional interactivity in the DOHR VR experience. Since the world of our VLE represents the lived experiences of the storytellers, students do not need agency in the sense of being able to take action that initiates a response from objects or characters in the virtual world. If they were able to do that, the world would no longer represent the storytellers’ perspectives, and would not help learners understand the difference between their own perspectives and those of the storytellers. It would also give students the illusion of having power to change the stories, which, since justice in the DOHR project depends on hearing stories that have not been heard before, would subvert the project’s aim of encouraging an active listening that may provoke new understandings of the past. It would undermine the students’ ability to consider how lessons from the past can contribute to future just relations, as required of them in the restorative plan they develop as part of the curriculum. In the DOHR VLE, interactivity is hence extremely limited in traditional VLE design terms. To the extent that it is available at all, it is designed to characterize learners as witnesses to the stories. Students are able to choose one story from each of the three former residents whose stories are represented, and then listen to it. However, learners are not inactive, because listening to the stories is itself an important cognitive activity that has been recognized, for example, in Indigenous studies as an active “inhabiting” of representational worlds (Ridington 1998), and in performance studies as an active self-reflection on one’s “role and experience as a spectator” (Rokem 2002). Bronwen Low, a member of the DOHR research team, has written about “the pedagogy of listening” in similar terms (Low 2015). Drawing upon the work of Jean-Luc Nancy, Low describes listening as a learning process that extends the ear towards the other. This is not a silent nor passive process, but rather, one that builds relations of deep listening between storyteller and listener (Low 2015, 270–75; Llewellyn and Cook 2017). We differentiate this form of “witnessing” from “co-presence” as it is understood in the field of VLE design: namely, as the perception of others in a virtual environment, and the perception that others perceive us, giving us the sense that we are “part of a group” (Bulu 2012, 155; see Goffman 1963; Slater, Sadagic, Usoh, and Schroeder 2000). In VLE design, “co-presence” normally refers to the presence of other users in the virtual environment, and it is achieved through synchronous user-manipulated avatar interactions. By contrast, we are interested in giving students a sense of relation not to other VR participants, but to the lived experience of the non-player characters of the storytellers, Gerry, Tony, and Tracy, who are not simultaneously present, but represented through pre-recorded media and pre-fabricated digital assets. Relational interactivity, then—the invitation to witness—places agency in the context of relationship. The learner’s power is to relate across differences in perspective.
In these different approaches to representational fidelity, identity construction, and interactivity, DOHR takes an approach that is also different from precedents in fields outside VLE design. For example, despite a shared lack of interest in in-world interactivity, our conception of “witnessing” also differs significantly from the concept of non-interactive “immersive witness” (Nash 2018) that has been taken up in 360º video-based journalism inspired by the work of Nonny de la Peña (2010). In this context, 360º video is understood as a means of simulating a distant event, and consequently as offering the experience, rather than a representation, of that event; this experience is “immersive witnessing” (Nash 2018). While immersive witnessing has been critiqued for its relative lack of interest in the distance or disinterestedness normally expected in journalistic reporting (Nash 2018; Reis and Coelho 2018), it has been taken up by activist and humanitarian organizations because it was thought to instill a sense of responsibility for others. However, immersive witness makes different assumptions than the DOHR project does about the nature and aims of witnessing. Immersive witness is interested in “providing the audience with something of an experience that is linked in various ways to the experiences of others” (Nash 2018) through passive reception of a photo-realistic simulation. DOHR, by contrast, avoids simulation to encourage the active exploration of differences of perspective arising from differences in lived experience.
Finally, another important difference between the DOHR VLE and other VLEs is that witnessing the different perspectives of former residents of the Home can be an uncomfortable experience. The stories are about the harms that former residents suffered there, and the resilience that they and other children drew upon to survive those harms. So, another difference between relational presence and traditional presence is that, although relational presence can be absorbing and lead to the kind of “flow” where students are fully engaged in their learning task, it does not necessarily offer the pleasant kind of self-satisfaction related to taking action within the VLE intended by Janet Horowitz Murray (1997) and others. Instead, relational presence can lead to affective dissonance, which is the discomfort we feel when we experience difficult knowledge (Zembylas 2015; Zembylas and Bekerman 2008; Simon 2015). That discomfort prompts thought-provoking questions for learners, providing a different opportunity and experience of agency, which learners explore in the fuller classroom curriculum. These questions include: How could this have happened? Why didn’t I know about this before? What is my responsibility now that I know these stories? That kind of questioning is a learning agency—the agency to inquire and to reconsider how we act in and through relationship with others in the world.
Designing the DOHR VR experience has suggested to us that presence need not necessarily be understood as a simulation-based forgetting that we are witnessing an illusion, nor as an erasure of our awareness of the technology that delivers it. The project does not use a model of presence that requires the reconstruction of spaces for us to “be” in, identifying with representations of ourselves or others, and feeling satisfying agency by interacting with place and social context. Instead, we think of presence as the unsettling agency to witness a different perspective on meaning, which offers an opportunity to consider, and possibly change, our actual-world understanding and behavior as a result. If presence is thought of relationally, an alternative model of effective VLE design emerges. Instead of acting in a simulation, a learner occupies a relational place to witness a story. The resulting sense of relational presence fosters forms of agency and affect that are critical for learners to inquire further and seek restorative actions for justice in their actual-world contexts. For DOHR to achieve relational presence, our VLE needed to offer the opportunity to witness a past world described by those who lived it and provoke questions, based on the opportunity to witness, that would otherwise be impossible to formulate. In the case of the DOHR project, relational presence was achieved by means of a mixed-mode, impressionistic representation of the lived experience of a real-world environment, which avoided avatars, and limited interactivity to opportunities to witness (see Table 1).
Traditional Presence
Relational Presence
simulation (representational fidelity)
impression (representation of meaning)
interactivity
witnessing
identity construction (recognizing self)
self-awareness, awareness of difference
agency
interpretation, inquiry
satisfaction
affective dissonance
Table 1. A comparison of traditional and relational approaches to presence in VR-based VLE design.
Conclusion
In the DOHR VR experience, we have developed a theory of relational presence, and one approach to achieving it, which have yet to be validated through empirical study of student learning, or in other VR-based VLE design projects. At the time of this publication, the DOHR team has conducted a study of the DOHR curriculum, including the VR experience, and is analyzing the data. Preliminary results from the data indicate positive learning outcomes. Students reported sensations that indicated they did experience a strong sense of flow, and acquired important new knowledge, despite our unconventional approach to designing the VLE. Future study of the delivery of the curriculum will help the team to understand how diverse social and geographical factors affect learning outcomes, and the need to address the accessibility limitations of our current design, beyond physical and auditory enhancements. An analysis of the data from classroom implementation will provide us with the evidence required to determine how the VR experience with a focus on relational presence, embedded within the curriculum as a whole, may lead to learners’ increased relational competency; that is, to an increase in students’ ability to engage in the work of building more just relations in their worlds.
Two avenues for further research have already suggested themselves to the DOHR team. First, there may be productive research to be done on the role of aesthetics as a support for learning in VLEs. The concept of representational fidelity has so far been limited to a very narrow subsection of what might be more fully understood as the representational aesthetics of a virtual learning experience (by which we mean the intentional manipulations of the media of expression to both represent and generate experiential phenomena). This may be a result of the strong influence, to date, of STEM disciplines on the development of existing VLEs; STEM-based work typically thinks of aesthetics as a means of enhancing user experience and usability (e.g. Tuch et al. 2012). In the DOHR VR experience, we found that a significant investment in representational aesthetics was essential to the pedagogical goals of the project—both in terms of the theatre-based design experts gathered to work with former residents of the Home and other members of the DOHR steering committee on the VR development team, and also in terms of financial and material resources. What the DOHR build lacks in traditional presence-inducing features, it perhaps makes up for in aesthetic features. This may also explain why the DOHR team found some preliminary precedents for some components of relational presence in Indigenous studies and performance studies, two fields that are deeply invested in what we might think of as a twenty-first century development of what Nicholas Bourriaud first termed “relational aesthetics.” For Bourriaud, “the possibility of a relational art (an art taking as its theoretical horizon the realm of human interactions and its social context, rather than the assertion of an independent and private symbolic space), points to a radical upheaval of the aesthetic, cultural and political goals introduced by [its predecessor] modern art” (2002, 14; emphases original). The DOHR VR experience’s stylistic gesture of intermedial, impressionist representation to achieve relational presence may be one way VLEs can begin to explore social systems (or “formations,” as Bourriaud calls them) in much richer ways than the field of VLE design has yet done. Similarly, DOHR’s engagement of a relational approach may encourage others to explore the ways in which Bourriaud’s articulation of “relational aesthetics” might be altered or expanded to better serve the aims of projects fostering restorative justice.
In addition, there is certainly more work to be done on the contextualization of VR-based VLEs within classroom-based curricula and with reference to in-class teaching strategies. Work in this area currently consists of a decade or more of advocacy in pedagogical game studies (see, most recently, Hébert and Jenson 2019 for both context and evidence of best practices). Although VLE designers have devoted a great deal of energy to the technical design of stand-alone VLEs, the field has not yet taken full advantage of the opportunity to apply best practices in pedagogical design to VLEs (Fowler 2015). This could be done in stand-alone VLEs but could also be approached by situating a VLE as one in a series of classroom learning activities, as the DOHR project has done. A significant advantage of considering VLEs in the context of an overall blended (in-person and virtual) curriculum design is that it avoids “technological determinism” (Reis and Coelho 2018, 1093) whereby virtual experiences are “considered both a product and an outcome of technology” (Reis and Coelho 2018, 1093), rather than an outcome of the ways designers have manipulated the technologies in question. Understanding a VLE as one learning activity in the context of a larger curriculum necessarily makes its technology secondary and emphasizes the agency of educators to design and use the VLE in the ways that best serve their students. Relational presence is one offering that DOHR can make to the larger project of reconsidering the role of VLEs in K–12 and public education, with a view to addressing issues of pedagogy, representation, and justice that are not yet well accounted-for in the field.
Notes
[1] The DOHR VR experience was designed using a process that aligns generally with the principles of “co-design” articulated in the seminal work of Steen (2013), whereby parties characterized as “stakeholders” are actively involved throughout the design process and afterwards (Steen 2013); as distinct from “participatory design,” in which stakeholders are consulted only at key points (Schuler and Namoika 1993; Björgvinsson, Pelle, and Hillgren 2010). However, DOHR’s process differs from this and other activist, participatory artistic practices leveraging digital media (e.g. Gubraim, Harper, and Otañez 2015) in its centering of a relational approach to all project activities. The full citation for the DOHR VR experience, acknowledging specific roles of individual co-design participants, can be found in our reference list under Roberts-Smith et al. 2019.
[2] See Steuer 1992 (also cited below) for an example of an influential early technology-focused work; Chittaro 2013 for an example of work using psychological concepts to better understand human-computer action; and Riva 2018 for a compelling example of the integration of philosophy, human-computer interaction, and psychology in current work. Lombard and Ditton 1997 offer a survey of early 1990s trends; Fowler 2015 and Reis and Coelho 2018 critique the outcomes of the emphasis on technology in particular.
[3] We note, however, that the classical conditions under which virtual experience is advantageous (i.e. where embodied experience is “expensive, dangerous, or impossible” [Dalgarno and Lee 2010]) may be as likely to occur at the active experimentation stage as at the concrete experience stage.
[4] An important additional factor, beyond the scope of our discussion here, is immersive tendency, which operates outside of the context of the VLE itself. Immersive tendency refers both to the pre-disposition of some participants “to involve and focus on the [sic] common activities in real life” (Bulu 2012, 159), and also to participants’ desire to immerse because they “have specific expectations about what the outcome should be” (Shin 2017, 71; citing Weibel et al. 2010; Burns & Fairclough 2015; Hou, Nam, Peng, and Lee 2012).
[5] Kwon’s scale addresses only the five most familiar senses. There is also a great deal of work being done on proprioception in research related to motion sickness in VR, which Kwon acknowledges as a counter-indication of presence. For a substantial review of the relevant literature, see Weech, Kenny, and Barnett-Cowan 2019.
Bibliography
Archer, Dan and Katharina Finger. 2018. “Walking in Another’s Virtual Shoes: Do 360-degree Video News Stories Generate Empathy in Viewers?” TOW Reports. Columbia Journalism Review. Tow Center for Journalism, Columbia University Graduate School of Journalism. March 18.
Barfield, Woodrow, and Claudia Hendrix. 1995. “The Effect of Update Rate on the Sense of Presence within Virtual Environments.” Virtual Reality 1, no. 1: 3–15. https://doi.org/10.1007/BF02009709.
Bessa, Maximino, Miguel Melo, A. Augusto de Sousa, and José Vasconcelos-Raposo. 2018. “The Effects of Body Position on Reflexive Motor Acts and the Sense of Presence in Virtual Environments.” Computers & Graphics 71 (April): 35–41. https://doi.org/10.1016/j.cag.2017.11.003.
Biocca, Frank, and Ben Delaney. 1995. “Immersive Virtual Reality Technology.” In LEA’s Communication Series: Communication in the Age of Virtual Reality, 57–124. Hillsdale, New Jersey: Lawrence Erlbaum Associates, Inc.
Björgvinsson, Erling, Pelle Ehn, and Per-Anders Hillgren. 2010. “Participatory Design and ‘Democratizing Innovation.’” PDC ’10: Proceedings of the 11th Biennial Participatory Design Conference (November): 41–50. https://doi.org/10.1145/1900441.1900448.
Bulu, Saniye Tugba. 2012. “Place Presence, Social Presence, Co-Presence, and Satisfaction in Virtual Worlds.” Computers & Education 58, no. 1: 154–61. https://doi.org/10.1016/j.compedu.2011.08.024.
Dalgarno, Barney, and Mark J W Lee. 2010. “What Are the Learning Affordances of 3‐D Virtual Environments?” British Journal of Educational Technology 41, no. 1: 10–32. https://doi.org/10.1111/j.1467-8535.2009.01038.x.
Epstein, Terrie, and Carla L. Peck. 2018. Teaching and Learning Difficult Histories in International Contexts a Critical Sociocultural Approach. New York: Routledge.
Fowler, Chris. 2015. “Virtual Reality and Learning: Where Is the Pedagogy?” British Journal of Educational Technology 46, no. 2: 412–22. https://doi.org/10.1111/bjet.12135.
Gibson, Lindsay, and Roland Case. 2019. “Reshaping Canadian History Education in Support of Reconciliation.” Canadian Journal of Education 42, no. 1: 251.
Goffman, Erving. 1963. Behavior in Public Places. New York: The Free Press.
Gubrium, Aline, Krista Harper, and Marty Otañez, eds. 2015. Participatory Visual and Digital Research in Action. Walnut Creek, California: Left Coast Press, Inc.
Kwon, C. 2019. “Verification of the possibility and effectiveness of experiential learning using HMD-based immersive VR technologies.” Virtual Reality 23: 101–118.
Llewellyn, Jennifer J. 2011. “Restorative Justice: Thinking Relationally about Justice.” In Being Relational: Reflections on Relational Theory & Health Law, edited by Jocelyn Downie and Jennifer J. Llewellyn, 89–108. Vancouver: UBC Press.
Llewellyn, Kristina R. and Sharon Cook. 2017. “Oral History as Peacebuilding Pedagogy.” In Oral History and Education: Theories, Dilemmas, and Practices, edited by Kristina Llewellyn and Nicholas Ng-A-Fook, 17–41. New York: Palgrave Macmillan.
Llewellyn, Kristina R. and Jennifer J. Llewellyn. 2015. “A Restorative Approach to Learning: Relational Theory as Feminist Pedagogy in Universities.” In Feminist Pedagogy in Higher Education, edited by Tracy Penny Light, Jane Nicholas, and Renee Bondy, 11–31. Waterloo, Ontario: Wilfrid Laurier UP.
Llewellyn, Kristina R. and Nicholas Ng-A-Fook, eds. 2017. Oral History and Education: Theories, Dilemmas, and Practices. New York: Palgrave Macmillan.
Llewellyn, Kristina R. and Nicholas Ng-A-Fook, eds. 2019. Oral History, Education, and Justice: Limitations and Possibilities for Redress and Reconciliation. New York: Routledge.
Low, Bronwen E., and Emmanuelle Sonntag. 2015. “Towards a Pedagogy of Listening: Teaching and Learning from Life Stories of Human Rights Violations.” In The Canadian Oral History Reader, edited by Kristina Llewellyn, Alexander Freund, and Nolan Reilly, 266–281. Montreal and Kingston: McGill-Queen’s University Press.
Mabrook, Radwa, and Jane B. Singer. 2019. “Virtual Reality, 360° Video, and Journalism Studies: Conceptual Approaches to Immersive Technologies.” Journalism Studies 20, no. 14: 2096–112. https://doi.org/10.1080/1461670x.2019.1568203.
Malkowski, Jennifer, and Treaandrea M. Russworm. 2017. Gaming Representation: Race, Gender, and Sexuality in Video Games. Bloomington: Indiana University Press.
McCreery, Michael P., P. G. Schrader, S. Kathleen Krach, and Randy Boone. 2013. “A Sense of Self: The Role of Presence in Virtual Environments.” Computers in Human Behavior 29, no. 4: 1635–40. https://doi.org/10.1016/j.chb.2013.02.002.
Murray, Janet Horowitz. 1997. Hamlet on the Holodeck: The Future of Narrative in Cyberspace. Cambridge, Massachusetts: MIT Press.
Nash, Kate. 2018. “Virtual Reality Witness: Exploring the Ethics of Mediated Presence.” Studies in Documentary Film 12, no. 2: 119–31. https://doi.org/10.1080/17503280.2017.1340796.
Nowak, Kristine. 2001. “Defining and Differentiating Copresence, Social Presence, and Presence as Transportation.” A Comparison of Four New Communication Technologies, Proceedings of HCI International Conference on Human-Computer Interaction, Lawrence Erlbaum Associates.
Papanastasiou, George, Athanasios Drigas, Charalabos Skianis, Miltiadis Lytras, and Effrosyni Papanastasiou. 2018. “Virtual and Augmented Reality Effects on K-12, Higher and Tertiary Education Students’ Twenty-First Century Skills.” Virtual Reality 23, no. 4: 425–36. https://doi.org/10.1007/s10055-018-0363-2.
Reis, António Baía, and António Fernando Vasconcelos Cunha Castro Coelho. 2018. “Virtual Reality and Journalism: A Gateway to Conceptualizing Immersive Journalism.” Digital Journalism 6, no. 8: 1090–100. https://doi.org/10.1080/21670811.2018.1502046.
Roberts-Smith, Jennifer, Paul Cegys, William Chesney, Arda Kizilkay, Colin Labadie, Robert Plowman, Tracy Dorrington-Skinner, Gerald Morrison, Tony Smith, Kristina Llewellyn, Jennifer Llewellyn, and the DOHR Team. The Home: The Digital Oral Histories for Reconciliation Virtual Reality Experience. Pilot Release. Auburn Drive High School, Dartmouth, and Hants North Rural High School, Kennetcook, Nova Scotia, Canada, Oct 28–Nov 8, 2019. Beta Release. Nova Scotia Home for Colored Children Restorative Inquiry and Nova Scotia Black Cultural Centre, Dartmouth, Nova Scotia, Canada, Apr 11–12, 2019. Alpha Release. Halifax, Nova Scotia, Canada, July 2018.
Roberts-Smith, Jennifer. 2017. “Théâtre et humanités numériques. Du développement des outils au design expérimental. [Theatre and the Digital Humanities: From Tool Development to Experience Design].” Trans. Ioana Galleron. Revue d’Historiographie du Théâtre 4. Special Issue: Études théâtrales et humanités numériques.
Roberts-Smith, Jennifer, Shawn DeSouza-Coelho, Paul Stoesser, and the SET Research Team. 2016. “Cambridge Revisited? Methodological and Phenomenological Theatricalities in Simulated Environments for Theatre History Research.” In Early Modern Studies and the Digital Turn: New Technologies in Medieval and Renaissance Studies, edited by Laura Estill, Diane Jakacki, and Michael Ullyot, 318–44. Toronto: Iter Press.
Roberts-Smith, Jennifer, and the SET Research Team. 2013. “Visualizing Theatrical Text: From Watching the Script to the Simulated Environment for Theatre (SET).” Digital Humanities Quarterly 7, no. 3.
Sanchez-Vives, Maria V., and Mel Slater. 2005. “From Presence to Consciousness through Virtual Reality.” Nature Reviews Neuroscience 6, no. 4: 332–39. https://doi.org/10.1038/nrn1651.
Schroeder, Ralph. 2002. The Social Life of Avatars: Presence and Interaction in Shared Virtual Environments. Berlin: Springer Verlag.
Schuler, Douglas, and Aki Namioka. 1993. Participatory Design: Principles and Practices. Hillsdale, NJ: L. Erlbaum Associates Inc.
Scoresby, Jon, and Brett E. Shelton. 2011. “Visual Perspectives within Educational Computer Games: Effects on Presence and Flow within Virtual Immersive Learning Environments.” Instructional Science 39, no. 3 (May): 227–54. https://doi.org/10.1007/s11251-010-9126-5.
Simon, Roger I. 2015. A Pedagogy of Witnessing: Curatorial Practice and the Pursuit of Social Justice. Albany: SUNY Press.
Slater, Mel. 2009. “Place Illusion and Plausibility Can Lead to Realistic Behaviour in Immersive Virtual Environments.” Philosophical Transactions of the Royal Society B: Biological Sciences 364 (1535): 3549–57. https://doi.org/10.1098/rstb.2009.0138.
Slater, Mel, and Sylvia Wilbur. 1997. “A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments.” Presence: Teleoperators and Virtual Environments 6, no. 6: 603–16. https://doi.org/10.1162/pres.1997.6.6.603.
Slater, Mel, Amela Sadagic, Martin Usoh, and Ralph Schroeder. 2000. “Small-Group Behavior in a Virtual and Real Environment: A Comparative Study.” Presence: Teleoperators and Virtual Environments 9, no. 1: 37–51. https://doi.org/10.1162/105474600566600.
Steen, Marc. 2013. “Co-Design as a Process of Joint Inquiry and Imagination.” Design Issues 29, no. 22: 16–28. https://doi.org/10.1162/DESI_a_00207.
Sullivan, Elaine, Angel David Nieves, and Lisa M. Snyder. 2017. “Making the Model: Scholarship and Rhetoric in 3-D Historical Reconstructions.” In Making Things and Drawing Boundaries: Experiments in the Digital Humanities, edited by Jentery Sayers. Minneapolis: University of Minnesota Press.
Taylor, Nicholas, and Gerald Voorhees, eds. 2018. Masculinities in Play. Cham, Switzerland: Palgrave Macmillan.
Triberti, Stefano, and Giuseppe Riva. 2016. “Being Present in Action: A Theoretical Model About the ‘Interlocking’ Between Intentions and Environmental Affordances.” Frontiers in Psychology 6. https://doi.org/10.3389/fpsyg.2015.02052.
Tuch, Alexandre N., Sandra P. Roth, Kasper Hornbæk, Klaus Opwis, and Javier A. Bargas-Avila. 2012. “Is Beautiful Really Usable? Toward Understanding the Relation between Usability, Aesthetics, and Affect in HCI.” Computers in Human Behavior 28, no. 5: 1596–607. https://doi.org/10.1016/j.chb.2012.03.024.
Weech, Séamas, Sophie Kenny, and Michael Barnett-Cowan. 2019. “Presence and Cybersickness in Virtual Reality Are Negatively Related: A Review.” Frontiers in Psychology 10 (April). https://doi.org/10.3389/fpsyg.2019.00158.
Weibel, David, Bartholomäus Wissmath, Stephan Habegger, Yves Steiner, and Rudolf Groner. 2008. “Playing Online Games against Computer- vs. Human-Controlled Opponents: Effects on Presence, Flow, and Enjoyment.” Computers in Human Behavior 24, no. 5: 2274–91. https://doi.org/10.1016/j.chb.2007.11.002.
Zembylas, Michalinos. 2015. Emotion and Traumatic Conflict: Reclaiming Healing in Education. New York: Oxford University Press.
Zembylas, Michalinos and Zvi Bekerman. 2008. “Education and the Dangerous Memories of Historical Trauma: Narratives of Pain, Narratives of Hope,” Curriculum Inquiry 38, no. 2: 125–54.
About the Authors
Jennifer Roberts-Smith (Associate Professor, Theatre Performance, University of Waterloo) is an award-winning artist-researcher, whose transdisciplinary work in performance, digital media, design, education, and social justice has appeared in theatres, exhibitions, and scholarly publications internationally. She is currently director of the qCollaborative (the intersectional feminist design research lab housed in the University of Waterloo’s Games Institute), and of the Social Sciences and Humanities Research Council-funded Theatre for Relationality and Design for Peace projects. Since 2017, JRS has served as creative director and virtual reality cluster lead for the Digital Oral Histories for Reconciliation project.
Justin Carpenter is a PhD Candidate in English Language and Literature at the University of Waterloo. His current research traces the use of the term “generative” from literary to computational contexts, arguing that an understanding of this term opens up a variety of arguments around concepts such as authorship, agency, and emergence. He argues that such a genealogy can help situate game studies scholarship in dialogue with modernist and postmodernist literary studies, as well as cinema and other media. His other research interests include poetry, philosophy of technology, and aesthetics.
Kristina R. Llewellyn is Associate Professor of Social Development Studies at Renison University College, University of Waterloo. She is an expert in oral history, history education, history of education, and women’s history. Llewellyn has numerous award-winning publications, including The Canadian Oral History Reader (MQUP, 2015), Oral History and Education: Theories, Dilemmas, and Practices (Palgrave, 2017), and Oral History, Education, and Justice: Possibilities and Limitations for Redress and Reconciliation (Routledge, 2019). Llewellyn is a co-investigator on the SSHRC-funded project Thinking Historically for Canada’s Future, which is working to revise history education across Canada. She is the Principal Investigator and Director for Digital Oral Histories for Reconciliation: The Nova Scotia Home for Colored Children History Education Initiative project.
Jennifer J. Llewellyn is a Professor of Law and the Yogis and Keddy Chair in Human Rights Law at the Schulich School of Law, Dalhousie University. She is an expert in relational theory and a restorative approach. She served as a Commissioner on the Restorative Inquiry for the Nova Scotia Home for Colored Children. She directs the Restorative Approach International Learning Community and the Restorative Research, Innovation and Education Lab at Dalhousie University. She is a member of the Steering Committee for the Digital Oral Histories for Reconciliation project.
Tracy Dorrington-Skinner is a member and former co-chair of Victims of Institutional Childhood Exploitation Society (VOICES). She was a resident of the Nova Scotia Home for Colored Children. A member of the DOHR Team she was one of the three storytellers. Tracy was a member of the UJIMA Design Team for the Restorative Inquiry and a member of the Advisor Group for the Restorative Inquiry.
Gerald “Gerry” Morrison is a co-chair of Victims of Institutional Childhood Exploitation Society (VOICES). He was a resident of the Nova Scotia Home for Colored Children. A member of the DOHR Team, he was one of the three storytellers. Gerry was also a member of the UJIMA Design Team for the Restorative Inquiry and a Commissioner on the Restorative Inquiry.
Tony Smith is a co-chair of Victims of Institutional Childhood Exploitation Society (VOICES). He was a resident of the Nova Scotia Home for Colored Children. A member of the DOHR Team, he was one of the three storytellers. Tony was also a member of the UJIMA Design Team for the Restorative Inquiry and a Commissioner on the Restorative Inquiry. He served as the co-chair of the Council of Parties (Commissioners) for the Restorative Inquiry.
Email us at commonshelpsite@gmail.com so we can respond to your questions and requests. Please email from your CUNY email address if possible. Or visit our help site for more information:
