11 / May / 2021 /

A slide demonstrates TimelineJS used in a history classroom, with the header 'The Accidental Iconic Trend'.

Collaborative Digital Projects in the Undergraduate Humanities Classroom: Case Studies with TimelineJS

Spencer D. C. Keralis, University of Illinois at Urbana-Champaign

Courtney E. Jacobs, University of California, Los Angeles

Matthew Weirick Johnson, University of California, Los Angeles

Abstract

This article presents case studies for the use of TimelineJS in two types of courses: sophomore-level humanities survey courses at the University of North Texas (UNT), and senior capstone history seminars at the University of California Los Angeles (UCLA), both large land-grant research institutions. The case studies offer a framework for assignment scaffolding (including iteration and reflection), FERPA rights management, and describe models of faculty-librarian collaboration in assignment design and implementation. These assignments provide students an introduction to basic metadata and HTML markup skills and empower them to explore the historical contexts of primary sources by visualizing the chronology of historical periods.

As open-source digital tools for content creation and curation flourish, those engaged in higher education have a unique opportunity to apply these tools in support of undergraduate pedagogy. Harnessed appropriately, these tools can facilitate collaborations between teaching faculty and librarians focused on developing alternative durable research products that scale into various classroom settings, including remote teaching contexts. These projects offer the opportunity to incorporate primary source literacy into the curriculum and focus on active learning and critical inquiry while creating a digital project that teaches skills in data literacy, visual literacy, and citation, and incorporates the use of images as primary sources. Additionally, students develop skills in curation as well as writing interpretive or didactic text for external audiences.This article presents case studies for the use of TimelineJS in two types of courses: sophomore-level humanities surveys at the University of North Texas (UNT), and senior capstone history seminars at the University of California, Los Angeles (UCLA), both large land-grant research institutions.

The design of these assignments is aligned with the ethos of critical digital pedagogy. Informed by the work of educational theorists including bell hooks and Paulo Freire, critical pedagogy is, in Jesse Stommel’s terms “an approach to teaching and learning predicated on fostering agency and empowering learners.” Critical digital pedagogy brings this concept into digital space. Stommel and others have constructively wondered whether and how effectively this translation can occur. Given the dynamics of digital platforms and social media, one could ask “what is digital agency?” (Stommel 2014). In the context of the assignments described here, digital agency allows students to pivot quickly from being content consumers to knowledge producers, with digital tools providing the means of production, rather than being direct objects of learning in and of themselves.

In most undergraduate humanities classes, the primary learning goals for courses are not technical. As such, the decision to integrate digital projects must necessarily be balanced with the humanistic learning outcomes for a given course. Requirements for historical coverage, and exposure to diversity of genres and literary forms, as well as representation in terms of race and gender—all outcomes anticipated in survey courses—must come before adopting technical skills. Since survey courses frequently fulfill core curriculum or general education requirements, the make-up of these classes typically includes non-literature majors from across the curriculum. Capstone courses require students in a particular major to demonstrate mastery of the domain knowledge necessary for the Baccalaureate degree. In both cases, tech skills are not expected of students, nor should they be.

Applying the concepts of minimal computing can help instructors strike the balance between humanistic learning outcomes and technical requirements for digital assignments in their courses. Minimal computing describes a set of principles that foregrounds “fundamental questions about choice and necessity: ‘What do we need?’ ‘What don’t we need?’ ‘What do we want?’ ‘What don’t we want?’” (Sayers 2016). In the case of the assignments described here, we need a platform that allows students to collaboratively develop a project that complements the learning goals of the classes. We do not need a lot of technical overhead that would be intimidating for most learners. We want both a process and an end result that facilitates student learning by empowering students to produce and share content. And, again, we do not want to chew up a ton of class time for either the instructor or the students to learn the tech.

TimelineJS fulfills the requirements of minimal computing for the purposes of this assignment. Developed by Knight Lab at Northwestern University, TimelineJS is an open-source digital tool that creates flexible online interactive timelines. It uses an API linked to Google Sheets to provide an easy-to-use platform for an alternative digital research project. Clearly on the back-end TimelineJS is not minimal at all; the program is based on thousands of lines of code that drive a sophisticated interface between Google Sheets (itself a massive program) and the web browser that displays the information. But this complexity is hidden from the users (in this case students and teachers). This is the concept of minimal computing that Jentery Sayers describes as Minimal Visibility, in which developers “reduce the perceived intervention of technologies to facilitate interaction as well as the production/extraction of data from those interactions/behaviors.” The invisibility of the “intervention of technologies” between the Google Sheet and the web browser facilitates a very low technical bar to entry for end users, making Timeline JS ideal for incorporating a collaborative research project into humanities classes. Because Google Sheets allows multiple users to simultaneously populate and edit metadata in the worksheet, TimelineJS provides an opportunity for teachers to develop alternative collaborative assignments for asynchronous or synchronous remote learning settings.

In Dr. Keralis’s World Lit class, the interactive timeline allowed students to visualize a chronology of events without necessarily implying a teleological relationship between these events. The timeline helped students see how historical events correspond without necessarily influencing each other. For example, while the Tây Sơn rebellion took place in the 1770s, there is no reason to assume that the peasant uprising in Vietnam had any impact on the Declaration of Independence in the British colonies in North America. Alternatively, in the UCLA History Capstone course, the assignment facilitated a discussion about how canonical historical events both occur and are related to one another chronologically, as well as providing a grounding for discussing how the documentation and dissemination of information (as manifested in individual printed and written texts) relate to those political, social, and religious events. In the example below, Christopher Hanson’s initial interest in the Protestant Reformation and its impact on early modern Europe (see Figure 2), informed his research on a late seventeenth century text from an anonymous minister criticizing the Anglican church and its role in education (see Figure 3). Additionally, Christopher’s entry is co-located next to an entry from a fellow classmate using an image from an eighteenth century manuscript cookbook discussing access to exotic ingredients and the women’s role in household duties during a time of increasingly arduous baking methods. Thus the timeline underscores for students the correspondence between events and documentary sources, helping students use documentary evidence to understand historical events. Both classes use the tool to help meet the learning outcomes for each course, but with contrasting rhetorical approaches as to how the timeline represents historical information.

Additionally, TimelineJS serves as a flexible and easily implementable digital platform for institutions without the capacity or desire to invest in similar open source tools that require long-term hosting and stewardship commitments. Content management systems common in digital humanities such as Scalar, Omeka, and WordPress all require hosting on campus servers or on commercial servers, all of which will incur additional maintenance and other expenses. As many institutions have invested in institutional Google Apps licensing, many library staff, teaching faculty, and students benefit from existing familiarity with these tools (Google Sheets, Google Sites, etc.), decreasing the barriers for implementation.

Librarians have a role when advocating for both the principles of critical digital pedagogy and minimal computing as they consult with faculty to develop and implement digital assignments. Faculty are experts in their areas of study but might lack expertise in pedagogical areas of particular interest to librarians, such as critical information literacy, primary source literacy, and critical digital pedagogy. In their consultations and conversations with faculty, librarians should advocate for pedagogical approaches and learning activities that support student learning in these areas, advance the instructor’s goals for their courses, and incorporate our professional commitment to information literacy and research skills development. Doing this requires an extensive toolbox of assignments and learning activities as well as the specific skills to pursue assignment design during instructional consultations. This allows the librarian an opportunity to tailor assignments and activities to meet the needs of the instructor, the course, and the students, and then ensure those learning experiences are scaffolded within a course. This essentially requires and enables librarians to situate themselves as partners in instruction and course design. In these case studies, we have particularly advocated for the incorporation of TimelineJS as a digital assignment that bolsters these goals and commitments while supporting the needs and expectations of the instructor and students.

These case studies illustrate successful implementations of TimelineJS into the undergraduate humanities classroom through collaboration between librarians and teaching faculty. In these classes, TimelineJS supports course goals and supplies an engaging alternative to traditional research paper assignments that builds student knowledge, scaffolds learning, provides opportunities to learn digital humanities skills, and engages in alternative modes of scholarly research output.

TimelineJS in a World Literature Survey

The UNT assignment was originally designed to help students in a semester-long (fifteen-week) World Literature II course to help students understand the historical contexts for their assigned reading. Wide ranging survey courses such as World Lit can be challenging for both students and instructors because of the long durée nature of the syllabus; the World Lit I course at UNT goes from antiquity to the eighteenth century, and World Lit II covers the period from the eighteenth century to contemporary literature. Instructors teaching these courses—frequently graduate students or contingent faculty—are typically specialists in a specific period of English or American literature, and not generalists or non-Western literature specialists. World Lit surveys require instructors to familiarize themselves with huge swathes of historical time, as well as representative literatures from periods and cultures outside of their usual specialization. When Dr. Keralis taught the course in Spring of 2017 as an adjunct instructor for the Department of English, he was also serving as a Research Associate Professor and head for digital humanities and collaborative programs in the university library. As such, he wanted to incorporate a digital assignment into the class that would facilitate conversation about the historical contexts of the assigned texts.

The textbook for the course was The Norton Anthology of World Literature, Volume E, supplemented with plays and novels, including Larry Kramer’s The Normal Heart, Laura Esquivel’s Like Water for Chocolate, and Nnedi Okorafor’s Binti. The Norton anthology includes extensive materials to provide historical context for the works, but the stand alone materials largely did not include scholarly or contextual introductions. Once Dr. Keralis had identified the assigned texts for the syllabus, he compiled a list of historical topics that complemented those texts. For example, for the week in which the class read major figures from Romanticism including William Blake, Johann Wolfgang von Goethe, and John Keats, the timeline entry topics included Beethoven’s Emperor Concerto (1809 –11) and Caspar David Friedrich’s 1818 painting Wanderer above the Sea of Fog, to demonstrate the concept of the sublime across media. The process of developing the list of topics—three topics per student over the entire semester—was very labor intensive, taking about thirty hours to develop for the first iteration. Instructors adapting this assignment should recognize that it does require a significant amount of set up prior to the start of the semester.

Timeline entry with illustration of Wedgewood medallion of a kneeling slave. — Figure 1. Timeline entry for topic “Am I Not A Man And A Brother?” curated by World Literature student Sydney Kim. Used by Permission. CC BY-SA 4.0

The assignment was scaffolded to allow students the opportunity for revision and reflection, mirroring the concept of iteration from design thinking. There were four major components to the assignment: selection and citation of a media object related to their topics, composition of the interpretive text for the media object, an in-class presentation of their entries in which they connected the entries to the course readings, and revision and reflection. In the first class, each student chose three topics from the pre-prepared list. In subsequent classes, Dr. Keralis scaled back the number of required entries to two per student. As a writing assignment, the interactive timeline demands brevity. For each entry, students prepared a hundred-word descriptive entry for the timeline. This proved to be particularly challenging for students who were enthusiastic about their topics, and students who went over the word count were required to revise their work. Students were encouraged to include information that they had cut from their entries in their in-class presentation, rather than just reading the entries themselves. Because all twenty-eight students were working in the same Google Sheet, it allowed the instructor to troubleshoot errors in one sheet rather than many, sometimes on the fly in front of the class. The Google Sheets template provided by Knight Labs provides some error messages to assist with identifying and correcting errors. In addition, the exercise introduces students to skills in conscientious collaboration in a shared data set. Google Sheets’ version control functionality lowers the risk of having multiple hands in the same data set, since it is easy to restore an earlier version should data be inadvertently deleted.

One aspect of critical digital pedagogy in which Dr. Keralis has been particularly invested is the vindication of students’ labor and intellectual property rights. As he has written elsewhere, “student labor in the classroom is never not coerced” (Keralis 2018, 286), and students’ contribution to digital projects in classes often effaces their labor, hiding the actual cost of producing digital projects from funders and administrators. These concerns are also elaborated in the Student Collaborators’ Bill of Rights, developed at UCLA. To mitigate the coercive nature of classroom labor, Keralis designed three ways for students to complete the requirements of the assignment: online with full author attribution, online anonymously, or offline in a Google Slide. These options were discussed with students at the beginning of the semester, and all students signed off on the option they chose. For those who selected attribution, an HTML markup model was provided for them to include in their entry, an example of which can be seen in the student byline in Figure 1:

<p><small>Entry curated by [Student Name]</small></p>

Students opting for the Google Slide rather than doing the timeline were given a title and two column layout template which they would use to present their research to the class, but their work does not appear in the online timeline (Figure 2).

Google Slide mockup with illustration of Wedgewood medallion of a kneeling slave. — Figure 2. Mockup of Google Slide submission option for Sydney Kim’s timeline entry from Figure 1. CC BY-SA 4.0

Because the assignment required both primary and secondary source research that many first and second year students were unlikely to have experience doing, library instruction was provided by English subject librarian Carol Hargis at the beginning of the semester to introduce students to the data literacy and research skills necessary to successfully complete the writing for the assignment. The subject librarian was available to consult during the research process. Rather than discouraging or forbidding the use of Wikipedia as a source, the instructor and the subject librarian discussed how to use Wikipedia as a starting point for research, how to use the citation lists from Wikipedia entries as leads for sources, and how to correctly cite a Wikipedia entry. Further, many students used open access and public domain images from Wikimedia for their entries, which facilitated discussion of creator rights and licensing, and provided practice in citing non-textual media.

The students presented their timeline entries to the class prior to the discussion of the literary works. This allowed for a flipped classroom in which students shared the knowledge they had acquired through their research to provide contexts for the literature under consideration. Students participated in question-and-answer sessions with their peers. After discussion and feedback from their peers, students were allowed to revise their entries, and they submitted a final version of their interpretive text with complete citations for primary and secondary sources and media. This allowed students to demonstrate applied information seeking skills and citation practices for a variety of sources. As part of their written assignment, each student prepared an exam question based on their entries, several of which were selected for the mid-term and final exams for the course. The assignment ended with a reflective writing, in which students wrote about what they learned from their research and their engagement with the technology.

TimelineJS in a History Capstone Seminar

In incorporating TimelineJS into his courses at the University of North Texas, Dr. Keralis established clear templates and best practices for scaffolding this assignment model into curriculum to support learning outcomes. Courtney Jacobs borrowed heavily on this work to implement a similar assignment at UCLA in partnership with Dr. Muriel McClendon (associate Professor of History) who was interested in developing alternative research projects using special collections materials. In its first iteration, Jacobs collaborated with Dr. McClendon, Marisa Méndez-Brady (then English and History Librarian for UCLA Library) and Philip Palmer (then Head of Research Services for the Andrew W. Clark Memorial Library) to integrate TimelineJS and primary source literacy into a quarter-long (ten-week) history capstone seminar.[1] In years prior, Brady had worked with Dr. McClendon to build a robust partnership that embedded critical research skills and primary source literacy into the course curriculum for multiple classes, establishing a beneficial partnership open to new projects. The fall 2018 class provided the basic framework for scaffolding the TimelineJS assignment into the ten-week quarter while also allowing for hands-on research visits to UCLA’s Charles E. Young Research Library, Library Special Collections, and the Andrew W. Clark Memorial Library.

The following year, Jacobs collaborated with Matt Johnson (English and History Librarian) and Dr. McClendon to integrate TimelineJS and primary source literacy into McClendon’s fall capstone seminar exploring the topic of work and leisure in early modern England. As in previous iterations of this course, Professor McClendon was interested in building an upper-level history capstone class around library research skills, insisting that students develop primary source literacy and incorporate special collections materials into their research and assignments. As a strong library advocate and past collaborator, Professor McClendon was open to alternative project ideas that met the requirements of the capstone program, and had successfully incorporated TimelineJS into her previous year’s capstone seminar. Over the course of the quarter, the students would collaboratively define “early modern” as it applied to their class, establish an initial timeline of canonical events, then identify a topic, event, or individual for more in-depth research using primary sources to develop their final timeline entry. The final class deliverable was a collaborative digital exhibit focused on work and leisure in early modern England using TimelineJS.

Like many capstone seminars, Dr. McClendon’s class met once per week for a three-hour session. Of a total of ten class sessions, four were dedicated library sessions, scheduled during weeks four, five, seven, and ten of the quarter. Additionally, Jacobs visited the class during week two to introduce the TimelineJS assignment and begin the initial project scaffolding. Dr. McClendon assigned students related readings in preparation for this visit as well as a one-page essay in which they were asked to define their understanding of the term “early modern” and delineate and justify the chronological boundaries of the period. Additionally, she asked each student to develop and contribute a list of the ten most canonical (or impactful) events, topics, or individuals during the early modern period in England.

During this initial conversation, Jacobs introduced the class briefly to the TimelineJS platform, demonstrated the previous timeline created by the 2018 class, and outlined expectations for the assignment as well as their upcoming library visits. Next, students used their essay assignments to determine, as a group, the beginning and end dates of the early modern period in England (as it would apply to their project) before sharing their canonical lists. From this list, the students culled a working outline for their upcoming assignments. They each identified one canonical event/topic/individual to research using secondary and tertiary sources. This event/topic/individual would serve as the basis for their initial timeline JS entry, as well as a one-page essay identifying and outlining their research goals for their final assignment. Additionally, students populated a worksheet with descriptive metadata for their canonical entry and identified an illustrative and linkable reference image online. The essay and the complete entry form were due during the library visit in week four, allowing for two weeks of discussion and feedback with their instructor as well as any changes in topic.

Timeline entry with painting of Martin Luther nailing edicts to a church door. — Figure 3. Timeline entry for topic “Martin Luther’s Protestant Reformation” curated by History Capstone student Christopher Hanson.

Johnson led the first class session held in the library during week four. Prior to class, students completed an asynchronous workshop from UCLA Library’s Writing Instruction + Research Education (WI+RE) team (WI+RE Team n.d.) on Developing Research Questions and Creating Keywords to help students better prepare for searching and finding sources (Romero et al 2019). Despite being a capstone course, some of the students had never previously performed academic research nor used library resources. In order to prepare students for the research involved to create the digital exhibit, Johnson introduced students to library resources, such as book catalogs, interlibrary loan service, research assistance and support services, and relevant online databases. In class, students found one article relevant to their topic. Jacobs then provided an orientation on primary source literacy and information on accessing and requesting UCLA Library Special Collections materials. During the second half of the class, Jacobs and Johnson led a hands-on workshop where students collaborated to ingest the data they had developed for the initial TimelineJS entry into the class spreadsheet. This flipped classroom model, in which students work collaboratively to ingest the metadata from their timeline entry forms into the shared spreadsheet during class, encourages a peer-led learning experience rather than isolated and individual data-entry. Students are able to assist one another in problem-solving various tech issues that arise in working with a new tool and witness their content creation and problem-solving manifest in real-time on the resulting timeline.

Students visited Library Special Collections (LSC) and the William Andrews Clark Memorial Library during weeks five and seven to gain hands-on experience with primary sources, materiality, and book history. Jacobs worked with Devin Fitzgerald (Curator of Rare Books and Print Culture for Library Special Collections) and Anna Chen (Head Librarian at the Clark) to lead a series of hands-on lectures focused on book production and material culture in early modern England. The students learned about paleography and the basics of descriptive bibliography, and were provided methods to assess early modern books, manuscripts, maps, broadsides, and other textual artifacts as material objects. In preparation for a subsequent visit during week seven, students read works on curatorial practices, new museum theory, and best practices for writing exhibit text. They also visited a class-curated physical exhibit mounted in the Charles E. Young Research Library to familiarize themselves with these concepts in praxis.

Following this visit, the students were asked to identify a primary source artifact from either LSC or the Clark to serve as the topic of their second TimelineJS entry and final assignment. Students visited the Library, LSC, and the Clark throughout the remainder of the quarter to research their primary source item. Their final assignment consisted of a second TimelineJS entry on their primary item, an image of their item for digital display, object label text for their item, and a bibliography of further reading on their item or topic. Each of these entries required considerable research evidenced by both the bibliography and the presentation at the end of the course. The information literacy and primary source literacy skills developed early in the course for the initial canonical event entries were expanded on and implemented by students for the second entry and further analysis of the primary source artifact.

The class culminated in week ten with a student-led project launch where they presented their work to their peers, the public, and invited guests in the Charles E. Young Library’s conference room. During the first portion of the class, students led another TimelineJS workshop, collaborating to ingest the data for the final assignment into the class spreadsheet. Students then volunteered to upload the final datasets into a publicly hosted Google site to serve as the project’s final public-facing home. Through collaboration, they also added introductory text to contextualize the exhibit for viewers and additional sections for their bibliographies and further reading. After the exhibit was finalized, students used the latter portion of the class to present briefly on their selected primary source item, their research process, and their findings throughout the class.

Timeline entry with image of title page of A Second Letter to a Bishop from a Minister of His Diocese. — Figure 4. Timeline entry for topic “What Was Wrong with the Church” curated by History Capstone student Christopher Hanson.

Scaffolding Considerations

In both these implementations, students were afforded a unique opportunity to conduct their own individual research while participating in a collaborative durable digital project with their peers. In doing so, they were simultaneously participating in, and assessing their own knowledge creation throughout the course. The success of implementing these types of digital projects on the public web hinges upon pro-actively scaffolding various considerations inherent to this work, such as issues of accessibility, FERPA protections, and respect of student intellectual property and labor.

Accessibility

TimelineJS uses semantic HTML in a JavaScript shell, so the timelines resulting from these projects comply at least nominally with W3C guidelines for web accessibility. However, since the projects rely on Google Sheets, some students may face accessibility challenges in completing the projects. While Google Apps provide much of the same accessibility support that comparable software such as the Microsoft Office suite offers, students whose only internet access is via mobile phones or other devices may have difficulty working with the online app and thus have difficulty fulfilling the requirements of the assignment. Instructors would be well advised to privately poll students about their internet access and provide students with information about campus computer labs. Non-resident students may also be able to use the internet at public libraries. Offering the option of fulfilling the intellectual requirements offline in a PowerPoint or Word template may also be useful.

FERPA

A student’s enrollment in a class is protected information under the Family Education Rights and Privacy Act (FERPA) (20 U.S.C. § 1232g; 34 CFR Part 99). Thus, requiring students to publish class work on the internet—whether in blogs, social media, or platforms such as TimelineJS—necessarily involves exposure that is limited by the provisions of FERPA. Instructors may address this by asking students to sign a “Consent of Disclosure of Education Record“ form addressing the rationale for the online assignment, and offering the students the option to complete the assignment anonymously, or offline. An example form from the UNT assignment is cited below, and some Registrar’s offices have boilerplate for this release for instructors (Keralis, “Consent”). Ask your Registrar if a form is available that specifically addresses digital assignments.

Student intellectual property

In addition to addressing protected information under FERPA, instructors doing digital assignments online should directly address students’ intellectual property rights for their work. Generally, students’ in-class work is covered under universities’ Creator-Owned Intellectual Property policies, and copyright for work created for classes is retained by the student. Students in creative majors and in fields such as business or advertising may be very conscious of the appearance of their online presence, and may not want apprentice work in classes unrelated to their majors to be discoverable online. Conversely, students majoring in the field in which these assignments are offered may find it desirable to have a link-able digital project to include in their curriculum vitae or e-Portfolio. As such, students should be given the option to demonstrate authorship for their work with a by-line, complete the work anonymously online, or complete the work offline. An example of the HTML tag for a byline is included in the example “World Literature Timeline: Assignment and FERPA Release” cited in the bibliography. In any case, digital assignments provide instructors with a unique opportunity to have a conversation with students about their rights—to intellectual property, to privacy, and to protections for their educational record.

There are different ways of communicating issues surrounding privacy and intellectual property rights with students. In working with Dr. McClendon, Jacobs and Johnson deferred to the instructor’s preferred method of requiring an in-person meeting with each student prior to approving their enrollment into the capstone seminar. During this conversation, Dr. McClendon clarified the nature of the class assignment and her expectations with regards to student participation and the final digital exhibit, thus creating a process by which students were asked to opt into this work. Additionally, both Jacobs and Johnson are working within the library to develop a list of resources and best practices for faculty wishing to engage in similar public-facing digital projects that will include institution-specific guidance on these issues.

Conclusion

These case studies describe models of faculty-librarian collaboration in assignment design and implementation, providing templates for scaffolded assignments that introduce students to basic data literacy skills and empower them to explore the historical contexts of primary sources by visualizing the chronology of historical periods. By crafting student-centered assignments, instructors and librarians facilitate “digital agency” for students, honor their labor, and allow them to participate in a digital project under conditions of informed consent rather than coercion. In doing so, faculty and librarians create an environment “in which students and teachers co-author together the parameters for their individual and collective learning” (Stommel 2014), and develop a durable digital project that students and faculty can share as an example of collaborative teaching and learning.

Notes

[1] UCLA’s Capstone Initiative began in 2006 with an eye toward our centennial in 2019 (“UCLA’s Capstone Initiative” n.d.). These capstones connect well to the research mission of the university, turning students into scholars and researchers in their fields of study, and serving as a culminating education experience for students in a disciplinary course of study. The official capstone requirements of the initiative includes five criteria: (1) creative, inquiry-based learning, (2) an individual or group project with clearly delineated student work, (3) a product that can be archived for at least three years must be created, (4) the capstone must be at least four units (on a quarter system) in an upper-division course, and (5) there must be opportunities to share the finished product publicly (“Criteria & Options” n.d.). To this end, our TimelineJS project meets all of these requirements for a final product in a capstone course; it is arguably more effective for public display and archiving than regular (traditional) paper options.

Bibliography

Di Pressi, Haley, Stephanie Gorman, Miriam Posner, Raphael Sasayama, and Tori Schmitt, with contributions from Roderic Crooks, Megan Driscoll, Amy Earhart, Spencer Keralis, Tiffany Naiman, and Todd Presner. 2015. “A Student Collaborators’ Bill of Rights.” UCLA Digital Humanities. https://humtech.ucla.edu/news/a-student-collaborators-bill-of-rights/.

Family Education Rights and Privacy Act of (FERPA) (20 U.S.C. § 1232g; 34 CFR Part 99) https://www2.ed.gov/policy/gen/guid/fpco/ferpa/index.html.

Keralis, Spencer D. C. 2017. “World Literature Timeline: Assignment and FERPA Release.” Humanities Commons. https://hcommons.org/deposits/item/hc:31089/.

Keralis, Spencer D. C. 2018. “Disrupting Labor in Digital Humanities; or, The Classroom Is Not Your Crowd.” In Disrupting the Digital Humanities, edited by Jesse Stommel and Dorothy Kim, 237–94. Santa Barbara: Punctum Books.

Knight Lab. n.d. TimelineJS. http://timeline.knightlab.com/.

McCLendon, Muriel. n.d. “Work and Play in Early Modern England.” Accessed July 14, 2020. https://sites.google.com/view/mcclendonfa19exhibit/home.

Romero, Renee, Shannon Roux, Taylor Harper, Kian Ravaei, and Doug Worsham. 2019. “Developing Research Questions and Creating Keywords [Workshop].” Writing Instruction & Research Education (WI+RE). Accessed July 17, 2020. https://uclalibrary.github.io/research-tips/workshops/developing-research-questions-and-creating-keywords/.

Sayers, Jentery. 2016. “Minimal Definitions (tl;dr version).” Minimal Computing: a working group of GO::DH. October 2, 2016. https://go-dh.github.io/mincomp/thoughts/2016/10/03/tldr/.

Stommel, Jesse. 2014. “Critical Digital Pedagogy: A Definition.” Hybrid Pedagogy. November 17, 2014. https://hybridpedagogy.org/critical-digital-pedagogy-definition/.

University of California, Los Angeles, n.d. “Criteria & Options.” UCLA’s Capstone Initiative. Accessed July 17, 2020. http://capstones.ucla.edu/the-initiative/criteria-options/.

University of California, Los Angeles. n.d. UCLA’s Capstone Initiative. Accessed July 17, 2020. http://capstones.ucla.edu/.

WI+RE Team. n.d. “WI+RE (Writing Instruction + Research Education) [Home]”. Writing Instruction & Research Education (WI+RE). Accessed July 17, 2020. https://uclalibrary.github.io/research-tips/.

About the Authors

Spencer D. C. Keralis, Ph.D. is Assistant Professor and Digital Humanities Librarian with the University Library at the University of Illinois at Urbana-Champaign. They are Founder and Executive Director of Digital Frontiers, a non-profit organization that brings together the makers and users of digital resources for humanities research, teaching, and scholarly communication.

Courtney Jacobs is the inaugural Head of Public Services, Outreach, and Community Engagement for UCLA Library Special Collections. She is a co-founder of the 3DHotbed project, a multi-institution digital humanities project that explores the use of 3D printing technology to facilitate hands-on book history pedagogy.

Matthew Weirick Johnson is a Humanities and Social Sciences Librarian at UCLA Library’s Young Research Library and liaison to the English, History, and Comparative Literature departments. Johnson holds an MSLS from the University of North Carolina at Chapel Hill’s School of Information and Library Science. Prior to joining UCLA, Johnson worked in medical, academic, and public libraries and one non-profit in both on-site and remote roles.

Two charts compare interaction by videoconference meeting participants.

Issue Nineteen

Assessing the Effectiveness of Using Live Interactions and Feedback to Increase Engagement in Online Learning

Beth Porter, Riff Analytics

John Doucette, Riff Analytics

Andrew Reilly, New College of Florida

Dan Calacci, MIT Media Lab

Burcin Bozkaya, New College of Florida

Alex Pentland, MIT Media Lab

Abstract

Traditional, in-person instruction provides a social environment and immediate feedback mechanisms that typically ensure participants are successful. In comparison, online mediums for instruction lack these mechanisms and rely on the motivation and persistence of each individual learner, often resulting in low completion rates. In this study, we examined the effect of introducing enabling tools and live feedback into an online learning experience on learner performance and persistence in the course, as well as election to complete supplemental readings and assignments. The findings from our experiments show positive correlations with strong statistical significance between live interactions and all performance measures studied. Specifically, we find that students who consistently participated (at least once a week on average) received final grades 80% higher than those who did not and were twice as likely to earn a certificate of completion. Furthermore, the odds of a student receiving higher grades are increased by a factor of 23% for each additional hour spent using our enabling tools and the odds of passing the course by 35%. We contend that further exploration and implementation of such enabling tools will aid both instructors and learners, as the online platforms may continue to be the preferred choice of instruction even after the pandemic subsides.

Introduction

Achieving and maintaining consistent student engagement can be difficult in online learning contexts (Kizilcec and Halawa 2015; Allen and Seaman 2015, 61; Greene et al. 2015). There are many factors that impact learner engagement: course structure, amount of lecture time, user experience of the learning management system, and countless other variables. However, online educators and scholars of online learning have consistently found evidence that peer interaction and learner-instructor interaction are strongly related to learner engagement (Bryson 2016; Garrison and Cleveland-Innes 2005; Greene et al. 2015; Richardson and Swan 2003; Yamada 2009). Several studies have found that measures of social presence, a theoretical construct originating from communications studies, are significantly related to learners’ overall outcomes and perceived experience (Richardson and Swan 2003). Social presence refers to how “salient” an interpersonal interaction is—how readily one feels like they are “present” with another.

To achieve high social presence between students, researchers have discovered that the modality of online communication is extremely important (Yamada 2009; Kuyath 2008; Johnson 2006). The modality of communication refers to the types of channels used: text chat, forums, voice chat, video conference, etc. In communications, human-computer interaction, and online education literature, it has been shown that richer modes of communication between conversing members lead to higher rates of trust, cooperation, engagement, and social presence (Bos et al. 2002). For example, video conference, which includes real time video and voice, is a richer mode of communication than asynchronous forums, and has been shown to lead to higher rates of cooperation than text chat (Bos et al. 2002).

These advances provide strong evidence that the interaction patterns of people carry rich signals that correlate with a variety of outcomes important to personal growth, learning, and team success. By using modern machine learning methods and statistics of conversation patterns—who spoke when, for how long, etc.—even without knowing the content of these conversations, researchers have been able to predict speed dating outcomes, group brainstorming success, and group achievement on a test of “collective intelligence” (Pentland et al. 2006, Woolley et al. 2010; Dong and Pentland 2010; Jayagopi et al. 2010; Kim et al. 2008; Dong et al. 2012a, 2012b). By developing more sophisticated computational models of human interaction, it is possible to infer social roles, such as group leaders or followers (Dong et al. 2013).

Current study

To show the effectiveness of online learning systems that contribute to strong learning outcomes and learner performance via tools that foster participation and interaction, we present an experimental study in this paper conducted with an online class. Our learning platform, which is used to conduct our experiments, allows students to interact using video chat and text chat. In addition, students get continuous feedback on their participation and engagement in the video chat via a tool we named Meeting Mediator (MM). MM essentially provides feedback on how the class participant is doing in terms of measures of engagement such as speaking time, turn-taking, influencing or affirming other participants, and interrupting other speakers.

The Meeting Mediator integrated into our online learning platform is a further enhancement of the concepts and ideas introduced by Kim et al. (2008), who implemented a similar user feedback mechanism in a face-to-face setting with data collected via sociometric badges worn by users interacting in small groups. Our platform includes a new version of the MM, which introduces new metrics for key conversational events such as interruptions and affirmations. In this study, we validate the effectiveness of this online MM through a number of experiments we conduct.

Part of the innovation at the core of our study is the burgeoning science of quantitatively analyzing human social interaction. This small but exciting subfield of computational social science has shown immense promise in developing machine understanding of human interaction (Woolley et al. 2010). By instrumenting many different types of communication, such as face-to-face physical conversation, video conference, and text chat, researchers have been able to predict group performance on a variety of tasks.

Though this study was conducted prior to the 2020 pandemic, the challenge presented to the team carrying out the experiment was to understand the factors contributing to successful transition of programs to online environments. The platform was designed to foster virtual engagement, providing course developers and instructors new tools for offering online experiences likely to capture and keep learner interest. This is especially critical in a time when the pandemic has forced millions of students and professional learners around the globe to use online platforms, replacing the traditional in-person instruction.

An important point to emphasize is that we do not make a claim to replace in-person learning with technology-enabled online learning. We acknowledge that remote instruction may present certain technology-driven features and advantages, but there are many ways in-person instruction provides a superior learning experience. Through the experiments we conduct in this study and our findings, we contend that the remote learning experience can be enhanced and the gap between in-person versus online learning can be bridged to some extent using real-time feedback mechanisms such as the Meeting Mediator and interaction metrics. This is clearly good news in times of a pandemic like COVID-19, where instructors as well as learners are challenged in many unique ways.

Materials and Methods

We first describe our user-facing applications, supporting backend tools and the metrics we designed and employed for performance assessment. Then, we present our experiment design.

The Riff Platform and its user-facing applications

Our team developed three foundational user-facing applications and supporting backend tools as part of the Riff Platform. These are:

Video Chat (with Meeting Mediator)
Post-Meeting Metrics
Text Chat (which includes Video Chat and Post-Meeting Metrics)

These applications and tools were built for both commercial and research purposes, cloud-deployed in standardized ways, but customized over time to meet the changing needs of Riff customers and modified research goals. (An open version of Riff Video Chat, called Riff Remote, is available here: https://my.riffremote.com/.)

Video chat

Riff Video Chat (Figure 1) is an in-browser video chat application that uses the computer’s microphone to collect voice data for analysis and is present in all manifestations of the Riff Platform. Riff Video does not require plug-ins or a local application in order to run—participants navigate to a URL (either by entering it directly into the address bar or through an authenticated redirect from another application), enable their camera and microphone, and then enter the video chat room. Other features of the video chat include screen sharing, microphone muting, and the ability to load a document (typically one with shared access) side-by-side with the video on screen.

A key element of Riff Video is the Meeting Mediator (MM) (Figure 2), which gives participants of the video chat real-time feedback about their speaking time. Specifically, it provides three metrics:

Figure 1: The Riff Video browser-based video conferencing
client with Meeting Mediator shown. — Figure 1. Riff Video browser-based video conferencing client.

Engagement, as indicated by the shade of purple of the node in the middle of the visualization, which shows the total number of turns taken in running five–minute intervals throughout the chat—dark purple means many turns; light purple means fewer turns;
Influence, as indicated by the location of the purple node, which moves toward the person who has taken the largest number of turns in running five–minute intervals throughout the chat;
Dominance, as indicated by the thickness of the grey “spokes” running from the central purple node out to each of the participant nodes, which indicates the number of turns people have taken in running five–minute intervals.

Figure 2: The Riff Meeting Mediator, with hover information
“Name”, “Speaking Time” and “Turn Count” shown. — Figure 2. Hover states of the Meeting Mediator showing different metrics.

These metrics are continuously updated throughout the video chat, lagging one or two seconds behind the live conversation. In a meeting, this tool acts as a nudging mechanism, which is essentially an intervention by the system aimed at increasing a participant’s self and situational awareness, influencing a participant’s behavior towards a more rewarding learning experience and outcomes. The MM indirectly encourages users to maintain conversational balance by speaking up (if the user sees they have not spoken recently) or to engage others who have not been as active in the conversation.

Post-meeting metrics

Immediately after a Riff Video Chat meeting, Metrics (Figure 3) appear on screen to show participants measures associated with the meeting, specifically:

Speaking Time—a chart showing the percentages of turns taken by each participant;
Pairwise Comparisons—each of the following charts shows the pairwise comparisons of the individual participant with respect to each other participant, to show the balance of each pair in their interactions:
- Influences—a chart showing who spoke after whom in the video chat, which aggregates data across all types of spoken events;
- Interruptions—a chart showing interruptions (when someone cuts off another person’s speaking turn);
- Affirmations—a chart showing affirmations (when someone has a verbalization, but does not cut off another person’s speaking turn);
Timeline—an interactive view of the video chat meeting over time, on top showing what time each participant spoke and below when each pairwise interaction took place (“your” and “their” interruptions, affirmations, and influences).

Figure 3: Meeting metrics (timeline, speaking time, influences,
interruptions and affirmations). — Figure 3. Meeting metrics (timeline, speaking time, influences, interruptions and affirmations).

Additionally, each user can access their past history of meetings at any time, independent of the video chat itself (Figure 4).

Figure 4: Meeting history, in chronological order. — Figure 4. Meeting history, in chronological order.

Text chat

Riff Text Chat (also known as Riff EDU) is an environment that allows people to text chat with one another in open channels, private channels, and direct messages (Figure 5).

Riff EDU is built on top of the Mattermost platform, which is an open-source project (https://mattermost.com/). The Riff team developed a customized version of this platform which has three key additions (and several smaller modifications):

LTI integration, which allows the learner to move seamlessly from their learning environment (Canvas, Open edX, etc.) using the learning tools interoperability (LTI) protocol for authentication;
Video integration, which allows the learner to start an ad hoc video chat session from the Chat environment;
Metrics integration, which shows the learner views of their video meeting metrics within the Chat environment, as well as additional text chat data and recommendations.

Learner data privacy

At the outset of the course, learners were asked to consent to have data collected about their activity and performance. No learners declined to have data collected about them. Data collected during this experiment was shared directly with learners through the Meeting Mediator and Post-Meeting Metrics page. Everyone in a video chat got access to the data about their conversation, but video and text chat data was not shared with anyone else during the experience. Data analyzed for the purpose of the experiment was anonymized to preserve the privacy of participants.

Supporting accessibility

The experimental environment was designed to account for participants with accessibility needs. Following the WCAG 2.0 AA accessibility standard from the W3C (World Wide Web Consortium 2021) the software underwent an accessibility audit to identify potential issues, which were then remediated before the platform was released to learners. Accessibility compliance was documented using a publicly available Voluntary Participation Accessibility Template (Riff Analytics 2021). Though we did not have any students who let us know that they had specific accessibility needs, we were prepared for that eventuality.

Experiment design

To show the effectiveness of our video conferencing environment and our nudging tool, the Meeting Mediator (MM), we conducted an experiment with a major client of Riff Learning Inc. in Canada. In this experiment, test subjects were recruited to participate in an online course called “AI Strategy and Application” from a government subsidized incubator of tech companies headquartered in Toronto, Ontario, Canada. Students elected to take the course as a professional development experience, driven by interest in learning how to start an AI initiative either as an intrapreneurial or entrepreneurial activity. In most cases, the learners were paying for the course, but in some cases, either the course had been paid for by their employer or they were given free access to the course. Learners were either students in advanced degree programs or full-time professionals taking the course as a supplemental learning experience.

The eight-week course was delivered on the Open edX platform with the Riff Platform serving as the communications and collaboration platform for instructor and learner interactions, both structured and ad-hoc. Learners had opportunities to view videos and read original materials on how AI is applied to business problems, tackle hands-on coding exercises to learn basic machine learning techniques in a sandbox environment, and take assessments at the end of each learning unit.

The participants were divided into groups throughout the course to facilitate the following two types of activities:

Brainstorming activities (in pre-set groups of four to six people)—Each week during the first four weeks of the course, people met on video in their Peer Learning Groups (PLG) to collaborate on the assignment, and then create a shared submission.
Capstone project collaboration (in self-selected groups of four to six people, plus a mentor)—Each week during the last four weeks of the course, people met on video in their Capstone Groups to collaborate on developing a venture plan based on the winning pitches submitted by their peers in the course.

Learners were left to schedule video meetings with their group members on their own, and the course was time-released (one week of material at a time), but self-paced. However, the course support staff did participate in the first meeting of each group, to ensure that people didn’t have any technical issues and could make the video chats happen without issue.

The main goal of our experiment was to analyze the relationship between using video and text chat, and various outcomes among learners in the course. Furthermore, we aimed to analyze differences, if any, between early users (those who completed at least the first half of the course) versus the rest, to understand the effects of early usage on retention and grades.

Note that in this experiment, the Meeting Mediator (MM) was always present for video chats. Our rationale for that choice was that if our hypotheses that video with MM is positively correlated with any of performance, persistence, or satisfaction in online courses, then having it present for some students and not for others in a paid, graded learning experience might disadvantage the control population and be deemed inequitable.

In our experiment, we set out to test a number of hypotheses and hence show the (positive) relationship (or its lack thereof) between the use of video and various outcomes. Specifically, we explored the following questions:

Did students who used Riff Video Chat (and Meeting Mediator) more often receive higher grades in the course?
Did students who used Riff Video Chat (and Meeting Mediator) more often complete the course at a higher rate?
Is there a strong positive association between participating in additional Riff Video Chats during the first four weeks of the course and earning a certificate?
Is there a strong positive association between participating in additional Riff Video Chats during the first four weeks of the course and receiving higher grades at the end of the course?
Is there a strong positive association between participating in additional Riff Video Chats during the first four weeks of the course and completing more of the optional course assignments?
Is there a strong positive association between participating in additional Riff Video Chats during the first four weeks of the course and pitching a capstone project topic?

The first two research questions are explored for students who completed the entire course (n=62); we removed the chat records and performance output results for students who dropped part way. The remaining questions are explored for the entire cohort students of n=83. These students all started the course, but some of them dropped out and, hence, did not receive a final grade for completion or a certificate.

In our analysis of our experiment results, we used the following three constructs for reporting and interpreting our figures: correlations, odds ratios, and significance. For definitions of these metrics and their implications for our study, we refer the reader to Appendix B. With correlation analysis, we would like to further note that while one may observe the degree to which people who do one thing more (e.g., use video chat) also tend to do something else more (e.g., receive higher grades), such a relationship cannot be directly used to imply causality; they are merely indicators that two phenomena occur together (in a positive or negative relationship) or not.

Finally, we conducted an exit survey for all students who completed the course and a separate survey for those who failed to complete it. While the results of this survey are not directly related to the quantitative analysis we report in the next section, they do provide some qualitative feedback from users on the efficacy of the Riff Platform. We have included the list of survey questions and some key results we collected from these surveys in the Appendix.

Results

In this section, we report the results of our experimental study for students who participated in the course for its entire duration (questions 1–2, n=62) and for all students who started with the course, even if they dropped out (questions 3–6, n=83) separately.

Results for questions 1–2 for students who completed the course

To answer the first question in our study (“Did students who used Riff Video Chat more often receive higher grades in the course?”), we considered the following output variables:

Final grade earned
Coding exercise grade
Capstone exercise grade
Collaboration exercise grade
Pitch video completion

For the second question (“Did students who used Riff Video Chat more often complete the course at a higher rate?”), we simply considered whether or not the student earned his/her certificate of completion in the course.

The following table, Table 1, shows the correlation between each output variable listed above and the input variable of “video calls made,” which is the number of times a student connected to the video chat. The p-values were further corrected using Holm’s method to maintain a FWER of 0.05. We find that all p-values are significant at 95% confidence level or better, and correlation values point to reasonably high levels of relationship with most of the output variables.

Attribute	Correlation to # Riff Calls Made	n	p	Significance
Final Grades	0.50	62	1.56e-04	***
Coding Exercise Grades	0.41	62	2.54e-03	**
Capstone Exercise Grades	0.49	62	1.56e-04	***
Collaborative Exercise Grades	0.27	62	2.94e-02	*
Pitch Video Completion	0.37	62	5.25e-03	**
Certificate Earned	0.50	62	1.56e-04	***

Table 1. Correlation between Riff Video usage and various performance variables.
*p < 0.5, **p < 0.01, ***p < 0.001

We further calculated the odds ratio for two of these variables, as shown in Table 2. Here, in addition to the “Certificate Earned” binary variable, the “Grades” variable was created as another binary variable indicating the student has received (or not) a passing grade based on his/her final grade.

Attribute	Odds Ratio	n	p	Significance
Grades	1.23	62	16.92e-03	**
Certificate Earned	1.35	62	3.964-04	***

Table 2. Odds ratios for the effect of attending one additional Riff Video call on final grade and certificate attainment obtained by fitting a logistic regression model.
*p < 0.5, **p < 0.01, ***p < 0.001

Table 2 with odds ratios greater than 1.0 suggests that spending additional time with video chat increases the likelihood of earning a passing grade and a final certificate of completion (see the Discussion section for more details).

We further illustrate the relationship between these two variables and our input variable (“video calls made”) in Figures 6 and 7. The figures display positively sloped trends which confirm the positive correlations reported in Table 1. That is, as the learners spend more time with video chat, they are more likely to earn higher grades and a certificate of completion.

Figure 6: A scatter plot showing the relationship between the students’ total Riff video usage and final grade. There is a regression line included showing a positive relationship between the two variables. — Figure 6. Relationship between Riff Video usage and students’ final grades.

Figure 7: A scatter plot showing the relationship between the students’ total Riff video usage and whether the student received a certificate for the course. There is a regression line included showing a positive relationship between the two variables. — Figure 7. Relationship between Riff Video usage and certificate achievement.

Results for questions 3–6 for all students

We explored the remaining research questions regarding the early usage (first four weeks) of Riff Video using data we collected from all students in this course. We note again that some students completed the course only partially and dropped out. They did, however, complete the first four weeks of the course. Here we used the start of the Capstone Project as the cutoff date for tracking how often the students were present in channels in which video chats were offered. We correlated this variable to the same performance variables we considered in the previous section to explore the research questions 3–6:

Tables 3 and 4 show the correlation values between the variables similar to the way they are presented in the previous section. All correlations were computed with Pearson’s method, and p values were corrected again using Holm’s method to maintain a FWER of 0.05. We find that not only are the correlation values in Table 3 generally higher than those in Table 1, the p-value significance levels are also stronger. Furthermore, higher odds ratios in Table 4 suggest that early video chat usage is even more determinant in achieving successful performance in the course.

Attribute	Correlation to # Riff Calls Made	n	p	Significance
Final Grades	0.54	83	5.40e-07	****
Coding Exercise Grades	0.42	83	5.22e-05	****
Capstone Exercise Grades	0.50	83	3.89e-06	****
Collaboration Exercise Grades	0.52	83	2.07e-06	****
Pitch Video Completion	0.45	83	3.95e-05	****
Certificate Earned	0.50	83	3.89e-06	****

Table 3. Correlation between early Riff Video usage and various performance variables.
*p < 0.5, **p < 0.01, ***p < 0.001, ****p<0.0001

Attribute	Odds Ratio	n	p	Significance
Grades	1.79	83	1.03e-04	***
Certificate Earned	2.00	83	1.96e-05	****

Table 4. Odds ratios for the effect of attending one additional Riff Video call on final grade and certificate attainment obtained by fitting a logistic regression model.
*p < 0.5, **p < 0.01, ***p < 0.001, ****p<0.0001

The relationship between these two variables and our input variable (“Early video usage”) is further illustrated in Figures 8 and 9. Red points in these figures indicate students who dropped the course. Note that in one case, a student indicated they were dropping the course, but then completed all work and earned a certificate. Similar to Figures 6 and 7, we also find that as the early video chat usage increases, the likelihood of earning higher grades and a certificate of completion also increases.

Figure 8: A scatter plot showing the relationship between the students’ early Riff video usage and final grade. There is a regression line included showing a positive relationship between the two variables. — Figure 8. Relationship between early Riff Video usage and students’ final grades.

Figure 9: A scatter plot showing the relationship between the students’ early Riff video usage and whether the student received a certificate for the course. There is a regression line included showing a positive relationship between the two variables. — Figure 9. Relationship between early Riff Video usage and certificate achievement.

Summary, Discussion, and Conclusions

Summary

The results of the experiment may be summarized as follows:

Each additional Riff Video Chat made during the first four weeks of the course predicts a doubling of the odds that the student will receive a certificate of course completion. (Fig. 9)
Each additional Riff Video Chat made during the first four weeks of the course predicts an increase in the odds of receiving a high grade by 79%. (Fig. 8)
Most benefits are accrued after participating in the first four to five video chats. Students who participated in more than four calls (an average of one per week) received final grades 80% higher than those who did not and were twice as likely to earn a certificate. (Tables 3–4 and Fig. 8–9)
We find significantly stronger evidence, as reported in Tables 1–4 and Figures 6–9, of measured metrics linked to user performance, than those reported in the literature by Kim et al. (2008).

Discussion and Conclusions

In our analysis of experiment results, for the first group that completed the entire course, we found strong correlations between the time (in minutes) spent using video to communicate with peers, and every outcome of interest as shown in Table 1. Particularly strong correlations were found between students’ final grades and attainment of course certificates, which has significant commercial implications. Historically, open online courses have had very low completion rates around 5% (Lederman 2019); hence the use of novel tools within online courses can increase engagement, raise the rates of persistence, and improve outcomes. Because this is an observational design, these results do not establish a causal relationship between usage of the Riff Platform and these outcomes, but they suggest that such a relationship may exist. We found that the relationship between video usage and both of our response variables (grades and course completion) was well fit by a logistic curve, as shown in Table 2 and Figure 7. The upward pattern observed in Figures 6–7 simply indicates that the participants are more likely to achieve higher grades and earn a certificate of completion if they more frequently use Riff Video chat.

The reported odds ratios as extracted (reported in Table 2) from the coefficients of the logistic regression model fit suggest results that are statistically significant. The reported values correspond to increases in the odds of a student receiving higher grades by a factor of 23% for each additional hour spent using Riff Video chat and in the odds of passing the course by 35%. The analysis suggests that potential improvements in outcomes are realized with a total exposure of no more than approximately 15 hours over the course.

With the second group of students who remained enrolled during at least the first quarter of the course, the results summarized in Tables 3 and 4 demonstrate that early usage of Riff Platform has an even stronger relationship to the outcomes of interest than usage over the course as a whole. Particularly notable is the effect size for each additional hour of exposure to Riff Video during the first half of the course, which is associated with a doubling of the odds that the student completes the course and earns a certificate. Although this is an observational experiment, again, it suggests that the Riff Platform may be having a significant impact on students’ course completion rates and participation. Similar to Figures 6 and 7, we also observe an upward pattern in Figures 8 and 9, which indicates that this second group of students are also more likely to achieve higher grades in the course and earn a certificate of completion if they more frequently use video. We further note that the high positive correlation values relating measured metrics to user performance in the course are significantly better than those reported in the literature by Kim et al. (2008).

These results suggest that the platform and its nudging mechanisms ultimately benefit student learning as assessed via several exercises completed by the students. The high correlations described above suggest that there is a statistically significant difference in student performance when nudging features such as the MM are used versus not. We also contend that the Riff Platform helps students with self and situational awareness, as it presents the student with live as well as post-session information about their individual and group performance. This helps students adjust themselves in terms of balancing active contribution versus listening to others, and their level of engagement, via the real time feedback loop, when the course requires engagement to meet the learning goals.

These experiments and the corresponding results, though they signal strong correlations between the usage of Riff Platform and all measured output variables, are not without limitations. The results are based only on this set of experiments with this cohort of students in a single course. One could repeat these experiments in a variety of settings (using different cohorts, different countries, and different learning environments and tasks) to see if the results can be generalized. Secondly, we did not have any control variables (such as age, gender, job experience or other variables like prior online learning experience or performance in similar online or offline courses) on the subjects for whom we collected the data. It is possible that students who are successful using the Riff Platform would have been successful anyway because they have prior experience using online learning platforms or they are already high-performing students regardless of the learning environment or tools. Or some students may afford extra time outside the class to review and digest the material, explore additional resources and/or complete additional exercises while others may not. These effects should definitely be included in a more detailed future experimental study. This line of research would highly benefit from an A/B testing approach (e.g., using randomized controlled trials) where one would control for various factors. This requires a different learning environment to be set up where participants are not disadvantaged due to the parameters of the experiment in a legitimate online course where they expect to be provided with equal opportunities for learning.

Another limitation is the fact that the platform only keeps track of audio signals, not the actual content of what is spoken, and it classifies them into three categories as influences, interruptions or affirmations. It is possible that these classifications may not always be accurate, despite the rules we apply, due to backchannel noise. Future work could refine current techniques for identifying the three conversation events with additional hand-labeling, which could then be used to confirm the validity of the rules currently used in the platform.

To capture content being exchanged by participants, there is further opportunity to process spoken words using Natural Language Processing libraries and collect information on the scope and “quality” of the content (e.g., new topics or ideas introduced; sentiment of the words used). The platform could then reflect not only on the frequency or duration of one’s verbal participation, but also its quality.

One may also be concerned about the cost of using such a platform or integrating it with an existing learning management system (LMS). We note that because the platform readily comes as a complete platform solution, there is no additional cost of integrating it with existing LMS. Users simply login and use the features of the platform as needed. Some non-financial “costs” may exist, but they are limited to time and effort needed for training on how to use the system and understand its features such as the MM and Riff Metrics. We believe these costs are quite reasonable as the user interface has been designed with usability in mind.

Finally, though this experiment was within a course for adults interested in expanding their understanding of AI, the participants came from a variety of backgrounds including undergraduate and graduate students, early and mid-career professionals, and government employees. We see potential for this technology to address engagement in virtual settings for any type of collaborative work and any age group. (In all cases, it is important for people meeting in small groups to have an appropriate motivation for collaborating, such as brainstorming or problem solving.)

Even with the limitations noted, our results are a solid first step towards showing that online learning platforms with the right user-facing components providing relevant and prompt feedback to participants are indeed likely to be effective in enhancing the learning experience. This is important especially in the wake of growing demand for online learning platforms due to the COVID-19 pandemic. We believe tools like the Riff Platform and the Meeting Mediator will play an increasingly important role in educational and professional settings where online platforms foster more and more interactions.

Bibliography

Allen, Isabel Elaine, and Jeff Seaman. 2015. Tracking Online Education in the United States. Babson Survey Research Group. https://onlinelearningconsortium.org/read/online-report-card-tracking-online-education-united-states-2015/

Bos, Nathan, Judy Olson, Darren Gergle, Gary Olson, and Zach Wright. 2002. “Effects of Four Computer-Mediated Communications Channels on Trust Development.” Proceedings of CHI ‘02, SIGCHI Conference on Human Factors in Computing Systems: 135–40.

Bryson, Colin. 2016. Review of Engagement through Partnership: Students as Partners in Learning and Teaching in Higher Education, by Mick Healey, Abbi Flint and Kathy Harrington. International Journal for Academic Development 21, no. 1: 84–86.

Dong, Wen, and Alex “Sandy” Pentland. 2010. “Quantifying Group Problem Solving with Stochastic Analysis.” International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction (ICMI-MLMI ’10), Article 40: 1–4.

Dong, Wen, Bruno Lepri, and Alex “Sandy” Pentland. 2012a. “Automatic Prediction of Small Group Performance in Information Sharing Tasks.” Proceedings of Collective Intelligence Conference, arXiv: 1204.2991.

Dong, Wen, Bruno Lepri, Taemie Kim, Fabio Pianesi, and Alex “Sandy” Pentland. 2012b. “Modeling Conversational Dynamics and Performance in a Social Dilemma Task.” Proceedings of the 5th International Symposium on Communications Control and Signal Processing, 1–4.

Dong, Wen, Bruno Lepri, Fabio Pianesi, and Alex “Sandy” Pentland. 2013. “Modeling Functional Roles Dynamics in Small Group Interactions.” IEEE Transactions on Multimedia 15, no. 1: 83–95.

Garrison, Donn Randy, and Martha Cleveland-Innes. 2005. “Facilitating Cognitive Presence in Online Learning: Interaction Is Not Enough.” American Journal of Distance Education 19, no. 3: 133–148.

Greene, Jeffrey A., Christopher A. Oswald, and Jeffrey Pomerantz. 2015. “Predictors of Retention and Achievement in a Massive Open Online Course.” American Educational Research Journal 52, no. 5: 925–955.

Jayagopi, Dinesh Babu, Taemie Kim, Alex “Sandy” Pentland, and Daniel Gatica-Perez. 2010. “Recognizing Conversational Context in Group Interaction using Privacy-Sensitive Mobile Sensors.” Proceedings of the 9th International Conference on Mobile and Ubiquitous Multimedia (MUM ’10), Article 8: 1–4.

Johnson, Genevieve Marie. 2006. “Synchronous and Asynchronous Text-Based CMC in Educational Contexts: A Review of Recent Research.” TechTrends 50, no. 4: 46–53.

Kim, Taemie, Agnes Chang, Lindsey Holland, and Alex “Sandy” Pentland. 2008. “Meeting Mediator: Enhancing Group Collaboration using Sociometric Feedback.” Proceedings of the 2008 ACM conference on Computer Supported Cooperative Work: 457–466.

Kizilcec, René F., and Sherif Halawa. 2015. “Attrition and Achievement Gaps in Online Learning.” Proceedings of the Second (2015) ACM Conference on Learning @ Scale – L@S ’15: 57–66.

Kuyath, Stephen James. 2008. “The Social Presence of Instant Messaging: Effects on Student Satisfaction, Perceived Learning, and Performance in Distance Education.” PhD diss., The University of North Carolina at Charlotte.

Lederman, Doug. 2019. “Why MOOCs Didn’t Work, in 3 Data Points.” Inside HigherEd, Jan. 16, 2019. Accessed March 15, 2021. https://www.insidehighered.com/digital-learning/article/2019/01/16/study-offers-data-show-moocs-didnt-achieve-their-goals.

Pentland, Alex “Sandy,” Anmol Madan, and Jon Gips. 2006. “Perception of Social Interest: A Computational Model of Social Signaling.” Proceedings of the 18th International Conference on Pattern Recognition: 1080–1083.

Richardson, Jennifer C., and Karen Swan. 2003. “Examining Social Presence in Online Courses in Relation to Students’ Perceived Learning and Satisfaction.” Journal of Asynchronous Learning Network 7, no. 1: 68–88.

Riff Analytics. 2021. “Riff Learning Accessibility Conformance Report.” Accessed March 15, 2021. https://riffanalytics.ai/riff-edu-vpats-3/.

Sun, Huaiyi “Cici,” Dan Calacci, and Alex “Sandy” Pentland. 2018. “Patterns of Interaction Predict Social Role in Online Learning.” 2018 International Conference on Social Computing, Behavioral-Cultural Modeling & Prediction and Behavior Representation in Modeling and Simulation. Accessed April 21, 2021. http://sbp-brims.org/2018/proceedings/papers/latebreaking_papers/LB_11.pdf.

Woolley, Anita Williams, Christopher F. Chabris, Alex ‘Sandy’ Pentland, Nada Hashmi, and Thomas W. Malone. 2010. “Evidence for a Collective Intelligence Factor in the Performance of Human Groups.” Science 330, no. 6004: 686–688.

World Wide Web Consortium. 2021. “Web Content Accessibility Guidelines (WCAG) 2 Level AA Conformance.” Accessed March 15, 2021. https://www.w3.org/WAI/WCAG2AA-Conformance.

Yamada, Masanori. 2009. “The Role of Social Presence in Learner-Centered Communicative Language Learning using Synchronous Computer-Mediated Communication: Experimental Study.” Computers & Education 52, no. 4: 820–833.

Appendix A—Exit Survey Questions and Key Results

A1. Exit Survey Questions

How well did you understand the lessons at the start of the course?
How well did you understand the subject matter at the midpoint of the course?
How well did you understand the subject matter at the end of the course?
How satisfied were you with what you learned throughout the course?
How well did the instructional materials convey course expectations?
How well did course assignments test your understanding of course materials?
How effectively did the activities and assignments used in this course aid your learning?
Would you recommend this course to other students?
How well did course staff support your learning throughout the course?
How well were your technical questions addressed?
How well were your questions about the coding assignments addressed?
How well were your questions about the business assignments addressed?
How effectively did course staff direct you toward resources?
How well did the course staff support you overall?
How useful did you find the real-time feedback during video calls?
How clear was the real-time feedback during video calls?
Thinking about the video calls for your Peer Learning Group and your Capstone team, to what degree do you agree with the following statements?

The feedback caused me to participate more.
The feedback caused me to give other team members more of an opportunity to speak.
The feedback caused me to participate less.
The feedback caused the other team members to talk more.
The feedback caused the other team members to give other team members more opportunities to speak.
The feedback caused the other team members to talk less.

On average, approximately what percentage of time were you speaking during meetings?
How would you describe how the conversations played out?
After using the video tool and seeing the real-time feedback, to what degree do you agree with the following statements?

I found this to be a useful tool.
I will try to talk more in similar activities in the future.
I will try to make sure other people have an equal opportunity to talk.
I will try to talk less in similar activities in the future.
I would like to use this tool in the future (for learning or work).

Do you have any suggestions for how we can improve the real-time feedback?
How useful was the Riff Stats feedback?
How easy was it to interpret the Riff Stats feedback?
How did Riff Stats feedback change your behavior?
How useful was the Riff messaging feature (the channels where you connected with other team members)?
Considering all the aspects of the Riff Platform (video calling, messaging, real-time feedback, and post-meeting feedback) to what degree do you agree with these statements?

The Riff Platform helped me connect with my team members.
The Riff Platform helped me connect with other people in the course.
The Riff Platform helped me connect with course staff.
The Riff Platform enabled my team to work effectively together.
The Riff Platform enabled me to be more successful in the course.

The Riff Platform was easy to use.
The Riff Platform did not present technical challenges.
The course environment was easy to use.
The course environment did not present technical challenges.
The coding environment was easy to use.
The coding environment did not present technical challenges.
I found it easy to navigate between the course environment, the coding environment, and the Riff Platform.
I was able to use all the tools in the course without any difficulty.

A2. Key Results

General

73% of course close survey respondents would recommend the course to a friend
89% reported being Satisfied or Very Satisfied with what they learned in the course
63% felt that activities and assignments aided their learning
92% felt well or very well supported by course staff and mentors
Net Promoter Score (NPS) = 62

Collaborative exercises

85% of survey respondents reported that the exercises helped them connect with peers
65% of survey respondents reported that the exercises aided their business understanding

Appendix B—Definitions

Correlations: A correlation value quantifies the relationship between the values of two numeric variables. It ranges between -1 and +1, indicating negative and positive relationships, respectively. A correlation of 0 (or near-zero) means there is no discernable relationship between the two outcomes that are measured. As an example, if the correlation between Riff Video usage and passing a course is 0.70, that means the more a participant uses Riff Video, the more likely it is that this person will pass the course.

Odds Ratios: The odds ratio ranges from 0 to infinitely large. A ratio above 1 means that the odds improve when the change is made. A ratio below 1 means the odds get worse.
A ratio of 1 means the odds didn’t change at all. As an example, if the odds of passing the course for a student who never used the video were 1 to 5, and the odds of passing among students who used the video exactly once were 3 to 7, then the odds ratio associated with exposing a student to one additional video call is [(3/7):(1/5)] = 15/7, which is a little over 2. This would mean that the odds of the student passing the course roughly double when they are exposed to one additional video call. Notice that this is not the same as saying they are twice as likely to pass the course. It says the odds got twice as good.

Significance: The “p-value” is used to quantify the significance of a relationship. The smaller the p-value, the stronger the evidence that some null hypothesis (stating that a relationship between two variables does not exist) can be rejected. Typical threshold p-values (to reject a null hypothesis) are 0.001, 0.01 and 0.05. The strength of the relationship can be determined based on which threshold the p-value falls under.

Acknowledgments

This research is based upon work supported by the National Science Foundation under Grant No. 1843391, SBIR Phase I: Positive Effects of Feedback and Intervention for Engagement in Online Learning.

About the Authors

Beth Porter is the CEO of Riff Analytics. Formerly, Beth was the VP of technical platform at Pearson Education and VP of product engineering at edX. Beth has led research and development of products that seek to transform online teaching and learning, including driving the Open edX initiative at edX and architecting the original Texas OnCourse program. Her engagements in higher education include lecturer at Boston University Questrom School of Business.

John A. Doucette is a research scientist. He was an employee of Riff Analytics at the time this research was conducted. Prior to working at Riff Analytics, John was an assistant professor at New College of Florida. He holds degrees from the University of Waterloo, and Dalhousie University.

Andrew Reilly is a data scientist at Riff Analytics, where he drives data-centric product development. He is currently a student at New College of Florida completing a Master’s degree in Data Science, and holds a Bachelor’s degree in Psychology from Central Washington University. Prior to working with Riff, he worked as a graduate intern providing business analysis to the Economic Development Corporation of Sarasota County.

Dan Calacci is a PhD student at the MIT Media Lab studying how data and algorithms impact community behavior and governance. Their recent work involves studying how new sources of data and its stewardship can help communities and worker collectives advocate for a more just future. Their artwork and research have been exhibited and presented around the globe.

Burcin Bozkaya is a professor and director of data science at New College of Florida. Prior to joining New College, he worked as a professor of business analytics and director of Behavioral Analytics and Visualization Lab at Sabanci University and a visiting professor at MIT Media Lab. His main research interests include behavioral analytics with big data, computational social science, spatio-temporal modeling and analysis and vehicle routing and logistics.

Alex Pentland, professor at MIT, directs the MIT Connection Science initiative, and previously co-created the MIT Media Lab. He is one of the most-cited computational scientists in the world, member of the National Academy of Engineering, and co-leads the IEEE Council for Extended Intelligence. He is a co-founder of over a dozen companies, including Riff Analytics.

Issue Nineteen

The Help Desk as a Community-Building Tool for Online Professional Development

Salome Apkhazishvili, Media Education Lab

Serene Arena, Independent Scholar

Renee Hobbs, University of Rhode Island

Abstract

COVID-19 safety measures have forced professional development programs to pivot to online environments, which affects how participants interact and collaborate. When the University of Rhode Island hosted their annual, week-long teacher professional development event as a fully-online program, the staff of the Summer Institute in Digital Literacy provided an online, real-time help desk service, knowing that some participants would benefit from targeted, individualized support. Using evidence from the help desk incident log and post-event qualitative interviews, this research deepens understanding of what teacher professional development can look like in online environments. Through the provision of personalized, real-time assistance that created a relationship between the participant and the staff member, those who used the Help Desk reduced their feelings of isolation, increased a sense of connectedness, and demonstrated agency as co-learners in a professional development learning experience. By providing intrapersonal, technical, and navigational support, the help desk deepened a sense of community connectedness in an online professional development program for educators who faced a dramatic pivot to online learning as a result of the COVID-19 pandemic.

Closures and physical distancing measures due to COVID-19 have shifted the way we interact, forcing many organizations to eliminate programs in teacher professional development (TPD) or move them to online platforms for the first time. In this shift, educators have faced some obstacles and adjustments. Although online learning is not a new model for digital literacy education, the COVID-19 pandemic has changed how and to what extent educators are expected to utilize online platforms for learning and community, bringing with it challenges to and opportunities for growth.

Given this backdrop, we look to understand how current research in TPD translates for fully-online experiences, exploring principles of community-building to understand the affordances of online learning. Importantly, our work seeks to understand the possibility of successfully applying known, effective in-person practices to online learning and professional development. This study documents a key feature of the 2020 Summer Institute in Digital Literacy (SIDL), a TPD program affected by COVID-19 restrictions. In its eighth year, SIDL was held completely online for the first time, gathering around 150 participants—mostly from the United States but including more than two dozen from 10 countries around the world. Educators, school leaders, researchers, librarians, and media literacy advocates come together annually for the week-long intensive program to learn about digital literacy, practicing skills and instructional techniques that support student learning via digital platforms (Hobbs and Coiro 2019; 2016). When pandemic restrictions emerged in March, program planners decided to use a combination of synchronous and asynchronous learning, using a learning management system plus video conference meeting rooms, along with flexible scheduling

Because of the intensive nature of the program, with its focus on hands-on media production activities and the activation of digital literacy competencies, they also decided to add an online help desk component to act as a support mechanism. The help desk would rely on a dedicated Zoom video conference room and text service (Google Voice) staffed continuously to offer hands-on, real-time support throughout the six-day, 42-hour event. By visiting the Lounge/Help Desk, participants could hang out and engage in informal dialogue but also get questions answered or receive individualized coaching.

In this study, we aim to better understand the value of the SIDL Lounge/Help Desk as a component of a teacher professional development program. Through the provision of personalized, real-time assistance that created a relationship between the participant and the staff member, we wondered if it could replace the “elbow-to-elbow” support that the program embodies when implemented in face-to-face learning contexts, where faculty and participants work side-by-side to create to learn (Hobbs and Coiro 2016).

Literature Review

The academic scholarship most relevant to this work focuses on the characteristics of professional learning environments that address the identity of teachers as learners and the role of help desks in community-building for both face-to-face and online learning contexts.

Teachers as co-learners

COVID-19 restrictions have required educators to adopt online teaching methods not as an option but as a necessity, and the suggestion that “what works in effective traditional learning environments may or may not work in online environments” has proven true in the forced remote learning of the 2020 pandemic (McCombs and Vakili 2005, 1582). In these unusual circumstances, teachers must “unlearn” traditional concepts in order to be receptive to new approaches that work better in online settings. While some teaching and learning habits are useful, they can also be detrimental, especially in unpredictable and unstable moments in time. Not only must educators learn new forms of social engagement, they must also “unlearn habits that have been useful in the past but may no longer be valuable to the future” (McWilliam 2008, 263).

One of the most dynamic settings where a teacher can embrace the identity of the learner is a TPD program. Ann Lieberman (1995, 592) argued for teachers to be actively involved in their own learning, noting that “the ways teachers learn may be more like the ways students learn than we have previously recognized.” When teachers actively learn from each other, they may create communities of practice where participants share, reflect on, and build new knowledge (Darling-Hammond et al. 2017; Desimone 2009).

During professional development, educators are placed in student roles, where they may enter into a “troubling zone” that can be also described as a discomfort, and it is this discomfort that helps to build a critical inner reflection leading to openness and empathy (Fasching-Varner et.al. 2019).

In online learning contexts, the ability to critically reflect on the identity of the learner is crucial for the design of effective TPD (Baran, Correia, and Thompson 2011). A profound learning opportunity can be created by the temporary disequilibrium caused by switching from “expert” to “learner” (O’Mahony et al. 2019). An aggregate review of how to improve TPD for online and blended learning confirms this, stating that teachers must have “the opportunity to reflect on the roles that they ascribe to themselves and their students in (online) environments” (Philipsen et al. 2019, 1157). This empathy for learners creates critical awareness that can be used during times of acute situational adjustments, such as with COVID-19.

Help desks as spaces for online community building

Online learning creates many opportunities for communities to form. Smith (2013) notes that community is variously developed by place, interest, and communion and is built through tolerance, reciprocity, and trust. But community doesn’t make itself: Connections are made through interaction, thus enabling people to build those communities (Smith 2013).

So what does one do when interaction becomes virtual, such as occurred during the COVID-19 pandemic? Coryell (2013) contextualizes collaborative and comparative inquiry in cross-cultural adult learning by framing learning as participation (partaking in knowledge), rather than learning as acquisition (possessing it). In referring to Sfrad’s (1998) work, she argues that in learning-as-participation mode, learners recognize knowledge as an interactional journey (Coryell 2013).

Community cannot exist without shared experience, and TPD programs must activate a sense of community if they are to be successful. A sense of community informs the formation of collective identity, which “is demonstrated when group members work interdependently with a shared purpose and responsibility for collective success” (Vrieling et al. 2018, 3).

Help desks may be spaces that support collaborative learning. In the field of information science, computer help desks located in universities have been studied to understand their organizational or technical functions, with focus on staffing, training and other issues. Some researchers have explored how help desk activity is used to create, manage, and share knowledge (Halverson et al. 2004). But there is limited research on collective identity, participation, or co-learning in help desk scenarios. Only one study is especially relevant to our work: it looked at what kind of learning takes place between those who need support and those who offer it. In this help desk research, a consistent sequence of four phases emerged to support communication, learning, and engagement in a face-to-face help desk. The phases included the processes of introduction, knowledge establishment, conceptual change, and agency. Findings showed that these interactions (consisting of two professionals of different expertise) activated metacognition, a type of reflection, leading to learner agency and personal fulfillment (O’Mahony et al. 2019).

With this understanding of community-building via help desks, we can consider the unique opportunities and challenges of online learning environments, including for TPD. As a result of the rise of social media, digital interaction has become normative for most people around the world. Yet for many educators, online learning has been thought to be inferior to face-to-face learning. For example, researchers who conducted a meta-analysis of various TPDs and how they affect student outcomes found that TPDs with online components yielded lower student achievement than programs that were entirely face-to-face. Yet, in that same study, several online learning practices were associated with gains, including having space to “troubleshoot and discuss implementation” of digital tools (Hill et al. 2020, 54).

To prepare teachers for online learning, online TPD may be a powerful treatment. But an understanding of the full potential of online TPD is still in development. Based on participant comments regarding collaborative and face-to-face engagement in Collins and Liang’s (2015) study of online TPD, little advancement in both the approach and implementation of these programs seems to have occurred. They report:

A number of individuals expressed they did not find OTPD as effective or meaningful as traditional face-to-face protocols…hardly anyone mentioned the online environment as engaging or encouraging participation through support or collaboration. A high number explicitly expressed that interaction was lacking … and many reported that even though they appreciated online delivery and its accessibility … they still missed the dialogue and collaboration of face-to-face PD. (Collins and Liang 2015, 28–29)

Online learning pedagogies are still primarily viewed through a prism of limitations when it comes to community-building. But scholars and practitioners are beginning to reimagine the use of technology and digital devices for collaborative learning. Bhati and Song (2019) conceptualize the creation of a dynamic learning space (DLS) in combination with mobile collaborative experiential learning (MCEL) as a means to encourage “high-level learning” and personalization. To our knowledge, approaches that level up these experiences by using the collaborative value of peer-to-peer synergy—proven instrumental to successful social learning—have not yet been studied.

Research Methods

The purpose of the paper was to understand the role of the help desk in online TPD as a form of informal learning and community building. Because this is a form of exploratory research, we asked: How did adult learners experience the value of an online help desk in the context of teacher professional development?

Participants and program context

The Lounge/Help Desk was fully-integrated into the Summer Institute in Digital Literacy (SIDL), the six-day, 42-hour TPD program, which included 135 participants and fifteen staff members. SIDL is an established program with a long history (Hobbs and Coiro 2019; 2016) but 2020 was the first time the TPD program was offered as a fully online program. Thus, many features of the program required adaptations that were new to the event organizers, faculty, staff, and returning participants.

The SIDL Lounge/Help Desk was conceptualized as an informal gathering space, where participants could go to get help—but also to interact with other participants and staff. Describing the Help Desk as a lounge was also intentionally designed as a means to reduce the stigma of asking for help. Participants were reminded of the Lounge/Help Desk every day. Each morning of the six-day program, participants received an email with the links to the learning management system, where links to video conference Zoom rooms and the Lounge/Help Desk were provided. The first and second authors were responsible for staffing the Zoom room Lounge/Help Desk, and the third author served as their supervisor.

The Lounge/Help Desk was both a synchronous and asynchronous communication channel for program participants and faculty, open to join at any time throughout event hours (9 AM–5 PM). Participants joined the Zoom Room or sent texts or emails, and these were handled throughout the day as the TPD program was in operation. Program faculty also participated in the Lounge/Help Desk, joining the online Zoom room for 1–3 hour shifts. In cases where the staff could not answer questions, one member would reach out to program organizers via a private Signal chat, which was used as a backchannel tool, in order to gain information needed to answer questions or solve problems.

As Lounge/Help Desk staff members, we gradually came to recognize that we were teachers in the TPD program and that our role was truly educational. We were not just providing a transactional service: Through our interaction, we were demonstrating the depth of community building that is at the heart of the SIDL program (Hobbs and Coiro 2019). People came to the Lounge/Help Desk needing different kinds of personalized support. Some were clearly beginners in their use of technology, while others had considerable expertise. But each of these individuals were people that we had a chance to interact with and learn from; during other components of the program we sometimes encountered them, particularly in small breakout groups and informal discussions. Indeed, it was the awareness of our own experience as co-learners with the participants that inspired our interest in this research project.

Data collection and analysis

Incident Log

During the program, we logged every visit to the Help Desk in an incident log to identify each time a participant visited the Zoom room or interacted via Google Voice text messages. During the real-time TPD program, this practice helped event organizers to understand participant pain points for particular learning activities that involved digital media and technology. It also functioned to help staff contact participants when reaching out to those whose questions could not be resolved in real time. The log documented: who contacted the help desk; who assisted them; what the question or problem was; and how the resolution occurred. The incident log was not initially designed for research, as we merely imagined its function as a tool for formative assessment during the program implementation.

During the program, the Help Desk Zoom room was accessed 76 unique times by 41 different participants. Fourteen text messages were sent to Help Desk staff. In our first phase of data analysis, the first and second authors used data from the incident log to categorize our encounters with participants. We worked independently to develop categories to account for the variety of interactions in order to increase divergent interpretations and reduce confirmation bias. By reviewing the categories created by each researcher using simple description, we identified emerging themes like: “emotional support needed after confusion caused by new platforms,” “tech glitches,” and “wanting to be told what to do.”

Interviews

After the event, we reached out to 41 participants who had used the Help Desk and eight agreed to participate in a research interview; one male and seven females. In terms of race and ethnicity, six participants were White, one Black, and one Latina. Seven participants were from the United States, while one participant was from Great Britain. The average age of the participants varied from 40 to 65. The demography of the research participants closely represents the SIDL demography, with the majority of participants white, female, and based in the United States.

The interview was conducted through Zoom and included ten scripted questions regarding the participant’s experience using the Help Desk. Participants were asked to describe what led up to their decision to access the Help Desk, the emotions they could recall at play before, during, and after its use, and how the experience compared to other help desk services they may have experienced in the past. Interviews were conducted three weeks after the event. The University’s institutional review board approved the research and participants gave permission for audio recording.

In the second phase of data analysis, we analyzed both the transcribed interview data from the individual interviews and the incident log data collected during the TPD program. The interview data helped to more deeply contextualize the documentation in the incident log. For example, interviews suggested that areas first coded as “tech glitches” may also relate to “confusion,” and that participants who we initially perceived to be “needing to be told what to do” were navigating the social loss of community interaction.

Findings

Three themes emerged from this work which give insight into how informal learning was experienced in the context of using a Help Desk during an online teacher professional development program. Participants came with a variety of very specific questions and problems during the week-long program. Of the 76 visits to the Help Desk, many were easy to answer, requiring only a few minutes. Examples of these include finding a link to a Zoom room, recalling a password, or noting the day’s agenda and schedule. These were often merely a matter of visiting a web page and clicking a link.

But some questions required some additional form of co-learning as Help Desk staff needed to answer a question by modeling a learning process with a participant. Some of the questions that participants asked could not be easily answered by Help Desk staff. For example, one participant needed help learning how to edit a post on Wakelet, a digital curation tool, while another wanted a tutorial on ThingLink, a visual annotation tool. Neither staff member was familiar with these digital tools but both were able to demonstrate co-learning with participants to answer their question or solve their problem. Another participant struggled to find a solution to the microphone on her laptop, which suddenly stopped working. In each case, the Help Desk staff demonstrated through inviting the participant to share their screen, using coaching that enabled participants to solve their own problem with scaffolded support from a member of the staff. For questions that Help Desk staff could not solve on their own, they explained and modeled how they reached out for help from the larger faculty team. In those cases, staff were able to find answers within an hour or two of the request being made. Considering the nature of the help provided in the context of the participant interviews, we found that many of the Help Desk encounters created a rich interpersonal relationship between participant and staff member that functioned to reduce isolation, deepen a sense of community, and increase learner agency.

Co-learning as a journey borne of isolation

The Lounge/Help Desk reinforced the perception that the TPD program was a co-learning journey that involved the participants and the staff as collaborators. Many participants (and program faculty) were experiencing online TPD for the first time; it was a new experience for everyone.

While describing initial feelings and the scenarios leading up to accessing the Lounge/Help Desk, participants mentioned experiencing “confusion,” “nervousness,” and “anxiety.”[1] For example:

“Before [coming to the help desk], it was confusion and a little bit of…I wouldn’t go as far as to say panic, but close.”
“I was a little lost a couple of times in terms of where I was supposed to be going.”

In the TPD program, the novelty of a fully online event was made even more intense by the expectation that participants would be practicing the use of new digital tools, including Pathwright LMS, Adobe Spark, Padlet, and many other platforms. This may have exacerbated concerns that participants naturally have in new learning scenarios, except that, instead of being able to organically turn to the person next to you and ask questions, participants were, in that moment, alone.

Interview data clearly reveals that awareness of a sense of isolation was a precipitating incident. Participants noted feeling confused about “where” to go and when, unsure of which “Zoom room” they belonged in. At various points during the week, there was uncertainty regarding task details and/or deadlines for completion. These are common in learning environments, and the accessibility of the Help Desk acted as a bridge in lieu of the missing opportunity to “turn to your neighbor,” thus helping participants keep involved and engaged.

Some veteran SIDL participants (attending for a second or third time) hesitated in reaching out to the help desk out of concern for others, downplaying their own need for support. Feelings of demoralization and inadequacy were also referenced in the moment of realizing help was needed.

“[Y]ou think, ‘should I know the answer to this—is this something I can figure out myself?’ … my hesitancy was that people might need [the Help Desk] more than I did.”
“Everybody sort of doesn’t want to take time away from other people or you don’t want to bother people. So there’s always that, but I felt more comfortable using it after I used it the first time…”
“The feeling before I joined the lounge was ‘I’m “supposed” to be doing this, but I can’t.’”

One participant said that she felt much more comfortable coming to the help desk when she realized she knew one of the staff members. Clearly, such relationships and bonds can support not only successful learning but also continued community development.

Co-learning as a journey to connect

The decision to share Google Voice numbers with participants offered additional options to connect with the Lounge/Help Desk staff through calling or texting. One participant noted this as particularly helpful; as a non-native English speaker, it was easier for her to write her question. Because the help desk was continuously available during the six days of the program, it created a sense of immediacy, efficiency, and effectiveness, as participants saw how the help desk embodied the empathy of the program’s tagline: “Everyone Learns from Everyone,” a phrase that made adult learners feel welcomed as peers (Hobbs and Coiro 2019). For example, participants noted:

“The people there were very helpful and compassionate … about leading me through where something was and actually, one time, the assistant was confused as well. They didn’t quite know where to go. So we were learning together—how to navigate the site. So it felt like a very welcoming place.”
“I was very reassured. I was helped immediately; I wasn’t kept waiting … and I felt as though my concerns were being dealt with.”

Many participants had experienced help desks at their workplace or school. There, they encountered a generally asynchronous system: submit query, wait for response, hope for solution. But the SIDL Lounge/Help Desk was different. Participants who reached out for help mentioned appreciating the immediacy and liveliness of the help desk interaction. The help desk was an online “place” for congregation; after all, it doubled as The Lounge. Participants noted:

“Having a real person to talk to is a bonus. It’s better than either a chatbot or talking with somebody online—having somebody to actually talk to and have working through it is definitely a good thing.”
“When you contact a regular help desk, you feel like you’re just lost—your request is out there; you may or may not hear from anybody. That wasn’t the case here.”

During the interaction, some participants realized their initial confusion was a result of inattention. In being able to focus and talk through a concern and visualize it on a shared screen with the help desk staff, participants gained awareness of what they had overlooked. As they worked together, the missing piece of information would often be noticed by participants themselves. The sense of pleasure in solving a problem transformed the sense of isolation into a shared experience.

Co-learning as a journey toward agency

Interview subjects described the calm and confident feelings they experienced upon resolving their questions or concerns through the help desk interaction. Important to supporting this sense of agency was the ability for both the staff and the participants to share their screens. Screen-sharing enabled help desk staff to model the iterative process of learning to use digital platforms and the shared experience of confronting and solving a problem together built trust and independence for the participants. For example, participants noted:

“I could see things that I needed to see and know that I wasn’t missing anything.”
“Afterward, I had very clearly seen where to go. So it was a sense of relief that now I could do that by myself.”
“I learned that it wasn’t as complicated as I thought it to be. And that there was more than one way to approach the issue we were having.”

Almost all interviewed noted how their own struggles aligned with what their students may experience with online learning. In fact, contrary to Collins and Liang’s (2015) suggestion that honoring the adult is part of effective PD (the idea that while learners, they are first and foremost experienced adults and professionals), we found that participants who could embrace the role of learner—complete with the requisite insecurities, needs, problems and questions—gave them the opportunity to deepen empathetic connections to their own learners. This is one way to understand how an online help desk can provide value to adult learners in the context of teacher professional development.

We found that three forms of support—intrapersonal, technical, and informational—all contributed to increased participant agency as co-learners. Intrapersonal support occurred as participants entered the Lounge/Help Desk with strong feelings, the full range of feelings that manifest when something does not work as expected or when obstacles occur. Emotions varied from frustration to panic. Sometimes, these feelings emerged from intra-actions related to self-imposed expectations; in other cases, external pressures like time constraints were activating strong emotion. Feelings often coincided with information structure and technical scenarios, as when one is distracted or flustered and forgets simple things like how to log in. The ability to acknowledge and validate participant concerns in real-time provided an immediate sense of relief to participants—even when a solution wasn’t immediate.

Technical support included both hardware, software, and online platform glitches, as well as password problems. During the week-long program, a variety of forms of basic IT support was provided, such as updating software, changing passwords, checking settings, and restarting computers. In one instance, the Help Desk assisted a participant who was experiencing prohibitive technical problems (e.g. a poor network connection) by emailing PDF copies of online content. Participants learned more about their digital devices from the transparent way in which these forms of support were modeled by staff.

Navigation support was provided to participants in helping them find what they needed using the learning management system, which was unfamiliar to them. Help desk staff demonstrated how to find specific information, and in the process, they recognized that some of the challenges that participants were experiencing was the result of errors made by program staff, including mislabeled or broken links or poorly expressed language or wording. The help desk participants enabled the TPD faculty to recognize weaknesses in their own explanations of program activities. For example, in one instance, a set of Zoom links were presented using a red font color, which led them to be easily overlooked on a page full of text, even as the red color was intended to make them stand out visually. Help desk staff thanked participants for calling attention to the problem—but participants were equally grateful, expressing feelings of relief as they realized the problem was not “their fault.”

By supporting participants emotionally, technically, and navigationally, feelings of community emerged, because despite the lack of face-to-face encounter in this fully online TPD program, participants felt taken care of. As one experienced participant put it:

all the things that I think made Summer Institute special for me (in-person in past years) … were present this year … And the Help Desk was part of that. So the Help Desk was an even bigger part because without it, SIDL couldn’t have flowed—somebody could get lost.

Through the provision of personalized, real-time assistance, those who used the Lounge/Help Desk reduced their feelings of isolation, increased a sense of connectedness, and demonstrated agency as co-learners in an online professional development learning experience.

Discussion

Our findings provide strong support for the ability of help desks to function as vital components of online teacher professional development programs. SIDL’s Lounge/Help Desk enabled participants to move through an arc of learning-as-participation that not just supports but enhances learning. Rather than conceptualizing the help desk as a merely transactional experience, at the 2020 Summer Institute in Digital Literacy, it functioned as a meaningful part of the overall learning experience.

Of course, this study has several limitations: the small sample size and potential respondent and research bias must be considered as limitations, given the researchers’ own roles as staff during the TPD. We aimed to minimize this limitation by developing the initial analysis of the incident log separately in order to increase divergent interpretations and minimize confirmation bias. We recognize that our ideas of community-building in TPD are framed through an American, Westernized cultural lens, though effort was made to review work from across the globe. The research reviewed for this study is gleaned mostly from abled/neurotypical interactions of spoken or auditory communication, potentially limiting outreach and input.

This research makes a unique contribution to new knowledge by re-framing the online help desk as a novel feature of teacher professional development. Because the online help desk was available throughout the TPD, it functioned to engage participants much like in face-to-face interactions, qualifying it as space to troubleshoot and discuss implementation, a category found to be successful in creating student learning gains from teachers’ TPD learning (Hill et al. 2020).

Key features of the Help Desk design were critical for its use as such an informal learning space: it was called the Help Desk/Lounge, and it was designated specifically as a hangout place online, thus reducing the stigma of being perceived as a place for “people who need help.” For those educators with insecurities about their digital competencies, there was no shame associated with visiting the Help Desk. Thus, it connected and strengthened the program’s core value of “Everyone Learns from Everyone” (Hobbs and Coiro 2018).

The potential to build personalized engagement is another feature needed for a help desk to be part of successful TPD. As designed and implemented, the Help Desk provided the situational context needed to question and solve problems immediately and in real time, running in parallel to the formal program. It also exposed pain points in the event and platform infrastructure, offering a form of continuous evaluation of the TPD experience and enabling event producers to make adjustments during the event itself, further enhancing the program’s overall quality. This tailored approach, so aligned with teacher needs and experiences during COVID-19, enhanced the TPD’s sense of relevance for participants, a requisite dimension of effective training (Stein et al. 2011). The Lounge/Help Desk contributed to this sense of relevance by engaging one-on-one with individuals on the emotional, technical, and navigation challenges they were likely to face as educators heading into an unparalleled 2020-21 school year. The process of engaging with a help desk that offered individualized support offered participants the opportunity to develop understanding of possible hiccups that may be encountered in their own classes and the confidence to troubleshoot these problems themselves. This finding aligns with research that demonstrates the value of helping educators critically reflect on how they approach their work and consider their roles in the educational dynamics of learning (Baran et al. 2011).

While some researchers claim that TPD support must come “from an educational technologist or an expert within the field” (Philipsen et al. 2019, 1155), we found that a help desk intentionally staffed as a peer-supported environment was effective in modeling how to investigate problems together. In such paradigms, trust helps to bridge the implied power dynamics between the helper and the “helped.” Because the help desk staff positioned themselves as participants and partners in the process, they offered the support for collaboration so valued as a critical ingredient for teacher learning (Bates and Morgan 2018; Darling-Hammond et al. 2017) As Bates and Morgan (2018, 623) point out, “a co-learner stance” ultimately contextualizes and personalizes support, guaranteeing “that actual problems are addressed.” The question moves from an individual, isolated/ing concern to a social learning opportunity, something Vygotsky (1978) addresses as essential to meaning-making.

By viewing an online help desk as a shared learning experience with value as a programmatic feature of TPD, we will need to consider how it could be adapted in post-pandemic times, as teacher professional development returns to be provided in face-to-face contexts. The help desk offers the value of providing that “in the moment” experience for individualized grappling and reflecting on problems, helping to meet the needs of every learner. Because the online format was new to everyone involved, including the help desk staff, the co-learning journey in finding answers offered value to faculty, staff and participants alike. Although it was intended to provide individualized support for those experiencing technology problems, the Lounge/Help Desk actually became a part of the overall TPD experience, enabling it to be a programmatic feature that extended the value of the Summer Institute in Digital Literacy as a genuinely collaborative learning experience.

Notes

[1] The quotations in this section come from research interviews with 2020 SIDL participants (names withheld) and were administered by Salome Apkhazishvili and Serene Arena in August, 2020.

Bibliography

Bates, Celeste C., and Denise N. Morgan. 2018. “Seven Elements of Effective Professional Development.” The Reading Teacher 71, no. 5. (Mar/Apr): 623–626. https://doi.org/10.1002/trtr.1674.

Baran, Evrim, Ana-Paula Correia, and Ann Thompson. 2011. “Transforming Online Teaching Practice: Critical Analysis of the Literature on the Roles and Competences of Online Teachers.” Distance Education 32, no. 3: 421-439.

Bhati, Abhishek, and Insu Song. 2019. “New Methods for Collaborative Experiential Learning to Provide Personalized Formative Assessment.” International Journal of Emerging Technologies in Learning (iJET) 14, no. 7. http://doi.org/10.3991/ijet.v14i07.9173.

Coryell, Joellen E. 2013. “Collaborative, Comparative Inquiry and Transformative Cross-Cultural Adult Learning and Teaching: A Western Educator Metanarrative and Inspiring a Global Vision.” Adult Education Quarterly 63, no. 4: 299–320.

Collins, Linda J., and Xin Liang. 2015.“Examining High Quality Online Teacher Professional Development: Teachers’ Voices.” International Journal of Teacher Leadership 6, no. 1 (Fall): 18–34.

Darling-Hammond, Linda, Maria E. Hyler, and Madelyn Gardner, with assistance from Danny Espinoza. 2017. Effective Teacher Professional Development. Palo Alto: Learning Policy Institute.

Desimone, Laura. 2009. “Improving Impact Studies of Teachers’ Professional Development: Toward Better Conceptualizations and Measures.” Educational Researcher 38, no. 3: 181–199.

Fasching-Varner, Kenneth J., Michaela P. Stone, Roberto M. Mella, Francisco O. Henriquez, and Macarena Y. Palma. 2019. “‘…4542 Miles from Home…’: Repositioning English Language Learners as Power Brokers and Teachers as Learners in the Study Abroad Context.” Education Sciences 9, no. 2 1–13. MDPI. http://dx.doi.org/10.3390/educsci9020146.

Halverson, Christine A., Thomas Erickson, and Mark S. Ackerman. 2004. “Behind the Help Desk: Evolution of a Knowledge Management System in a Large Organization.” In Proceedings of the 2004 ACM conference on Computer Supported Cooperative Work, 304–313.

Hill, Heather C., Kathleen Lynch, Kathryn E. Gonzalez, and Cynthia Pollard. 2020. “Professional Development that Improves STEM Outcomes.” Phi Delta Kappan 101: 50–56.

Hobbs, Renee, and Julie Coiro. 2019. “Design Features of a Professional Development Program in Digital Literacy.” Journal of Adolescent & Adult Literacy 62, no. 4: 401–409.

Hobbs, Renee, and Julie Coiro. 2016. “Everyone Learns from Everyone: Collaborative and Interdisciplinary Professional Development in Digital Literacy.” Journal of Adolescent & Adult Literacy 59, no. 6: 623–629.

Lieberman, Ann. 1995. “Practices that Support Teacher Development.” Phi Delta Kappan 76, no. 8: 591–596.

McCombs, Barbara, and Donna Vakili. 2005. “E-Learner-Centered Framework for E-Learning.” Teachers College Record 107, no. 8: 1582–1600.

McWilliam, Erica. 2008. “Unlearning how to Teach.” Innovations in Education and Teaching International 45, no. 3: 263–269.

O’Mahony, Timothy, Jason Petz, Jonathan Cook, Karen Cheng, and Marco Rolandi. 2019. “The Design Help Desk: A Collaborative Approach to Design Education for Scientists and Engineers.” PLoS ONE 14, no. 5: e0212501. https://doi.org/10.1371/journal.pone.0212501.

Philipsen, Brent, Jo Tondeur, Natalie Pareja Roblin, et al. 2019. “Improving Teacher Professional Development for Online and Blended Learning: a Systematic Meta-Aggregative Review.” Education Tech Research Development 67: 1145–1174. https://doi.org/10.1007/s11423-019-09645-8.

Smith, Mark K. 2001, 2002, 2013. “Community.” The Encyclopedia of Pedagogy and Informal Education. https://infed.org/mobi/community/.

Stein, Sarah J., Kerry Shephard, and Irene Harris. 2011. “Conceptions of E-Learning and Professional Development for E-Learning Held by Tertiary Educators in New Zealand.” British Journal of Educational Technology 42, no. 1: 145–165. https://doi.org/10.1111/j.1467-8535.2009.00997.x.

Vrieling, Emmy, Antoine van den Beemt, and Maarten de Laat. 2019. “Facilitating Social Learning in Teacher Education: A Case Study.” Studies in Continuing Education 41, no. 1: 76–93.

Vygotsky, Lev S. 1978. Mind in Society: The Development of Higher Psychological Processes. Cambridge, Massachusetts: Harvard University Press.

About the Authors

Salome Apkhazishvili is a media and communication researcher from the country of Georgia where she coordinates the media and digital literacy program for the conflict-affected youth in the South Caucasus. She is a Fulbright communication graduate from the University of Southern Indiana. Apkhazishvili is a communications officer at the European Communication Research and Education Association Children, Youth, and Media section and a staff member of the Media Education Lab.

Serene Arena is a communication design expert focused on language use and collaborative development in communication and social systems. She has a Masters in Civic Media from Columbia College Chicago, where she studied social power dynamics and informal social spaces as foundations for community and personal identity.

Renee Hobbs is a professor of communication studies and director of the Media Education Lab at the University of Rhode Island’s Harrington School of Communication and Media. She has offered professional development to educators on four continents and authored 12 books and more than 150 scholarly publications on digital and media literacy.

A flow chart demonstrates an app's information architecture

Issue Nineteen

Interdisciplinary Approach to a Coping Skills App: A Case Study

Antigoni Kotsiou, The Pennsylvania State University

Erica Juriasingani, The Pennsylvania State University

Marc Maromonte, The Pennsylvania State University

Jacob Marsh, The Pennsylvania State University

Christopher R. Shelton, The Pennsylvania State University

Richard Zhao, University of Calgary, and

Lisa Jo Elliott, The Pennsylvania State University

Abstract

Rich learning opportunities exist when academic departments reach beyond their discipline. During the COVID-19 pandemic, we organized an interdisciplinary team to create a mobile app to measure and support mental health through better coping skills education in the local community of Erie County, Pennsylvania. Guidance on how to develop a professional level organization at an undergraduate academic institution for app creation is sparse. Best practices in developing this environment are needed. This article describes how we, a team of four educators and three students, created the mobile app. The process mimicked a professional development team with many adjustments. The arrangement of the team and the process taught the students teamwork and gave the educators an opportunity to collect meaningful data on the local population. The methodology included adaptations from industry in a project planning guide, requirements gathering processes, user testing processes, prototyping and iterations. Development encountered several unanticipated challenges with the need for two institutional review board approvals, consultation with an attorney, hosting challenges, and Google Play Store hurdles. We suggest that future academic teams plan for these challenges at the outset. This interdisciplinary experience is a complement to any digitally-oriented classroom and is a nice introduction for students to gain the needed skills to advance to Startup and Tech Accelerator programs already in place at many universities.

Introduction

Rich learning opportunities exist when academic departments reach beyond their discipline and engage with each other. Interdisciplinary approaches are key to success in independent business entities and they allow a team to “engage with their ideas, maintain productive interaction, and successfully implement these ideas” (Brodack and Sinell 2017, 10). Interdisciplinarity broadens the knowledge base of a project team, taking full advantage of the specialized knowledge of its members while avoiding the peripheral blindness often associated with such specialization. When managed correctly, creative problem-solving outcomes are enhanced (Moirano, Sánchez, and Štěpánek 2020), silos are merged, and focus limitations associated with specialization are removed (Blackwell et al. 2009). Working on a project within an academic environment is no different. Students have specialized knowledge and often a preconceived perception of a problem. Collaboration with those outside their field broadens their interpretation of and approach to a problem while allowing them to use their special skills with external input. Developing a mobile app requires the technical skills of computer scientists, the knowledge of subject matter experts, and the expertise of user experience researchers. Here, we present a case study of a successful app development process as a blueprint for others to follow with a discussion of tools, activities, time budgets, resources, challenges, and potential impact.

During the COVID-19 pandemic, two psychology faculty members (a clinical psychologist and an engineering psychologist) and one computer science faculty member collaborated along with the corporate liaison at the university to address the increased need for coping skills in the local and student community (Fernández et al. 2020; Naeem et al. 2020; Saltzman et al. 2020). We received a grant and hired two undergraduate students and one graduate student to work with us. This article documents our wins and lessons learned in order to help other academics bootstrap the process.

Research is clear on the significant negative effects resulting from the COVID-19 Pandemic. In addition to significant physical health risks, there are substantial increases in the rate that individuals are experiencing mental health symptomatology such as distress, anxiety, sadness, and isolation (Kar et al. 2020; Pierce et al. 2020). Unfortunately, there has been less focus placed on increasing our ability to address mental health concerns at a systems level even in the face of rising pathology (Kar et al. 2020).

As a result of the pandemic, there is a shift within the mental health field towards providing more services remotely (e.g., meeting with a therapist by webcam). However, that shift does not address the increasing demand for mental health services. Thus, there is a significant need to develop other digital avenues to try and reach individuals in need (Ho 2020). According to Ho (2020), apps developed for smartphones to provide users with psychoeducation, resources, and coping strategies may prove especially useful to help meet increased mental health needs during the pandemic. Previous research demonstrates the viability of using smartphones to integrate mental health services through technology. Digital apps have been created to monitor, record, and, in some cases, modify mental health, such as providing location-based services to alert users to the nearest mental health clinic, providing self-help mantras and guided meditations, and tracking mood ratings based on self-reporting (Luxton et al. 2011).

According to the Centers for Disease Control and Prevention (2021), engaging in appropriate coping strategies during the pandemic is important to maintaining one’s mental wellbeing. We developed the Serene app to help our students and the surrounding community cope with the pandemic while gathering information on the mental health of the community. Specifically, the app was developed to accomplish this goal through non-medical advice that engages users with behavioral activities, psychoeducation, motivational quotes, video exercises for relaxation and breathing, local and national professional resources, and connections to available, external, evidence-based mental health apps.

We set out to determine the best practices in developing a professional level organization for app creation. We anticipated the project would take four months, but it took eight months with an additional four weeks for the Google Play Store release and an additional eight weeks for media coverage.

Organization

Our group was separated into three pairs, each consisting of a faculty member and a student. Faculty members chose students in their discipline based on previous coursework, previous independent study, and their experience of the students in their courses. The faculty/student pairs are referred to as teams. The three teams were as follows:

UX (user experience) team (an engineering psychology professor and an undergraduate human factors psychology student who successfully completed the assignment in Appendix A),
Content team (a clinical psychology professor and a counseling graduate student), and
App development team (a computer science professor and an undergraduate computer science student).

The corporate liaison provided advice and guided compliance to the institutional mission and the funding agency’s mission.

User experience (UX) team

The UX team organized first to create the design for the minimally viable product (MVP) prototype. Eric Ries (2013) discusses the specifics of MVPs and how they can save development time. The UX student used her expertise in human factors and referred to the research-based best practices on Don Norman and Jakob Nielsen’s NN group website (Nielsen Norman Group 2020). Don Norman is a faculty member at the University of California San Diego and one of the forefathers of UX. Jakob Nielsen is an engineer and a principal at the NN group.

The UX and content students conducted a competitive analysis to discover what similar apps existed and what these apps provided to users as Jill DaSilva (2020) discusses. They created a spreadsheet of similar apps and their features. This was the basis of requirements gathering as Janet Six (2019) suggests. The team reviewed the spreadsheet and developed the requirements document using a version of the MOSCOW method (must, should, could, won’t) as discussed in ProductPlan (2020) and then refined this list. Some desired but untenable features were “connecting to a counselor on campus through a chat feature” and “talking to others using the app.” Both of these features would require infrastructure that was unavailable. Then, the UX student organized the architecture of the app discussed by Jen Cardello (2014) and used LucidChart to create the architecture as shown in Figure 1.

A flow chart showing the information architecture. — Figure 1. The information architecture.

After approval, the UX student generated pencil sketches of the screens and then developed the individual screens using the open-source material design pattern library (http://material.io/). Next, the student used the Invision App (https://www.invisionapp.com/) with a free educational license (https://www.invisionapp.com/education) to work out the navigation between the screens. At each stage of this process, her work was approved by the group. The prototype took two weeks longer than anticipated. The Serene app design is stored online at Invision (https://projects.invisionapp.com/share/8DXSLUJCRAK#/screens).

App development team

The development team participated in the discussions of the overall design, the design of the architecture, and the design of the UX. Following creation of the UX design by the UX team, the app development student created a prototype of Serene that followed the UX design and turned the prototype into a fully functioning app product using his expertise in computer science, following weekly discussions with the entire team. The development process consisted of the development of the back end, a Java server that handled the processing and storage of data, and the front-end, the app itself, built using HTML5 and JavaScript with Cordova (https://cordova.apache.org/), providing multi-platform support.

Content team

The content team helped the UX team to research similar apps. A spreadsheet was created to compare similar apps and their functionality. Following discussion with the whole team, the content student developed comprehensive resource lists to provide users with information regarding:

Mental health providers in Erie County, Pennsylvania. The list consisted of local agencies and organizations, their contact information, and the target population.
Nationwide mental health resources. The list consisted of national mental health hotlines and organizations for various populations.
Other mental health smart-phone applications. In collaboration with the UX team, a list was created with all the mental health applications that the team was able to find. The content team assessed the applications and chose a small number of evidence-based applications to suggest in the Serene app as additional applications.
Behavioral activities that users could consider doing. Based on psychological principles of behavioral activation (Kanter et al. 2010) to help increase well-being by remaining physically active, the content student used her expertise in counseling psychology to compile a list of various activities users could do across a variety of settings and circumstances (see Appendix B). Given some of the restrictions experienced due to the COVID-19 Pandemic, these resources provide users with ideas for activities they can engage in regardless of pandemic-related circumstances (e.g., socially distant outdoor activities or things to do at home if faced with a stay-at-home order).
Motivational quotes. Upon discussion with the whole team, it was decided that three categories of quotes (i.e., psychology quotes, I am… quotes, and motivational quotes) were needed. The content team sought to include at least 365 quotes in each of the three categories thereby ensuring a steady stream of new content (i.e., one new quote from each category for every day of the year) to help promote regular use of the app and gather information on how users were feeling. The final list consisted of approximately 380 quotes for each category.

The content student also used her expertise in counseling to research and write articles that provided users with evidence-based information regarding mental health and COVID-19, all accessible from within the Serene app. The mental health information discussed emotional reactions and stigmatization in mental health. The COVID-19 article was a comprehensive summary of the characteristics of the coronavirus, along with ways individuals can protect themselves. All sources used were either governmental (e.g., CDC and WHO) or other high-quality online resources (see Appendix C; e.g., information from the Bill and Melinda Gates Foundation). These articles and resources assist users in finding valid and reliable information about mental health and the COVID-19 pandemic.

Providing users with this type of information, also referred to as psychoeducation, is a very important component of multiple therapeutic models in mental health services. That is to say, we need to provide information related to the individual mental health concerns of mental health consumers in order to raise awareness and offer a sense of reality and control. Since the Serene app was created as a tool to assist its users with mental health struggles in isolation during the COVID-19 pandemic, our articles are meant to provide users with information about basic mental health concepts like stigma and emotions, as well as information about the coronavirus. Further, many unreputable online resources spread misinformation and inaccuracies that may confuse or even disturb individuals. Therefore, providing reliable resources and psychoeducation to users of the Serene app may also help to decrease potential distress that individuals may face if they were to search for and receive this same information from other, potentially unreliable or misinformative sources.

Finally, the content student’s expertise in counseling helped her to create mindfulness, meditation, and progressive muscle relaxation exercises that users could freely access within the Serene app. For this purpose, appropriate audio recording equipment (Studio Condenser USB Microphone with Adjustable Scissor Arm Stand) was purchased. Moreover, the content student searched online resources to find and adapt scripts and soundtracks for use within the app. The content student recorded the audio for each of the exercises, mixed the audio with background music and visuals, and uploaded each exercise to the team’s YouTube channel, for use within the app. The majority of the content student’s hours were spent in the creation of these exercises because significant time was needed for the student to locate appropriate scripts and soundtracks and to get familiar with the recording equipment and software. By the end, a five-minute recording would take approximately two hours to complete from start to finish.

Student Work and Time

As this was the first time that we had developed an application together, we had many questions about how much time the students should spend and what they should be doing during those hours. Based on our experiences, 50 percent of person-hours were devoted to the app development student, followed by 36 percent to the content student, and 14 percent to the UX student. To help future academic development teams determine a budget, we have included the actual student time/activities as concrete guidance.

UX team

Students were screened in an “Introduction to Human Factors” class on a prototype development assignment (see Appendix A). One of the challenges we had was budgeting for student hours. While these times may not work for every project, here is the time breakdown for the UX student.

UX student time	Activity
8 hours	Competitive app research
2 hours	Helping with content
2.5 hours	Information architecture
16.25 hours	Meetings—requirements development, review sessions, organizing the project
30 hours	Prototyping
8 hours	User testing and reporting
2 hours	Miscellaneous

Table 1. The time that it took the UX student excluding final user testing.

App development team

Students were screened in a computer science class where coding assignments were a major component. Students’ assignments were reviewed based on their performance, which included the correctness, efficiency, and organization of their written programming code. One student was invited to join the team based on his performance and his availability in the schedule of the development.

App development student time	Activity
12 hours	Back-end development, including the storage of data on a server and server setup*
205 hours	Front-end app development, including the app interface and the connection to the back end
6 hours	Miscellaneous
18 hours	Meetings

Table 2. The time that it took the app development student excluding submission of the final version to Google Play. *Server setup requires the support of the IT department at the institution, which may take days or weeks depending on the institution. This is not counted in the development table.

Content team

Given the mental health nature of the content to be created for this application, it was important to find a student with expertise in both the research and practice of clinical psychology. Therefore, the student for the content team was hand selected from a clinical psychology graduate program on campus. Prior to working on this project, the student worked as a research assistant for the content team lead. Through this work the student demonstrated several key qualifications for this position, including: a passion for mental health advocacy, a mastery of the material, and the ability to work efficiently and effectively both as part of the team and on an individual level.

Content student time	Activity
16 hours	Creating a database of county-wide mental health resources (e.g., providers) as well as select nationwide resources (e.g., the National Suicide Prevention Lifeline)
3.5 hours	Researching other mental health smartphone applications to list within this app to provide users with additional wellness resources
9 hours	Creating a list of behavioral activities users could access to help find things to do across a range of current circumstances (e.g., things to do at home, if faced with a stay-at-home order due to the pandemic; socially distant outdoor activities)
34 hours	Collating several lists of positive and inspirational quotes
91.5 hours	Producing video content for the app (e.g., mindfulness exercise videos)
16 hours	Attending team meetings
3 hours	Miscellaneous

Table 3. The time that it took the content student.

Project Development and Implementation

Initial development began in late April 2020 with team organization for the summer and an application for grant funding. Project planning was done using a mix of free templates. For planning purposes, the UX work and content work happened in the first three months. App programming began concurrently in the second month once the initial prototype screens had been determined. User testing began in the third month along with iterations to solve the issues that were discovered. Figure 2 outlines the order in which the development occurred as well as the stages of ideation, solidification, and implementation within each phase. Review was ongoing in each phase. Development finished in the sixth month with submission to Google Play Store in the seventh month. The app was approved and deployed in the eighth month.

A diagram showing the development cycle with four components: specification, design, development, and deployment. Each component is further divided into ideate, solidify, and implement. The four components form one iteration of the cycle. — Figure 2. The development cycle.

Resulting Impacts

In order to keep the team focused and establish a collaboration, a project plan was our first action. Within the project plan was the rationale, the project scope, team composition, team responsibilities, team deliverables, milestone activities, communication management plan, the contact information for each team member, the budget, how the meetings would be conducted, and a quality baseline commitment. All members reviewed and revised the project plan until it was agreeable. The document was critical to the interdisciplinary focus and prevented role drift where one person tries to take over all the roles in the development process.

The Serene app included three opportunities for learning. The first opportunity was to learn more about how a multidisciplinary team could be structured to deliver specific applied skills in an academic setting. The second was the learning environment of developing the app itself. The third occurs in deploying the app to the larger community and learning more about the community’s mental health.

The first learning opportunity gave us a greater appreciation of how each discipline perceived the work and structured priorities. For example, the content group had a great interest in gathering mental health data. At first, the UX and app development teams failed to realize how important biological sex is to mental health data collection and analysis. The content team explained a long-standing issue in the mental health field pertaining to a need to research and better understand biological sex differences in relation to psychopathology as Cynthia Hartung and Thomas Widiger (1998) and Cynthia Hartung and Elizabeth Lefler (2019) discuss. As a result, it is important to ensure that data such as biological sex is collected and analyzed in mental health research to help elucidate whether any potential findings vary by sex. Even through this simple occurrence, the students learned the value of the various perspectives provided by an interdisciplinary team. The team included five options to report sex: prefer not to say, male, female, intersex, and other.

The second opportunity happened both during and after development as we learned to coordinate our expertise. The members of UX, app development, and content teams used their expertise to move the project forward. Faculty mentors coached students on teamwork skills and developed the students’ expertise in separate meetings. The weekly team meetings were opportunities for joint design decisions, review of the work, and progress maintenance by following the four we’s: (a) this is what we were thinking in our role, (b) this is what we did to move the project forward, (c) this is what we think should be done next, (d) what do we think? The forming, storming, norming, and performing stages are well documented (Tuckman 1965), however, in this project a deeper sense of inquiry was necessary to convey respect for each role’s effort. This respect freed individuals from preliminary criticism that would hamper their motivation yet allowed the team to critique the project at critical milestones. For example, the UX team struggled to devise the weekly, monthly, and yearly graphs. During weekly reviews, the team settled on an earlier solution. The UX team enjoyed the freedom to exercise their expertise and intellectually explore the options before the final design decision was made by the team.

The third learning opportunity is ongoing and comes from anonymous user data being collected to identify and address potential mental health inadequacies prevalent in the regional community. Typically, mental health needs outpace the resources available. By creating this app, we not only provided valuable resources to the community during a time of immense need, but we also gained valuable insight into the ongoing needs of our community. For instance, these data allow us to analyze the anonymous, self-reported, mental health data, across time throughout and after the pandemic. It also allows us to examine and better understand what types of local resource content are most applicable to our community members. Ultimately, these data will allow the team to better understand the specific needs of our area, as reported by the community, and can serve to tailor engagement towards addressing specific community needs.

While we often put students in teams in class, they rarely participate in teams across disciplines. This project was a beneficial example of how to construct an interdisciplinary team. Each student responded positively in their comments (see Appendix D): “The biggest challenge was taking what suggestions the development team had and giving them life”; “Nonetheless, working with a large group of experienced professors and students allowed for a painless development process”; and “The main sense that remains with me after the completion of Serene is that of working and communicating with people from various fields who all used their own language, interests, and expertise for the same project.”

Challenges

We encountered several challenges through our development process. They range from technical—data warehousing, quality assurance, design, to legal—terms of service, to managerial—content. The following sections describe each of them.

Data warehousing

One of the first challenges was where to host the programming code as the project developed. Two back-end server hosting solutions were considered. The first was a cloud-based solution, Amazon Web Services (AWS). However, this solution was abandoned due to its ongoing costs associated with storage (AWS Simple Storage Service), computing (AWS Elastic Compute Cloud), and communication (AWS Data Transfer). We chose the second solution, hosting with an internal institutional server using Windows Server. This solution required:

Help from IT support from the university in setting up server
University computing and storage resources
Compliance with university, including accessibility
Access to the Google Play Store from a university-owned account

Content

A considerable amount of time was spent in finding mindfulness scripts, soundtracks, and images with no copyrights. Also, the composition of the audio files was quite challenging, and particularly the pairing of the soundtrack with the narrative. The first few recordings took many hours to complete. Finally, the list with the quotes was unexpectedly time consuming, as the content team had to proof-read the quotes and ascertain the authors for all the quotes that appeared “unknown” during the search.

Many of the resources were kept in a spreadsheet file. The team decided to use html tags so the app could easily access the spreadsheet resources and use those resources, as is, within the app. The content team easily learned the tags and adapted.

Institutional Review Board

We found that we needed two reviews for human subjects research. One was for user testing during development. The other was for using the data that the app gathered. As this second review of data had information that was not identifiable to a person, it was determined that this was not human subjects’ data.

Attorney services

There were many questions about how to best navigate terms of service and data use. We worked closely with the legal department at our institution through the corporate liaison officer to implement a terms of service appropriate to the general nature of the app and for the information on the data and how it would be used. Since the Serene app has a mental health focus, it was essential to ensure that it was not used as a substitute for medical care and that the development team and the university could not be held liable for any such misuse. Therefore, the corporate liaison consulted with the university’s offices of risk management and general counsel. These offices helped to craft simple, understandable terms of service and data usage language. As the app was produced by the students, we chose to provide that information on the accompanying website rather in the app itself.

Design of charts

Other challenges included problems related to a specific content area. In UX/UI- there was quite a bit of work on how the charts would look. Initially, we considered a complex line and bar chart such as in Figure 3.

Figure 3. A line plot charts 'anxious', 'boredom' and 'anger.' While 'anger' hovers at the lowest level, rising toward the end, 'anxious' comes to overtake 'boredom' at the end, after an uneven dip below. — Figure 3. Initial prototype for tracking emotions.

However, this chart was too complex for the small real estate on a mobile phone screen and did not capture the weekly, monthly, and yearly changes. Then, we tried three different charts as shown in Figures 4 to 6.

Figure 4. A line plot of weekly progress showing how three types of emotions change on a weekly basis. One emotion starts with a high value and then drops to a low value towards the end. — Figure 4. Prototype of weekly progress.

Figure 5. A bar graph of monthly progress showing how three types of emotions change on a monthly basis. Each emotion is represented by a differently-colored bar. The emotion represented by the orange color starts with a high value and then drops to a low value towards the end. — Figure 5. Prototype of monthly progress.

Figure 6. A filled area graph of yearly progress showing how three types of emotions change on a yearly basis. Each emotion is represented by a differently-colored area. The values of the three emotions change from year to year. — Figure 6. Prototype of yearly progress.

User testing

In order to discover how well the design was understood, we conducted user testing with five undergraduate students from the psychology course testing pool which was approved by the IRB. The UX student used standard user testing methodologies as recommended on NN group’s user testing videos (Nielsen Norman Group 2020). In the user testing, participants were asked to do three tasks: Find a mindfulness video, find an activity to do on the phone/computer, and find a local resource for depression. Then, the researcher took note of any problems the participant had and how long it took each participant to do each task along with satisfaction ratings as described by Erik Frøkjær, Morten Hertzum, and Kasper Hornbæk (2000). Participants in user testing found that some of the labels were unclear and there was confusion about where to find specific tasks within the app. There were also questions about the use of a password and if users would be able to use the app on their Android phone and on the web. Users said that they would like a password, but this would invalidate some of the anonymity of the data and could cause some late-stage development changes. Thus, we decided to leave the password issue to the future releases and comments were gathered.

Quality assurance team

Once the code is working, there needs to be a dedicated team that tests the app, looks for the weaknesses and finds out if there are bugs that might break the app. We did not have such a team and instead functioned as our own quality-assurance team along with friends who volunteered their time. This method took longer and a professional-level assurance team should be included in the budget.

Conclusions and Future Directions

Despite the challenges encountered during the development of the app, the combination of talents into one interdisciplinary team allowed the creation of a completed product that far exceeded what one discipline could accomplish alone. One purpose of the project was to give the students the experience of working in a structured and distributed environment with a team that was segregated by roles but followed an agile software development approach, where development happened in an iterative way. Professors spent additional hours mentoring the students on teamwork and communication as well as learning about the other roles. Working together while maintaining a constant line of communication was key to its success. Having regular stages of reviews during the development cycle helped guide the process in the right direction. Using the right collaboration tools, such as Microsoft Teams, Google Docs, and GitHub, made team work much easier. The Serene app is available through Google Play and can be accessed on the Serene website.

Knowing that the project would result in an app launched to a large community of potential users provided ample motivation to meet the learning and performance needs for successful completion of the project. Learning requirements were high and extended well beyond knowledge gained in coursework. Also, it did not go unnoticed by the students that the audience for this project was external to the university, and it motivated them to take extra care and expend extra effort in their work. Working within an interdisciplinary team that extends beyond students to include university faculty and staff helped the students to broaden their perspective of app development work to include the work of the other teams, focus their thinking on alignment of the project with the mission of the university, understand potential legal responsibilities, and value meeting the expectations of external stakeholders. For example, the funding agency has a strong local focus, so the students had to be sure to target a sufficient portion of the app’s functionality toward a regional audience.

Our university is highly focused on being an Open Lab (Birx, Ford and Payne 2013), an interdisciplinary living laboratory where learning and discovery are applied to solve problems defined in partnership with external stakeholders. The Open Lab concept evolved from the idea of research clusters working on pressing local problems. The Open Lab is a win for students, faculty, and the external organizations: students gain career-building, real-world experience; faculty enjoy the ability to keep their skills relevant and transfer their networks to students; and external partners benefit from the energy and ingenuity of student talent. More information can be found at the Open Lab website through the university homepage (https://behrend.psu.edu). This project aligns with that focus, as it creates an outward-facing product that receives and engages with feedback from the external community, adding motivation and accountability to the students’ work. It is notable that all students acknowledged the value that this experience provided to them and their career development. Students involved in this team left individual comments regarding their experience (see Appendix D).

We plan on continuing an extension of this project. For future work, psychology content can be updated and expanded with additional mental health or COVID-19 focused information, resources, or the creation and addition of new wellness exercises that users can freely access from within the Serene app. We have deployed the app and are monitoring user feedback. Additional user testing will be conducted based on user feedback. We also plan to deploy the app to iOS. When recruiting undergraduate students, it is a good idea to have overlapping years. There should be some second-year students, some third-year students, and some senior students. This approach ensures smooth transitions for projects having a multi-year life expectancy.

Bibliography

Birx, Donald L., Ralph M. Ford, and Carrie A. Payne. 2013. “The University as an Open Laboratory.” Journal of Research Administration 44, no. 2: 11–37.

Blackwell, Alan F., Lee Wilson, Alice Street, Charles Boulton, and John Knell. 2009. “Radical Innovation: Crossing Knowledge Boundaries with Interdisciplinary Teams.” Department of Computer Science and Technology Technical Reports UCAM-CL-TR-760, November 2009. https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-760.html.

Brodack, Franziska, and Anna Sinell. 2017. “Promoting Entrepreneurial Commitment: The Benefits of Interdisciplinarity.” Technology Innovation Management Review 7, no. 12: 6–13. http://doi.org/10.22215/timreview/1123.

Cardello, Jen. 2014. “The Difference between Information Architecture and Navigation.” NN group, November 26, 2020. https://www.nngroup.com/articles/ia-vs-navigation/.

Centers for Disease Control and Prevention. 2019. “Coping with Stress.” CDC, January 22, 2021. https://www.cdc.gov/coronavirus/2019-ncov/daily-life-coping/managing-stress-anxiety.html.

DaSilva, Jill. 2020. “A Guide to Competitive Analysis for UX Design.” Adobe, June 12, 2020. https://xd.adobe.com/ideas/process/user-research/guide-to-competitive-analysis-ux-design/.

Fernández, Rodrigo S., Lucia Crivelli, Nahuel Magrath Guimet, Ricardo F. Allegri, and Maria E. Pedreira. 2020. “Psychological Distress Associated with COVID-19 Quarantine: Latent Profile Analysis, Outcome Prediction and Mediation Analysis.” Journal of Affective Disorders 277: 75–84. https://doi.org/10.1016/j.jad.2020.07.133.

Frøkjær, Erik, Morten Hertzum, and Kasper Hornbæk. 2000. “Measuring Usability: Are Effectiveness, Efficiency, and Satisfaction Really Correlated?” Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 2, no.1: 345–352.

Hartung, Cynthia M., and Elizabeth K. Lefler. 2019. “Sex and Gender in Psychopathology: DSM-5 and Beyond.” Psychological Bulletin 145, no. 4: 390–409. https://doi.org/10.1037/bul0000183.

Hartung, Cynthia M., and Thomas A. Widiger. 1998. “Gender Differences in the Diagnosis of Mental Disorders: Conclusions and Controversies of the DSM–IV.” Psychological Bulletin 123, no. 3: 260–278. https://doi.org/10.1037/0033-2909.123.3.260.

Ho, Cyrus Sh, Cornelia Yi Chee, and Roger Cm Ho. 2020. “Mental Health Strategies to Combat the Psychological Impact of COVID-19 Beyond Paranoia and Panic.” Annals Academy of Medicine, Singapore 49, no. 1: 1–3.

Kanter, Jonathan W., Rachel C. Manos, William M. Bowe, David E. Baruch, Andrew M. Busch, and Laura C. Rusch. 2010. “What is Behavioral Activation?: A Review of the Empirical Literature.” Clinical Psychology Review 30, no. 6: 608–620. https://doi.org/10.1016/j.cpr.2010.04.001.

Kar, Sujita Kumar, SM Yasir Arafat, Russell Kabir, Pawan Sharma, and Shailendra K. Saxena. 2020. “Coping with Mental Health Challenges During COVID-19.” In Coronavirus Disease 2019 (COVID-19). Medical Virology: From Pathogenesis to Disease Control, edited by Shailendra K. Saxena. Singapore: Springer.

Luxton, David D., Russell A. McCann, Nigel E. Bush, Matthew C. Mishkind, and Greg M. Reger. 2011. “mHealth for Mental Health: Integrating Smartphone Technology in Behavioral Healthcare.” Professional Psychology: Research and Practice 42, no. 6: 505–512.

Moirano, Regina, Marisa Analía Sánchez, and Libor Štěpánek. 2020. “Creative Interdisciplinary Collaboration: A Systematic Literature Review.” Thinking Skills and Creativity 35, no. 100626: 1–14. https://doi.org/10.1016/j.tsc.2019.100626.

Naeem, Farooq, Muhammad Irfan, and Afzal Javed. 2020. “Coping with COVID-19: Urgent Need for Building Resilience Through Cognitive Behaviour Therapy.” Khyber Medical University Journal 12, no. 1: 1–3. https://doi.org/10.35845/kmuj.2020.20194.

Nielsen Norman Group. 2020. “User testing.” NN group, June 3, 2020. https://www.nngroup.com/videos/user-testing-jakob-nielsen/.

Pierce, Matthias, Holly Hope, Tamsin Ford, Stephani Hatch, Matthew Hotopf, Ann John, Evangelos Kontopantelis et al. 2020. “Mental Health Before and During the COVID-19 Pandemic: a Longitudinal Probability Sample Survey of the UK Population.” The Lancet Psychiatry 7, no. 10: 883–892. https://doi.org/10.1016/S2215-0366(20)30308-4.

ProductPlan. 2020. “MoSCoW Prioritization.” ProductPlan, November 25, 2020. https://www.productplan.com/glossary/moscow-prioritization/.

Ries, Eric. 2013. “Building the Minimum Viable Product.” Stanford Technology Ventures Program, August 28, 2013. https://youtu.be/1FoCbbbcYT8.

Saltzman, Leia Y., Tonya Cross Hansel, and Patrick S. Bordnick. 2020. “Loneliness, Isolation, and Social Support Factors in Post–COVID-19 Mental Health.” Psychological Trauma: Theory, Research, Practice, and Policy 12, no. S1: S55–S57. https://dx.doi.org/10.1037/tra0000703.

Six, Janet M. 2019. “Eliciting Business Requirements.” UX Matters, April 23, 2019. https://www.uxmatters.com/mt/archives/2019/04/eliciting-business-requirements.php.

Tuckman, Bruce W. 1965. “Developmental sequence in small groups.” Psychological Bulletin 63, no. 6: 384–399.

Appendix A: UX Screening Assignment

Instructions</h3
For this project, I want you to imagine that you have been hired to create a prototype for a mythical application. The goal of the application is to reach out to people who are feeling anxious and provide them with mental health resources and exercises to help. Boredom is part of anxiety, so it also addresses boredom. The red route in this app is that people should be able to report their level of anxiety and then get a breathing exercise and then get the phone numbers of mental health professionals. Please use one of these programs: Adobe XD (inside Adobe Creative Suite), or Figma, or Invision to create a prototype for this app. Your prototype must be designed for an Android phone and have at least one screen for each box in the information architecture outline which is here in Figure 7.

Figure 7. The architecture that the assignment used.

Each screen must show the choices that would bring you to the next screen (yes/no). The screens must use the Material design pattern library which is here: https://material.io/. Once you are finished with all the screens and they are linked so that they work as an app would, please turn in the URL. I don’t want the screens or the file, just the URL for the prototype.

Appendix B: Activities Provided to Users Within the Serene App

Activity Category Type	Activity
Indoors	Cook/Bake a new recipe
Indoors	Take a nap (or two)
Indoors	Do a jigsaw puzzle
Indoors	Organize your room
Indoors	Take a (long) bath/shower
Indoors	Clean your room/house
Indoors	Try out DIY crafts
Indoors	Organize your cabinets
Indoors	Throw away expired items
Indoors	Redecorate your room/house
Indoors	Repaint your room/house
Indoors	Reorganize your closet (check out Marie Kondo)
Indoors	Fix broken items
Indoors	Plan your outfits (even your “Zoom” ones)
Indoors	Clean your electronic devices
Indoors	Do your laundry
Indoors	Clean your fridge
Indoors	Create an emergency kit
Indoors	Cook/Bake for your friends or co-workers
Indoors	Rearrange your furniture
Indoors	Make a pillow fort
Indoors	Make a cardboard house
Indoors	Take care of your plants
Indoors	Sing around the house
Indoors	Cook a Michelin worthy meal
Indoors	Meal prep for the week/month (it will change your life)
Indoors	Change your bedsheets
Indoors	Organize your workspace
Indoors	Shave
Indoors	Cook an international cuisine
Indoors	Make home-made “fast food” (pizza, tacos, etc.)
Indoors	Grow an indoor kitchen garden
Indoors	Make your own peanut butter and jam
Outdoors	Start a vegetable garden
Outdoors	Clean out your car (Beware!)
Outdoors	Plant flowers
Outdoors	Find a place to volunteer
Outdoors	Go to church
Outdoors	Sell stuff you don’t need
Outdoors	Go on a solo date
Outdoors	Go for a walk/run
Outdoors	Go for a bike ride
Outdoors	Plan and go on a scavenger hunt
Outdoors	Make a cardboard house
Outdoors	Birdwatch
Outdoors	Go for a drive
Outdoors	Have a bonfire (and roast marshmallows, of course)
Outdoors	Sunbathe (don’t forget your sunscreen)
Outdoors	Take care of your plants
Outdoors	Fly a kite
Outdoors	Go camping (and roast marshmallows, again)
Outdoors	Walk on the beach/riverfront
Outdoors	Go roller-skating
Outdoors	Go hiking
Outdoors	Go out for dinner/lunch to a new restaurant
Outdoors	Go fishing
Outdoors	Gaze at the stars (appease the romantic in you)
Outdoors	Go on a picnic
Outdoors	Go to a coffee shop (no, not the drive-thru)
Outdoors	Have a barbecue
Outdoors	Spend time in nature
Outdoors	Go to home opens
Outdoors	Walk around the city
Outdoors	Mow your lawn
Outdoors	Build a bird house/feeder
Outdoors	Go to a scenic spot and enjoy the view
Outdoors	Learn tricks with a jumping rope
Outdoors	Turn your yard into an outdoor cinema
Outdoors	Take your dog to the park
Outdoors	Check out geocaching (yes, it is still a thing)
Outdoors	Let out some energy by screaming or running around like crazy (not recommended if you live near people)
Outdoors	Build a hammock
Outdoors	Watch the sunset/sunrise
Outdoors	Build a sandcastle
Entertainment	Watch YouTube videos
Entertainment	Binge-watch a new TV show
Entertainment	Play a video game
Entertainment	Read a book/magazine
Entertainment	Blast some music
Entertainment	Discover new music
Entertainment	Visit museums virtually
Entertainment	Watch a documentary
Entertainment	Read your favorite blogs/find new ones
Entertainment	Listen to your favorite podcast/find new ones
Entertainment	Play online games with your friends/family
Entertainment	Make a new playlist
Entertainment	Make a playlist for every mood
Entertainment	Download fun apps
Entertainment	Listen to an audiobook
Entertainment	Watch a Disney movie
Entertainment	Learn a magic/card trick
Entertainment	Dig out old board games
Entertainment	Listen to the radio
Entertainment	Re-watch your all-time favorite movies
Entertainment	Have a movie marathon
Entertainment	Read a comic book (DC or Marvel?)
Socializing	Text/call someone you haven’t talked for a long time
Socializing	Play online games with your friends/family
Socializing	Take on a new challenge with your friends/family
Socializing	Ask your parents and grandparents about their childhood
Socializing	Find a place to volunteer
Socializing	Go to church
Socializing	Throw a themed Zoom party
Socializing	Call your grandparents
Socializing	Plan your next vacation/get-away (visualize that the Earth and people are fine again)
Socializing	Contact a distant relative
Socializing	Talk with your family
Socializing	Plan/Go on a road trip
Socializing	Go old school and get a pen pal
Socializing	Have a class reunion (Zoom makes it easier)
Socializing	Don’t take your loved ones for granted and remind them that you love them
Socializing	Plan a Zoom trivia night
Socializing	Teach a skill to someone
Socializing	Plan a surprise for someone
Socializing	Get to know your neighbors
Socializing	Spread some positive energy and give someone a genuine compliment
Pen & Paper	Draw/Paint/Doodle
Pen & Paper	Do a painting tutorial
Pen & Paper	Create a bucket list
Pen & Paper	Write thank-you cards
Pen & Paper	Start a journal
Pen & Paper	Create a healthy meal plan (and follow it)
Pen & Paper	Schedule your week/month/year
Pen & Paper	Solve brainteasers/crosswords
Pen & Paper	Make a list of your favorite quotes (Serene can help you out with this)
Pen & Paper	Make a travel bucket list
Pen & Paper	Write a letter to your future self
Pen & Paper	Start a gratitude journal
Pen & Paper	Color an adult coloring book
Pen & Paper	Make a list with all the things that make you happy
Pen & Paper	Create a list with all the things you don’t know and want to Google
Pen & Paper	Write a poem/essay/story/song
Pen & Paper	Design your dream house (maybe log in your Sims account?)
Pen & Paper	Plan your next vacation/get-away (visualize that the Earth and people are fine again)
Pen & Paper	Make a pros-cons list to help you make a decision
Pen & Paper	Plan/Go on a road trip (Google themed road trips; you won’t regret it)
Pen & Paper	Learn calligraphy
Pen & Paper	Follow a writing prompt
Pen & Paper	Document all the self-isolation days by photography or writing for the future generations to see
Personal Growth	Read a book/magazine
Personal Growth	Visit museums virtually
Personal Growth	Watch a documentary
Personal Growth	Organize your finance
Personal Growth	Start a journal
Personal Growth	Create a healthy meal plan (and follow it)
Personal Growth	Make a list of your goals with 3 logical and feasible steps to achieve them
Personal Growth	Learn a new language (well, get started at least)
Personal Growth	Update your resume
Personal Growth	Watch TED-Talks
Personal Growth	Start a gratitude journal
Personal Growth	Learn how to play an instrument
Personal Growth	Listen to an audiobook
Personal Growth	Apply for a new job
Personal Growth	Plan your future education
Personal Growth	Look for online/free certificates
Personal Growth	Make a list with all the things that make you happy
Personal Growth	Expand your vocabulary (appease your intellectual self)
Personal Growth	Do the one thing you have been putting off (you know what we are talking about)
Personal Growth	Make a plan to pay out your debt
Personal Growth	Learn how to build up a good credit
Personal Growth	Find a place to volunteer
Personal Growth	Practice your religion
Personal Growth	Learn about spirituality
Personal Growth	Research ways to make your living situation more sustainable and “green”
Personal Growth	Research about other cultures
Personal Growth	Update your LinkedIn
Personal Growth	Create a vision board
Personal Growth	Start a money saving challenge
Personal Growth	Take a fun online course
Personal Growth	Google things that interest you
Personal Growth	Finish unfinished projects
Personal Growth	Research fitness/wellness videos/blogs
Personal Growth	Learn more about finance and budgeting
Personal Growth	Make a pros-cons list to help you make a decision
Personal Growth	Create a savings plan
Personal Growth	Learn a new skill
Personal Growth	Learn a graphic design program
Personal Growth	Learn first aid
Personal Growth	Explore career options
Personal Growth	Buy a newspaper to read with your morning coffee instead of checking your phone
Personal Growth	Teach a skill to someone
Personal Growth	Spread some positive energy and give someone a genuine compliment
Personal Growth	Dance (like no one is watching)
Personal Growth	Play with your pet or teach it a new trick
Personal Growth	Exercise
Personal Growth	Practice a new physical activity
Personal Growth	Do yoga
Personal Growth	Stretch (daily if possible)
Personal Growth	Go for a walk/run
Personal Growth	Go for a bike ride
Personal Growth	Do aerobics (remember Zumba?)
Personal Growth	Try out martial arts/self-defense
Personal Growth	Go swimming
Personal Growth	Go roller-skating
Personal Growth	Go hiking
Personal Growth	Play your favorite sports
Personal Growth	Walk around the city
Personal Growth	Learn tricks with a jumping rope
Computer/Phone	Play a video game
Computer/Phone	Visit museums virtually
Computer/Phone	Back-up your computer
Computer/Phone	Play online games with your friends/family
Computer/Phone	Make a new playlist
Computer/Phone	Make a playlist for every mood
Computer/Phone	Watch TED-Talks
Computer/Phone	Declutter your emails
Computer/Phone	Download fun apps
Computer/Phone	Create a TikTok video
Computer/Phone	Delete old contacts from your phone
Computer/Phone	“Get lost” with Google Sky and Google Maps
Computer/Phone	Search for birthday gifts for your loved ones
Computer/Phone	Unsubscribe your email from newsletters
Computer/Phone	Sell stuff you don’t need
Computer/Phone	Organize your documents
Computer/Phone	Google things that interest you
Computer/Phone	Research fitness/wellness videos/blogs
Computer/Phone	Leave a positive review on Amazon (because we all need some positive in our lives)
Computer/Phone	Make a wish-list on Amazon
Computer/Phone	Make a to-watch list on IMDB
Computer/Phone	Make a to-read list on Goodreads
Computer/Phone	Update your social media bio(s)
Computer/Phone	Learn a graphic design program
Computer/Phone	Start a blog
Other	Organize your pictures
Other	Start a new challenge
Other	Practice relaxation techniques
Other	Meditate
Other	Look into your family tree
Other	Put together a family history book
Other	Go through old pictures
Other	Stay hydrated and drink more water
Other	Practice breathing techniques
Other	Do a picture challenge-take pictures with certain themes
Other	Care for your pet
Other	Do a pet photoshoot
Other	Sew something
Other	Patch up an old blanket
Other	Make a short movie
Other	Try out a new makeup look
Other	Try out a new hairstyle
Other	Keep track of your alcohol/caffeine intake
Other	Start a collection (coins, shells, stamps, etc.)
Other	Organize your picturesTry embroidery/cross stitching/crocheting/knitting
Other	Go to a beauty salon (applies to all genders)
Other	Babysit for a friend/neighbor (kids are fun)
Other	Daydream like everything is possible
Other	Spend a day with children
Other	Play dress-up
Other	Light up candles and relax
Other	Do some research for the best deal on the things you want to buy
Other	Do a favor for someone
Other	Donate blood
Other	Turn off your electronic devices for an hour
Other	Blow bubbles
Other	Try out origami
Other	Do something nostalgic (listen to old songs, watch old pictures, etc.)

Appendix C: Sources Used for the Mental Health and COVID-19 Content Articles

BBC. “Coronavirus global update” bb.co.uk
https://www.bbc.co.uk/programmes/w13xtv39

Bill and Melinda Gates Foundation. “COVID-19” gatesfoundation.org
https://www.gatesfoundation.org/TheOptimist/coronavirus

Centers for Disease Control and Prevention (CDC). “Coronavirus (COVID-19)” cdc.gov
https://www.cdc.gov/coronavirus/2019-nCoV/index.html

COVID-19 facts. “COVID-19 facts” covid-19facts.com
https://www.covid-19facts.com/

Department of Homeland Security. “Master question list for COVID-19 (caused by SARS-CoV-2)” dhs.gov
https://www.dhs.gov/publication/st-master-question-list-covid-19?fbclid=IwAR0R7XbMPaANzMYahmW311zv2Iekk-eVIn97xlk8VPUvD_hgZAlKS9qPASU

Inside Higher Ed. “Live updates: Latest news on coronavirus and higher education” insidehighered.com
https://www.insidehighered.com/news/2020/05/29/live-updates-latest-news-coronavirus-and-higher-education

Law librarians of Congress. “Coronavirus resource guide” loc.gov
https://blogs.loc.gov/law/2020/03/coronavirus-resource-guide/

National Center for Biotechnology Information (NCBI). “LitCOVID” ncbi.nlm.nih.gov
https://www.ncbi.nlm.nih.gov/research/coronavirus/

National Institute of Health (NIH, official website). “Coronavirus (COVID-19)” nih.gov
https://www.nih.gov/coronavirus

National Institute of Health (NIH). “Open-Access Data and Computational Resources” nih.gov
https://datascience.nih.gov/covid-19-open-access-resources

Occupational Safety and Health Administration (OSHA). “COVID-19” osha.gov
https://www.osha.gov/SLTC/covid-19/

Pennsylvania Department of health. “Coronavirus (COVID-19)” health.pa.gov
https://www.health.pa.gov/Pages/default.aspx

Rapid reviews. “Rapid reviews: COVID-19” rapidreviewscovid19.mitpress.mit.edu
https://rapidreviewscovid19.mitpress.mit.edu/

Surgo Foundation. “Bringing greater precision to the COVID-19 response” precisionforcovid.org
https://precisionforcovid.org/

The U.S. Census Bureau. “COVID-19 Demographic and economic resources” covid19.census.gov
https://covid19.census.gov/

The U.S. Food and Drug Administration (FDA). “Coronavirus disease 2019 (COVID19)” fda.gov
https://www.fda.gov/emergency-preparedness-and-response/counterterrorism-and-emerging-threats/coronavirus-disease-2019-covid-19

The U.S. Department of Health and Human Services (HHS). “Coronavirus (COVID-19)” coronavirus.gov
https://www.coronavirus.gov/

Very Well Mind. “Emotions and types of emotional responses” verywellmind.com https://www.verywellmind.com/what-are-emotions-2795178#citation-1

World Health Organization (WHO). “Coronavirus disease (COVID-19) pandemic” who.int
https://www.who.int/emergencies/diseases/novel-coronavirus-2019

Appendix D: Student Feedback

UX student

When we created the app, the challenge was how to make the app feel calm and soothing while the potential users were in the app. Research on what apps were already available and what features they offered helped me to shape what I wanted the app to look like—leading to the blues and nature theme throughout the app. The biggest challenge was taking what suggestions the development team had and giving them life. There were several changes along the way as we got into the development of content. I think that is one of the interesting points of being a UX researcher, is the continued changes that must happen along the way as we progress with the application. The journey from starting the app to finishing my part with the development of the app has been a really great experience and it has given me skills that I can build upon and take into my career with me. An exciting part of this app development for me is that this will not only be available to university students but the whole community as well, which is a point that helped me to create a mental health app that is visually soothing and helpful for those users.

App development student

While developing the application, I learned many new skills and faced just as many challenges. Throughout college, I have never worked on a project of this scale. This forced me to apply my, somewhat entry level, skills as a web/application developer and build upon them immensely. This included countless hours of experimentation and research in concepts that were new to me. If I had to choose two skills that I am grateful for learning through the development process, it would be working with Amazon Web Services and Windows Server. It is one thing to develop an application on your computer at home, but it is a completely new experience when the application is running on a server for everyone to enjoy. Amazon Web Services makes the process of hosting an application simple but working with a Windows Server proved to be a much greater challenge. This required working within the limitations of the server’s security and, on many occasions, discussing with the IT department to make changes to the server that I did not have clearance to access. Nonetheless, working with a large group of experienced professors and students allowed for a painless development process. Overall, I have gained valuable experience and knowledge that will be beneficial to my future and I enjoyed it along the way.

Content student

The first unexpected challenge was the creation of the “Quote” list. I had to track down the authors of the quotes that appeared as “by unknown” during the search and also check the background of each author to assure that a certain identity was real and the person had a capacity that allowed them to say the specific quote.

The next and biggest challenge was the creation of mindfulness exercises. First, I had to familiarize myself with the digital audio editing software (Audacity). Second, during the recording, I had to assure that the words were being pronounced correctly. As an international student with English being my second language, I had to re-record the same sentence multiple times or even record word by word until I had a final output where my accent was as indistinct as possible. Finally, pairing a recording with a soundtrack had its own difficulties, as the soundtracks were typically shorter than the recordings and I had to assure that the transition from one track to another was smooth and did not interfere with the recording.

The final challenge was the composition of the two brief articles about mental health. This task required a significant amount of merely thinking, trying to narrow down to specifics all the knowledge I had as a clinical psychology student. I had a large amount of information available, but my task was to provide a very specific and, at the same time, comprehensive summary of it all. I also tried to avoid writing based on my own biases and opinions. Finally, I had to use everyday language to explain scientific terms and concepts that would make sense to people unfamiliar with the field.

What I had not realized until I started working on the tasks was that when I was providing the users with every piece of information I had to be completely valid, reliable, and accurate at all levels. I had to be as meticulous as I could. The main sense that remains with me after the completion of Serene is that of working and communicating with people from various fields who all used their own language, interests, and expertise for the same project. In a very short period of time, I was able to gain an experience valuable for my academic and professional future.

Acknowledgments

This project was funded by the Erie County Gaming Revenue Authority. We thank the two reviewers for their insightful comments, and also thank Kris McLain.

About the Authors

Antigoni Kotsiou was the primary content developer on the project. She graduated from Penn State Behrend with an MA in Applied Clinical Psychology. She works as a therapist providing treatment to children, adolescents, and young adults with trauma history and/or other mental health and behavioral concerns. Her research interests include therapeutic processes, techniques, and models, and psychopathology. She is interested in qualitative research and the subjective experiences of those involved in psychotherapy and mental health services. Her responsibility in the Serene project was to create the content based on various psychotherapy theories and models, such as cognitive therapy and mindfulness.

Erica Juriasingani was the primary UX/UI developer on the project. She is a human factors psychology student at Penn State Behrend and is currently completing her last semester. She is working as a UX Researcher with Innovation Commons at Behrend and plans to continue pursuing UX work after her graduation.

Marc Maromonte was the primary software developer on the project. He is an engineering student at Penn State University and is currently completing his Bachelor of Science in computer science. He is currently working as an Application Developer with Innovation Commons at Penn State Behrend. He plans to continue pursuing software development after his graduation.

Jacob Marsh is the Industry Relations Coordinator at Penn State Behrend. Jacob has a bachelor’s degree in biochemistry from Grove City College, a history in virology research at Penn State Hershey, and a master’s degree in project management from Penn State World Campus. He was instrumental in founding, and currently oversees, the Innovation Commons at Penn State Behrend, a product design and rapid prototyping center staffed by undergraduate students, as part of the Invent Penn State initiative. Jacob also helps develop, fund, and manage various other programs involving entrepreneurship, economic development, and industrial partnerships with Penn State Behrend.

Christopher R. Shelton is an Assistant Professor of Clinical Psychology and the director of the Virtual/Augmented Reality Lab at Penn State Behrend. He has significant clinical experience providing diagnosis, assessment, and treatment for mental health concerns across a wide spectrum of the population. His current research focuses on: (a) examination of ADHD and Sluggish Cognitive Tempo; (b) development of digital mental health assessments and interventions to increase treatment availability; and (c) the use of immersive technologies, such as augmented and virtual reality, across a range of domains. Dr. Shelton earned his Ph.D. in Clinical Psychology from the University of Wyoming.

Richard Zhao is an Assistant Professor in the Department of Computer Science at the University of Calgary. He led the app development team on the Serene project. His current research group focuses on serious games for training and education where he utilizes artificial intelligence, virtual reality, and eye-tracking technologies for this purpose. He received his M.S. and Ph.D. in Computing Science from the University of Alberta. Dr. Zhao was a faculty member at Penn State Behrend.

Lisa Jo Elliott is an Assistant Teaching Professor at Penn State Behrend where she directs the Laboratory for Usability and Interactive Systems – LUIS lab. This lab and Innovation Commons lead a multi-million-dollar grant for a UX-first product design lab. This initiative is one of the first UX-centric product design labs in the United States. It trains UX, UI, interaction design, and experience design students to be future product designers and developers in the engineering, DIGIT, and psychology programs at Penn State Behrend. Dr. Elliott has a Ph.D. from New Mexico State University, Las Cruces, NM, USA.

Screenshot of protestors with signs. An excerpt from a course reading is to the right of the image.

Issue Nineteen

“The Future Started Yesterday and We’re Already Late”: The Case for Antiracist Online Teaching

David L. Humphrey Jr., University of Michigan

Camea Davis, Georgia State University

Abstract

Using Black critical theoretical perspectives and pedagogical examples from our experiences teaching in online learning environments, this article articulates a case for antiracist online education. In the midst of the deadliest convergence of three devastating global “pandemics”— the COVID-19 pandemic, the continued murdering of Black bodies, and abnormal environmental disasters precipitated by global warming—educational technology could be a vehicle of liberation yet it remains an apparatus of control, further exacerbating inequality, especially for Black students. The absence of specific references to antiracist pedagogical orientations in the extant literature and theory of online education is emblematic of the normativeness of anti-Black racism and white normativity in online education. An antiracist pedagogy for online education begins with creating spaces that bring attention to race, class, gender, and ability. The authors conclude with a call to action for a shift to antiracist online teaching for all learners.

If you hear this message, wherever you stand
I’m calling every woman, calling every man
We’re the generation
We can’t afford to wait
The future started yesterday and we’re already late
—John Legend, “If You’re Out There”

Due to technology’s rapid innovations and reimagining in the social sphere, each time education makes a strong push forward, it seems we’re already late. Even in the midst of the deadliest convergence of three devastating global “pandemics”—the COVID-19 pandemic, the continued murdering and “fungibility” (la paperson 2017, 15) of Black bodies, and abnormal environmental disasters precipitated by global warming—educational technology could be a vehicle of liberation yet it remains an apparatus of control, further exacerbating inequality, especially for Black students. It is the goal of this paper to shake educators out of the slumber of white heterosexist monotonous and disembodied teaching and offer a vision for antiracist online teaching. As two critical Black scholars (one cis-het woman, one cis-het man) with doctorates in Critical Education Policy Studies and Educational Technology, we are convinced that online teaching is either antiracist and liberating or racist and dehumanizing; there is no in-between (hooks 1994; Love 2019; Kendi 2019). This paper will articulate a case for antiracist online education using both Black critical theoretical perspectives and pedagogical examples from our experiences teaching in online learning environments. We conclude with a call to action for a shift to antiracist online teaching for all learners.

The COVID-19 pandemic has forced most teaching and learning to online platforms. At many institutions, this shift has been met with anxiety, frustration, and in some cases stubborn refusal to conform to the virtual realities, limitations, and possibilities that online teaching provides. Many K–12 school districts continue to struggle to equip their teachers to make the shift to online teaching, leaving teachers to fend for themselves (Lambert and Rosales 2020). The shift to a virtual space has also left ill-prepared teachers, parents, and students fatigued (Holladay 2020). Higher education spaces are not exempt from these issues. Some believe the sudden shift to online teaching and learning is having both negative affective and cognitive effects on students as they work to negotiate the newness of it all (Burke 2020). But the fact still remains that the shift online has not removed the racist norms that were normative in face-to-face classrooms.

Cathy Davidson wrote a blog post entitled “The Single Most Essential Requirement in Designing a Fall Online Course.” In her post, she made an impassioned plea for educators to radically change their approach to (online) teaching during fall 2020, in light of all of the things students will be carrying into their classes because of the social situation. She argued that effective fall 2020 teaching required that summer preparation foreground the reality that “our students are learning from a place of dislocation, anxiety, uncertainty, awareness of social injustice, anger, and trauma” (Davidson 2020). Moreover, she added that informed solutions to creating a more humane and student-centered learning environment during the pandemic would mean “being sensitive to the devastating historical moment in which we are now living and offering students a way forward beyond it.” While the year 2020 will certainly go down as a unique point in the annals of history, many Black students would have entered the fall classrooms from a space of “dislocation, anxiety, uncertainty … anger, and trauma” as well as perpetual “awareness of social injustice” even if the COVID-19 pandemic had not occurred. The pandemic merely heightened what has always been there. Black people in the US continue to live in what Saidiya Hartman called “the afterlife of slavery—skewed life chances, limited access to health and education, premature death, incarceration, and impoverishment” (2007, 6). Any way forward that does not account explicitly for the normativeness of anti-Black thinking in education will merely ensure a racist and anti-Black future in education.

Inspired by the words of Cathy Davidson, while at the same time, captivated by the ancestral rhythm of resistance and freedom inherent in blackness (Cone 2018; Moten 2013; Dillard 2006; Morrison 1993), we argue for a deeper response to the current historical moment. In this article, we address Black non-being and exclusion that is the norm for education in general, but online education more specifically, in the US. This conversation is critical for online education because it is an irrefutable mode of all education moving forward and a space ripe with possibility for antiracist innovations that could unbound the limitations of physical classrooms.

What is Critical Black Theory and Why Is It Important for Online Schooling in the US?

For the sake of this article, critical Black theory will be used to zero in on the signification of blackness in historical and contemporary considerations of online schooling in the US. While other critical theories, like critical race theory (CRT), provide in-depth and intersectional analysis on race and racism, critical Black theory, or “BlackCrit” focuses on a “theorization of blackness” (Dumas and ross 2016, 416). BlackCrit is a “metatheory,” used to explicate the hidden whitened discursive context that undergirds and drives most theories, even theories that consider themselves to be “critical” (Wilderson III 2020, 14). Put simply, the goal of BlackCrit is to shine light on the anti-Black soul of the United States: to lay bare the levers that drive the racist, sexist, classist engine of capitalism. BlackCrit provides an avenue to see past the elusive and often confusing racially ambiguous language such as “people of color,” “diverse group” or person, “minority,” or “underrepresented group” when one is speaking explicitly about Black people (Wilderson III 2020, 41). It compels us to name things for what they are instead of using racially ambiguous or colorblind language, metaphors, or figures of speech that mask, under-emphasize, or erase Black pain and suffering. As Fred Moten (2016) eloquently put it, “We can’t go around this. We gotta go through this” BlackCrit argues that the keys to being liberated out of this racial caste system is acknowledging that it still exists; that the episteme and libidinal nature of the plantation continues unhindered and fundamentally shapes society.

At the same time, BlackCrit also speaks to the ways that blackness signifies a being and deep embodied knowing. Fred Moten (2013) argued that there lives a rich and emancipating hope out of the hopeless condition in understanding blackness. Moten believes that in the signification of blackness outside of the discourse of humaneness, and therefore the realm of being, theorizing blackness exceeds understanding and therefore cannot be reduced to a single thing. Its rich, unmappable essence carries with it the “absolute overturning, the absolute turning of this motherfucker out” (2013, 742). Therefore, BlackCrit forces a consideration of what is possible out of the binary and either/or constraints inputted by a white western colonial imaginary and instead invites an orientation that positions being and knowing as circular (Spillers 2003). Online learning remains an uncharted and underutilized discursive space for addressing anti-blackness and engaging in antiracist praxis (Asenbaum 2019; Bonilla and Rosa 2015; Bondy, Hambacher, Murphy, Wolkenhauer, and Krell 2015; Guthrie and McCracken 2010). In fact, decolonizing and antiracist visions are already embedded within the colonized racist machines of online learning (la paperson 2017; Collins and Bilge 2016).

Online Classrooms Are Not Race-Neutral: How Online Education Eludes Race

The use of technology as a mode of learning has been a part of the US educational infrastructure since the beginning of the nineteenth century (Reiser 2001)—a beginning during which it was criminal for Black people to be formally educated. In the US historically, schoolhouses were created by power-holding whites to sanction and reify anti-Black racism, sexism, white Anglo-Saxon Protestant values, and later prepare a docile workforce to maintain economic disparities (Rury 2009). While American society has come a long way from nineteenth-century schools, traditional public schools have fundamentally maintained nineteenth-century learning practices and structures committed to anti-blackness.

In a historical overview of the structure of schooling between 1890–1900, Larry Cuban reminded educators “the apparent uniformity in instruction irrespective of time and place appears connected to the apparent invulnerability of classrooms to change” (1995, 1). Cuban identified minuscule change in the structures of schooling over time and observed that innovation in schools tended to be reserved for a subcategory of students such as the gifted, but not implemented with the main population (1995). Cuban’s analysis neglects to detail the ways schools’ documented invulnerability to change maintains anti-blackness in structures, curricula, and personnel. These realities are transposed into online learning environments and digital learning tools that are assumed to be race-neutral. For example, Borje Holmberg formed a theory of distance teaching that advocates for personalized distance education but avoids the ways race informs individual learners, tech tools, or online environments (1995). The major players in online education still hold tightly to racist Enlightenment ideals of rugged individualism and the belief in the disembodied articulation of the self (Asenbaum 2019). In other words, white supremacy and its chief actor, whiteness, still maintain a hegemonic hold on online learning. More explicitly, race-neutral language transposes whiteness to educational technology as normative, reifying that white people are the standard for humanity, thus relegating blackness to sub-human. Online education operates with race-neutral rhetoric that obscures how race informs everything.

Despite the prevalence of anti-blackness in online education, collaborative technology platforms like Twitter, Snapchat, and Facebook have provided disruptive spaces of resistance due to their ability to transcend traditional forms of networking and collaboration. This is largely due to these technologies’ user-centered platforms, which allow users to design social communities and develop and disseminate their own innovations. Facebook and other Web 2.0 collaborative platforms have become impactful spaces to negotiate and transform the traditional “boundaries of the classroom,” where the teacher directs and designs all learning and students merely respond to often irrelevant, dated content within the confines of the physical classroom space (Dennen 2018, 239). Disruptive social media platforms on the other hand allow learners to design thinking, select relevant topics of interest, and engage in expedient dialogue and response to real world issues. For example, with the widely-publicized killings of unarmed Black people at the hands of law enforcement and security personnel—Daunte Wright, Rayshard Brooks, George Floyd, Breonne Taylor, Atatiana Jefferson, Trayvon Martin, Philando Castile, Tamir Rice, Yvette Smith, Rekia Boyd, and Botham Jean, and Michelle Cusseaux, just to name a few—people form across the globe organized through online activism to resist racial injustice. Online activism is indicative of the power of digital tools to fight racism, yet online education broadly has yet to seriously move toward antiracist pedagogies.

Valcarlos, Wolgemuth, Haraf, and Fisk (2018) queried all peer-reviewed articles from the past 11 years to ascertain the presence of anti-oppressive pedagogies in scholarship related to online education. Of three thousand articles, they found ten that dealt specifically with anti-oppressive pedagogies in online education. Out of those ten articles, four common themes emerged: legitimizing students’ epistemologies (personal narratives, emotions, and culture), requiring reflection and discussion, establishing expectations of critical awareness, and democratizing educator and student roles (351). With the exception of these articles, the online learning community has been almost mute on critical social justice concerns (Valcarlos, Wolgemuth, Haraf, and Fisk 2020). Even fewer articles explicitly name the role of antiracist pedagogy in online education.

Bridging the Gap: The Potential of an Antiracist Future in Online Education

The absence of specific mention to antiracist pedagogies in the extant literature and theory of online education is emblematic of the normativeness of anti-Black racism and white normativity in online education. A corrective is essential to equip students with the requisite forms of racial literacy to constantly reflect upon their world in a responsible manner (Tarrant and Thiele 2014). Racism is a system of historical oppression that is built on an hegemony of power and domination that privileges certain groups as inferior to the dominant group (Harrell 2000). In the hegemony of racism, certain ideas shape the construction of the constituent parts of society (business, education, law, and medicine) and give rise to accepted behaviors and belief systems (Gramsci 1989). To be neutral on racism is to be complicit in racist ideas; there is no in-between (hooks 1994; Kendi 2019). In what follows we will provide a framework for thinking through an antiracist pedagogy for online education.

We take up Zachary Casey’s (2016) framing of pedagogy to help shape our understanding of an antiracist pedagogy. In his book, A Pedagogy of Anticapitalist Antiracism: Whiteness, Neoliberalism, and Resistance in Education, Casey defined pedagogy as an action to “foster (political, partial, humanizing) learning, in ways that acknowledge the political nature of human interactions and the varying context(s) in which we live” (18). Shaped by the liberatory visions of Paulo Freire, bell hooks, Audre Lorde, and others, Casey argues that every choice is political: that each action, each choice we make requires making particular judgements and conceptualizations about reality and knowledge that emanate from our ethical framework. An antiracist pedagogy acknowledges the political and partial nature of teaching; there is no such thing as neutrality. When we separate our being from our doing, our knowing from our doing, we are adhering to a racist, colonialist imagination. How we teach is a byproduct of how we were taught to view the world.

David Gillborn posited that “Anti-racism has not failed—in most cases; it simply has not been tried yet” (2006, 17). He observed that antiracist work in education arose as a response to the performative liberal practices used to serve Black children and their families but were deeply conservative in nature, contained no awareness of the systemic nature of oppression, and were actually rooted in deficit perspectives of Black people. Antiracism work is about dismantling racism, but it is much more than that. Racism takes many forms and so antiracist actions must be flexible and constantly adapt to the complex nature of reality. To be antiracist means to constantly be about the work of dismantling the racist ingrained nature of teaching.

Nevertheless, without a clear framework for antiracism, the use of antiracism becomes empty rhetoric, what Sara Ahmed (2004) calls non-performative. Often, antiracist statements themselves become the only actions that an institution takes. Or they become the very barrier to developing an antiracist ethos in an institution or classroom and provide no actionable way to diagnose racist versus antiracist praxis. When this happens, explicitly naming racist practices, and examining complicity in racist ideas become non-existent because the institution and the people in it have branded themselves as antiracist. To continue calling the institution racist after the institution has made a commitment to being antiracist becomes undesirable. Likewise, racism then becomes the boogey woman. No one wants to utter its name. But taking up antiracism is a process not a destination. You can act in antiracist ways one day and the next day act in racist ways or uphold racist ideas (Kendi 2019).

Antiracist education accepts the presence of bias and stereotypes but requires employing diligent and consistent investigation into the source of racism and how racist ideas manifest structurally, culturally, politically, and interpersonally (Troyna 1987; Collins 2017). An antiracist pedagogy for online education begins with creating spaces that bring attention to race, class, gender, and ability. An antiracist online environment begins first with an articulation of online learning as an embodied digital discursive space. In other words, enacting an antiracist pedagogy in online learning begins in the body. It requires students and teachers alike to bring their full selves (this is a collective self, not an individual self) into the online learning environment (Dillard 2006).

Specific Antiracist Pedagogies for Online Education

In this section, we describe classroom practices, activities, and experiences we employ in our online classrooms as examples of antiracist pedagogy for online education.

Showing up and the power of the aesthetic: Jazz, freedom dreaming, and liberatory teaching

In each of my (David’s) courses, I start with an artifact exercise that can be done synchronously on the first day of class or asynchronously using a video platform such as Flipgrid or YouTube to facilitate sharing, and a discussion board prompt to debrief the activity (Figure 1). The artifact exercise is also coupled with several modes of engagement: pre-course survey, at least two readings that provide context for a discussion on identity, a related video, and a few thoughts that foreground group values for the course. Providing this level of scaffolding for the artifact exercise is essential to create an environment that encourages authenticity and transparency and ensures students have adequate context for the ensuing dialogue. This exercise serves multiple purposes. First, it is an icebreaker, an opportunity for students to ease into the new semester and get to know each other. Secondly, the goal of the exercise is to foreground very early in the course that teaching and learning are not neutral acts. That to show up in embodied ways means to give attention to the weight that your raced, gendered, and classed selves takes up in the classroom. This exercise invites all parties involved to see each other in embodied ways (Hill 2017).

Finally, the activity is an opportunity for students to share something from their life-world that is important to them and reveals an aspect of their culture they believe is important for our sacred learning experience. For example, after everyone has shared, I invite the class to identify connections or themes across the shared stories. After the connection phase, I emphasize the uniqueness and interconnected nature of our stories. I also point to how the stories reveal values passed down to us and therefore the “presence” of our ancestors;many students actually share pieces of jewelry and pieces of cloth that were given to them by their now deceased grandparents and great grandparents. I inform students that the fact these values still shape how we will interact and engage content in the course is what makes the learning environment sacred and also a space of potential conflict.

Screenshot of computer screen with words detailing course announcements. — Figure 1. Screenshot taken from the Canvas learning management system of one of David’s pre-course announcements to students about the artifact exercise and supported activities.

The types of artifacts that students bring are often connected to certain values their parents, grandparents, and others have passed down to them. These values and worldviews shape how the students engage each other and make sense of course material—this point is also consistently made by students as they share during the artifact exercise. For example, in one of my classes I had a student share an artifact from a grandparent that emphasized the idea of collectivity. In that same class, another student shared an artifact that a great aunt gave them which reinforced the value of individualism—the exact words were, “pulling yourself up by the bootstraps and getting your work done.” Both students offered conflicting values that were central to their unique worldviews and approaches to learning. The artifact exercise makes it explicit that the online classroom experience is an intergenerational contested space full of imaginations, images, cultures, and values that shape the ontological, epistemic, and ethical visions of the learning space for the teacher and students (Sheppard 2017). The intergenerational contested reality of the classroom necessitates that cultural wars are constantly being waged, most of them occurring undetected behind the scenes.

Antiracist pedagogies are anticipatory in nature. To be antiracist is to anticipate and welcome conflict as a companion in the learning process. As conflict occurs, antiracist pedagogues must be intentional in explicitly naming what is happening. In the example above with the two students, I used their examples to invite students into a connections and synthesis phase of the artifact activity where we engaged in dialogue about issues of identity, power, and perspective taking—we reference the articles they would have read before class to support this movement—in order to understand and draw connections between the values expressed by students and the ways of knowing and being that are privileged in education (e.g. rugged individualism vs. collectivism).

I also begin each class session with approximately five minutes of an invocation. The invocation experience—which is actually done first at the very beginning of class—and the artifact exercise flow together to concretize the sacredness of our collective learning task and the fact that learning is an intergenerational experience. One of the amazing Black women in my research who identified as womanist first introduced a “pedagogy of invocation” to me (Humphrey Jr. 2020, 93). Often this first five minutes will consist of a song, something soulful and rhythmic like Nina Simone’s “I Wish I Knew How It Would Feel to Be Free” or Thelonious Monk’s “Round Midnight.” Invocation simply means to invoke someone or something for assistance or authority. In the Black prophetic tradition, this act is used to acknowledge that what we are about to embark upon is bigger than ourselves. That we are not alone in this moment. That this moment is fixed in dialectical union with the present and future. That we bring our full selves (a collective self) into this space including our mind, our body, our memories, dreams, and the sacred witness of our ancestors (their lessons passed down to us, both good and bad, whether we want to admit to it or not and whether we realize or remember these messages or not). During the invocation, I instruct students to do the following: “Please take this moment and listen to the words, rhythm, and melody of the song. Meditate on what you are hearing; be sensitive to what your mind and body are saying to you as you listen. After the song concludes, we will begin our collective task.” After the second or third class, I invite students to share their favorite songs and lead the invocation. I always bridge the mindfulness moment with the course content of the day. While most effective in the synchronous online class, this activity can be adapted for asynchronous learning environments by posting the invocation as a required first task in a module using YouTube or any other video platform that allows you to pre-record video and a discussion post-feature.

Collaboratively innovating content with students

By pairing collaborative digital tools with contemporary, relevant racial justice content through an embodied pedagogy, online education can become antiracist. An effective antiracist approach in online learning is to connect all content to the racialized reality every student and teacher is traversing. In my classroom, I (Camea) do this by inviting students to make real-world connections between course content and racial justice. This is central to my antiracist teaching strategies and not a cursory exercise or attempt at culturally relevant gimmicks (Love 2019). This is exemplified in my online spring 2021 doctoral qualitative research methods course titled, “Critical Ethnographic Methods for Social Justice Research,” in which student researchers used critical ethnography research tools to design research studies that contributed to changing conditions toward greater freedom and equity; to amplify minoritized participant experiences; and to humanize the research act. This course was created in response to COVID-19 and the anti-Black violence and other social unrest occurring in the US, and was delivered synchronously with weekly meetings on zoom. In this course, students were invited to make sense of their own racial identities by exploring researcher positionality, rapport with research participants in the field, and accessing and exiting the field.

For example, a white woman student researcher in a midterm presentation made connections between the 2020 presidential executive order banning critical race theory in federally funded trainings and the critical ethnographic research to demonstrate that the latter was an effective example of social justice research. Figure 2 is a screenshot from the student’s Zoom presentation that details her analysis. In doing this presentation, the student researcher reflected on her own whiteness and how it informed the choice of book she selected for her midterm and how it impacted her positionality as a researcher. By inviting student researchers to make connections between the course content, themselves, and the real world in class, I created space for authentic engagement with race and racism, and allowed students to innovate paths towards antiracism. This is the work.

The pedagogical elements of the assignment design and the creation of a classroom community that could hold this type of learning were grounded in student choice, criticality, and a learning community where students were safe to introspectively reflect on how race impacts all we do and learn.

Creating an antiracist online-class Zoom ethos

Additionally, when attempting to employ antiracist pedagogy in an online context it’s imperative that the ethos of the digital space exude the brilliance and intellectual rigor of Black and other minoritized peoples and cultures. The class ethos is apparent by the way students are made to feel. My online courses center an ethos of Black creativity by playing an upbeat song from various Black music traditions such as hip hop, pop, neo-soul, or gospel as students log onto Zoom. During this time the music plays in the background and students welcome one another verbally or in the chat. This is a tone-setting exercise that grounds the space in Black aesthetics to communicate that Blackness is celebrated here. The songs are not discussed; they just exist like the wall decor in a physical classroom. Similarly, when I use slides (which is rare because my pedagogy is dialogic and interactive in nature) I intentionally use slides that have artwork of Black and Brown faces and use cultural icons in the designs. Again, this is an aesthetic choice that communicates that celebrating Black culture is a part of how we do everything; nothing is race-neutral. This practice invites students to feel safe to share their own cultural artifacts when they present.

Furthermore, I co-create an antiracist class-zoom ethos with students by setting norms that are particularly valuable for antiracist praxis such as: “own our subjectivity,” “challenging each other with respect,” and “ question your own lens/perspectives without fear.” These norms are essential to disrupt the notion that race is a taboo or scary topic in the classroom. Likewise, I lead the class agenda from an Africanist (King 2019) perspective of time grounded in abundance. I remind students there is always enough time. In my fluidity I use a structure that always includes space for intuition and possibility. I require all cameras be on during discussion therefore I can gauge if students are confused, excited, or tired, and I always have the time to respond to those human aspects of their learning in the moment.

My class-zoom ethos is strengthened because students are positioned as co-facilitators.The syllabus includes student voice and choice in each assignment and each student is required to co-facilitate a discussion. Students are encouraged to fuse their own identities and interests with interactive technological tools. For example, one bi-lingual Latinx student used www.getepic.com to share the children’s e-book “Salsa” by Jorge Argueta, which depicted visual art and a “cooking poem” about a Mexican-American family creating salsa as a metaphor for engaging Mexican-American research participants. The student screen-shared the e-book pages and read the book to the class in Spanish unapologetically to model the purpose and need for research participants being their authentic selves. Beyond the content of his presentation, the fact that this doctoral student felt safe and welcomed enough to share his home culture and language is evidence of an embodied pedagogy from which all learners can grow.

Drawing of 2 children sitting around a table. — Figure 3. Screenshot of Epic digital resource.

Conclusion

To thrive in the afterlife of COVID-19, abnormal environmental disasters, and the continued murdering of Black bodies, all education—especially online education—must become antiracist. Neither online education nor digital tools are exempt from the deeply anti-Black racism that are the bones of US education systems. Educators in online contexts collaborating with students are well-positioned to create digital learning spaces that intentionally (above all else) work to eradicate the insidiousness of racial violence perpetuated through the myth of race-neutral learning theories and pedagogical practices.

This is great teaching for all students. We use BlackCrit to guide this discussion, we invite our readers to understand that the imaginative resistance and freedom in blackness is a guiding light for all educators and learners in all contexts. In the spirit of Dubois, we believe that addressing the issues that perpetuate racism and anti-blackness are the key to liberating all humanity. The pedagogical principles shared in this paper are meant to reconcile the disembodied nature of online education as it exists.

Writing as two scholars who frequently facilitate antiracist spaces occupied by white colleagues, we anticipate the oppressive question, “Can white people use these pedagogical tools?” We invite these well-intended inquirers to ask themselves what about their socialization and relationship to teaching and learning prompts them to center whiteness? Moreover, we point out this question is never asked about white scholars or scholarship that originates from Eurocentric locations. What if the answer to this question was grounded in decentering whiteness? Instead of needing to center anything and employing hierarchical language, we challenge (online) educators to think interstitially (Spillers 2003). This invites a vision of learning that is not governed by power but is motivated by humanizing the community through an embodied teaching approach.

We can’t afford to wait. The future started yesterday and we’re already late.

Bibliography

Ahmed, Sara. 2004. “Declarations of Whiteness: The Non-Performativity of Anti-Racism.” Borderlands E-Journal 3, no. 2.

Asenbaum, Hans. 2019. “Rethinking Digital Democracy: From the Disembodied Discursive Self to New Materialist Corporealities.” Communication Theory qtz033: 1–33. https://doi.org/10.1093/ct/qtz033.

Bernard, April, and Onwubiko Agozino. 2012. “Free Space and Inner Space: A Place for Reconstructing Self and Other.” Journal of Pan African Studies 5, no. 6: 56–66.

Bonilla, Yarimar, and Jonathan Rosa. 2015. “#Ferguson: Digital Protest, Hashtag Ethnography, and the Racial Politics of Social Media in the United States.” American Ethnologist 42, no. 1: 4–17.

Bonilla-Silva, Eduardo. 2009. Racism without Racists: Colorblind Racism and the Persistence of Racial Inequality. 3rd ed. Lanham, Maryland: Rowman & Littlefield.

Bonk, Curtis J. 2009. The World is Open: How Web Technology is Revolutionizing Education. San Francisco: Jossey-Bass.

Burke, L. 2020. “Moving Into the Long Term,” Inside HigherEd, October 27, 2020. https://www.insidehighered.com/digital-learning/article/2020/10/27/long-term-online-learning-pandemic-may-impact-students-well.

Casey, Zachary A. 2006. A Pedagogy of Anticapitalist Antiracism: Whiteness, Neoliberalism, and Resistance in Education. Albany: State University of New York Press.

Collins, Patricia H. 2017. “The Difference that Power Makes: Intersectionality and Participatory Democracy.” Investigaciones Feministas 8: 21–41.

Collins, Patricia H., and Sirma Bilge. 2016. Intersectionality. Cambridge, UK: Polity Press.

Cone, James H. 2018. Black Theology and Black Power. Maryknoll, New York: Orbis Books.

Cuban, Larry. How Teachers Taught: Constancy and Change in American Classrooms, 1890-1990. New York: Teachers College Press, 1993.

Davidson, Cathy. 2020. “The Single Most Essential Requirement in Designing a Fall Online Course.” Hastac, May 11, 2020.https://www.hastac.org/blogs/cathy-davidson/2020/05/11/single-most-essential-requirement-designing-fall-online-course.

Dillard, Cynthia B. 2006. On Spiritual Strivings: Transforming an African American Woman’s Academic Life. Albany: State University of New York Press.

Dumas, Michael J., and kihana m. ross. 2016. “‘Be Real Black for Me’: Imagining BlackCrit in Education.” Urban Education 51, no. 4: 415–442. https://doi.org/10.1177/0042085916628611.

Gillborn, David. 2006. “Critical Race Theory and Education: Racism and Antiracism in Educational Theory and Praxis.” Discourse: Studies in the Cultural Politics of Education 27, no. 1: 11–32.

Gramsci, Antonio. 1989. Selections from the Prison Notebooks. New York: International Publishers Co.

Harrell, Shelley P. 2000. “A Multidimensional Conceptualization of Racism-Related Stress: Implications for the Well-Being of People of Color.” American Journal of Orthopsychiatry 70, no. 1: 42–57.

Hartman, Saidiya. 2007. Lose Your Mother: A Journey Along the Atlantic Slave Route. New York: Farrar, Straus, and Giroux.

Humphrey, David L., Jr. 2020. “In the Wake of our Womanist Foremothers: Resistance as Signif(y)er Among Self-Identified Womanist Scholars in Higher Education.” Ph.D. dissertation, Educational Studies, Ball State University (2413531391).

Hill, Dominique C. 2017. “What Happened When I Invited Students to see me? A Black Queer Professor’s Reflections on Practicing Embodied Vulnerability in the Classroom.” Journal of Lesbian Studies 21, no. 4: 432–442.

Holladay, Hilary. 2020. “OC Teachers Speak of Frustration, Exhaustion at School Board Meeting.” Orange County Review, September 17, 2020. https://dailyprogress.com/community/orangenews/oc-teachers-speak-of-frustration-exhaustion-at-school-board-meeting/article_30543406-f848-11ea-b95a-eff92d9c054d.html.

Holmberg, Borje. 1995. Theory and Practice of Distance Education. New York, NY: Routledge.

hooks, bell. 1994. Teaching to Transgress: Education as the Practice of Freedom. New York: Routledge.

Kendi, Ibram X. 2019. How to Be an Antiracist. New York: One World.

King, Joyce, Thais Council, Janice Fournillier, Valora Richardson, Chike Akua, Natasha Mcclendon, Adrian Neely, Glenda Chisholm, Tiffany Russell, Fernanda Vieira da Silva Santos, and Mikala Streeter. 2019. “Pedagogy for Partisanship: Research Training for Black Graduate Students in the Black Intellectual Tradition.” International Journal of Qualitative Studies in Education. 32: 1–22.

la paperson. 2017. A Third University Is Possible. Minneapolis: University of Minnesota Press.

Lambert, Diana, and Betty M. Rosales. 2020. “California School Districts Struggled to Prepare Teachers for Distance Learning this Fall.” EdSource, October 13, 2020. https://edsource.org/2020/california-school-districts-struggled-to-prepare-teachers-for-distance-learning-this-fall/641442.

Love, Bettina L. 2019. We Want to Do More Than Survive: Abolitionist Teaching and the Pursuit of Educational Freedom. Boston: Beacon Press.

Mbembe, Achille. 2019. Necropolitics. Durham: Duke University Press.

Morrison, Toni. 1993. Playing in the Dark: Whiteness and the Literary Imagination. New York: Vintage.

Moten, Fred. 2013. “Blackness and Nothingness (Mysticism in the Flesh).” The South Atlantic Quarterly 112, no. 4. https://doi.org/10.1215/00382876-2345261.

Moten, Fred, Saidiya Hartman , J. C. Carter, and Sarah J. Cervenak. 2016. “The Black Outdoors: Humanities Futures after Property and Possession.” John Hope Franklin Humanities Institute, Duke University, 2:04:02.
https://fhi.duke.edu/videos/black-outdoors-fred-moten-saidiya-hartman.

Palmer, Parker. J. 2017. The Courage to Teach: Exploring the Inner Landscape of the Teacher’s Life. San Francisco: Jossey-Bass.

Reiser, Robert A. 2001. “A History of Instructional Design and Technology: Part I: A History of Instructional Media.” Educational Technology Research and Development 49, no. 1: 53–64.

Rury, John L. 2009. Education and Social Change: Contours in the History of American Schooling. New York: Routledge.

Sheppard, Phillis I. 2017. “Hegemonic Imagination, Historical Ethos, and Colonized Minds in the Pedagogical Space: Pastoral Ethics and Teaching as if our Lives Depended on it.” Journal of Pastoral Theology 27, no. 3: 181–194.

Simonson, Michael, Charles Schlosser, and Anymir Orellana. 2011. “Distance Education Research: A Review of the Literature.” Journal of Computing in Higher Education. 124–142.

Spillers, Hortense J. 2003. Black, White, and in Color: Essays on American literature and Culture. Chicago: The University of Chicago Press.

Troyna, Barry. 1987. “Beyond Multiculturalism: Towards the Enactment of Antiracist Education in Policy, Provision, and Pedagogy.” Oxford Review of Education 13, no. 3: 307–320.

Valcarlos, María M., Jennifer R. Wolgemuth, Samantha Haraf, and Nathan Fisk. 2020. “Anti-Oppressive Pedagogies in Online Learning: A Critical Review.” Distance Education 41, no. 3: 345–360. https://doi.org/10.1080/01587919.2020.1763783.

About the Authors

David L. Humphrey, Jr. is a jazz and justice-loving scholar-practitioner, who is guided by a radical love ethic. He currently serves as the Chief Diversity and Inclusion Officer for the School of Education at the University of Michigan, Ann Arbor. In his current role, David is a strategic partner and thought leader for the work of diversity, inclusion, justice, and equity (dije) in the SoE. David’s research sits at the nexus of curriculum theory, BlackCrit (fugitive) and liberatory traditions, and student learning and development.

Camea Davis is the assistant director of the Center for Equity and Justice in Teacher Education and a research assistant professor at the College of Education & Human Development, in the Department of Middle and Secondary Education. Her research focuses on racial justice in teacher education, critical collaborative ethnography, and critical poetic inquiry. Davis has published in Qualitative Inquiry; Equity & Excellence in Education; The Journal of Middle School Education; Art/Research International: A Transdisciplinary Journal; Ubiquity: The Journal of Literature, Literacy, and the Arts; The Journal of Hip Hop Studies; and The Journal of School and Society.