Issue Twenty

Conference attendees are pictured from the side, under dramatic Russian Orthodox church windows.
0

A Conversation on International Collaboration in Digital Scholarship

Translations

Russian translation by Kseniia Tereshchenko | Russian Translation (PDF).
Arabic translation forthcoming.

Introduction

The conversation published here among three “bridge” figures in global digital humanities took place on January 25, 2021, over Zoom as part of the program of Saint Petersburg Digital Humanities Week (SPbDH) 2021. Each of the speakers has each been instrumental in organizing digital humanities events in their respective communities: NYCDH Week, SPbDH Week and the NYU Abu Dhabi Winter Institute in Digital Humanities (WIDH). They have also cooperated with each other in sharing experiences, expertise, and in some cases programming. They all met in person at a NYCDH Week event hosted in early 2020 by the NYU Abu Dhabi Institute entitled “International and Interdisciplinary: Collaborations in DH Research.” SPbDH week has traditionally been a place-based event, as has the NYCDH Week; both events even carry the name of the city in their title. Over time, these events (along with the WIDH in Abu Dhabi) have served a community-building function, making them all the more meaningful. In 2020–2021, the pandemic put this focus on place under a significant amount of stress, making it impossible to convene in person. Yet, as difficult as this period has been, we do believe that it offered a unique opportunity to reflect on questions of international, institutional cooperation among like-minded colleagues, thinking particularly about how dialogue about the internationalization of DH ultimately feeds back to our local communities. The moderator of the panel was Lada Zimina.

Dialogue

Lada: Hello, dear guests and colleagues! I am very happy to welcome you to the opening panel for the third annual Saint Petersburg Digital Humanities (SPbDH) Week. The topic of our discussion today is Think digitally, act humanely: building DH communities locally and globally.” Our speakers are Kimon Keramidas, Associate Professor at Experimental Humanities and Social Engagement, NYU and also co-director of the International DHLab at Informational Technology, Mechanics, and Optics University (ITMO University); David Joseph Wrisley, Associate Professor of Digital Humanities at NYU Abu Dhabi; and Antonina Puchkovskaia, a director of the DH Center at ITMO University and the founder of SBbDH Week. To begin today, Antonina, I would like to ask you to tell us a bit about your experience in the global DH community and your work consolidating the local DH community in Saint Petersburg? How do DH weeks, both NYC and SPb, fit into your story with DH?

Antonina: Thanks for a very good question. I suppose everyone who is doing DH has their own story about how everything started. With a Ph.D. in Cultural Studies, I ended up teaching at a very STEM-based University like ITMO (Information Technology, Mechanics and Optics), and I’d started to think about how I may bridge two different universes—Humanities and Computer Sciences. I did some research and discovered a pretty promising co-location known as Digital Humanities. I dug deeper and was nicely surprised and excited about the manifesto, agenda, and most importantly the community, which from the onset seemed quite open and friendly. I started looking for events that gather DH people together. Within the first lines of my Google search was NYCDH week, the event which later became a model for our SPb DH week. I proposed a workshop on how to turn your Humanities course into a digital one. Not only was I accepted and got to run my very first DH workshop, but I met all those wonderful people (among them was Kimon) who were very supportive and helpful, answering all my questions about the fundamentals of DH.

When I returned to Saint Petersburg, the first thing I did was to talk to our first vice rector about launching a DH lab. Being a great supporter of different interdisciplinary initiatives at the university, she gave the green light to establish a small DH research lab, and Kimon continued to advise me through the whole process. A bit later, our DH lab team was awarded a grant from our university for developing an interactive map of Saint Petersburg with a specific focus on relations between the landmarks and famous people associated with them. This project now exists as a web and mobile application, and it also contributed to building a DH community in Saint Petersburg, especially by integrating the local into the international.

In four years, we have succeeded in launching a Master’s program in Data, Culture, and Visualization (DCV)  with a track in Digital Humanities, and the DH lab grew into an International DH Center co-directed by Kimon. Still having great support from our university, we are now more focused on community development and recruitment for our Master’s program at the intersection of Humanities and Computer Science. We have run the annual SPbDH Week for three years now and the trend is quite convincing: we tripled the number of workshops, lectures and lightning talks within the week (despite the global pandemic) and, most importantly, increased our community fivefold. The geography of the participants is also very impressive: people come from Moscow, Perm, Vladivostok, and other Russian cities either to give or attend a workshop. This year, we went even further and organized an international panel. The DH community is very helpful and friendly and that is why we will continue to develop our local community hand in hand with the international community, dedicating our resources and energy to make more DH projects collaboratively, making more data available, open, and shareable, and creating more interesting discourses around it.

Lada: Thank you, Antonina! Talking about SPbDH Week, I would like to add a small personal observation of my own. As a member of the organizing committee, I noticed that in Russia quite often people who use digital methods in their own area of expertise and who are willing to offer a workshop are not really familiar with the term “DH.” In that perspective, the event is really about community building, because it shows the researchers that they are not alone and that what they do actually has a name and a community behind it. Moreover, we have many cases when speakers or guests stay in touch with the Center or even work with us as lecturers.

Kimon, can you tell us how and why DH week started in New York? What was your motivation as an organizer? And how’s it going?

Kimon: Within New York there are many large institutions: Columbia, NYU, the City University of New York (CUNY) Graduate Center, Fordham University, Hofstra University, and Pace University, and all of the cultural heritage organizations as well. Unfortunately, what happens is that these places are big enough to become insular. With NYCDH, we’re trying to foster a really open community outside of those bounds. NYCDH is a non-institutional, inter-institutional organization, so we’re not reliant on any single university or organization as a center. Running NYCDH in this way is based on the recognition that good work in DH often necessitates collaboration between different types of people. David, Antonina, and I all are used to collaborations, whether those collaborations are with other professors, data services, or librarians, and the events we organize are an opening for these kinds of collaborations.

NYCDH Week started because we wanted to share existing workshops at different institutions, but it was complicated by different curricula. So, we decided to create a structure completely outside of that. It began small, but has grown beyond our wildest expectations. In 2020, we had more than forty sessions and over 800 registrations. This year we only had thirty-two sessions—because of COVID—but we ended up with more than 1500 registrations, and included sessions being run both through NYCDH in New York and others organized at NYU Abu Dhabi. We were even thinking of aligning with Saint Petersburg DH Week to create an even greater sense of global simultaneity. People are starving for these kinds of gatherings and look forward to NYCDH week every year.

NYCDH Week is a movement based on the idea that we can change DH work within the academy by developing a community of people who are willing to give their time, be open with their work, share information, and create networks of participation. I know of an enormous amount of connections between colleagues that have been created through NYCDH Week, our graduate student awards, and web platforms.

The next step is figuring out how you manifest those experiences and interests into programmatic changes back at your institution. At ITMO, we’ve made a significant amount of progress, going from virtually nothing to having a successful Master’s program in just a few years. There are further possibilities in Moscow and in Perm to expand this in Russia, and David has been doing this work in the Middle East for twenty years in both Abu Dhabi and Beirut. Our hope is that we can bring together these three places that aren’t often in collaboration in Russia, the Middle East, and in the US or New York.

Lada: Thank you, Kimon! Do you have something to add, David?

David: Thank you for the invitation to join the conversation. As I listen to the two of you speak about the way the community is emerging in Russia, and particularly in the context of ITMO, I am really pleased to see what you have achieved. It’s quite remarkable that the institutionalization of DH has taken place so quickly. It is a testament to the passion and focus with which you have worked, but also to the openness of the future-oriented approach of your institution. In many places in the world, there’s a lot of interest in digital research in the humanities, but not a lot of action. It can be, for many people, a scary jump.

I remember some ten years ago when we had our first event in the Middle East, in Beirut, and then participating in some of the different community models that Kimon brought up: the Digital Humanities Summer Institute (DHSI) (University of Victoria, Canada), the European Summer University of Digital Humanities (ESUDH) ( Germany), NYCDHweek, and then returning home to think about how to adapt those community models to our local environment. Localization is not an easy process.

You’ll have lots of eyes on Russian DH this year with the European Association of Digital Humanities (EADH) annual conference. So, congratulations on getting this going!

Lada: Thank you, David! We are actually really excited about the upcoming conference in Krasnoyarsk, as our DH Center together with the colleagues from the State Hermitage Museum, the Institute of Russian Literature, and the Museum of the History of Religion are organizing a panel aimed at community building, entitled “Digital Humanities Researchers and Cultural Institutions: Towards Productive Interaction” (see Puchkovskaia et al. 2021). We think it’s important to connect DH researchers from academia and specialists from the Galleries, Libraries, Archives, and Museums (GLAM) sector. I really hope the panel will help to consolidate the local DH community and maybe even some collaborative projects.

David, do you think DH brings people together, or does it create new forms of isolation and specialization? Could you also tell us more about your personal experience with DH research in the field of medieval studies and if the digital methods are welcome in that community?

David: My research interests have been diverse. First, I am trained as a comparative medievalist, but I have also lived and worked in Arab countries for most of my career. In the last decade, it’s not surprising that I have been working on new ways to bring the digital to those two communities: medieval studies and the Arabic-speaking world. Sometimes they overlap for me, but for the most part they do not.

I have experienced both an expansion of my community and some forms of isolation in my career. Integrating the digital into one’s speciality subfields is an entirely different endeavor than working to build a transdisciplinary DH community. There is a vibrant community of digital medievalists in the world, but they are still somewhat marginalized within medieval studies. I believe that the situation is slowly changing, and we may be less isolated post-pandemic, but it is fair to say that most medievalists are not involved in digital research. In specific specializations—say, digital medieval studies in Arabic, French, or Latin—the percentage drops significantly. What this means is that if you are doing digital humanities community building aimed at a larger audience, then your audience is much wider than it would be in a specialized field.

Shared methodologies most certainly bring people together. Imagine that a medievalist’s daily interlocutors could be Americanists, art historians, computer scientists, or linguists working in entirely different languages. It is often said that in universities, the digital is increasingly providing common vocabularies for a wide variety of research and community building amongst researchers. There’s a commonality in the challenges that we face—in the ways we produce our data and manage our research—which is becoming increasingly difficult to ignore.

Working simultaneously in a transdisciplinary space, where our colleagues have very different ways of working, and in a specialist subdomain, interacting with domains close to us in the organization of knowledge, can be both challenging and very rewarding.

Lada: Thank you, David, for setting such an encouraging tone to the discussion. As for Saint Petersburg, and maybe even Russia in general, I feel that we are not quite there either. Hopefully big events, like EADH2021, will promote DH in Russia and boost the community-building process. Kimon, being a New Yorker, what are your thoughts on the same question? Does DH unite us, or does it separate us?

Kimon: I’ve learned a lot watching David because he has reached out internationally, building coalitions and relationships in what can sometimes be a provincial community. New York is large enough that people can often find enough relationships to stay working within that community. The same can be said for DH communities around Washington, D.C., and Virginia, and Southern California, where there are enough people that a kind of regional DH identity even begins to take shape. But as you begin looking at the situation globally, with DHSI in Canada, in Leipzig in Germany, and of course David’s work across the Middle East, you realize that it takes different textures of collaboration to work with different people in different communities. This is particularly important in this continuing period of transition, as people doing DH are still facing resistance in traditional disciplinary environments. That’s where occasions like SPbDH Week, NYCDH Week, and WIDH are so valuable, because they highlight that there are networks you can tap into to find like-minded people.

One thing Antonina and I talk a lot about with regards to ITMO is that the path to growth here is from the technical to the humanities. It’s the reverse from how it occurs predominantly in the US at least, and it is an intriguing challenge as you determine how best to transmit these ideas to an unfamiliar audience and how to engineer these collaborations.

What I appreciate about DH communities is how different people can come together and look for balance. You can engage with someone and say, “You do something completely different than I do but we probably have some common thoughts and make each other’s work better.” I think that that eagerness is probably a global trend that still benefits a lot of people. And the people who are eager to put that foot forward tend to be slightly more gregarious and open to the conversation rather than resisting. It takes that kind of nature, so it creates a good community right now.

Antonina: I would like to add something about how at ITMO we are trying to complement our skillset with the competencies we lack. Our institution is very STEM-based and we lack a strong Humanities faculty, so our way of bridging the gap between Humanities and digital technologies was to engage with the GLAM sector in Saint Petersburg. Luckily, we have plenty of cultural institutions in our city and they have shown a willingness to collaborate on questions of common interest, such as database architecture for collecting and storing data, building user-friendly interfaces, expanding a user base through digital storytelling, etc. The additional benefit is that we have also found a great number of people who are motivated and encouraged to learn more and to collaborate on DH projects. I would also like to highlight that establishing these relationships is an ongoing process, and it is very time- and energy-consuming. But it is really worth it. And in our case, working with such wonderful institutions as the Museum of the History of Religion, Mayakovsky Central Public Library, the Institute of Russian Literature, and many others, helps us both to build and develop the DH community and to represent cultural heritage and memory in digital environments. Furthermore, these cultural institutions become places for our students to do internships. It is a win-win for both academia and the GLAM sector.

Lada: Antonina, you mentioned the Master’s program that you run at ITMO. What are the challenges that you face while designing and running a Master’s program in DH? How do you explain digital humanities to your administration and other faculty at ITMO and how do you teach the field to students who may be completely unfamiliar with its methodologies and terminology?

Antonina: That’s a very good question and I believe an open one. I would like to answer it from two perspectives, both from the position of an academic leader of our Master’s program in Data, Culture, and Visualization and from the role of a DH educator. Designing a two-year Master’s program at the intersection of Humanities and Computer Sciences, I had to take a lot into consideration. First of all, I meant to build such a program for students with various backgrounds. So, developing a curriculum I focused on including more project-based modules, which allow students to work collaboratively and complement each other’s skill sets. Second of all, I tried to design a curriculum around fundamentals of digital technologies adapted to students’ varied skill levels, such as the most popular programming languages (Python and R), database development, data mining, UX/UI, visualization, text analysis, project management, etc. Moreover, I always wanted to introduce our students to a vibrant, multidimensional, and multilingual world of DH, by regularly inviting colleagues to participate in various hands-on workshops, guest lectures, reading groups, and summer schools.

Starting from the second semester, our students choose from two tracks provided at the program: one is in Digital Humanities taught in Russian, and the other is Cultural Analytics taught in English. Beyond the difference in the language of instruction, these two tracks also differ given the nature of the data students work with. In the first case, it is digitized data and in the second, born-digital data. So, choosing from these two options students are basically choosing between working with cultural heritage or digitized literary texts, in the former, or content found in the media, in the latter. Depending on the track, students are likely to learn about OCR (optical character recognition) and NLP (natural language processing), or digital storytelling and network analysis. As an academic leader of the DCV program I do believe that diversity is key, but this is especially true with DH education. So, my goal is to provide as many options as possible, so every student will be able to build an individualized educational trajectory, thereby fully realizing their potential. Teaching and co-teaching various DH courses at ITMO and Tartu University and giving lots of guest lecturers has allowed me to expand my teaching philosophy in DH. Most of my teaching has been focused around developing and curating DH projects. This includes not only acquiring necessary practical skills in data mining, data analysis, and visualization, but also in building a strong argument by asking research questions and analyzing what particular digital technologies are needed in each case and why. I call it “critical DH thinking” and usually highlight that, even though the applied skills are also very important, developing critical thinking will help to build more sustainable DH projects. To sum it up, the more I teach or do DH, the more I discover what is still to be learned and redesigned.

Lada: Thank you, Antonina! It is, indeed, so wonderful that we have three professional educators in DH at the table today. David, what are your thoughts on how to teach DH?

David: I love the topic of how the digital is changing the way that we teach and learn! One answer to your question of how to teach DH lies in the values embraced by your institution and how they are evolving. To give you an example from my own position: I arrived at NYU Abu Dhabi having worked for a long time in relative isolation on my digital projects at my previous institution, the American University of Beirut. There wasn’t institutional momentum there to create new courses and to imagine teaching as integrative and connected to faculty research. In my current position, there are multiple stimuli that have encouraged me to bring my research into the classroom. I have benefitted from the presence of an Arabic natural language processing (NLP) group, a vibrant Art Center with projects and commissions in digital or data-driven art, an active interactive media program, as well as the research center of the nearby Louvre Abu Dhabi. Those partners in and around my institution have most definitely contributed to my growth as a teacher-scholar.

There are lots of DH syllabi that you can find on the web and copy, and there are lots of methods that can be transferred to most parts of the world. I certainly did this for a certain phase of my teaching. But teaching DH has become less of a methodological checklist and more of an engagement with the common values of my institution and communities as they grow. When your DH courses align with those higher goals, there is much more room for DH to contribute, and to be valued, at home.

Antonina: I totally agree with that point that cooperation with other departments at ITMO, in particular Computer Science, helps us a lot, not only in teaching various courses at the DCV program but also working together on projects. We also collaborate and co-teach some DH courses together with colleagues from other DH centres in Russia bringing together students and cherishing diversity and openness. Moreover, as I have mentioned previously, we are closely connected with our partners from cultural institutions in Saint Petersburg and these collaborations have resulted in student internships, paid internships, and the expansion of student portfolios. This enlarging network is what makes us a community and allows us to make DH projects more meaningful and widely publicly available.

Lada: I would also like to jump in and add a little from the perspective of both a student and aspiring young researcher who is making her first steps in the field. First, when it comes to collaboration and education in DH, I think that in the program my fellow students and I have a lot to teach each other. Given that we come from different backgrounds, some of us are better at coding and others are better at finding good research questions and have a deeper understanding of cultural data. DH taught me that it’s ok not to know something—and for someone with an A-student complex, that was quite a discovery. I really love how helpful the community is, because really when you dive into DH, you push yourself far out of your comfort zone, no matter what background you come from.

And Kimon, I suspect you have something to add to this discussion.

Kimon: I’ve been working on these questions since I started my PhD program at the City University of New York in 2001, before the term “digital humanities” entered popular use. As a theater history PhD candidate, I noticed that we were for the most part studying play scripts because that’s what’s left from a production, and this ephemerality is one of the reasons it is hard to study the history of theatre. I began working in CUNY’s Interactive Technology and Pedagogy Certificate program to consider ways to gather all the artifacts from a performance to bring them to public history. Many of the faculty in that program were involved in creating the textbook Who Built America, which tracks the history of America from a labor perspective, rather than following the traditional “great white men” narrative. Along with the printed book version, this textbook was available as an interactive CD-ROM in the late 80s. So here we can see—decades before we even started using the term DH—that academics were doing work  driven by the idea of expanding education and the reach of academic discourse.

So my perspective has always been to get information into people’s hands, using the new media we have to best contextualize the knowledge we produce. In building my DH syllabi, I wanted to avoid saying, “Here are the big names and tools in DH that you have to learn about,” and instead focus on how living in a complex world is intrinsically affected by the digital, and how do we understand that through method. So every time we talk about databases for the humanities, we also consider that our personal and health records are also somewhere in a database—and what does that structure mean to the way you are viewed by systems as a series of data points? How does the setup of that information affect you in your daily life? The goal therefore is to continually keep everything in a larger context and perspective.

Lada: Thank you, it’s really fascinating to follow your discussion, so let’s keep on talking about education and dissemination of DH. In light of the digital having become an integral part of our everyday lives, how do mixed methodologies in DH open the possibilities for scholarship that are more public, open, and designed to engage with the larger audiences?

Kimon: In the last decade in the US, we’ve seen how profoundly digital media can affect daily life, as social media has expanded and everything has been consumed by the internet. In my teaching and research I have been trying to figure out how the digital landscape affects us daily, how it affects the culture of consumerism, and how as academics and producers of knowledge we can bring all those points together. To these conceptual methodologies I would add visualization and website development, to reveal the potential of design in creating more engaging modes of narrative and storytelling. Because, in the end, we are all living some part of our lives in these public digital spaces, which, despite the problems they are creating, massively open up our ability to create innovative teaching platforms, share information, and build communities.

My connection with the theater, material culture, and web design has put me in a place where I want students to know how they can reach audiences through web development and design, which has arguably a lower threshold than more complicated analytical techniques. To do very advanced text analysis you really need to understand the fundamentals of statistics and the algorithms, but to do web development, you start to understand the discursive and aesthetic challenges and possibilities that come into play for presenting your work for a broader range of audiences. I teach a course on the American Hardcore Punk movement, and we focus on understanding one particular cultural moment to better understand how we use different modes of expression to generate activism, critique society, and carve out a space for our voices today. With that kind of platform, we can then look at subsequent related movements like Queercore and Riot Grrrl as well as reflecting on current flows in art and activism to understand their historical progenitors and the uniqueness of movements of the moment.

I ask students to think about how they can use the ethos of Hardcore to reach out and tell their own story. The most recent iteration of the course developed an amazing website, which was an experimental product of creative ingenuity that far exceeded what a stack of research papers alone would have accomplished.

Lada: We have a little bit of time to talk about our current concerns, and I would like to address the next question to David. So, David, how did the pandemic influence the global academic community and what perspectives do you see for international collaboration under the current circumstances?

David: Thank you for the question. Pre-pandemic, there were many DH events taking place in local environments around the world. An example of one such event we have heard about is NYCDH Week. Now, in New York it’s easy enough to move around the city from institution to institution for a week to experience this co-learning, co-teaching event. Not all cities have such a large community, however. In places like the UAE or Lebanon, where I have spent the majority of my professional life, the number of DH practitioners is very limited and we often rely upon international relationships to grow our projects and our community. The advice that I got a decade ago from a senior practitioner in the field when I asked how to start working in DH was to do three things: attend a THATCamp, enroll in a DH summer school like DHSI, and attend the Association of Digital Humanities Organizations (ADHO) annual conference. Back then, to attend such a faraway event required many months of advance planning. I imagine that we could do oral histories about the DH week-like events that have taken place around the world, focusing on how they played a transformative role in introducing researchers to digital methods.

The pandemic did not bring about a total shift in the ways that people have accessed training in digital methods—there were certainly some events taking place online before—but it has accelerated the process, and there has been a marked increase in people who want to engage in such learning. Over the course of 2020, with the large-scale adoption of remotely broadcasted events, I have witnessed a rapid opening up of local environments to global audiences. There have been all kinds of opportunities to listen in on many different conversations in different parts of the world now —far too many for a single person to follow. An interesting question to ask ourselves is which of these learning opportunities will remain virtual and which will pivot back to in person meetings? I believe that a professor of digital humanities should not only do research of their own, but also model new forms of research for their local environment, and in this respect, like Antonina, I spend quite a lot of time learning about what is going on in the world of DH in order to bring it back home. In 2020, the sheer abundance of conversations going on in digital humanities research and their connectedness has been challenging, but also exciting, to follow. As I do so, I am always asking myself how to match what a growing local community wants and needs and how to match it with global research trends. It’s a process of connecting the dots.

Another interesting question for me is what will a world of scholarly communication look like post-pandemic? There are so many people envisaging, and even prototyping, different modes of communicating or different modes of scholarly dissemination and communication. Even if we have not been traveling for this last year, we will no doubt get back on planes to go to professional meetings. I do wonder how we are going to spend our time and how we are going to manage the very new kinds of access that we have experienced. I think there’s a lot to be said about this last topic of post-pandemic scholarly communication, which stretches beyond the topic of today’s panel, but which will impact the ways that international collaborations continue to be founded and sustained. I guess that the way forward for research that I imagine, especially in remote places where there are smaller numbers of local researchers, will be fewer in-person conferences and meetings, replaced instead by longer stays or exchanges abroad for teaching, team building, or intensive exchange. Synchronous video conversation platforms like Zoom are, after all, great for exchanging ideas, in pick-up conversations with people you already know, coming to consensus through team discussion, or for certain kinds of scholarly dissemination, but what about all the other ways we communicate and do scholarship?

I don’t think we have fully considered what it will mean to be in person anymore, what the added value of being in the same space will be, let alone how connected to the rest of the world we will be when we are back in person in our local environments. I’m sure that there will continue to be great value in teaching in person, but for research, we still have to figure that out. Will we be able to have hackathons and sprints online with the same results? Will we meet in person for more reflective activities, such as ideation, planning, and writing? Or simply to foster the social bonds that support collaboration? What will happen to our colleagues who joined in the conversation during the pandemic for whom passport privilege and crossing borders are not a given?

During the pandemic I experienced so many interesting new kinds of interactions and experimental engagements. Obviously, not all of them will continue as we transition back to our offices and campuses. It is worth thinking, however, about how we will do digital humanities when we are together as opposed to when we are apart, as well as the role of the international collaborator in the local area. New forms of hybrid collaboration are certainly on the horizon beyond our current circumstances.

Kimon: I think the sprints are a good point. Sometimes when you really want to accomplish a significant amount, you need to be around your colleagues for three days in a row, eight hours a day working, including in-person time over dinners. So it’ll be interesting to see now that we’ve had this extended experience with telecommuting, how we will move forward. We haven’t gone through something like this trauma in the world for over a hundred years and we’re going through it now in a very different kind of time with specific technical, socio-economic, and pedagogical challenges. There has been a lot gained from this period in getting things working in a short timeframe, but we also need to sit back and consider which changes have been for the better and which really shouldn’t be repeated.

It’s hard to teach workshops in general using teleconferencing tools; some such as photogrammetry were extremely difficult to do. When I teach a workshop I move a lot around the room. I’m looking at everyone’s screens troubleshooting problems, showing students what they might have in common and how they can share knowledge, and making the learning environment engaged and participatory. On the flip side, while we had fewer sessions for NYCDH Week 2021, we had hundreds of more people register for workshops. So there is a new interesting mix of variables that we have to consider and that we will have to adapt to. Hopefully, we’ll find a really fruitful middle ground that balances all of these things. As David notes, a lot of us are thinking about it already: what’s the best way to transition things. We are fortunate in our positions that we have already looked to transcend geographical challenges, and now we are armed with better tools and more familiarity with approaches to closing the gap between these three locations: Saint Petersburg, Abu Dhabi, and New York. So trying to find ways to mitigate those distances while remembering the benefits of locality and presence is going to be the trick.

Antonina: Yeah, and I would like to add a little on that topic. I agree completely that professional networking is the most valuable thing that happens at a conference. In the end, this scholarly network is the most valuable thing you have. What you pay for when you travel is to build these relationships, and from there you can develop them through these digital tools and means like email and Zoom, but it’s really hard to establish strong relationships and community by only communicating online.

You can just work using emails, Zooms, Microsoft Teams, but I think that this is a very good lesson the pandemic taught us, that not all the things can be shifted online. We still have this “humanities” impact. We even have in Russia this funny word for “madness”— безумие/безZOOMие (a madness created by Zoom)—which has been in the media for a while. That’s true, it’s madness, it’s too many Zooms, all the calendars are packed, and people can call you at nine in the morning and that’s fine because they live far away.

I like that we’re finishing up this discussion before we move on to questions thinking about the future because DH is also about the future. We try to preserve things, we try to make them available and that’s an important part of our work as well. We want to preserve our community and chart a path for the future as well.

Lada: Dear colleagues, thank you so much for such an interesting and insightful discussion—we covered some perennial issues of DH as well as some current challenges and concerns. Now we have a little bit of time left to open our discussion and attend some questions from the audience.

Question from Nadezhda Povroznik, Head of the Center for Digital Humanities, Perm State University: When it comes to DH, is it possible to be only a medieval scholar or just a historian focusing on a narrow topic, or is it more encouraged to find the solutions for a wider range of questions? If so, how does this impact the way that we think about the focus on tools? 

David: Your question reminds me of the question I received earlier where I was asked to reflect on how DH supports either integration or isolation, or perhaps both, in scholarship, and whether it only makes an intervention in a field of speciality or instead it builds some kind of larger community. For me, the way that I responded to a changing world of digital humanities around me depended on the moment of my career (I am currently Associate Professor). When I launched my first independent digital research, I made the strategic decision to work on a project on my own. I wanted to build my skills, make a first foray into digital humanities, and to be able to link the conclusions to the scholarly field that I knew so well. I started in digital humanities without an active community in my immediate surroundings; my collaborative research began with faraway scholars. There’s this very interesting shift in my publications that took place around 2013/2014 when I began to co-publish. It’s not that I abandoned single-authored research, but since this particular work was collective, it made sense to publish together. With colleagues in computer science we now tend to publish a variety of articles on the same project for different audiences, highlighting the multidisciplinary contribution that our work makes. It’s tricky to straddle knowledge cultures, but it has become the way that I balance my career now. You can do that when you’re a little bit more advanced in your career. When you’re younger, it’s harder to do, for sure.

In the end, it is about taking risks that your institution is willing to support. One of the key things in North America we have been witnessing is how, increasingly, professional organizations are creating guidelines for review that include digital scholarship in the evaluation process. This effort really relies on senior colleagues, because they sit on the boards of those professional organizations and on the editorial boards of journals. Mid-career faculty who value digital scholarship have an important role to play also in mentorship and advocacy.

Question from Larisa Krayeva, PhD, Assistant Professor at the faculty of Technological Management and Innovation, ITMO University: It’s more of an observation than a question and I’m not sure if it’s relevant…. Well, you see, I teach philosophy to students from all sorts of technical backgrounds at ITMO, and during your discussion it occured to me how my personal experience applying different approaches teaching philosophy to different groups of students—like engineers, chemists, medical student etc.—can be similar to your experience, finding the common ground with scholars from other humanities backgrounds in DH.

David: I have something to say about the different kinds of students who are not necessarily specialized in questions of the humanities. I am lucky to have my position in the Arts and Humanities Division, but I don’t teach students exclusively who are concentrating in topics in the humanities. There is definitely an advantage to this model, namely that you are able as you say to “build common ground” across disciplines and tap into very different interests and skillsets for a truly interdisciplinary experience. In fact, I have heard over and over from students who are not majoring in humanities subjects that they never knew that such a field of inquiry existed, and that it was amazing to be able to think across and between fields. Of course, one of the minor disadvantages of having only individual courses offered in a “standalone” fashion is that they are not developmental or sequential, allowing students to build skills over time as you would want if you had students specializing in the domain.

Question from the audience: What is the most important thing when you get into the DH? What should you focus on first?

Kimon: It’s very important to inspect the tools and to critically evaluate the methodologies before starting. Many of these tools have hidden processes that you need to understand the implications of in order to do competent research. You can’t just download an NLP package and start throwing stuff into it. You can download topic modeling tools and you can run them over the web or locally and then you can start trying to figure out what those words mean, but topic modeling is a complex statistical process that does very specific things to text to create a certain kind of output. If you don’t know what it does to create that output, you may arrive at incorrect or ill-informed conclusions. That’s why it doesn’t hurt to have technical knowledge. It doesn’t hurt to understand what an algorithm is; it doesn’t hurt to have some programming language. We don’t just start using a database. So becoming aware of the texture of the tools and methodologies and establishing a critical sensibility is the strongest foundation you can have.

David: Working with data, and working with the digital more generally, takes us outside of our comfort zone. So my suggestion might be to work to create some data about something that you know well and do so in familiar surroundings, and then try to evaluate as a group how well those data represent the thing that you think you know. It’s important to recognize that in the digital world we necessarily distort things by representing them as abstraction or by formalization. So I think it’s worth thinking at that level about the relationship between the digital objects that we’re creating and that which we know.

Afterword

The ongoing phases of the pandemic have affected different parts of the world in different ways and at different times, exposing inequalities and different states of readiness and ability for communities to mobilize in moments of crisis and necessity. In the digital humanities, we have also listened with a critical ear to changes discussed within the academy: maintaining certain types of hybrid learning, alternative delivery of content, and even remote education as a means of expanding markets for higher education. These proposed initiatives suggest both change and instability in the years to come.

Our conversation about the practice of international collaboration was carried out in a period of flux where there was not as much time for reflection as we would have liked. Obvious points of discussion arising from this roundtable discussion which deserve further attention include the future of the digital humanities community event as national or regional scholarly cultures expand from basic training toward more specialized work, as well as the potential of hybridization of in-person events both to increase access in the scholarly community and to reduce the carbon footprint of our academic endeavors. On the other hand, we feel that it would be unfortunate if local digital scholarship were the only sort to prevail in the coming years. We hope that our thoughts published here will keep the discussion of international collaboration in digital scholarship moving forward, encouraging others to explore what kinds of new roles it might play.

We recognize that collaborative international dialogue takes a lot of energy and labor, as well as a certain kind of personality and commitment which the academy does not often support. The speakers at this panel are quite proud of their cooperation, and of how their efforts have influenced communities of practice far away from the location of their academic positions. For this reason, we have decided to publish the text of this conversation along with Arabic and Russian translations of it to foster critical discussion with JITP’s readership and the new worlds of digital scholarship coming into focus in these languages and cultures. We trust that this translingual initiative will further the goal of transnational dialogue in global digital humanities and will inspire others to build such bridges.

References

Puchkovskaia, Antonina, Lada Zimina, Pavel Aleksandrovich Tugarinov, and Maria Sergeevna Bakhteeva. 2021. “Digital Humanists and Cultural Institutions: Achieving Efficient Collaboration.” Panel session. European Association of Digital Humanities 2021, Krasnoyarsk, 23 September 2021.

About the Authors

Antonina Puchkovskaia is an Associate Professor of Digital Humanities at ITMO University (Saint Petersburg, Russia), where she teaches various Digital Humanities courses. Her research interests include cultural heritage representation and visualization, spatial humanities, and digital infrastructures. She is a director of the International Digital Humanities Center and also a program leader of ITMO’s MSc in Digital Humanities, which aims at creating well-rounded data professionals who have strong statistical and technical skills combined with strengths in research, communication, and design. Antonina is one of the 2018–2019 Willard McCarty Fellowship holders at the DH Department at King’s College London. She is a founder of Saint Petersburg DH week and has been organizing it for four years.

Kimon Keramidas is Clinical Associate Professor of Experimental Humanities and Social Engagement and Affiliated Faculty in International Relations at New York University. He is also Co-Director of the International Digital Humanities Center at ITMO University. Kimon’s research and pedagogy take place at the intersection of media and technology studies, cultural history, sociology of culture, and experimental humanities. Kimon’s work includes cultural heritage and public history initiatives with international organizations such as the Smithsonian Institution, the State Hermitage, Rubin Museum, Zayed University in Dubai, and the University of Leicester. He is a co-founder of New York City Digital Humanities (NYCDH) and has been organizing NYCDH Week for eight years.

David Joseph Wrisley is Associate Professor of Digital Humanities at NYU Abu Dhabi. His research interests include comparative approaches to medieval literature in European languages and Arabic, digital spatial approaches to corpora, neural methods for handwritten text recognition across writing systems, and open knowledge community building in the Middle East, where he has lived and researched since 2002. He co-founded the Arab world’s first two digital humanities training events: the Digital Humanities Institute Beirut (dhibeirut.wordpress.com) in Beirut in 2015 and the NYU Abu Dhabi Winter Institute in Digital Humanities (wp.nyu.edu/widh) in Abu Dhabi in 2020.

Lada Zimina is an Assistant Professor at the International DH Center at ITMO University (Saint Petersburg, Russia). She has recently graduated from the masters program in Digital Humanities that enabled her to apply IT onto her Bachelor background in Museology and Heritage Management. Her research interests include memory studies, cultural heritage representation, and digital collections. She is a co-founder of a biennial conference on Digital Local Studies.

A young person, in shadow and profile, gazes at their own video feed as they take a class online from home, faced with a webcam.
0

Introduction

We are now nearing the two-year mark of a global pandemic that has had such a profound effect on every aspect of our lives. As students, educators, administrators, and researchers, we have had to adapt our academic practice in ways that blurred the lines between our public personas and our private lives. We have had to learn about and embrace various forms of technology in order to enable remote teaching, learning, and collaborations, all with little control over the scale and extent of the invasiveness made possible by these technologies. It is at this crucial conjuncture that we offer this Themed Issue of the Journal of Interactive Technology and Pedagogy on surveillance and educational technologies.

As we said in the call for papers for this issue: “The COVID-19 pandemic has served as a magnifying glass, revealing all the ways our systems are broken.” Indeed, social fault lines have not only been exposed and exacerbated in the harsh light of the pandemic response, but even more so through the ways many institutions chose to ignore it while hoping to continue with some version of business as usual. But, just as a magnifying glass reveals faults, it shows us opportunities for repair: we cannot simply fix what is broken, but also must work toward eliminating systems that are not “broken” but working as designed—to the detriment of marginalized and vulnerable populations. Thus, while we develop counternarratives and critiques, we can also draw on more expansive visions of abolition, which demand “that we change one thing, which is everything” (Gilmore 2018).

Bluntly, many of these surveillance systems and computational tools shouldn’t exist. While the emergency circumstances under which people and institutions have adopted them during the pandemic make such developments somewhat understandable, now that we have a better understanding of the concrete consequences in our pedagogy and in our students’ lives, we really have no justification to continue using these tools in these ways. We must change not only the punitive technology we use, but the educational mindset and broader world that rationalizes it.

This issue is not the first, nor will it be the last, collection of such critical work, but as we spend more time within this pandemic paradigm, we are accumulating clearer and stronger evidence and narratives of the harms surveillance-oriented educational technology brings, making it much less understandable for justice-oriented educators to excuse their use. The pandemic has crucially highlighted the need for consent, compassion, and care, and one of the striking things about many of the pieces in this issue is that they are self-reflective rather than analytical. Many of the authors situate themselves in a fraught system of monitoring and punishment and analyze or question their roles in bringing potentially harmful surveillance to bear on others, especially the students they are meant to nurture. It’s also notable how many of these projects address remote proctoring and Learning Management Systems (LMSs). For many institutions, the pandemic supercharged the already pervasive use of LMSs and proctoring systems in the transition to remote instruction. As the uptake of these tools and protocols increased, so did the outcry around the invasiveness and consequences of their use. Consider these pieces both as scholarly research, and as a call to action for justice, within and beyond education.

In “Toward Abolishing Online Proctoring: Counter-Narratives, Deep Change, and Pedagogies of Educational Dignity,” Charles Logan invokes Audrey Watters’ notion of the “edtech imaginary” as a way of exploring how remote-proctoring companies develop powerful narratives about the necessity and usefulness of their products, and how we might establish counternarratives that move us closer to the abolition of these discriminatory technologies and their effects.

In “Back Doors, Trap Doors, and Fourth-Party Deals: How You End up with Harmful Academic Surveillance Technology on Your Campus without Even Knowing,” Autumm Caines and Sarah Silverman alert us to the dangers and complications of allowing fourth-party vendors access to institutional data through backdoors created by third-party relationships. With Proctorio as the primary example, they unpack these relationships in an accessible and clear way, while outlining the different kinds of fourth-party partnerships that institutions might unknowingly find themselves in. Caines and Silverman also lay out a harm index, a useful framework to measure the levels and scale of harms that remote proctoring services can cause. The authors include an example of their collaborative autoethnographic reflection, which provides a glimpse into the tedious but necessary steps needed to thwart corporate control over faculty and student data.

Jessica Kester and Joel Schneier’s “Soft Surveillance: Social Media Filter Bubbles as an Invitation to Critical Digital Literacies” discusses having students engage with the surveillance-derived filter bubbles of their own social media feeds in order to develop critical digital literacies—a way for students to “critically look at their digital practices through their own digital practices.”

In “Resisting Surveillance, Practicing/Imagining the End of Grading,” Marianne Madoré, Anna Zeemont, Joaly Burgos, Jane Guskin, Hailey Lam, and Andréa Stella assert that grading systems are an element of larger systems of surveillance at educational institutions and that grading is incompatible with antiracist pedagogies. They offer a variety of experiences where they either individually or collectively operated against or outside the schema of grading, and push us to “reimagine the purpose of schooling” in light of these struggles.

For Issue 20, we also wanted to create space to explore issues around educational surveillance that wasn’t constrained by the formality of more traditional journal articles, so we invited submissions to our Views from the Field section. We are very pleased to present five thought-provoking pieces that critically engage with the experience of being surveilled by educational technology and the potential consequences of this surveillance on our collective wellbeing.

We start off with “Why Don’t You Trust Us?”, a compelling piece from undergraduate student Sinéad Doyle, who generously shares her own experience of being subjected to additional surveillance during the pandemic and how this sort of invasive surveillance can blur the lines between public and private in counterproductive ways. Lance Eaton’s “The New LMS Rule: Transparency Working Both Ways” imagines what it would look like if we turned the tables and gave students the same level of access to instructor activity on LMSs as these platforms give instructors to student activity, noting the power imbalances built into conventional LMSs. In “Pedagogy and the Expansion of Surveillance at the City University of New York,” Marc Kagan continues the exploration of the potentially insidious nature of LMSs by pointing out the dangers of allowing unfettered and unregulated administrative access to online courses, highlighting the potential role of labor organizations in challenging this threat. “Black Mirror Pedagogy: Dystopian Stories for Technoskeptical Imaginations,” by Daniel G. Krutka, Autumm Caines, Marie K. Heath, and K. Bret Staudt Willet, provides a way to help students interrogate their own techno-optimism through the use of Black Mirror-inspired speculative-fiction narrative building. And finally, Chris Miciek’s creative text, “Field Notes from the Education to Employment Pipeline: A Career Development Perspective,” gives us a bird’s-eye view history of the contested imbrication of education and labor-market requirements, highlighting the historical and ongoing processes wherein students are inured to the use of technological surveillance in readiness for workplace surveillance.

In addition to the pieces on surveillance in education, we are pleased to include two general-interest articles before we pause publication for our migration to a new publishing platform.

The first, “Authoring an Open-Source Game for a Faculty Open Educational Resources Workshop: A Case Study” by Katherine Foshko Tsan, is an excellent piece on using Twine, an open-source interactive narrative building tool, for faculty development focused on OER. This piece highlights how using these sorts of narrative tools can be a compelling way to engage with faculty while opening new space for them to learn about OER.

Our second general-interest article, “Poetry in Your Pocket: Streaming Playlists and the Pedagogy of Poetic Interpretation” by Stephen Grandchamp, shares how the use of Spotify playlists made poetry more accessible to students and helped to recontextualize poetry in a more contemporary setting. This approach helped students understand and participate in the shifting meaning and significance of poetry, and gives hope for those of us who find interpreting poetry a little intimidating.

We want to acknowledge the patient and incredible work that managing editor, Patrick DeDauw, and editorial assistant, Chanta Palmer, have done to keep us on track and wrangle the many moving pieces that needed to come together to produce this issue. Our deep gratitude to the members of the JITP editorial collective for all their behind-the-scenes work and support. We also want to acknowledge the reviewers who took time out of their busy schedules to provide valuable feedback to our authors and note that it has been a privilege to be able to work with the authors to bring you Issue 20. We are deeply grateful that we were all able to come together during this pandemic to give shape and space to this important conversation, and we hope you will join us in doing what we can to ensure an equitable and surveillance-free educational future.

References

Gilmore, Ruth Wilson. 2018. “Making Abolition Geography in California’s Central Valley.” Interview with Léopold Lambert. The Funambulist 21. https://thefunambulist.net/magazine/21-space-activism/interview-making-abolition-geography-california-central-valley-ruth-wilson-gilmore.

About the Editors

Chris Gilliard is a Visiting Research Fellow at the Harvard Kennedy School Shorenstein Center. His ideas have been featured in The New York Times, The Washington Post, Wired Magazine, The Chronicle of Higher Ed, and Vice Magazine. He is a member of the UCLA Center for Critical Internet Inquiry Scholars Council, and a member of the Surveillance Technology Oversight Project community advisory board.

sava saheli singh is an independent researcher who just completed a postdoctoral fellowship with the eQuality Project and the AI + Society Initiative, both at the University of Ottawa. She created the award-winning Screening Surveillance, a series of short, near-future speculative fiction films. This public education and knowledge translation project calls attention to the potential human consequences of big data surveillance. She co-produced the first three films as a postdoctoral fellow with the Surveillance Studies Centre at Queen’s University in Kingston, Ontario, and is currently in post-production on the fourth film in the series which she also co-wrote and co-produced. sava received her PhD from New York University’s Educational Communication and Technology program. As an interdisciplinary scholar, her current research interests include educational surveillance; digital labour and surveillance capitalism; restorative justice and abolition; speculative fiction; and critically examining the effects of technology and techno-utopianism on society.

A telephone poll in black and white at night has a poster on it that reads 'Big Data is Watching You.'
0

Table of Contents

Introduction
Chris Gilliard and sava saheli singh

Themed Articles

Toward Abolishing Online Proctoring: Counter-Narratives, Deep Change, and Pedagogies of Educational Dignity
Charles Logan

Back Doors, Trap Doors, and Fourth-Party Deals: How You End up with Harmful Academic Surveillance Technology on Your Campus without Even Knowing
Autumm Caines and Sarah Silverman

Soft Surveillance: Social Media Filter Bubbles as an Invitation to Critical Digital Literacies
Jessica Kester and Joel Schneier

Resisting Surveillance, Practicing/Imagining the End of Grading
Marianne Madoré, Anna Zeemont, Joaly Burgos, Jane Guskin, Hailey Lam, and Andréa Stella

Themed Views from the Field

Why Don’t You Trust Us?
Sinéad Doyle

The New LMS Rule: Transparency Working Both Ways
Lance Eaton

Pedagogy and the Expansion of Surveillance at the City University of New York
Marc Kagan

Black Mirror Pedagogy: Dystopian Stories for Technoskeptical Imaginations
Daniel G. Krutka, Autumm Caines, Marie K. Heath, and K. Bret Staudt Willet

Field Notes from the Education to Employment Pipeline: A Career Development Perspective
Chris Miciek

General Articles

Authoring an Open-Source Game for a Faculty Open Educational Resources Workshop: A Case Study
Katherine Foshko Tsan

Poetry in Your Pocket: Streaming Playlists and the Pedagogy of Poetic Interpretation
Stephen Grandchamp

Issue Twenty Masthead

Issue Editors
Chris Gilliard
sava saheli singh

Editorial Assistant
Chanta Shenell Palmer

Managing Editor
Patrick DeDauw

Copyeditors
Param Ajmera
Patrick DeDauw
Kelly Hammond
Jojo Karlin
Benjamin Miller
Brandon Walsh
Nicole Zeftel
Dominique Zino

Staging Editors
Inés Vañó García
Jojo Karlin
Anne Donlon
Kelly Hammond
Laura Wildemann Kane
Teresa Ober
Danica Savonick
Luke Waltzer

A faceless figure taking a picture is reflected in the convex lens of the camera they're photographing.
9

Toward Abolishing Online Proctoring: Counter-Narratives, Deep Change, and Pedagogies of Educational Dignity

Abstract

A future with ubiquitous academic surveillance is not sealed, not yet. In this essay, I discuss how online proctoring companies sell their technology with stories inspired by the edtech imaginary. Higher education institutions, in turn, often repeat these narratives, as evidenced by the ways institutions frame the technology as neutral, convenient tools for facilitating assessments. I propose a possible path toward abolishing online proctoring by authoring counter-narratives. I identify two spaces for constructing counter-narratives. First, we can apply a cognitive perspective to policy implementation to shift individual educators’ understanding of online proctoring through dissonance-producing institutional resources. Second, we can build collective partnerships between administrators, staff, faculty, and students to achieve deep change in our assessment practices. This potential path forward is guided by dual commitments: to reject online proctoring and the intersectional harms endured by students forced to use the technology; and to uproot the underlying pedagogies of policing and punishment that support online proctoring and replace them with pedagogies of educational dignity. I end my essay with a call to adopt an abolitionist approach to ridding education of online proctoring. By exercising abolitionist principles of refusal and care, along with a rejection of reform as an acceptable middle ground, we can move closer to creating the kinds of learning environments and relationships that cultivate students’ educational dignity.

The story of online proctoring is difficult to disentangle from surveillance and policing. Companies with names like Honorlock and Respondus Monitor conjure images of a patriarchal panopticon. Then there’s Proctortrack’s origin story. The chief technology officer for Verificient Technologies, the company that developed Proctortrack, arrived at the idea for the online proctoring technology after working on a Transportation Security Administration (TSA) project that included searching video footage for facial expressions deemed abnormal (Singer 2015). A version of the TSA’s security theater, online proctoring is further evidence of Sasha Costanza-Chock’s observation that “The same cisnormative, racist, and ableist approach that is used to train the models of the millimeter wave scanners [used by the TSA] is now being used to develop AI in nearly every domain” (2020, 5). I worry that the mechanized dehumanization experienced by individuals from nondominant groups at airport security is now being normalized in education due to online proctoring.

The attempts to make prejudiced technology prosaic are facilitated by online proctoring companies and their commitment to an edtech imaginary and its powerful storytelling. Audrey Watters describes the edtech imaginary as a collection of “stories we invent to explain the necessity of technology, the promises of technology; the stories we use to describe how we got here and where we are headed” (2020). Read the statements from online proctoring CEOs and the claims made by companies on their websites, and you can see the edtech imaginary at work. Online proctoring is supposedly necessary because, in the words of ProctorU’s CEO, without it, cheating will increase and pose “a severe threat to all higher education’” (Feathers and Rose 2020). The hollowness of the edtech imaginary is further illustrated in the diminishing story sold by Proctorio. Beginning in January 2019, the company promised institutions their “software eliminates human error [and] bias” (Proctorio 2019). The company’s homepage declared their software’s impressive capability until April 2021. On April 19, 2021, the Federal Trade Commission warned companies not to claim their algorithms can erase bias (Jillson 2021). Within days Proctorio’s promise of unbiased technology shrank to “Our software attempts to remove human bias and error” (Proctorio 2021). Visit the company website today, and you will find the edited sentence has disappeared. The edtech imaginary features many such revisionist narrators.

I want to consider the other elements of the edtech imaginary described by Watters: how we got here and where we might go. I’m trying to understand how institutions and people in power too often come to believe edtech’s glossy narratives about the past, present, and future. I’m also searching for the sites where we can share our counter-narratives. Alongside counter-narratives, I’m seeking ways we might uproot the pedagogies of policing and punishment that make online proctoring possible and replace them with pedagogies of educational dignity.

Educational dignity is critical for enacting a just present and future (Espinoza and Vossoughi 2014; Espinoza et al. 2020). Educational dignity is “the multifaceted sense of a person’s value generated via substantive intra- and inter-personal learning experiences that recognize and cultivate one’s mind, humanity, and potential” (Espinoza et al. 2020, 326). Online proctoring and its shallow definitions of learning are incompatible with educational dignity because of the technology’s hostility toward every individual forced beneath a webcam’s glare. The technology can harden internalized oppression, especially for nondominant students (Bali 2021), through its built-in racism and ableism. Further, online proctoring positions educators as police officers and students as criminals, straining inter-personal learning experiences. Online proctoring, I should note, is not the sole source of negative intra-personal and inter-personal learning experiences. Acknowledging its encoded opposition to educational dignity, however, can encourage us to view its abolishment as a part of a larger project to help educators develop and practice pedagogies of educational dignity.

I turn now to possible ways of conducting that larger project. I will review the institutionalization of online proctoring; describe the importance of how institutional resources frame online proctoring; offer a case for how to create deep change in the ways educators understand online proctoring and its alternatives; and conclude with a call to take an abolitionist approach to ridding online proctoring from education.

A future with ubiquitous academic surveillance is not sealed, not yet.

How Did We Get Here?

The critiques of online proctoring are numerous. Online proctoring replicates inadequate assessment methods (Leafstadt 2017). Online proctoring can exacerbate a student’s anxiety, particularly a student with high anxiety (Woldeab and Brothen 2019), which in turn can have a negative impact on students’ ability to demonstrate their learning (Eyler 2018). Online proctoring technology is racist (Feathers 2021; Swauger 2020); ableist (Brown 2020; Zhu 2021); and it invades students’ privacy (Cahn et al. 2020; Germain 2020). Put another way, online proctoring not only reinforces ineffective, harmful pedagogies; it’s also a deeply unethical technology.

Joining these critiques are the thousands of students who have documented and shared their experiences with online proctoring:

I know that I’m going to have to try a couple times before the camera recognizes me…I have a light beaming into my eyes for the entire exam…That’s hard when you’re actively trying not to look away, which could make it look like you’re cheating…[The software] is just not accurate. So I don’t know if it’s seeing things that aren’t there because of the pigment of my skin.
—Femi Yemi-Ese, student at the University of Texas at Austin (quoted in Caplan-Bricker 2021)

I’ve despised using this software…. On one occasion, I was “flagged” for movement and obscuring my eyes. I have trichotillomania triggered by my anxiety, which is why my hand was near my face. Explaining this to my professor was nightmarish.
—Bea, student at Tarrant County College (quoted in Retta 2020)

It’s really cruel to have students come to class and expect to learn, and then treat them, essentially, like criminals and make them install programs that look for all their information and force them to give tours of their home.
—Anonymous, student at the University of Washington (quoted in Hipolito 2020)

The chorus of student criticism has apparently not done enough to slow institutions and faculty from deploying the technology against students. For example, Proctorio’s CEO claimed his company helped to proctor 25 million exams at 1,000 institutions in 2020 (Harwell 2020).

To help to explain the growth of such an apparently toxic technology, it’s important to note that institutional use of online proctoring predates the coronavirus pandemic. The existing institutional knowledge and resource infrastructure, combined with the coronavirus pandemic’s demands for quick and cheap solutions to complex teaching and learning problems, meant online proctoring could take root farther and faster than might otherwise have been the case. The upheaval also presented educational technology companies an opportunity to activate the edtech imaginary and present themselves as partners ready and able to assist institutions’ pivot to remote emergency teaching. In some cases, non-online proctoring companies joined together with online proctoring companies, marketing their wares directly to educators for free (Top Hat 2020). The companies’ beneficence can be understood as an attempt to deepen their connections to institutions as well as to circumscribe what we imagine when we envision online learning and its possibilities.

Once a technology becomes well-established at an institution, it can be difficult to uproot (Arthur 1994). As a consequence of the pandemic, institutions have made substantial financial investments in online proctoring technology. The University of California at Santa Cruz, for instance, spent $200,000 for online proctoring in 2020–2021, and the institution’s leadership plans to continue to fund online proctoring (Harwood 2021). In addition to the monetary cost, institutions and their employees incurred a labor cost, too. Staff members had to learn how to use and support the technology. Faculty who decided to use the technology learned enough to do so, or they may have relied on staff and graduate students to troubleshoot technical problems, which meant any staff member or graduate student called upon to troubleshoot must have known how to fix the problems, and if not, they may have turned to the companies themselves for help. And finally, students, who rarely have a say in the matter, learned how to use the technology if they wanted to pass a class.

The money and labor sunk into online proctoring moves the institution, its employees, and its students further down the online proctoring path in a process of increasing returns (Pierson 2000) and software sedimentation (Weller 2020) so that change is difficult to contemplate let alone implement. As we’ve seen, the edtech imaginary is invested in software sedimentation. In response to criticism, online proctoring CEOs have promised friendlier interfaces and faster loading times (Deighton 2021), design “upgrades” presumably meant to make online proctoring more acceptable and ready for further sedimentation.

Another sedimentation tactic used by online proctoring’s defenders is to argue students have long been surveilled (Global Silicon Valley 2021). Since surveilling students is not new, these advocates observe, then contemporary warnings about academic surveillance are unfair. I read this argument as an attempt to make online proctoring more palatable—and thus more profitable—by conflating the technology with in-person proctoring. However, online proctoring is invasive in ways in-person proctoring is not (Fitzgerald 2021).

An in-person proctor does not demand to view a student’s bedroom. An in-person proctor is not an unflinching gaze trained to interpret students’ behavior through the singular lens of suspicion. When online proctoring executives and other adherents of online proctoring collapse the differences between in-person and online proctoring, they are reaching into the edtech imaginary. The story that emerges is a history of assessment practices meant to make their technology appear to be an uncontroversial extension of how students have always completed homework, quizzes, and tests. Do not trust online proctoring companies to be credible narrators. Their business depends on selling a specific tale of how we got here and where we should be going, and if nothing else, their public relations version of education history should be met with profound skepticism.

Elsewhere, online proctoring has been equated with older online learning technologies like “poorly recorded video lectures [and] inactive LMS discussion boards” (Selwyn et al. 2021, 13). I am concerned about the ways the edtech imaginary is succeeding to shape the discourse and frame online proctoring as a misunderstood, humdrum technology. I do not want racist, ableist academic surveillance to be a practice educators and students shrug off as an unfortunate but necessary part of learning. I do agree with Selwyn et al. (2021) that online proctoring demands we “develop counter-narratives that push back against the imagining of public education as simply a ‘tech issue’” (14). Where and how these counter-narratives emerge is an urgent question.

From Neutrality to Dissonance

Before exploring online proctoring counter-narratives, I want to consider how higher education institutions normalize online proctoring. Of 100 randomly selected US and Canadian college and university websites chosen from a sample of 2,155, “none took a critical stance toward proctoring tools or addressed the ethics of student surveillance” (Kimmons and Veletsianos 2021). Official institutional policy appears to treat online proctoring tools as neutral educational technology. The finding is perhaps unsurprising. While exceptions do exist (e.g., “Proctoring and Equity” from the Center for Innovative Teaching and Learning at Indiana University Bloomington), institutions that have invested money and labor into bringing online proctoring to campus may be hesitant or unwilling to criticize the same technology on public-facing websites. Neutrality is therefore a strategic choice. And because education is politics (Nieto 1999; Shor & Freire 1987), neutrality is a political choice too, one that aligns institutions with online proctoring companies.

Disrupting this neutrality becomes even more difficult because educators and institutions, perhaps unaware of the technology’s harms, often provide students with guiding language written by the online proctoring companies themselves. For example, Respondus Monitor offers instructors a template titled “Using LockDown Browser and a Webcam for Online Exams,” which instructors can copy and paste into a syllabus (Respondus n.d.). The syllabus template suggests to students that they “[t]ake the exam in a well-lit room and avoid backlighting, such as sitting with your back to a window” (Respondus n.d.). Missing from this recommendation and others like it is the reason why students must be in a well-lit room, sometimes having to resort to shining a bright light directly into their faces (Chin 2021): because many online proctoring companies use facial recognition technology. Not only do these technologies struggle to detect dark skin (Simonite 2019), they are built using biased datasets, leading to racialization and dehumanization (Stevens and Keyes 2021).

Online proctoring companies also shape the perception of their harmful technology at the institutional level. Just as individual educators might depend on the companies for ways to describe to students how to use the technology, so too do institutional how-to resources and websites. Institutional support pages are too often little more than hyperlinks to help guides and video tutorials created by the companies. In addition, an institutional resource page might repackage a company’s recommendations to students, such as one example when the Respondus Monitor syllabus language about lighting reappears on an institutional resource page warning students, “You may need to add more lighting to your workspace when using Respondus Monitor to ensure the program can recognize your face during the assessment” (Northwestern University n.d.). Once more, the reason why students need to add more lighting is glaringly absent. Online proctoring companies can continue to control the narrative about their technology as long as institutional resource pages are indistinguishable from the frequently asked questions websites produced by online proctoring companies. Thus, online proctoring companies have succeeded in making their technology appear benign by attempting to collapse the distinct differences between in-person and online proctoring. Companies have also benefited from instructors and institutions who frame the technology as neutral, often parroting company copy on syllabi and how-to webpages.

Taking lessons from a cognitive approach to learning and policy implementation can help explain why changing people’s understanding about online proctoring might be especially hard when the technology is presented in such a way that its functionality appears both commonplace and unambiguously advantageous.

We draw on prior knowledge and existing beliefs when interpreting new information (Bransford, Brown, and Cocking 2000). A problem arises because “New ideas either are understood as familiar ones, without sufficient attention to aspects that diverge from the familiar, or are integrated without restructuring of existing knowledge and beliefs, resulting in piecemeal changes in existing practice” (Spillane et al. 2002, 398). What does this mean for educators encountering online proctoring for the first (or fifth) time? When neutral or positive language masks the technology’s harms, then online proctoring can appear not to be so much a new idea but instead a logical, if imperfect, extension of an educator’s existing beliefs and practices.

That online proctoring can either be an outgrowth of, or seem an outgrowth of, existing beliefs and practices is evidence of a larger problem: the beliefs and practices themselves. Pedagogies of policing and punishment are the soil sustaining online proctoring. It’s not enough to weed out online proctoring. Instead, what we could use is a controlled burn.

To light a fire that removes online proctoring from higher education, start by revising institutional websites and resources to explicitly name and describe online proctoring’s harms. These revisions—these counter-narratives—need to produce cognitive dissonance in educators in order to disrupt the narrative of online proctoring as a necessary, innocuous technology. This dissonance can force educators to confront both the technology itself and the underlying beliefs about learning that help educators rationalize deploying academic surveillance against their students. A goal is to help educators “recognize an existing model as problematic and, then, to focus resources and support on attempts to make sense of the novel idea, restructuring existing beliefs and knowledge” (Spillane et al. 2002, 418). In other words, sparking a shift in a person’s thinking begins with illuminating the ways online proctoring is a problem both as a technical solution and as a pedagogical practice. A dissonance-producing institutional resource about online proctoring might look like Figure 1:

A hypothetical institution webpage introduces educators to online proctoring. The upper left of the page contains a quote from an anonymous student that reads: “‘It’s really cruel to have students come to class and expect to learn, and then treat them, essentially, like criminals and make them install programs that look for all their information and force them to give tours of their home.’” The upper right of the page contains an image of a camera's lens. The middle of the page contains the words: “Online proctoring is racist, ableist, and privacy-invading.” Beneath these words, appears a quote which reads: “‘If we understand teaching as consisting primarily of social relationships and as a political commitment rather than a technical activity, then it is unquestionable that what educators need to pay most attention to are their own growth and transformation and the lives, realities and dreams of their students.’ - Sonia Nieto”. Beneath the quote from Sonia Nieto appears the words: “How might we abandon pedagogies of policing and commit to pedagogies of educational dignity for ourselves and our students?” followed by a bullet-pointed list with the items: “Embrace a more holistic view of learning. Develop authentic assessments. Adopt ungrading practices. Foster partnerships between administrators, staff, faculty, and students.” At the page's bottom, two sentences read: “Read more about the harms of online proctoring and how to transition away from the technology.” and “Make an appointment with the Center for Teaching and Learning, an instructional designer, and/or an educational technologist.” The word “References” is at the bottom of the page, followed by two references. The first citation is: Hipolito, Matthew. 2020. “‘Going Through Your Things’: Remote Proctoring Software ‘Demeaning’ and ‘Cruel,’ Students Say.” The Daily, October 29, 2020. https://www.dailyuw.com/news/article_8b14f13e-197a-11eb-8730-c7459eeb446a.html. The second citation is: Nieto, Sonia. 1999. The Light in Their Eyes: Creating Multicultural Learning Communities. New York: Teachers College Press.
Figure 1. A hypothetical institutional webpage that uses a critical framing when introducing educators to online proctoring.

I recognize it’s unlikely the above resource will be adopted by institutions of higher education across the land. So I have another suggestion. Before providing a reader with installation directions and other troubleshooting tips, which is what many institutional resources do, the resource could prompt an educator to reflect on the technology and its effects by asking:

  • Do you believe students with dark skin should have to shine a bright light on their faces to be recognized as having a face by the online proctoring technology?
  • Do you believe diabetic students should be too afraid to check their blood sugar levels or eat a snack for fear of being labeled suspicious by the online proctoring technology?
  • Do you believe students should allow a stranger to have remote access to their personal computer?
  • Would you want to show a stranger your office or bedroom before an exam begins or while taking an exam?
  • Is your pedagogy founded on distrust, policing, and punishment?

Institutional resources about online proctoring may appear to play a seemingly small role in the larger conversation about the technology and its impacts on teaching and learning. However, understanding the resources as a vehicle the edtech imaginary uses to influence teaching and teachers themselves emphasizes the need to attend to how the resources frame online proctoring. Institutional resources about online proctoring can be understood as a policy technology—a technology about technology, if you will—or a means designed to implement policy. Other policy technologies include curricula and assessments (Spillane et al. 2019). The problem with institutional resources adopting a neutral or positive framing of online proctoring is thus twofold. First, as previously discussed, uncritical resources can produce, reinforce, and normalize academic surveillance and pedagogies of distrust, policing, and punishment by being assimilated into educators’ preexisting beliefs and practices.

A second damaging consequence exists. When an educator’s pedagogy is pushed toward policing and punishment, practices enabled in part by uncritical resources, their sense of themselves as a teacher risks being corrupted. Here Stephen Ball’s observation that “we do not do policy, policy does us” (2015, 307) helps to articulate why focusing on the resources’ language is so important. Because if our pedagogy is an outgrowth of our identities as educators, and policy shapes our sense of self, then a pedagogy of punishment wants us to become punishers. The policy “does us” by defining who we can be as teachers and who our students can be as learners. Recall a student forced to submit to online proctoring felt like a criminal because the technology positions students as inherently suspicious. And if a policy of online proctoring transforms students into criminals, then it turns teachers into police officers—and cops, I believe, should be banned from campuses.

A Story of Reform

Overcoming online proctoring and the pedagogies that maintain its use might begin with the individual, but we improve the chances of abolishing the technology when we join together to unlearn harmful pedagogies and replace them with pedagogies of educational dignity. To grow pedagogies of educational dignity, we can couple a cognitive approach to policy implementation with a stance toward learning as a fundamentally social experience (Lave and Wenger 1991; Vygotsky 1978). Many educators concerned about online proctoring have realized the social nature of learning by organizing events to learn with and from one another. Examples of collective meaning making include the Teach-In #AgainstSurveillance (Gray 2020) and the #AnnotateEdTech events (Logan and Caines 2021). These online gatherings, while vital for building community and solidarity, may nonetheless struggle to bring about the systemic change at institutions many of us seek.

To accomplish change at scale—a favorite word, I know, of the edtech imaginary—the movement against online proctoring can address the depth of educators’ beliefs and practices; the sustainability of changes over time; the spread of changes throughout an institution; and a shift in ownership over the new ideas from external to internal sources (Coburn 2003). Remember that changing an individual’s understanding requires resources and support (Spillane et al. 2002). Combine these additional resources and support with the elements for achieving deep change at scale (Coburn 2003), and the project of ridding online proctoring from campuses appears daunting.

Nonetheless, we can turn to the efforts of a coalition of administrators and staff at the University of Michigan-Dearborn for an example of institutional change at scale. In March 2020, the University shifted to remote emergency teaching. At the same time, the Office of the Provost and deans decided to publicly oppose online proctoring, and though the administrators did not ban the technology, they did strongly recommend faculty not use it (Silverman et al. 2021). In the months that followed, the staff at the Hub for Teaching and Learning Resources (the Hub) worked to implement the reform through a combination of depth, sustainability, spread, and shift (Coburn 2003). See Table 1 for how the University tried to accomplish the different dimensions of reform implementation.

Dimension of Reform Implementation How Administrators and the Hub’s Staff at the University of Michigan-Dearborn Tried to Accomplish the Dimension of Reform Implementation
Depth

Changes to educators’ beliefs, educators’ interactions with students, and educators’ pedagogies.

  • provided individual consultations to faculty new to online teaching.
  • assisted faculty to develop authentic assessments.
  • hosted a virtual guest speaker, an expert in authentic assessments with a speciality in the STEM disciplines.
  • organized multiple faculty development programs throughout the year.
Sustainability

A long-term commitment to nurturing educators’ development over time.

  • hired two additional instructional designers on two-year contracts.
  • hired graders to aid faculty in high-enrollment courses to grade and provide feedback on more time-intensive authentic assessments.
Spread

The diffusion of reform-related pedagogical principles within a course, department, and institution.

  • provided individual or group email responses from the Hub, the Office of Digital Education, and the Office of the Provost with a consistent message that the decision to avoid online proctoring was due to student privacy and equity concerns.
  • Silverman et al. (2021) acknowledge communications with faculty “could have been better wrapped into a cohesive, campus-wide message” (121) to improve spread.
Shift

Ownership of the reform transitions from an external reform to an internal reform with authority for maintaining the change left to groups and individuals.

  • Silverman et al. (2021) recommend designing experiences to “develop a shared critical digital literacy between instructors and students by discussing the ethical problems associated with remote proctoring and building a shared understanding of academic integrity in the digital age” (126).
Table 1. Applying Coburn’s (2003) concept of scale to the University of Michigan-Deaborn’s approach to shifting educators’ beliefs and practices regarding online proctoring.

We need counter-narratives. However, a strategy for abolishing online proctoring built only on counter-narratives risks ceding the terms of the debate to those set by the online proctoring companies. For this reason, we also need stories that aren’t defined solely in opposition to the likes of online proctoring CEOs and the edtech imaginary they’re entranced by. The coalition against online proctoring that emerged at the University of Michigan-Dearborn is an instructive example of one such alternative narrative, a story of how we might achieve deep change by developing a partnership organization founded on relationships (Logan 2020). What started as co-authoring a counter-narrative about online proctoring at the University became, over time, a new narrative about partnerships between administrators, staff, faculty, and students to develop equitable, authentic assessments.[1]

The example set by the University of Michigan-Dearborn demonstrates that when administrators offer support and financial resources to reimagine teaching and learning, trusting staff and faculty along the way, resistance to and refusal of online proctoring can generate a community that rejects pedagogies of policing and embraces people and our immutable educational dignity.

Where Might We Go from Here?

The future of online proctoring is still being written. Appealing to institutions’ and students’ fears, online proctoring CEOs tell their tales of worthless coronavirus diplomas (Harwell 2020) tarnishing an institution’s brand and raising questions about a student’s employability. The narrative belongs to the larger playbook drawn up by the corporate education movement and its vision of learning as human capital development (Williamson 2017). I believe learning cannot be reduced to a datapoint to be quantified, a credential to be protected at all costs.

Online proctoring companies possess a paltry view of education that produces and reinforces pedagogies of punishment. When confronted with the intersectional damages inflicted upon students by their technology, online proctoring companies insist their products are necessary, claiming the technology is an engine of equity (Norris 2021). Yet as Chris Gilliard argues, “A better remote proctoring system isn’t on the way—it can’t be—because they are all built on the same faulty and invasive ideas…about pedagogy, surveillance, and control” (@hypervisible, April 6, 2021).

Online proctoring is not like in-person proctoring. Online proctoring is not like a badly lit lecture video or an underused discussion board. Online proctoring is a manifestation of what Ruha Benjamin calls the New Jim Code, or “the employment of new technologies that reflect and reproduce existing inequities but that are promoted and perceived as more objective or progressive than the discriminatory systems of a previous era” (2019, 5–6). When institutions and educators frame online proctoring with market-based stories and their boogeymen, they risk being duped by online proctoring companies and their unreliable narrators selling dubious promises of objectivity and equity as evidence the technology works. In contrast, when the story of online proctoring is framed as the instantiation of the New Jim Code and its racist, ableist surveillance, then those who experience the technology’s inequities—students—emerge as trustworthy narrators with heartbreaking accounts of the humiliations they’ve had to endure. Their stories should be part of the evidence we use as we seek to rid online proctoring from schools.

Including online proctoring as part of the New Jim Code offers another possibility: that of the abolitionist imaginary. Abolitionist practices, suggested sava saheli singh (Pasquini 2021), can be a generative source of imagination, politics, organizing, and action in the struggle against online proctoring and other problematic educational technologies. Abolitionism’s emphasis on refusal alongside care and collectivity (Kaba 2021), for example, is essential if we are to develop pedagogies of educational dignity.

In addition, the fight against online proctoring takes on greater urgency when we understand online proctoring as the latest example of white supremaist surveillance technologies designed and deployed to police and punish. Like previous racializing information technologies used to surveil and control people (Browne 2015), online proctoring’s harms are experienced disproportionately by Black people as well as other nondominant populations. This longview of online proctoring is vital, for as Bettina Love notes, “An ahistorical understanding of oppression leads folx to believe that quick fixes to the system, such as more surveillance, more testing, and more punishment, will solve the issues of injustice and inequality” (2019, 92). It also means adopting the abolitionist stance that reform, even at the scale accomplished by the University of Michigan-Dearborn, cannot be where the story of online proctoring ends.

If the story of online proctoring is to end in freedom, we can start by telling counter-narratives and fashioning new narratives altogether. I am hopeful these stories will include accounts of honest institutional resources and websites. Of administrators who abandon online proctoring, despite paying for its false promises, and invest instead in providing support and funding for faculty and staff to develop authentic assessments. And I am hopeful we will share stories of lasting partnerships between educators and students, coalitions that accomplish deep change and grow pedagogies of educational dignity.

Notes

[1] An emphasis on authentic assessment is an essential element for building pedagogies of educational dignity. Authentic assessments are characterized by self-reflection and collaboration with others (Conrad and Openo 2018). Prioritizing self-reflection and embracing individuals’ genuine, complex selves can support educational dignity through intra-personal learning. Authentic assessment can also help students experience educational dignity through its frequent use of learning with and from other people, a crucial design choice upon which educational dignity relies (Espinoza et al. 2020).

References

Arthur, W. Brian. 1994. Increasing Returns and Path Dependence in the Economy. Ann Arbor: University of Michigan Press.

Bali, Maha. 2021. “On Proctoring & Internalized Oppression #AgainstSurveillance.” Reflecting Allowed, March 13, 2021. https://blog.mahabali.me/educational-technology-2/on-proctoring-internalized-oppression-againstsurveillance/.

Ball, Stephen. 2015. “What Is Policy? 21 Years Later: Reflections on the Possibilities of Policy Research.” Discourse: Studies in the Cultural Politics of Education 36, no. 3: 306–313. https://doi.org/10.1080/01596306.2015.1015279.

Benjamin, Ruha. 2019. Race After Technology: Abolitionist Tools for the New Jim Code. Cambridge, UK: Polity.

Bransford, John, Brown, Ann, and Rodney Cocking, eds. 2000. How People Learn: Brain, Mind, Experience, and School. Washington, DC: National Academy Press.

Brown, Lydia X. Z. 2020. “How Automated Test Proctoring Software Discriminates Against Disabled Students.” Center for Democracy and Technology, November 16, 2020. https://cdt.org/insights/how-automated-test-proctoring-software-discriminates-against-disabled-students/.

Browne, Simone. 2015. Dark Matters: On the Surveillance of Blackness. Durham: Duke University Press.

Cahn, Albert, Caroline Magee, Eleni Manis, and Naz Akyol. 2020. “Snooping Where We Sleep: The Invasiveness and Bias of Remote Proctoring Services.” Surveillance Technology Oversight Project, November 11, 2020. https://www.stopspying.org/snooping.

Caplan-Bricker, Nora. 2021. “Is Online Test-Monitoring Here to Stay?” New Yorker, May 27, 2017. https://www.newyorker.com/tech/annals-of-technology/is-online-test-monitoring-here-to-stay.

Chin, Monica. 2021. “ExamSoft’s Proctoring Software Has a Face-Detection Problem.” The Verge, January 5, 2021. https://www.theverge.com/2021/1/5/22215727/examsoft-online-exams-testing-facial-recognition-report.

Coburn, Cynthia. 2003. “Rethinking Scale: Moving Beyond Numbers to Deep and Lasting Change.” Educational Researcher 32, no. 6: 3–12. https://doi.org/10.3102/0013189X032006003.

Conrad, Diane, and Jason Openo. 2018. Assessment Strategies for Online Learning: Engagement and Authenticity. Edmonton: Athabasca University Press.

Costanza-Chock, Sasha. 2020. Design Justice: Community-Led Practices to Build the Worlds We Need. Boston: The MIT Press.

Deighton, Katie. 2021. “Online Proctoring Programs Try to Ease the Tensions of Remote Testing.” The Wall Street Journal, April 13, 2021. https://www.wsj.com/articles/online-proctoring-programs-try-to-ease-the-tensions-of-remote-testing-11618304400.

Espinoza, Manuel, and Shirin Vossouhi. 2014. “Perceiving Learning Anew: Social Interaction, Dignity, and Educational Rights.” Harvard Educational Review 84, no. 3: 285–313.

Espinoza, Manuel, Shirin Vossouhi, Mike Rose, and Luis Poza. 2020. “Matters of Participation: Notes on the Study of Dignity and Learning.” Mind, Culture, and Activity 27, no. 4: 325–347. https://doi.org/10.1080/10749039.2020.1779304.

Eyler, Joshua. 2018. How Humans Learn: The Science and Stories Behind Effective College Teaching. Morgantown: West Virginia University Press.

Feathers, Todd. 2021. “Proctorio Is Using Racist Algorithms to Detect Faces.” Vice, April 8, 2021. https://www.vice.com/en/article/g5gxg3/proctorio-is-using-racist-algorithms-to-detect-faces.

Feathers, Todd, and Janus Rose. 2020. “Students Are Rebelling Against Eye-Tracking Exam Surveillance Tools.” Vice, September 24, 2020. https://www.vice.com/en/article/n7wxvd/students-are-rebelling-against-eye-tracking-exam-surveillance-tools.

Fitzgerald, Bill. 2021. “There Is No Such Thing as an ‘Online Proctoring System.’” FUNNYMONKEY, August 21, 2021. https://www.funnymonkey.com/2021/08/there-is-no-such-thing-as-an-online-proctoring-system/.

Germain, Thomas. 2020. “Poor Security at Online Proctoring Company May Have Put Student Data at Risk.” Consumer Reports, December 10, 2020. https://www.consumerreports.org/digital-security/poor-security-at-online-proctoring-company-proctortrack-may-have-put-student-data-at-risk/.

Gilliard, Chris (@hypervisible). 2021. “A better remote proctoring system isn’t on the way—it can’t be—because they are all built on the same faulty and invasive ideas of about pedagogy, surveillance, and control.” Tweet, April 6, 2021. https://twitter.com/hypervisible/status/1379501433763078145.

Global Silicon Valley. 2021. “The Academic Surveillance Debate.” Filmed August 2021 at ASU+GSV 2021, San Diego, CA. Video, 40:09. https://www.youtube.com/watch?v=cX_svCuqzyU.

Gray, Brenna Clarke. 2020. “Against Surveillance Teach-in.” YouTube video, December 1, 2020. https://www.youtube.com/watch?v=tAuoj3VYy_s.

Harwell, Drew. 2020. “Cheating Detection Companies Made Millions During the Pandemic. Now Students Are Fighting Back.” Washington Post, November 12, 2020. https://www.washingtonpost.com/technology/2020/11/12/test-monitoring-student-revolt/.

Harwood, Zoe. 2021. “Surveillance U: Has Virtual Proctoring Gone Too Far?” YR Media, n.d. Accessed June 24, 2021. https://interactive.yr.media/has-virtual-proctoring-gone-too-far/.

Hipolito, Matthew. 2020. “‘Going Through Your Things’: Remote Proctoring Software ‘Demeaning’ and ‘Cruel,’ Students Say.” The Daily, October 29, 2020. https://www.dailyuw.com/news/article_8b14f13e-197a-11eb-8730-c7459eeb446a.html.

Jillson, Elisa. 2021. “Aiming for Truth, Fairness, and Equity in Your Company’s Use of AI.” Federal Trade Commission, April 19, 2021. https://www.ftc.gov/news-events/blogs/business-blog/2021/04/aiming-truth-fairness-equity-your-companys-use-ai.

Kaba, Mariame. 2021. We Do This ‘til We Free Us: Abolitionist Organizing and Transforming Justice, edited by Tamara K. Nopper. Chicago: Haymarket Books.

Kimmons, Royce, and George Veletsianos, G. 2021. “Proctoring Software in Higher Ed: Prevalence and Patterns.” EDUCAUSE Review, February 23, 2021. https://er.educause.edu/articles/2021/2/proctoring-software-in-higher-ed-prevalence-and-patterns.

Lave, Jean, and Etienne Wenger. 1991. Situated Learning: Legitimate Peripheral Participation. Cambridge, UK: Cambridge University Press.

Leafstedt, Jill. 2017. “Online Courses Shouldn’t Use Remote Proctoring Tools. Here’s Why.” EdSurge, April 19, 2017. https://www.edsurge.com/news/2017-04-19-online-courses-shouldn-t-use-remote-proctoring-tools-here-s-why.

Logan, Charles, and Autumm Caines. 2021. “Creating Counter-Media Texts in the Open With #AnnotateEdTech.” Presentation at OERxDomains21, April 27, 2021. https://docs.google.com/presentation/d/18zlOTBLdRni2Ul173Jpa_iwJD6gYO4aEnMFJzyhnjFk/edit?usp=sharing.

Logan, Charles. 2020. “Refusal, Partnership, and Countering Educational Technology’s Harms.” Hybrid Pedagogy, October 21, 2020. https://hybridpedagogy.org/refusal-partnership-countering-harms/.

Love, Bettina. 2019. We Want to Do More Than Survive: Abolitionist Teaching and the Pursuit of Educational Freedom. Boston: Beacon Press.

Nieto, Sonia. 1999. The Light in Their Eyes: Creating Multicultural Learning Communities. New York: Teachers College Press. 

Norris, Ashley. 2021. “Exam Integrity Is an Issue of Equity.” American Consortium for Equity in Education. Accessed June 23, 2021. https://www.ace-ed.org/exam-integrity-is-an-issue-of-equity/.

Northwestern University. n.d. “Respondus Lockdown Browser and Monitor.” Accessed October 19, 2021.https://canvas.northwestern.edu/courses/1580/pages/6-dot-8-respondus-lockdown-browser-and-monitor.

Pasquini, Laura. 2021. “Between the Chapters #25: Searching for the Commons in the Wasteland with @savasavasava & @audreywatters.” Between the Chapters podcast episode, April 29, 2021. https://share.transistor.fm/s/442cc432.

Pierson, Paul. 2000. “Increasing Returns, Path Dependence, and the Study of Politics.” The American Political Science Review 94, no. 2: 251–267. https://doi.org/10.2307/2586011.

Proctoring and Equity. (n.d.). Center for Innovative Teaching and Learning. Accessed June 23, 2021. https://citl.indiana.edu/teaching-resources/diversity-inclusion/proctoring/index.html.

Proctorio. 2019. “Homepage on January 26, 2019.” Accessed October 19, 2021. https://web.archive.org/web/20190126031131/https://proctorio.com/.

Proctorio. 2021. “Homepage on April 30, 2021.” Accessed June 24, 2021. https://web.archive.org/web/20210430212733/https://proctorio.com/.

Respondus. n.d. “Respondus Monitor Resources.” Accessed October 19, 2021. https://web.respondus.com/he/monitor/resources/.

Retta, Mary. 2020. “Exam Surveillance Tools Monitor, Record Students During Tests.” Teen Vogue, October 26, 2020. https://www.teenvogue.com/story/exam-surveillance-tools-remote-learning.

Selwyn, Neil, Chris O’Neill, Gavin Smith, Mark Andrejevic, and Xin Gu. 2021. “A Necessary Evil? The Rise of Online Exam Proctoring in Australian Universities.” Media International Australia: 1–16. https://doi.org/10.1177/1329878X211005862.

Shor, Ira, and Paulo Freire. 1987. A Pedagogy for Liberation. Westport, CT: Bergin & Garvey.

Silverman, Sarah, Autumm Caines, Christopher Casey, Belen Garcia de Hurtado, Jessica Riviere, Alfonso Sintjago, and Carla Vecchiola. 2021. “What Happens When You Close the Door on Remote Proctoring? Moving Towards Authentic Assessments with a People-Centered Approach.” To Improve the Academy: A Journal of Educational Development 39, no. 3: 115–131. https://doi.org/10.3998/tia.17063888.0039.308.

Simonite, Tom. 2021. “The Best Algorithms Struggle to Recognize Black Faces Equally.” Wired, July 22, 2019. https://www.wired.com/story/best-algorithms-struggle-recognize-black-faces-equally/.

Singer, Natasha. 2015. “Online Test-Takers Feel Anti-Cheating Software’s Uneasy Glare.” New York Times, April 5, 2015. https://www.nytimes.com/2015/04/06/technology/online-test-takers-feel-anti-cheating-softwares-uneasy-glare.html.

Sonnemaker, Tyler. 2020. “Tech Companies Promised Schools an Easy Way to Detect Cheaters During the Pandemic. Students Responded by Demanding Schools Stop Policing Them Like Criminals in the First Place.” Business Insider, November 1, 2020. https://www.businessinsider.com/proctorio-silencing-critics-fueling-student-protests-against-surveilalnce-edtech-schools-2020-10.

Spillane, James P., Brian J. Reiser, and Todd Reimer. 2002. “Policy Implementation and Cognition: Reframing and Refocusing Implementation Research.” Review of Educational Research 72, no. 3: 387–431. https://doi.org/10.3102/00346543072003387.

Spillane, James P., Jennifer L. Seelig, Naomi L. Blaushild, David K. Cohen, and Donald J. Peurach. 2019. “Educational System Building in a Changing Educational Sector: Environment, Organization, and the Technical Core.” Educational Policy 33, no. 6: 846–881. https://doi.org/10.1177/0895904819866269.

Stevens, Nikki, and Os Keyes. 2021. “Seeing Infrastructure: Race, Facial Recognition and the Politics of Data.” Cultural Studies 35, no. 4–5: 833–853. https://doi.org/10.1080/09502386.2021.1895252.

Swauger, Shea. 2020. “Our Bodies Encoded: Algorithmic Test Proctoring in Higher Education.” Hybrid Pedagogy, April 2, 2020. https://hybridpedagogy.org/our-bodies-encoded-algorithmic-test-proctoring-in-higher-education/.

Top Hat. 2020. “Top Hat Partners With Proctorio to Deliver Free, Secure Online Proctored Tests and Exams Remotely. Top Hat, April 2, 2020. https://tophat.com/press-releases/top-hat-partners-with-proctorio/.

Vygotsky, Lev Semyonovich. 1978. Mind in Society: The Development of Higher Psychological Processes. Michael Cole, Vera John-Steiner, Sylvia Scribner, and Ellen Souberman, eds. Cambridge: Harvard University Press.

Watters, Audrey. 2020. “The Ed-Tech Imaginary.” Hack Education. June 21, 2020. http://hackeducation.com/2020/06/21/imaginary.

Weller, Martin. 2020. 25 Years of Ed Tech. Edmonton: Athabasca University Press.

Williamson, Ben. 2017. Big Data in Education: The Digital Future of Learning, Policy and Practice. Thousand Oaks, CA: Sage.

Woldeab, Daniel, and Thomas Brothen. 2019. “21st Century Assessment: Online Proctoring, Test Anxiety, and Student Performance.” International Journal of E-Learning and Distance Education 34, no. 1. http://www.ijede.ca/index.php/jde/article/view/1106/1729.

Zhu, Eva. 2021. “‘Foolproof’ Exam Software Creating Barriers for Students With Disabilities.” Healthy Debate, March 1, 2021. https://healthydebate.ca/2021/03/topic/exam-software-disabilities/.

About the Author

Charles Logan is a PhD student in learning sciences at Northwestern University. A former high school English teacher and university educational technologist, his research interests include critical digital pedagogy, co-authoring counter-narratives to oppose sociotechnical and edtech imaginaries, and designing learning experiences to support educational dignity. He is on Twitter @charleswlogan.

A bookcase opens to reveal a secret passageway.
4

Back Doors, Trap Doors, and Fourth-Party Deals: How You End up with Harmful Academic Surveillance Technology on Your Campus without Even Knowing

Abstract

In this paper we describe fourth-party vendor relationships between remote proctoring tools and other companies in higher education with a specific focus on the remote proctoring company Proctorio. We unpack the problematic nature of such relationships in general but note that they are exacerbated when dealing with technologies as harmful as remote proctoring. Fourth-party relationships are particularly troublesome because those who work at institutions of higher learning are often unaware of their existence or can do little to impact or change them. We present a “harm index” reviewing literature around the harms of remote proctoring systems. We describe the nature of different types of fourth-party relationships and perform a content analysis of the partnerships listed on Proctorio’s website. We use an autoethnographic approach to share our experience as instructional designers, at an institution which has taken steps to limit the use of remote proctoring, and of attempting (and succeeding) to get the fourth-party integration of Proctorio removed from our learning management system’s integration with McGraw Hill Connect. The paper concludes with a discussion of the discourse and rhetoric used to rationalize this harmful technology, and our recommendations for how institutions might exert more control over fourth-party integrations with harmful surveillance technology.

Introduction

The year is 2026, and Regional State University (RSU) has bounced back from the enrollment woes that resulted from the COVID-19 pandemic of 2020 and 2021. After intensive faculty development efforts during the pandemic, the majority of instructors on campus became skilled online instructors. RSU set itself apart during the pandemic by rejecting remote proctoring systems by issuing several strong recommendations against such tools to their faculty and offering faculty development opportunities around authentic assessments. RSU was featured in several national news articles for their stance on remote proctoring and celebrated for their attention to student privacy. New students were attracted to RSU because of its reputation for having strong online programs and also because of RSU’s commitment to student privacy and authentic assessments.

Aisha is a new student who specifically came to RSU because of its reputation for providing an excellent education with a personal touch. She had a friend, a year older than her, who had ended up at one of the for-profit online education schools and was routinely subjected to various forms of invasive surveillance. So, when Aisha’s math professor told the class that they would be using a remote proctoring system for some of their homework assignments she was a little put off. She knew the school did not have an official ban on such technologies but she had read the news articles and the Provost’s and the President’s statements advocating for student privacy. She tried to talk to her professor and asked specific questions about what data would be collected and how long it would be held but the professor only directed her to a long “terms of service” and privacy policy that was hard to understand. She decides to bite the bullet and just use the technology for this one class.

On her first attempt to use the technology she has to pass an identity check using facial recognition. She keeps getting an error that the software cannot detect her face and decides to call RSU’s technology help desk to see what the problem might be but no one there seems to know about this software. They ask her how she got access to this system and in tracing her steps she mentions that she started by clicking a link in the Learning Management System (LMS). So they transfer her to Summer, an administrator of the LMS. Summer tells Aisha that though she got to this proctoring system through the LMS that the school does not have any administrative access to this remote proctoring system and cannot help her in any way—that she will need to contact the remote proctoring company’s help line.

Little does Aisha know that Summer is a student privacy advocate and is shocked to hear that some students are being subjected to a remote proctoring system. Summer begins the process of trying to figure out what is going on but while waiting to hear back from the customer support rep at the math homework system she sees that RSU is in the national news—headline: “RSU Student Humiliated.” The article features Aisha who is recounting her experience with the remote proctoring company’s help line that Summer referred her to, and it just sounds horrible. They asked Aisha to jump through various hoops including shining a light on her face and even asked her to remove her hijab. Matters are made worse when a week later the proctoring company suffers a major data breach potentially exposing a massive amount of student data on the dark web. Summer is at a complete loss for what to do. Protecting student privacy is part of her job, but the company violating students’ privacy has no real relationship or accountability to RSU. She finds a clause buried in the university’s contract with the math homework system saying that they consider the contract to be binding for themselves but also for any partnerships that they enter into. Summer wonders if this means that the math homework company is legally responsible for the remote proctoring company—even so, it sure feels like the damage to student trust will fall back on the university.

The story above is a semi-fictional speculative account of potential harms that can come from fourth-party vendor relationships, in which outside companies partner with one another with little oversight by the educational institution. Ross (2017) explains how “speculative methods are particularly important for the study and analysis of digital education because of its rapidly changing nature, and the need to anticipate potential ‘unintended consequences’ of such rapid changes.” We based this story on real events that we experienced in our own professional context and the documented experiences of other students, faculty, and staff, which are reviewed in our remote proctoring Harm Index (Table 2). In this paper we will define fourth-party vendor relationships and investigate how the harms of remote proctoring technology are particularly pernicious in the context of a fourth-party deal.

We describe the nature of different types of fourth-party relationships with a focus on the remote proctoring company Proctorio’s partnerships. Our decision to focus on Proctorio is due to our practical experience working with the product through an integration with McGraw Hill Connect, a program that provides electronic textbooks and other learning materials. This paper adds to the larger body of literature that takes a critical view of educational technologies, in that these technologies often act as “mechanisms of economic capture, surveillance, and control” (Paris et al. 2021) and often work against the core pedagogical goals of educational institutions. In that institutions of higher education have a responsibility to treat students’ data responsibly and monitor the data privacy practices of vendors with access to student data, we suggest that fourth-party partnerships obstruct educational institutions’ regular oversight of student information.

Methodologies

Literature review

As remote proctoring technologies are relatively new and knowledge about them and their harms is rapidly evolving, the literature we have reviewed includes not just scholarly works but also news stories, especially accounts from students who have been exposed to remote proctoring technology in educational settings. Because we aim to define and detail the harms of these technologies, both generally and in the context of fourth-party deals, we found it necessary to focus on literature by and about students, faculty, and staff who had been negatively impacted by these technologies.

Content analysis

Our methodology includes content analysis of Proctorio’s website, help documentation, and publicly available information about their partnerships with other education technology companies (Hsieh and Shannon 2005). We analyze these materials to understand the nature of the relationships between Proctorio and its partner companies. The appendix outlines the documents analyzed in our content analysis.

Authoethnography

We use collaborative autoethnography to analyze our experience identifying the existence of a fourth-party surveillance company operating on our university campus, and requesting that the third party restrict the fourth-party from working with our campus users (faculty and students). Autoethnography is a method used to systematically analyze and interpret personal experiences (Ellis, Adams, and Bochner 2011). Collaborative autoethnography enables researchers to work together to combine their shared and personal narratives to make meaning out of experiences (Chang, Ngunjiri, and Hernandez 2016). Our collaborative ethnography was constructed using alternating personal narratives, in which we engage in critical reflection on the experience of identifying and ultimately removing a fourth-party proctoring service from the campus where they are both employed as instructional designers. Our perspective is one of current instructional design and educational technology professionals in higher education, employed at University of Michigan–Dearborn (a small, undergraduate-focused campus of the U of M). Both of us were already opposed to remote proctoring for the reasons detailed in Table 2 before the events in our autoethnography unfolded and had participated in public conversation about the harms of remote proctoring on social media. We additionally co-authored a peer-reviewed paper with colleagues about authentic assessment as an alternative to remote proctoring and issued a press release with Fight for the Future about the paper and expressing our difficulties with the McGraw Hill partnership with Proctorio. These experiences shape our narrative in that we likely had more prior knowledge of the harms of remote proctoring and the potential roadblocks to removing it from our campus. As our narrative will show, the process was complex, frustrating, and time-consuming despite our prior knowledge and preparation.

Defining the Fourth-Party Relationship

Three circles show a kind of venn diagram. Circle 1 is labeled Originating Party (1st and 2nd party): Institution and Direct Constituents - Students/Faculty/Public. Circle 2 is overlapping with Circle 1 and is labeled The 3rd Party - Outsourced Vendors. The overlap between these two circles is meant to show their relationship and reads Contract negotiated by institution; support staff from institution. Circle 3 is positioned under Circle 2 and is overlapping Circle 2. The overlap between these two is meant to show their relationship and reads Contract negotiated between vendors. In the lower left of the image there is text that reads Lack of relationship between Originating Party and 4th Party(ies) with a large arrow pointing to the lack of overlap in circles between Circle 1 and Circle 3
Figure 1. Diagram of fourth party relationships.

Third party services are common in the educational technology industry. For example, instead of building a platform on which to administer online courses, a university will often contract with a company to provide a Learning Management System (LMS). The company hosts and maintains the software, and the university pays a fee to use it. A fourth-party relationship comes about when vendor A, with whom the institution has an established relationship, partners with vendor B with whom the institution does not have an established relationship (Aldoriso 2020). Language can be confusing here because the vendor may refer to the partners as their third-party partners. This is because they see themselves as the originating party. However, this is an erasure of the institution as the originating party.

This kind of vendor relationship in higher education is a problematic one in that it is once removed from a direct contractual relationship with the higher education institution. Contracts and or user agreements with institutions may simply note that external services may be offered through Vendor A’s product, but the use of Vendor B’s product is governed by Vendor B’s terms of use. Note that these clauses are general, referring to a hypothetical Vendor B, rather than a specific one. For example, McGraw Hill Education’s terms of use states,

The Products may contain links or connections to third-party websites, services, or other technology provided by third parties that are not owned or controlled by us (“Third-Party Services”). When you access Third-Party Services, you accept that there are risks in doing so, and that we are not responsible for such risks. A Third-Party Service may include or be accompanied by a separate service, license, privacy policy, or other agreement (“Third-Party Agreement”), in which case that Third-Party Service is provided solely under the terms of that separate Third-Party Agreement. (McGraw Hill Education 2020)

Here, we see McGraw Hill (a third party to the university) referring to other services as a third party. This is confusing because what they refer to as a third party in these terms of use is the same as what we refer to in this paper as a fourth party because we recognize the university as the originating party. Even though McGraw Hill’s “third parties” (fourth parties to the university) are yet unknown to the user, McGraw Hill has already absolved itself of responsibility for these products. While it is entirely possible that in the process of using an add on (such as a Proctorio to McGraw Hill) the user will have to accept the fourth parties terms of use, it is only the end user (likely a faculty member or student) that comes in contact with those terms, rather than a representative of the institution with expertise in learning technology and student data privacy.

Fourth-party partnerships almost always introduce a new feature or functionality to the third-party product. Depending on the product and where any particular individual exists in the university hierarchy, new features from this relationship may be perceived as desirable or neutral (Gogia 2021). However, these partnerships merit increased suspicion when the fourth-party product does not have an obvious connection to the third-party product, or does not have an educational value. While such spurious partnerships could be viewed as simple money-making opportunities, these relationships can have dangerous implications when they involve problematic technologies that have a history of harming students, such as remote proctoring, because these technologies make their way into classrooms with little institutional input or support.

Jones et al. (2020) argue that institutions of higher education are examples of “information fiduciaries” (Balkin 2016), meaning that they have a particular obligation to treat the data of their primary stakeholders (students) responsibly. Fourth-party agreements constitute a threat to educational institutions’ responsibility to their students in terms of data privacy because, as we describe above, they expose students and their information to technologies that may harm their students or operate with bias.

Doorways Between Companies and Institutions

There are many examples of university maintained systems that allow for the possibility of integration with third-party or fourth-party vendors. For example, Google Apps for Education allow for various integrations with their suite of tools and Zoom has a marketplace where vendors can be allowed to offer integrations. One common option for a university-maintained Learning Management System (LMS) has long been the possibility to use plug-ins, and more recently the Learn Tools Interoperability (LTI) protocol (Severance, Hanss, and Hardin 2010), for integrating tools. There is an important distinction between a “true” fourth-party partnership and an integration option with a university maintained system. In instances where there is a possibility for integration, a university has some direct influence to work with the vendor to negotiate and navigate university policies around such integrations; for instance data privacy or procurement protocols. In some way (either by a dedicated employee or by a university committee) the university works directly with the vendor. This is different from fourth-party relationships which are inherently relationships between vendors which bypass the university entirely. It is also important to note that these kinds of integrations can actually act as the mechanism through which fourth-party vendors can come in; considering that once a third party is integrated that third party can then easily partner with a fourth party.

Relationships between a third-party and a fourth-party vendor can vary widely depending on the business partnership but ultimately these relationships act as doors into the institution. Just as there are many different types of doors: french doors, sliding doors, split doors; which can be in multiple states (open, closed, ajar), there are many different kinds of relationships between third- and fourth-party vendors. It is important to understand these relationships as they can deeply alter the power dynamics that are at play. Below are several relevant examples of relationships that can apply to partnerships between educational technology companies:

    • Integration Possibility – an option for two technologies to work together when an institution has separate agreements with both of them. Often a precursor to a fourth -party integration but not necessarily a fourth-party integration in and of itself.
    • Free – the third party offers the fourth-party service or product for free within their system.
    • Freemium – the third party offers some basic functions of the fourth party’s service or product for free with an option to pay for more advanced features. The free product in the Freemium model may not be totally free for the user—they may “pay” using an alternative currency known as “mind share,” or the “development of awareness for the provider’s brand and the consideration for purchase of future commercial products and services” (Pujol 2010). The free version, while a product or service in itself priced at $0, also serves as a type of marketing device for the paid product.
    • Resell – the third party offers the fourth party’s service or product at a cost. The cost can be charged to the institution, but is also frequently placed on the student.

Identifying the Doors: A Content Analysis of Proctorio’s Fourth-Party Partnerships

The remote proctoring company Proctorio lists a number of partnerships with other companies on its website, which are all presented under the category of “assessment platforms.” Our content analysis of these partnerships is summarized in Table 1 and details of the sites reviewed can be found in the Appendix.

Partner Nature of relationship to Proctorio (free, freemium, resell, integration) How partner defines their business (assessment, learning) Pricing for students/institutions How partner describes what Proctorio does
TopHat Student Resell – Top Hat resells Proctorio by directly charging students Active Learning Platform Each student pays $30 for access to TopHat Pro and then $10 per course for access to Proctorio “Partnership ensures higher education institutions transitioning to remote teaching can preserve the integrity of their tests and exams.”
McGraw Hill Connect Freemium/Student Resell – McGraw Hill Connect provides a freemium Proctorio plan directly to instructors, resells a more advanced product directly to students Course Management and Adaptive Learning Free for all instructors to require for their students. Instructors can additionally require a premium option for $15 a semester paid by students “You’re in control. Ensure your course’s academic integrity.”
Cirrus Assessment Integration possibility – Integration is possible with separate agreement with Proctorio End to end assessments platform “To make use of Proctorio you should have an agreement with Proctorio. Once the agreement is final, Proctorio will share K&S Keys for Cirrus to setup. Once the K&S details are received by Cirrus it will take a maximum of 24 hours to setup.” “Recordings of the exam sessions can be viewed and cheating behaviour is automatically flagged by the AI.”
Ans* Integration possibility – Integration is possible with separate agreement with Proctorio “Ans* is designed to support paper, digital and hybrid examinations. With a click of a button, you can convert a digital test into a face-to-face exam and vice versa.” “When a licence has been acquired, Ans* will support you in setting up the configuration by following these steps:…” “By enabling the integration with Proctorio, the security of the administration of the exam is increased. Proctorio collects information that can be used to handle fraud procedures of your institution. With online proctoring, you’re able to administer exams in a more secure way anyplace, anywhere.”

“Within the gradebook, suspicious behavior of the student is flagged.”

Ascend Learning Integration possibility – Ascend can facilitate the integrations without a separate agreement with Proctorio Integrated software, assessment, and analytics solutions Unknown “Proctors are monitoring for odd or disruptive behavior. Do not engage in misconduct or disruption. If you do, you will be dismissed, and your exam will not be scored.”
Derivita Freemium – unclear how much is paid for extra features and who pays “Derivita is a first of its kind STEM technology platform and complete computer algebra system.” “Lockdown” at no cost, other features “Derivita has been fully integrated with Proctorio’s remote proctoring platform. This enables educators to administer STEM assignments within the LMS, using Derivita’s content and technology, while ensuring rigorous adherence to academic integrity standards.”
EvaExam Unknown – While the partnership with EvaExam is listed on Proctorio’s site, EvaExam’s site does not detail the cost structure or nature of the partnership Testing Platform Unknown “ID verification, recording video, audio, the participant’s screen, and any web traffic on the system used may be centrally controlled, automated, and are legally compliant with the additional Proctorio plugin.”
QuestionMark Unknown – likely freemium. Questionmark offers an AI proctoring service called “record and review” that is facilitated by Proctorio, but also offer a live remote proctoring service Assessment platform Unknown “The automated system observes and records the exam session on video, for potential review later. The system flags potential anomalies, such as a second person on screen. When the system flags an anomaly, the customer can review it or send it to Questionmark for inspection. This makes it harder for a test-taker to cheat or to copy the exam questions to pass onto others.”
Table 1. Overview of Proctorio partnerships with “Assessment Platforms.”

The Proctorio website groups all of these partnerships together as “assessment platforms,” a decision which obscures the significant differences between the services that the partner provides (e.g. textbooks, assessments, active learning) but also what the nature of the partnership is between the company and Proctorio. For example, at the time of this writing, Proctorio’s partnership with McGraw Hill Connect allows any instructor who has adopted a McGraw Hill text to enable Proctorio on student activities that are assigned through the Connect platform (and additionally to enable a paid premium product for students), while Proctorio can only be used on the Cirrus Assessment platform when an institution has a separate (paid) agreement with Proctorio—an agreement that likely requires some attention from someone at the university.

Loose Hinges: Inconsistent Messaging in Fourth Party Relationships

Fourth-party relationships can introduce confusion about the purpose and functionality of the products involved. Proctorio defines itself as a tool for promoting academic integrity, which gives instructors information that they can use to make their own decisions about whether cheating has occurred during a given assessment. However, because many instructors encounter and use Proctorio’s product through these other fourth-party products, they may receive all of their information about Proctorio through the help documentation of this other vendor. These Proctorio partners and resellers can describe the purpose of Proctorio’s product in their documentation in ways that are not entirely congruent with Procotrio’s stated purpose. For example, at the time of this writing, Cirrus Assessment’s help documentation page titled Integration with Remote Proctoring from Proctorio states that “Cheating behaviour is automatically detected by the AI” (see Appendix).

Screenshot of Cirrus Assessment website. Text reads: On Friday the 6th (last week) we released an integration with the remote proctoring solution of Proctorio. We've run several successful pilots this week and are now opening up the possibility of remote proctoring through Proctorio for all our customers. The integration is seamless and is fully automated. Highlighted text reads: Cheated behavior is detected by the built-in AI. (red arrow points to highlighted text) How does it work? Remote proctored exams cannot be started unless in a proctored mode.
Figure 2. Screenshot from Cirrus help documentation.

Although we strongly object to the idea that any behavior detected by an AI can represent “cheating behavior,” this statement from Cirrus is notably at odds even with Proctorio’s public statements about their product. On their FAQ page under the question “Does Proctorio utilize algorithmic decision making?” Proctorio states that “No,… Proctorio’s software does not perform any type of algorithmic decision making such as determining if a breach of exam integrity has occurred. All decisions regarding exam integrity are left up to the exam administrator or institution” (see Appendix).

Screenshot showing text from Proctorio FAQ with Proctorio logo in upper left corner. Text reads No, Proctorio only uses algorithms for face and gaze detection if certain settings are enabled on an assessment. Proctorio's software (bolded) does not (end bolded text) perform any type of algorithmic decision making, such as determining if a breach of exam integrity has occurred. All decisions regarding exam integrity are left up to the exam administrator or institution.
Figure 3. Proctorio’s FAQ question about AI decision making.

Additionally on the Proctorio FAQ page for the question “How do you decide what behaviour counts as ‘cheating’?”, Proctorio themselves state, “Only the exam administrator or the institution can dictate what type of behaviour they want to monitor over the course of an exam. Exam administrators will then review exam attempts to determine whether any flagged behavior was truly infringing on the integrity of the exam” (see Appendix). Moreover, Proctorio’s own “acceptable use policy” prohibits punishment of students based solely on Proctorio reports. The policy states that “Institutions and their representatives are prohibited from making any negative decisions regarding exam integrity or from imposing any other negative consequence or detriment on an End User based partly or entirely on Proctorio’s analysis” (see Appendix).

Screenshot of Proctorio website showing the page URL and the Proctorio logo. Text on page reads Institutions and their representatives are prohibited from making any negative decisions regarding exam integrity or from imposing any other negative consequence or detriment on an End User based partly or entirely on Proctorio’s analysis. Institutions and their representatives must conduct their own independent, non-automated review, and analysis of any relevant data (including any available audio, video, screen recordings or images that were the subject of Proctorio’s analysis or that the Institution otherwise deems relevant) prior to imposing any such negative consequence or detriment
Figure 4. Excerpt from Proctorio’s Acceptable Use Policy.

Miscommunications like these about what the product is for and how it can be used, show how misleading fourth-party relationships can be. But while fourth-party partnerships are complex they are not inherently bad. Returning to our speculative narrative from the introduction, the math homework system we mention could integrate a digital graphing calculator from an outside vendor to assist students with their work with little negative consequence. However, when fourth-party deals are made with companies that offer harmful products these relationships become particularly problematic.

What’s the Harm?

Given that fourth-party integrations can expand the functionality of a third-party tool, often for free, what is the harm? Why would some schools not want free access to more functions? To explore this we review literature around the various harms that have been caused by remote proctoring systems. Reports of the many harms of remote proctoring systems have been widely documented especially since the beginning of the COVID-19 pandemic. Walker et al. (2020) outlined harms of remote proctoring technologies specific to nursing students, the nursing profession, and to the public. To address the important question of “What is the harm?” we have indexed harms from a broad perspective to create our Harm Index of Remote Proctoring Systems, which we present in Table 2. Here we are exploring three different levels of potential harm: harm to the student, harm to the institution, and harm to larger society.

Legend:
Harm to student – *
Harm to institution – +
Harm to larger society – ^
Data Security

  • Some remote proctoring companies store student recordings for years (Gogia 2020) *.
  • Remote proctoring companies have experienced data breaches (Abrams 2020)*.
  • Teaching students to install software that undermines the integrity of their computing environment can ingrain poor data security habits (Fox Cahn et al. 2020)*^.
Does Not Prevent Cheating/There are better kinds of assessments

  • Students have found ways around proctoring software and publicize these hacks (Geiger 2021)+.
  • Real-life tasks are a better assessment of real-life skills – students are assessed the the most effective ways (Crosslin 2021; Feathers 2020b; Silverman et al. 2021)*.
Test Anxiety

  • Remote proctoring systems can raise student anxiety (Chin 2020)*.
  • Student performance can suffer when they have test anxiety (Woldeab and Brothen 2019)*.
Accessibility

  • Remote proctoring features may not be compatible with adaptive technologies such as screen readers(Office of Information Technology 2021)*+.
  • Basic access to these technologies are often difficult (Feathers 2020a)*+.
Bias – Race, Ability, Gender

  • Those with certain kinds of disabilities can trigger cheating flags of no fault of their own – tics, eye movements, self-massage, needing to go to the bathroom (Brown 2020)*+^.
  • Flagging these behaviors is a feature not a bug of AI proctoring systems – they are designed to look for “atypical” behavior (Patil and Bromwich 2020) *+^.
  • Algorithmic proctoring uses facial recognition/detection technology which can fail to recognize those with dark skin (Clark 2021)*+^.
  • Students can be locked out of exams if a face is not detected in the frame (Chin 2021)*.
  • Reaching out to support can lead to degrading practices to “troubleshoot the problem” like being asked to shine a light on your face (Caplan-Bricker 2021)*.
  • AI identification methods can be compromising for trans and non-binary students (Swauger 2020)*+ .
Invasion of Privacy

  • Room scans are invasive and intrusive – they can reveal personal information the student doesn’t wish to share (Harwell 2020a)*.
  • Product features sometimes ask students to show parts of their bodies (their lap) in inappropriate ways (Harris 2020) *+.
Cost

  • Proctoring can cost upwards of $500,000 a year (Harwell 2020b)+.
  • High costs of proctoring borne by students or budget-squeezed institutions (Malone 2019; Wan 2019)+^.
  • Additional costs from the possibility of court cases and public relations (McKenzie 2021)+.
Liability

  • Proctoring services may not always be in compliance with state or local laws about student surveillance or collection of biometric data (Long 2021)*+^.
  • Human proctors are “alone” with students and may harass or otherwise harm them while the student is involved in course activities (Bhat 2021)*+.
Digital Divide/Digital Redlining

  • Many remote proctoring technologies require expensive hardware (laptop, webcam, microphone) that students may not have or software (a certain browser, a browser extension) that students may not consent to installing (Selinger and Gilliard 2021; Yun 2020)*.
  • Internet bandwidth is not the same everywhere and some students may struggle with connections (Flaherty 2021)*.
Larger harms to freedoms and society

  • Chilling effect on academic freedom – regarding research and choice in teaching
    • Australian researchers found that their research was hindered due to the litigious nature of proctoring companies and the larger negative climate around remote proctoring (Selwyn et al. 2021) +^.
    • Academic Integrity Researcher Phillip Dawson had to return grant funds because he could not find a remote proctoring company that would let him research their tool to see if it actually prevented cheating (CRADLEdeakin 2020)+^.
    • Some instructors are not given a choice about using this technology *+^.
  • Normalization of surveillance on students and faculty
    • Surveillance technologies are used in conjunction with human rights violations all over the word – proctoring normalizes surveillance for students (Fox Cahn et al. 2020)*^.
    • Short distance from surveillance of students in learning activities to surveillance of faculty during official university business (teaching, communications, etc.) (@hypervisible 2020)*+^
  • Degrading Trust
    • Remote proctoring systems have implications for eroding student trust (Stewart 2020)*+^.
Table 2. Harm index of remote proctoring systems.

Closing the Backdoor

The following is an autoethnographic reflection between us (the two authors) about our experience discovering and investigating a fourth-party proctoring option at our university, which had a stated anti-proctoring technology stance (Silverman et al. 2021). These events took place between February 5th and May 15th of 2021 and it is of importance to note this is the time it took to have a technology removed that was never vetted or approved by the university in any way. McGraw Hill’s partnership with Proctorio was established as early as February of 2020 (@mheducation 2020) potentially in response to the mass transition to online teaching during the COVID-19 pandemic. However, as this autoethnography will show, no one at the institution seemed to have any knowledge of it for a year. The partnership began to draw wider attention when a parent’s group published an open letter asking McGraw Hill to end their partnership with Proctorio (Ongweso 2020).

Autumm: In February of 2021 I was browsing twitter when I came across conversations about a McGraw Hill Connect (MHConnect) partnership with Proctorio. I did some googling and came across their webpage outlining this partnership and stating that the proctoring options would be part of any text with a 2019 copyright or newer. Days prior, I had been in a Canvas administrators meeting for our institution where the MHConnect integration had been discussed. It was noted that the new integration was mostly a “pass through” over to the publisher’s private platform, which they controlled. I had thought this was a good thing at first, that perhaps it meant that less of our student’s data would be passing between systems but then realized when I saw the Proctorio partnership that this would mean that our students could be subjected to this harmful technology without much oversight from our institution. This upset me for personal reasons but also because our campus had specifically rejected remote proctoring since the beginning of the pandemic. I reached out to Sarah on Twitter DM to vent.

Sarah: When Autumm contacted me to tell me about the McGraw Hill Connect partnership with Proctorio and how it might affect our students I was first and foremost angry. But then I immediately thought about our campus decision to reject remote proctoring (as Autumm mentions) and how that should carry some weight in our working relationship with McGraw Hill. After all, we are a customer and partner of theirs, and it is very common for customers of technology companies to ask to have the product configured to their liking (just take a look at several different universities’ instances of the Canvas LMS to see an example of edtech customization). I suggested to Autumm that we attempt to have Proctorio removed from MH Connect platform for UM-D users, assuming that this would be an easy request to fulfill.

Autumm: When I contacted Sarah I was feeling that we were somewhat at a dead end because of the nature of the integration that I had heard about in the admin meeting days prior. It did not sound like other integrations that I’d had experience with (having LMS admin experience from previous roles) that would bring outside functionality to the LMS—and over which an LMS admin would have more control. Rather, this integration just seemed to pass credentials over to another McGraw Hill controlled platform. Talking with Sarah she questioned my assumptions and asked many questions about configurations and customizations that might be possible. She made good points and I felt like it was a long shot but I reached out to our institution’s lead LMS administrator to point out this new partnership and question if customizations or a shut-off might be possible. He was not aware of this partnership but agreed that it was concerning and said that he would reach out to McGraw Hill. After waiting about two weeks and hearing nothing I reached back out to him and he said he would reach out again. Two days later he wrote to me to say he had met with representatives of McGraw Hill who said that they would turn the integration off for our school but that there were conditions: (1) that they wanted to look and see if there was anyone using the integration and (2) that they needed two weeks to turn it off. I was unhappy about the two week waiting period, especially since we had already waited two-weeks to initially get a response, and I never fully understood why it was required. Later that day one faculty member was identified as already using the integration and McGraw Hill additionally requested an email from the associate provost to perform the shut off. Arrangements were made to work with the faculty member who was using the system to find alternatives and the official email from the associate provost requesting the shut off went out. However, a month later Sarah was working with a faculty member who was using MHConnect and found that the options for proctoring were still there.

Sarah: That particular faculty member hadn’t wanted to use the proctoring feature but noticed that it became available. I think it is important to note that it was a complete coincidence that I was working with an instructor that was using MHConnect, and thus was able to verify that the proctoring feature had not been turned off. Absent my working relationship with this faculty member, none of the instructional designers on our campus would have access to Connect or any fourth-party tools that are connected to it. Together, Autumm relayed that the proctoring feature was not turned off to the LMS administrator and our associate provost, who contacted McGraw Hill again. At this point, we were assuming there was some sort of technical misunderstanding. We eventually heard back that while McGraw Hill could turn off the integration for users that accessed Connect through the LMS, it could not turn it off for those that logged in directly through the Connect site. Late in April I tweeted, “Frustrating day for resisting surveillance and e-proctoring. Found out that McGraw Hill cannot disable the Proctorio integration in “Connect” for all our users. This integration is built on the presumption that eproctoring is an uncomplicated value-add to any course. It is not.” Evidently, someone from McGraw Hill saw this tweet, and reached out to our associate provost by email offering to have a Zoom call in which we could clarify the details of McGraw Hill’s position.

Sarah and Autumm: We accepted the offer for a Zoom call, and decided to use it as an opportunity to better understand how Proctorio partners with McGraw Hill in addition to discussing how Proctorio could be removed for our campus users. We discussed a plan for McGraw Hill to deactivate Proctorio for all our campus users, both those that log in through our LMS and those that log in through the McGraw HIll Connect site. We then inquired as to whether the Data Privacy Agreement (DPA) we had signed with McGraw Hill covered other partnerships that they chose to make, such as the one with Proctorio. They responded that they viewed the DPA as applicable to any other technology companies that they partner with, meaning that our original DPA covered Proctorio being used through McGraw Hill Connect by our users. They also reiterated that they did not want to force Proctorio on anyone, and that they were happy to pursue various avenues to restrict its use on our campus if that is what we desired. In response to our dissatisfaction that they had integrated Proctorio into our users accounts without informing us or asking permission, they maintained that if we had a campus policy against remote proctoring, it was primarily our responsibility to inform faculty of the policy and enforce it.

Discussion and Recommendations

We have described the problematic nature of current fourth-party partnerships, but there is potential for more problematic future partnerships. Fourth-party partnerships may exist explicitly to circumvent campus decisions or policies (such as administrative policies, or faculty governance) or to respond to budget and purchasing constraints (Gogia 2021). The loopholes which are created as part of these deals can do real harm. Considering how we speak about and rationalize such technologies is an important part of analyzing how they end up existing on our campuses.

Caines (2021) described a weaponization of care around surveillance technologies where they are sold and rationalized under a rhetoric of care and gives specific examples from remote proctoring companies. Adjacent to this frame we also see remote proctoring companies using what Herzog (2010) called the “banality of surveillance.” With this construct we see companies making the case that the technology is essential and, though it may not be perfect, we need to suffer with the drawbacks because the good outweighs the bad (McFarland 2021). Third parties that form relationships with remote proctoring companies also implicitly make the case that remote proctoring is essential (and harmless) by integrating it into their product without user or institutional consent. But the banality of surveillance around remote proctoring is nothing but smoke and mirrors, as our Harm Index (Table 2) shows. The harms suffered by students, institutions, and larger constructs such as academic freedom from these technologies is very real. Additionally, multiple educators have pointed out that other, more authentic kinds of assessments do not even require exams (Crosslin 2021; Silverman et al. 2021).

Relationships between educational technology companies and educational institutions can be fraught. Often, there is a disconnect between how institutionally approved technologies are chosen and whether they support the kind of education the institution wants to provide (Cohn 2021). Our first recommendation for institutions to have more control over fourth-party integrations is for universities to consider not only which technologies to adopt, but which technologies, and specifically what functionalities, they do not want to adopt. Methods and tools need to be created to assist with alignment of the university’s mission, strategic plan, or other guiding principles for evaluating not just the benefits but also the harms of technologies. The addition of new functionalities to an existing educational technology tool through a fourth-party relationship can be particularly subtle and they often come with a techno-utopian sales rhetoric that fails to imagine that this addition could be anything but a good thing. Without consideration of where boundaries exist for the institution in regards to what technology is acceptable and what is not it is impossible to express issues with fourth-party relationships. Passing resolutions and offering recommendations against such technologies can go a long way against limiting the use of harmful technology.

Our second recommendation is for institutions to leverage their influence as direct paying customers, or as the provider of a sales environment of these tools, to demand that the third party remove the surveillance functions provided by the fourth-party companies. Enterprise systems are regularly configured to customers specifications. Shutting off these integrations is technically possible. Success in getting a fourth-party integration removed may vary depending on the specifics of the fourth-party partnership (as discussed above). We speculate that McGraw Hill was willing to grant our request for several reasons. For one, on balance it is better for them to retain us as a satisfied customer than to insist on proctoring functionality in our school. In addition, there is the name recognition of our institution and our relationship with our flagship university. Finally, it is important to note that we did take steps to bring public light on our situation. We wrote a peer-reviewed paper about our experiences and issued a press release (Fight for the Future 2021), which could have impacted our ability to successfully get this integration removed. While resisting harmful fourth-party integrations is a difficult, time-consuming, and unpredictable endeavor, we hope that other educators feel empowered to do so based on our experiences.

References

Abrams, Lawrence. 2020. “ProctorU Confirms Data Breach after Database Leaked Online.” Bleeping Computer. August 9, 2020. https://www.bleepingcomputer.com/news/security/proctoru-confirms-data-breach-after-database-leaked-online/.

Aldoriso, Jeff. 2020. “What Is Fourth-Party Vendor Risk Management? What You Need to Know.” SecurityScorecard. June 25, 2020. https://securityscorecard.com/blog/what-you-need-to-know-fourth-party-vendor-risk-management.

Bhat, Prajwal. 2021. “NMIMS Students Allege Harassment by Online Proctor during Exams.” The News Minute, January 10, 2021. https://www.thenewsminute.com/article/nmims-students-allege-harassment-online-proctor-during-exams-141207.

Brown, Lydia. 2020. “How Automated Test Proctoring Software Discriminates Against Disabled Students.” Center for Democracy and Technology. November 16, 2020. https://cdt.org/insights/how-automated-test-proctoring-software-discriminates-against-disabled-students/.

Caines, Autumm. 2021. “The Weaponization of Care.” Real Life Magazine. May 24, 2021. https://reallifemag.com/the-weaponization-of-care/.

Caplan-Bricker, Nora. 2021. “Is Online Test-Monitoring Here to Stay?” The New Yorker, May 27, 2021. https://www.newyorker.com/tech/annals-of-technology/is-online-test-monitoring-here-to-stay.

Chang, Heewon, Faith Ngunjiri, and Kathy-Ann C Hernandez. 2016. Collaborative Autoethnography. Routledge.

Chin, Monica. 2020. “Exam Anxiety: How Remote Test-Proctoring Is Creeping Students Out.” The Verge, April 29, 2020. https://www.theverge.com/2020/4/29/21232777/examity-remote-test-proctoring-online-class-education.

———. 2021. “ExamSoft’s Proctoring Software Has a Face-Detection Problem.” The Verge. January 5, 2021. https://www.theverge.com/2021/1/5/22215727/examsoft-online-exams-testing-facial-recognition-report.

Clark, Mitchell. 2021. “Students of Color Are Getting Flagged to Their Teachers Because Testing Software Can’t See Them.” The Verge, April 8, 2021. https://www.theverge.com/2021/4/8/22374386/proctorio-racial-bias-issues-opencv-facial-detection-schools-tests-remote-learning.

Cohn, Jenae. 2021. “Advice | Who Chooses What Ed Tech to Buy for the College Classroom?” The Chronicle of Higher Education. June 3, 2021. https://www.chronicle.com/article/who-chooses-what-ed-tech-to-buy-for-the-college-classroom.

CRADLEdeakin. 2020. Assessment in a Postdigital World. CRADLE Conference 2020: Symposium. https://www.youtube.com/watch?v=5PzmS-1AFBc&t=2248s.

Crosslin, Matt. 2021. “Proctoring Is Not Essential.” ELearningInside News. April 13, 2021. https://news.elearninginside.com/proctoring-is-not-essential/.

Ellis, Carolyn, Tony E Adams, and Arthur P Bochner. 2011. “Autoethnography: An Overview.” Historical Social Research/Historische Sozialforschung. JSTOR, 273–90.

Flaherty, Colleen. 2021. “U of Illinois Says Goodbye to Proctorio.” Inside Higher Ed, February 1, 2021. https://www.insidehighered.com/news/2021/02/01/u-illinois-says-goodbye-proctorio.

Feathers, Todd. 2020a. “Students Have To Jump Through Absurd Hoops To Use Exam Monitoring Software.” Motherboard – Tech by Vice, November 9, 2020. https://www.vice.com/en/article/88anxg/students-have-to-jump-through-absurd-hoops-to-use-exam-monitoring-software.

———. 2020b. “Colleges Say They Don’t Need Exam Surveillance Tools to Stop Cheating.” Motherboard – Tech by Vice, November 16. https://www.vice.com/en/article/88ag8z/colleges-say-they-dont-need-exam-surveillance-tools-to-stop-cheating.

Fight for the Future. 2021. “University Advocates E-Proctoring Alternatives, but Struggles to Remove e-Proctoring Option from McGraw-Hill Connect Platform.” Fight for the Future. April 1, 2021. https://www.fightforthefuture.org.

Fox Cahn, Albert, Caroline Magee, Eleni Manis, and Naz Akyol. 2020. “Snooping Where We Sleep: The Invasiveness and Bias of Remote Proctoring Services.” Surveillance Technology Oversight Project. https://static1.squarespace.com/static/5c1bfc7eee175995a4ceb638/t/5fd78bac79515d2e1fde4bb7/1607961518518/Snooping+Where+We+Sleep+Final.pdf.

Geiger, Gabriel. 2021. “Students Are Easily Cheating ‘State-of-the-Art’ Test Proctoring Tech.” Motherboard – Tech by Vice, May 5, 2021. https://www.vice.com/en/article/3an98j/students-are-easily-cheating-state-of-the-art-test-proctoring-tech.

Gogia, Laura. 2020. “Online Proctoring: A Call for Action for Academic Technology Leaders.” The Tambellini Group. October 26, 2020. https://www.thetambellinigroup.com/online-proctoring-a-call-for-action-for-academic-technology-leaders/.

———. 2021. “Hiding in Plain Sight: Online Proctoring as ‘Just Another Feature.’” The Tambellini Group. April 7, 2021. https://www.thetambellinigroup.com/hiding-in-plain-sight-online-proctoring-as-just-another-feature/.

Harris, Margot. 2020. “A Student Says Test Proctoring AI Flagged Her as Cheating When She Read a Question out Loud. Others Say the Software Could Have More Dire Consequences.” Insider. October 4, 2020. https://www.insider.com/viral-tiktok-student-fails-exam-after-ai-software-flags-cheating-2020-10.

Harwell, Drew. 2020a. “Mass School Closures in the Wake of the Coronavirus Are Driving a New Wave of Student Surveillance.” Washington Post, April 1, 2020. https://www.washingtonpost.com/technology/2020/04/01/online-proctoring-college-exams-coronavirus/.

———. 2020b. “Cheating-Detection Companies Made Millions during the Pandemic. Now Students Are Fighting Back.” Washington Post, December 12, 2020. https://www.washingtonpost.com/technology/2020/11/12/test-monitoring-student-revolt/.

Herzog, Todd. 2010. “The Banality of Surveillance: Michael Haneke’s Cache and Life after the End of Privacy.” Modern Austrian Literature 43 (2). Austrian Studies Association: 25–41.

Hsieh, Hsiu-Fang, and Sarah E. Shannon. 2005. “Three Approaches to Qualitative Content Analysis.” Qualitative Health Research 15, no. 9: 1277–88. doi:10.1177/1049732305276687.

@hypervisible. 2020. “End of Essay Brings up Important Point. There Are Countless Reasons Instructors Should Reject These Surveillance Systems, but One Is Self-Preservation. It’s Only a Matter of Time before These Systems Are Turned on Instructors—in Some Cases They Already Are. Https://T.Co/IPlGFAMm0V.” Tweet. Twitter. September 11, 2020. https://twitter.com/hypervisible/status/1304394596902993924.

Long, Waverly. 2021. “NU Faces Lawsuit for Storing Students’ Biometric Data.” The Daily Northwestern, February 19, 2021. https://dailynorthwestern.com/2021/02/18/campus/nu-faces-lawsuit-for-improperly-capturing-and-storing-students-biometric-data/.

Malone, Grace. 2019. “Students Required to Pay for Online Exam Proctoring Service.” The Daily Illini, October 28, 2019. https://dailyillini.com/news/2019/10/28/students-required-to-pay-for-online-exam-proctoring-service/.

McFarland, Scott. 2021. “Proctoring Online Exams or Tests Is Essential.” ELearningInside News. March 23, 2021. https://news.elearninginside.com/proctoring-online-exams-or-tests-is-essential/.

McGraw Hill Education. 2020. “Terms of Use: Consumer Purchase Terms.” December 1, 2020. https://www.mheducation.com/terms-use.html.

McKenzie, Lindsay. 2021. “DePaul Sued Over Online Proctoring Tool.” Inside Higher Ed, March 11, 2021. https://www.insidehighered.com/quicktakes/2021/03/11/depaul-sued-over-online-proctoring-tool.

@mheducation. 2020. “Thanks to a New Partnership with @proctorio, Our Connect Platform Now Has Additional Capabilities – like Browser Locking and Remote Proctoring – That Support Academic Integrity and Assessment Security. Read More: https://t.co/mDzCIbKJay.” Tweet. February 4, 2020. https://twitter.com/mheducation/status/1224797280462692352.

Office of Information Technology. 2021. “Proctorio – Accessibility.” University of Colorado – Boulder. Accessed June 5, 2021. https://oit.colorado.edu/services/teaching-learning-applications/proctorio/accessibility.

Ongweso, Edward Jr. 2020. “2,000 Parents Demand Major Academic Publisher Drop Proctorio Surveillance Tech.” Motherboard – Tech by Vice, December 21, 2020. https://www.vice.com/en/article/88am8k/2000-parents-demand-major-academic-publisher-drop-proctorio-surveillance-tech.

Patil, Anushka, and Jonah Engel Bromwich. 2020. “How It Feels When Software Watches You Take Tests.” The New York Times, September 29, 2020. https://www.nytimes.com/2020/09/29/style/testing-schools-proctorio.html.

Proctorio. n.d. “Frequently Asked Questions.” https://proctorio.com/faq.

Pujol, Nicolas. 2010. “Freemium: Attributes of an Emerging Business Model.” Available at SSRN 1718663.

Selinger, Evan, and Chris Gilliard. 2021. “Abolish A.I. Proctoring.” Medium. OneZero. April 7, 2021. https://onezero.medium.com/abolish-a-i-proctoring-c9e017dd764f.

Selwyn, Neil, Chris O’Neill, Gavin Smith, Mark Andrejevic, and Xin Gu. 2021. “A Necessary Evil? The Rise of Online Exam Proctoring in Australian Universities.” Media International Australia, April 1. doi:10.1177/1329878X211005862.

Severance, Charles, Ted Hanss, and Joseph Hardin. 2010. “IMS Learning Tools Interoperability: Enabling a Mash-up Approach to Teaching and Learning Tools.” Technology, Instruction, Cognition and Learning 7 (3–4): 245–62.

Silverman, Sarah, Autumm Caines, Christopher Casey, Belen Garcia de Hurtado, Jessica Riviere, Alfonso Sintjago, and Carla Vecchiola. 2021. “What Happens When You Close the Door on Remote Proctoring? Moving Toward Authentic Assessments with a People-Centered Approach.” To Improve the Academy: A Journal of Educational Development 39, no. 3.

Stewart, Bonnie. 2020. “Online Exam Monitoring Can Invade Privacy and Erode Trust at Universities.” The Conversation, December 3, 2020. http://theconversation.com/online-exam-monitoring-can-invade-privacy-and-erode-trust-at-universities-149335.

Swauger, Shea. 2020. “Our Bodies Encoded: Algorithmic Test Proctoring in Higher Education.” Hybrid Pedagogy, April 2, 2020. https://hybridpedagogy.org/our-bodies-encoded-algorithmic-test-proctoring-in-higher-education/.

Walker, Rachel K., Jessica Dillard-Wright, Em Rabelais, and Anna M. Valdez. 2020. “Surveillant #EdTech Harms Nursing Students, the Profession, and the Public.” Medium. December 21, 2020. https://rachelkwalker.medium.com/surveillant-edtech-harms-nursing-students-the-profession-and-the-public-6b225c57a7b3.

Wan, Tony. 2019. “As Online Learning Grows, So Will Proctors. Case in Point: Examity’s $90M Deal.” EdSurge. April 30, 2019. https://www.edsurge.com/news/2019-04-30-as-online-learning-grows-so-will-remote-proctors-case-in-point-examity-s-90m-deal.

Woldeab, Daniel, and Thomas Brothen. 2019. “View of 21st Century Assessment: Online Proctoring, Test Anxiety, and Student Performance | International Journal of E-Learning & Distance Education / Revue Internationale Du e-Learning et La Formation à Distance.” International Journal of E-Learning & Distance Education 34, no. 1. http://www.ijede.ca/index.php/jde/article/view/1106/1727.

Yun, Tom. 2020. “Math Students at Wilfrid Laurier Furious after Department Orders Them to Buy External Webcams for Exams.” Toronto Star. May 8, 2020. https://www.thestar.com/news/gta/2020/05/08/math-students-at-wilfred-laurier-furious-after-department-orders-them-to-buy-external-webcams-for-exams.html.

Appendix: Documents Analyzed from Proctorio’s Site and Partner Sites

Company Name Page Name URL
Proctorio Integration https://proctorio.com/about/integration

Archived https://web.archive.org/web/20210617040222/https://proctorio.com/about/integration

Proctorio Frequently Asked Questions https://proctorio.com/faq

Archived https://web.archive.org/web/20210617040359/https://proctorio.com/faq

Proctorio Acceptable Use Policy https://proctorio.com/policies#all&all&aup&section-acceptableusepolicy
TopHat TopHat Partners with Proctorio https://tophat.com/press-releases/top-hat-partners-with-proctorio/

Archived https://web.archive.org/web/20210310041915/https://tophat.com/press-releases/top-hat-partners-with-proctorio/

TopHat Professor: Remotely Monitored (Proctorio) Tests https://success.tophat.com/s/article/Teaching-Online-Remotely-Proctored-Tests

Does not archive properly

TopHat Choose the Right Plan for Your Course https://tophat.com/pricing/

https://web.archive.org/web/20210414132914/https://tophat.com/pricing/

McGraw Hill Connect Online Assessment Integrity https://www.mheducation.com/highered/connect/proctorio.html

Archived https://web.archive.org/web/20210616055843/https://www.mheducation.com/highered/connect/proctorio.html

Cirrus Assessments Integration with Remote Proctoring from Proctorio https://cirrus.zendesk.com/hc/en-us/articles/360012584799-Integration-with-remote-proctoring-from-Proctorio

Archived https://web.archive.org/web/20210617040838/https://cirrus.zendesk.com/hc/en-us/articles/360012584799-Integration-with-remote-proctoring-from-Proctorio

Cirrus Assessments Automated Proctoring by Proctorio https://cirrus.zendesk.com/hc/en-us/articles/360012590719-Automated-proctoring-by-Proctorio

Archived https://web.archive.org/web/20210617041251/https://cirrus.zendesk.com/hc/en-us/articles/360012590719-Automated-proctoring-by-Proctorio

Ans* Set up the Proctorio Integration https://support.ans.app/hc/en-us/articles/360011850058–Set-up-the-Proctorio-integration

Archived https://web.archive.org/web/20210301032302/https://support.ans.app/hc/en-us/articles/360011850058-Set-up-the-Proctorio-integration

Ascend Learning The ATI TEAS Exam with Proctorio https://www.atitesting.com/teas/the-ati-teas-exam-with-proctorio

Archived https://web.archive.org/web/20210413071454/https://atitesting.com/teas/the-ati-teas-exam-with-proctorio/

Ascend Learning About https://www.ascendlearning.com/about/

Archived
https://web.archive.org/web/20200923134618/https://www.ascendlearning.com/about/

Derivita Math with Integrity https://www.derivita.com/proctoring

Unable to be archived

Derivita Home https://www.derivita.com/

Trouble archiving https://web.archive.org/web/20201031103007/https://www.derivita.com/

EvaExam Online Exams https://evasys.de/en/online-exams/

Trouble archiving https://web.archive.org/web/20210617041927/https://evasys.de/en/online-exams/

QuestionMark Frequently Asked Questions https://www.questionmark.com/platform-services/faqs/
QuestionMark How do I take a “record and review” proctored assignment? https://support.questionmark.com/content/taking-a-record-and-review-proctoring-assessment
QuestionMark Proctoring Hub https://www.questionmark.com/platform-services/proctoring/

About the Authors

Sarah Silverman is an instructional designer at the Hub for Teaching and Learning Resources at the University of Michigan–Dearborn. In addition to educational technology criticism, her interests include Universal Design for Learning and Disability Studies. A scientist by training, she received her PhD in Entomology from UC Davis and worked in teaching and learning support at UC Davis and UW Madison before coming to UM Dearborn. She currently resides in New Haven, CT.

Autumm Caines is an instructional designer at the University of Michigan–Dearborn. Autumm’s scholarly and research interests include blended/hybrid and online learning, open education, digital literacy/citizenship with a focus on equity and access, and online community development. This blend of interests has led to a concern about mounting ethical issues in educational technology and recent publications and presentations on topics concerning educational surveillance, student data collection, and remote proctoring. Autumm has taught honors students at small liberal arts colleges as well as traditional students, working professionals, and veterans at a regional public university. More at autumm.org.

css.php
Need help with the Commons? Visit our
help page
Send us a message
Skip to toolbar