Introduction
Rich learning opportunities exist when academic departments reach beyond their discipline and engage with each other. Interdisciplinary approaches are key to success in independent business entities and they allow a team to “engage with their ideas, maintain productive interaction, and successfully implement these ideas” (Brodack and Sinell 2017, 10). Interdisciplinarity broadens the knowledge base of a project team, taking full advantage of the specialized knowledge of its members while avoiding the peripheral blindness often associated with such specialization. When managed correctly, creative problem-solving outcomes are enhanced (Moirano, Sánchez, and Štěpánek 2020), silos are merged, and focus limitations associated with specialization are removed (Blackwell et al. 2009). Working on a project within an academic environment is no different. Students have specialized knowledge and often a preconceived perception of a problem. Collaboration with those outside their field broadens their interpretation of and approach to a problem while allowing them to use their special skills with external input. Developing a mobile app requires the technical skills of computer scientists, the knowledge of subject matter experts, and the expertise of user experience researchers. Here, we present a case study of a successful app development process as a blueprint for others to follow with a discussion of tools, activities, time budgets, resources, challenges, and potential impact.
During the COVID-19 pandemic, two psychology faculty members (a clinical psychologist and an engineering psychologist) and one computer science faculty member collaborated along with the corporate liaison at the university to address the increased need for coping skills in the local and student community (Fernández et al. 2020; Naeem et al. 2020; Saltzman et al. 2020). We received a grant and hired two undergraduate students and one graduate student to work with us. This article documents our wins and lessons learned in order to help other academics bootstrap the process.
Research is clear on the significant negative effects resulting from the COVID-19 Pandemic. In addition to significant physical health risks, there are substantial increases in the rate that individuals are experiencing mental health symptomatology such as distress, anxiety, sadness, and isolation (Kar et al. 2020; Pierce et al. 2020). Unfortunately, there has been less focus placed on increasing our ability to address mental health concerns at a systems level even in the face of rising pathology (Kar et al. 2020).
As a result of the pandemic, there is a shift within the mental health field towards providing more services remotely (e.g., meeting with a therapist by webcam). However, that shift does not address the increasing demand for mental health services. Thus, there is a significant need to develop other digital avenues to try and reach individuals in need (Ho 2020). According to Ho (2020), apps developed for smartphones to provide users with psychoeducation, resources, and coping strategies may prove especially useful to help meet increased mental health needs during the pandemic. Previous research demonstrates the viability of using smartphones to integrate mental health services through technology. Digital apps have been created to monitor, record, and, in some cases, modify mental health, such as providing location-based services to alert users to the nearest mental health clinic, providing self-help mantras and guided meditations, and tracking mood ratings based on self-reporting (Luxton et al. 2011).
According to the Centers for Disease Control and Prevention (2021), engaging in appropriate coping strategies during the pandemic is important to maintaining one’s mental wellbeing. We developed the Serene app to help our students and the surrounding community cope with the pandemic while gathering information on the mental health of the community. Specifically, the app was developed to accomplish this goal through non-medical advice that engages users with behavioral activities, psychoeducation, motivational quotes, video exercises for relaxation and breathing, local and national professional resources, and connections to available, external, evidence-based mental health apps.
We set out to determine the best practices in developing a professional level organization for app creation. We anticipated the project would take four months, but it took eight months with an additional four weeks for the Google Play Store release and an additional eight weeks for media coverage.
Organization
Our group was separated into three pairs, each consisting of a faculty member and a student. Faculty members chose students in their discipline based on previous coursework, previous independent study, and their experience of the students in their courses. The faculty/student pairs are referred to as teams. The three teams were as follows:
- UX (user experience) team (an engineering psychology professor and an undergraduate human factors psychology student who successfully completed the assignment in Appendix A),
- Content team (a clinical psychology professor and a counseling graduate student), and
- App development team (a computer science professor and an undergraduate computer science student).
The corporate liaison provided advice and guided compliance to the institutional mission and the funding agency’s mission.
User experience (UX) team
The UX team organized first to create the design for the minimally viable product (MVP) prototype. Eric Ries (2013) discusses the specifics of MVPs and how they can save development time. The UX student used her expertise in human factors and referred to the research-based best practices on Don Norman and Jakob Nielsen’s NN group website (Nielsen Norman Group 2020). Don Norman is a faculty member at the University of California San Diego and one of the forefathers of UX. Jakob Nielsen is an engineer and a principal at the NN group.
The UX and content students conducted a competitive analysis to discover what similar apps existed and what these apps provided to users as Jill DaSilva (2020) discusses. They created a spreadsheet of similar apps and their features. This was the basis of requirements gathering as Janet Six (2019) suggests. The team reviewed the spreadsheet and developed the requirements document using a version of the MOSCOW method (must, should, could, won’t) as discussed in ProductPlan (2020) and then refined this list. Some desired but untenable features were “connecting to a counselor on campus through a chat feature” and “talking to others using the app.” Both of these features would require infrastructure that was unavailable. Then, the UX student organized the architecture of the app discussed by Jen Cardello (2014) and used LucidChart to create the architecture as shown in Figure 1.
After approval, the UX student generated pencil sketches of the screens and then developed the individual screens using the open-source material design pattern library (http://material.io/). Next, the student used the Invision App (https://www.invisionapp.com/) with a free educational license (https://www.invisionapp.com/education) to work out the navigation between the screens. At each stage of this process, her work was approved by the group. The prototype took two weeks longer than anticipated. The Serene app design is stored online at Invision (https://projects.invisionapp.com/share/8DXSLUJCRAK#/screens).
App development team
The development team participated in the discussions of the overall design, the design of the architecture, and the design of the UX. Following creation of the UX design by the UX team, the app development student created a prototype of Serene that followed the UX design and turned the prototype into a fully functioning app product using his expertise in computer science, following weekly discussions with the entire team. The development process consisted of the development of the back end, a Java server that handled the processing and storage of data, and the front-end, the app itself, built using HTML5 and JavaScript with Cordova (https://cordova.apache.org/), providing multi-platform support.
Content team
The content team helped the UX team to research similar apps. A spreadsheet was created to compare similar apps and their functionality. Following discussion with the whole team, the content student developed comprehensive resource lists to provide users with information regarding:
- Mental health providers in Erie County, Pennsylvania. The list consisted of local agencies and organizations, their contact information, and the target population.
- Nationwide mental health resources. The list consisted of national mental health hotlines and organizations for various populations.
- Other mental health smart-phone applications. In collaboration with the UX team, a list was created with all the mental health applications that the team was able to find. The content team assessed the applications and chose a small number of evidence-based applications to suggest in the Serene app as additional applications.
- Behavioral activities that users could consider doing. Based on psychological principles of behavioral activation (Kanter et al. 2010) to help increase well-being by remaining physically active, the content student used her expertise in counseling psychology to compile a list of various activities users could do across a variety of settings and circumstances (see Appendix B). Given some of the restrictions experienced due to the COVID-19 Pandemic, these resources provide users with ideas for activities they can engage in regardless of pandemic-related circumstances (e.g., socially distant outdoor activities or things to do at home if faced with a stay-at-home order).
- Motivational quotes. Upon discussion with the whole team, it was decided that three categories of quotes (i.e., psychology quotes, I am… quotes, and motivational quotes) were needed. The content team sought to include at least 365 quotes in each of the three categories thereby ensuring a steady stream of new content (i.e., one new quote from each category for every day of the year) to help promote regular use of the app and gather information on how users were feeling. The final list consisted of approximately 380 quotes for each category.
The content student also used her expertise in counseling to research and write articles that provided users with evidence-based information regarding mental health and COVID-19, all accessible from within the Serene app. The mental health information discussed emotional reactions and stigmatization in mental health. The COVID-19 article was a comprehensive summary of the characteristics of the coronavirus, along with ways individuals can protect themselves. All sources used were either governmental (e.g., CDC and WHO) or other high-quality online resources (see Appendix C; e.g., information from the Bill and Melinda Gates Foundation). These articles and resources assist users in finding valid and reliable information about mental health and the COVID-19 pandemic.
Providing users with this type of information, also referred to as psychoeducation, is a very important component of multiple therapeutic models in mental health services. That is to say, we need to provide information related to the individual mental health concerns of mental health consumers in order to raise awareness and offer a sense of reality and control. Since the Serene app was created as a tool to assist its users with mental health struggles in isolation during the COVID-19 pandemic, our articles are meant to provide users with information about basic mental health concepts like stigma and emotions, as well as information about the coronavirus. Further, many unreputable online resources spread misinformation and inaccuracies that may confuse or even disturb individuals. Therefore, providing reliable resources and psychoeducation to users of the Serene app may also help to decrease potential distress that individuals may face if they were to search for and receive this same information from other, potentially unreliable or misinformative sources.
Finally, the content student’s expertise in counseling helped her to create mindfulness, meditation, and progressive muscle relaxation exercises that users could freely access within the Serene app. For this purpose, appropriate audio recording equipment (Studio Condenser USB Microphone with Adjustable Scissor Arm Stand) was purchased. Moreover, the content student searched online resources to find and adapt scripts and soundtracks for use within the app. The content student recorded the audio for each of the exercises, mixed the audio with background music and visuals, and uploaded each exercise to the team’s YouTube channel, for use within the app. The majority of the content student’s hours were spent in the creation of these exercises because significant time was needed for the student to locate appropriate scripts and soundtracks and to get familiar with the recording equipment and software. By the end, a five-minute recording would take approximately two hours to complete from start to finish.
Student Work and Time
As this was the first time that we had developed an application together, we had many questions about how much time the students should spend and what they should be doing during those hours. Based on our experiences, 50 percent of person-hours were devoted to the app development student, followed by 36 percent to the content student, and 14 percent to the UX student. To help future academic development teams determine a budget, we have included the actual student time/activities as concrete guidance.
UX team
Students were screened in an “Introduction to Human Factors” class on a prototype development assignment (see Appendix A). One of the challenges we had was budgeting for student hours. While these times may not work for every project, here is the time breakdown for the UX student.
| UX student time | Activity |
|---|---|
| 8 hours | Competitive app research |
| 2 hours | Helping with content |
| 2.5 hours | Information architecture |
| 16.25 hours | Meetings—requirements development, review sessions, organizing the project |
| 30 hours | Prototyping |
| 8 hours | User testing and reporting |
| 2 hours | Miscellaneous |
App development team
Students were screened in a computer science class where coding assignments were a major component. Students’ assignments were reviewed based on their performance, which included the correctness, efficiency, and organization of their written programming code. One student was invited to join the team based on his performance and his availability in the schedule of the development.
| App development student time | Activity |
|---|---|
| 12 hours | Back-end development, including the storage of data on a server and server setup* |
| 205 hours | Front-end app development, including the app interface and the connection to the back end |
| 6 hours | Miscellaneous |
| 18 hours | Meetings |
Content team
Given the mental health nature of the content to be created for this application, it was important to find a student with expertise in both the research and practice of clinical psychology. Therefore, the student for the content team was hand selected from a clinical psychology graduate program on campus. Prior to working on this project, the student worked as a research assistant for the content team lead. Through this work the student demonstrated several key qualifications for this position, including: a passion for mental health advocacy, a mastery of the material, and the ability to work efficiently and effectively both as part of the team and on an individual level.
| Content student time | Activity |
|---|---|
| 16 hours | Creating a database of county-wide mental health resources (e.g., providers) as well as select nationwide resources (e.g., the National Suicide Prevention Lifeline) |
| 3.5 hours | Researching other mental health smartphone applications to list within this app to provide users with additional wellness resources |
| 9 hours | Creating a list of behavioral activities users could access to help find things to do across a range of current circumstances (e.g., things to do at home, if faced with a stay-at-home order due to the pandemic; socially distant outdoor activities) |
| 34 hours | Collating several lists of positive and inspirational quotes |
| 91.5 hours | Producing video content for the app (e.g., mindfulness exercise videos) |
| 16 hours | Attending team meetings |
| 3 hours | Miscellaneous |
Project Development and Implementation
Initial development began in late April 2020 with team organization for the summer and an application for grant funding. Project planning was done using a mix of free templates. For planning purposes, the UX work and content work happened in the first three months. App programming began concurrently in the second month once the initial prototype screens had been determined. User testing began in the third month along with iterations to solve the issues that were discovered. Figure 2 outlines the order in which the development occurred as well as the stages of ideation, solidification, and implementation within each phase. Review was ongoing in each phase. Development finished in the sixth month with submission to Google Play Store in the seventh month. The app was approved and deployed in the eighth month.
Resulting Impacts
In order to keep the team focused and establish a collaboration, a project plan was our first action. Within the project plan was the rationale, the project scope, team composition, team responsibilities, team deliverables, milestone activities, communication management plan, the contact information for each team member, the budget, how the meetings would be conducted, and a quality baseline commitment. All members reviewed and revised the project plan until it was agreeable. The document was critical to the interdisciplinary focus and prevented role drift where one person tries to take over all the roles in the development process.
The Serene app included three opportunities for learning. The first opportunity was to learn more about how a multidisciplinary team could be structured to deliver specific applied skills in an academic setting. The second was the learning environment of developing the app itself. The third occurs in deploying the app to the larger community and learning more about the community’s mental health.
The first learning opportunity gave us a greater appreciation of how each discipline perceived the work and structured priorities. For example, the content group had a great interest in gathering mental health data. At first, the UX and app development teams failed to realize how important biological sex is to mental health data collection and analysis. The content team explained a long-standing issue in the mental health field pertaining to a need to research and better understand biological sex differences in relation to psychopathology as Cynthia Hartung and Thomas Widiger (1998) and Cynthia Hartung and Elizabeth Lefler (2019) discuss. As a result, it is important to ensure that data such as biological sex is collected and analyzed in mental health research to help elucidate whether any potential findings vary by sex. Even through this simple occurrence, the students learned the value of the various perspectives provided by an interdisciplinary team. The team included five options to report sex: prefer not to say, male, female, intersex, and other.
The second opportunity happened both during and after development as we learned to coordinate our expertise. The members of UX, app development, and content teams used their expertise to move the project forward. Faculty mentors coached students on teamwork skills and developed the students’ expertise in separate meetings. The weekly team meetings were opportunities for joint design decisions, review of the work, and progress maintenance by following the four we’s: (a) this is what we were thinking in our role, (b) this is what we did to move the project forward, (c) this is what we think should be done next, (d) what do we think? The forming, storming, norming, and performing stages are well documented (Tuckman 1965), however, in this project a deeper sense of inquiry was necessary to convey respect for each role’s effort. This respect freed individuals from preliminary criticism that would hamper their motivation yet allowed the team to critique the project at critical milestones. For example, the UX team struggled to devise the weekly, monthly, and yearly graphs. During weekly reviews, the team settled on an earlier solution. The UX team enjoyed the freedom to exercise their expertise and intellectually explore the options before the final design decision was made by the team.
The third learning opportunity is ongoing and comes from anonymous user data being collected to identify and address potential mental health inadequacies prevalent in the regional community. Typically, mental health needs outpace the resources available. By creating this app, we not only provided valuable resources to the community during a time of immense need, but we also gained valuable insight into the ongoing needs of our community. For instance, these data allow us to analyze the anonymous, self-reported, mental health data, across time throughout and after the pandemic. It also allows us to examine and better understand what types of local resource content are most applicable to our community members. Ultimately, these data will allow the team to better understand the specific needs of our area, as reported by the community, and can serve to tailor engagement towards addressing specific community needs.
While we often put students in teams in class, they rarely participate in teams across disciplines. This project was a beneficial example of how to construct an interdisciplinary team. Each student responded positively in their comments (see Appendix D): “The biggest challenge was taking what suggestions the development team had and giving them life”; “Nonetheless, working with a large group of experienced professors and students allowed for a painless development process”; and “The main sense that remains with me after the completion of Serene is that of working and communicating with people from various fields who all used their own language, interests, and expertise for the same project.”
Challenges
We encountered several challenges through our development process. They range from technical—data warehousing, quality assurance, design, to legal—terms of service, to managerial—content. The following sections describe each of them.
Data warehousing
One of the first challenges was where to host the programming code as the project developed. Two back-end server hosting solutions were considered. The first was a cloud-based solution, Amazon Web Services (AWS). However, this solution was abandoned due to its ongoing costs associated with storage (AWS Simple Storage Service), computing (AWS Elastic Compute Cloud), and communication (AWS Data Transfer). We chose the second solution, hosting with an internal institutional server using Windows Server. This solution required:
- Help from IT support from the university in setting up server
- University computing and storage resources
- Compliance with university, including accessibility
- Access to the Google Play Store from a university-owned account
Content
A considerable amount of time was spent in finding mindfulness scripts, soundtracks, and images with no copyrights. Also, the composition of the audio files was quite challenging, and particularly the pairing of the soundtrack with the narrative. The first few recordings took many hours to complete. Finally, the list with the quotes was unexpectedly time consuming, as the content team had to proof-read the quotes and ascertain the authors for all the quotes that appeared “unknown” during the search.
Many of the resources were kept in a spreadsheet file. The team decided to use html tags so the app could easily access the spreadsheet resources and use those resources, as is, within the app. The content team easily learned the tags and adapted.
Institutional Review Board
We found that we needed two reviews for human subjects research. One was for user testing during development. The other was for using the data that the app gathered. As this second review of data had information that was not identifiable to a person, it was determined that this was not human subjects’ data.
Attorney services
There were many questions about how to best navigate terms of service and data use. We worked closely with the legal department at our institution through the corporate liaison officer to implement a terms of service appropriate to the general nature of the app and for the information on the data and how it would be used. Since the Serene app has a mental health focus, it was essential to ensure that it was not used as a substitute for medical care and that the development team and the university could not be held liable for any such misuse. Therefore, the corporate liaison consulted with the university’s offices of risk management and general counsel. These offices helped to craft simple, understandable terms of service and data usage language. As the app was produced by the students, we chose to provide that information on the accompanying website rather in the app itself.
Design of charts
Other challenges included problems related to a specific content area. In UX/UI- there was quite a bit of work on how the charts would look. Initially, we considered a complex line and bar chart such as in Figure 3.
However, this chart was too complex for the small real estate on a mobile phone screen and did not capture the weekly, monthly, and yearly changes. Then, we tried three different charts as shown in Figures 4 to 6.
User testing
In order to discover how well the design was understood, we conducted user testing with five undergraduate students from the psychology course testing pool which was approved by the IRB. The UX student used standard user testing methodologies as recommended on NN group’s user testing videos (Nielsen Norman Group 2020). In the user testing, participants were asked to do three tasks: Find a mindfulness video, find an activity to do on the phone/computer, and find a local resource for depression. Then, the researcher took note of any problems the participant had and how long it took each participant to do each task along with satisfaction ratings as described by Erik Frøkjær, Morten Hertzum, and Kasper Hornbæk (2000). Participants in user testing found that some of the labels were unclear and there was confusion about where to find specific tasks within the app. There were also questions about the use of a password and if users would be able to use the app on their Android phone and on the web. Users said that they would like a password, but this would invalidate some of the anonymity of the data and could cause some late-stage development changes. Thus, we decided to leave the password issue to the future releases and comments were gathered.
Quality assurance team
Once the code is working, there needs to be a dedicated team that tests the app, looks for the weaknesses and finds out if there are bugs that might break the app. We did not have such a team and instead functioned as our own quality-assurance team along with friends who volunteered their time. This method took longer and a professional-level assurance team should be included in the budget.
Conclusions and Future Directions
Despite the challenges encountered during the development of the app, the combination of talents into one interdisciplinary team allowed the creation of a completed product that far exceeded what one discipline could accomplish alone. One purpose of the project was to give the students the experience of working in a structured and distributed environment with a team that was segregated by roles but followed an agile software development approach, where development happened in an iterative way. Professors spent additional hours mentoring the students on teamwork and communication as well as learning about the other roles. Working together while maintaining a constant line of communication was key to its success. Having regular stages of reviews during the development cycle helped guide the process in the right direction. Using the right collaboration tools, such as Microsoft Teams, Google Docs, and GitHub, made team work much easier. The Serene app is available through Google Play and can be accessed on the Serene website.
Knowing that the project would result in an app launched to a large community of potential users provided ample motivation to meet the learning and performance needs for successful completion of the project. Learning requirements were high and extended well beyond knowledge gained in coursework. Also, it did not go unnoticed by the students that the audience for this project was external to the university, and it motivated them to take extra care and expend extra effort in their work. Working within an interdisciplinary team that extends beyond students to include university faculty and staff helped the students to broaden their perspective of app development work to include the work of the other teams, focus their thinking on alignment of the project with the mission of the university, understand potential legal responsibilities, and value meeting the expectations of external stakeholders. For example, the funding agency has a strong local focus, so the students had to be sure to target a sufficient portion of the app’s functionality toward a regional audience.
Our university is highly focused on being an Open Lab (Birx, Ford and Payne 2013), an interdisciplinary living laboratory where learning and discovery are applied to solve problems defined in partnership with external stakeholders. The Open Lab concept evolved from the idea of research clusters working on pressing local problems. The Open Lab is a win for students, faculty, and the external organizations: students gain career-building, real-world experience; faculty enjoy the ability to keep their skills relevant and transfer their networks to students; and external partners benefit from the energy and ingenuity of student talent. More information can be found at the Open Lab website through the university homepage (https://behrend.psu.edu). This project aligns with that focus, as it creates an outward-facing product that receives and engages with feedback from the external community, adding motivation and accountability to the students’ work. It is notable that all students acknowledged the value that this experience provided to them and their career development. Students involved in this team left individual comments regarding their experience (see Appendix D).
We plan on continuing an extension of this project. For future work, psychology content can be updated and expanded with additional mental health or COVID-19 focused information, resources, or the creation and addition of new wellness exercises that users can freely access from within the Serene app. We have deployed the app and are monitoring user feedback. Additional user testing will be conducted based on user feedback. We also plan to deploy the app to iOS. When recruiting undergraduate students, it is a good idea to have overlapping years. There should be some second-year students, some third-year students, and some senior students. This approach ensures smooth transitions for projects having a multi-year life expectancy.
'Interdisciplinary Approach to a Coping Skills App: A Case Study' has 2 comments
May 11, 2021 @ 10:18 am Introduction /
[…] Recognizing the trauma students were likely to face during the pandemic, Antigoni Kotsiou, Erica Juriasingani, Marc Maromonte, Jacob Marsh, Christopher R. Shelton, Richard Zhao, and Lisa Jo Elliott describe a collaborative and interdisciplinary process of creating a mobile app for students to track their emotions and develop coping strategies in “Interdisciplinary Approach to a Coping Skills App: A Case Study.” […]
May 11, 2021 @ 10:17 am Table of Contents /
[…] Interdisciplinary Approach to a Coping Skills App: A Case Study Antigoni Kotsiou, Erica Juriasingani, Marc Maromonte, Jacob Marsh, Christopher R. Shelton, Richard Zhao, and Lisa Jo Elliott […]