For almost a year, Hong Kong’s education sector has gone through unprecedented challenges. The social unrest in 2019 followed by waves of pandemic strikes in 2020 have resulted in periods of class suspension, restricted travels, compulsory quarantines, and social distancing measures. Under these circumstances, schools and higher education institutions moved from face-to-face to fully online delivery. More recently, a dual-mode teaching and learning strategy has been introduced to allow students who wish to attend classes on campus to be able to do so and students who are overseas or choose to stay home to be able to receive the same kind of education. With these changes in place, teachers have worked actively to provide a learning experience as fruitful as it was used to be. Video-conferencing technology has been used extensively to conduct synchronous lessons, yet more is to be done to provide an overall positive learning experience.

One of the concerns of learning remotely is the inability to provide students with the experience of the physical environment through digital means. There have been increased discussions and exploration into the use of innovative technology for teaching and learning. While immersive technology, including virtual reality (VR) and augmented reality (AR), has been around for more than half a century, its main applications lie in the areas of videography, entertainment, and simulated training (Biocca, Kim, & Levy, 1995; Javornik, 2016; Poetker, 2019). In the last decade or so, we might have seen sporadic experimental cases of VR or AR use in formal education settings, yet the scalability and sustainability of such applications have not always been proven to justify the costs and expertise involved.

We are very fortunate, with the support of Professor Ricky Kwok (Department of Electrical and Electronic Engineering) and Technology-Enriched Learning Initiative (TeLI), to have the opportunity to develop and experiment using immersive technology in our Common Core (CC) course, The Science of Crime Investigation. The course is designed to use criminal cases as a platform to teach students the scientific knowledge and inquiry, critical thinking, and interdisciplinary collaboration. Same as the other CC courses, we have large classes with students from different disciplines and levels of study. The diverse learning needs bring extra challenges to the teaching and learning of science and scientific inquiry. Over the course of four years, we took multiple steps to revamp the decade-long, face-to-face course into a flipped design that includes online learning videos (Figure 1), asynchronous online learning activities, and synchronous face-to-face classroom activities. This gives us more quality time with the students in the classroom, allowing learning to become interactive and collaborative.

Figure 1. Sample of online learning video

One of the major components of the course is a group project on a real-life criminal case. In the case, students take on the role of the investigation team and investigate the scene, interview witnesses, and collect and examine evidence, to develop theories to explain the mishaps of the victim(s). Because of legal and practical limitations, the case study is retrospective yet we preserved the explorative nature of a real-life crime investigation in the exercise. Upon receiving the information provided through the emergency hotline, the team will ask questions, devise hypotheses and a plan of actions to approach the case, using knowledge of criminal investigation procedures and the science behind the processing of forensic evidence students learned in the course. In response to their investigation plan, we will provide text-based documents in the form of witness statements or scientific reports from the original forensic file. During the investigation, teams are asked to submit a few interim reports to us. Such reports allow us to keep track of the learning progress and inform us about the forensic documents required. We would also provide point-by-point feedback in the interim reports to help students sharpen their inquiry skills. To give students an immersive learning experience with timely feedback, we gamified the coursework by building a mobile app that involves chatbots, supported by artificial intelligence (AI), to imitate witnesses and experts at the scene (Figure 2), and an interactive, three-dimensional crime scene to allow students to explore and collect evidence (Figure 3). While the components in the app could be completed individually, we designed a physical game board, enhanced by AR, to get students to come together and collaborate in the project (Figure 4). The group progress was made visible, with the aid of learning analytics, in the forms of log books and progress bars in the user interface to allow students to keep track of their own learning. Preliminary findings suggest that on average, students asked 30 times more questions talking to chatbots than participating in class discussions. Using the course grading criteria, we found a good 60% of these questions were “meaningful questions” that demonstrate deep understanding. Watch this video to hear what students said about their experience.

Figure 2. Sample of AI-chatbot

Figure 3. Interactive 3D scene

Figure 4. Students working together to activate the game app through the AR-enhanced game board

Learning from the first round of classroom implementation and students’ feedback, we made some changes to the game app. These included a QR code complementary to the game board to allow students to access the scene at home. In order to maintain the collaborative nature of the project, a colour-coded progress bar that shows individual members’ contributions to the group work was also added (Figure 5). While not intended for fully online delivery, these developments came in handy when all teaching and learning had to be done remotely during the pandemic! Immersive technology has been an added value in our course and could be extended to other learning contexts where simulations of the physical environment are involved. The downside: we need the expertise to help us plan and put things together. Fortunately, there are experts on campus (e.g., the TeLI team) and easy-to-use apps on the market that will make the process possible for beginners.

Figure 5. Colour-coded progress bars to show individuals’ contribution to the group work

While the search of the new normal is ongoing, we can only make our best efforts to speculate and prepare for unforeseeable changes that lie ahead. There are a few options that we have tried and could be the solutions for the interim period when situations can be uncertain and unexpected. We will continue to disseminate course content online supported by synchronous sessions to facilitate interaction and learning habits in the fixed timetable. Students are proficient in using shared drives and shared documents in collaboration. As on-campus sessions are now allowed, a number of hands-on sessions can be resumed, with appropriate hygiene and social distancing measures. Moving away from a confined classroom space can be an option, and believe it or not, we had done it previously when we were not able to secure a classroom for a small lab activity (Figure 6). Before we see some stability again, let’s all stay vigilant, be flexible, creative, and open to experimentation and failure, and continue to love the work we are doing.

Figure 6. Lab setup in an open-space corridor in the Main Building, HKU

Acknowledgement

The game project has not been possible without the support of:

• Dr Philip Beh, Department of Pathology, the founder and course coordinator of CCST9010
• Prof Ricky Kwok, Department of Electrical and Electronic Engineering
• The amazing and talented team of Technology-Enriched Learning Initiative

References

• Biocca, F., Kim, T., & Levy. M.R. (1995). The vision of virtual reality. In F. Biocca & M.R. Levy (Eds.), Communication in the Age of Virtual Reality (pp. 11-19). Hillsdale, NJ: Lawrence Erlbaum Associates.
• Javornik, A. (2016, October 4). The mainstreaming of augmented reality: A brief history. Harvard Business Review. Retrieved from https://hbr.org/2016/10/the-mainstreaming-of-augmented-reality-a-brief-history
• Poetker, B. (2019). The Very Real History of Virtual Reality (+ A Look Ahead). Retrieved from https://learn.g2.com/history-of-virtual-reality

Wincy Chan

Department of Pathology
The University of Hong Kong