Use of instructional videos in leadership education in higher education under COVID-19: A qualitative study

RESEARCH ARTICLE Use of instructional videos in leadership education in higher education under COVID- 19: A qualitative study Daniel T. L. ShekID*, Tingyin WongID, Xiang Li, Lu Yu Department of Applied Social Sciences, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong, China * daniel.shek@polyu.edu.hk Abstract The use of online teaching mode has grown rapidly in recent years, particularly under the COVID-19 pandemic. To promote the learning motivation of students and teaching effec- tiveness, development of attractive online teaching material such as videos is important. In the present study, we developed 15 theory-related videos and 9 case-based videos in the context of a leadership course focusing on psychological well-being and psychosocial com- petence. Using a qualitative research methodology via focus groups (N = 48 students) to evaluate these videos, six themes emerged from the data, including video arrangement, design of videos, content of videos, benefits to students’ pre-lesson self-learning, benefits to students’ learning of course content, and contribution to students’ class participation. The findings suggest that the videos can elicit positive perceptions of the students in a flipped classroom arrangement. Students also benefit from the videos in terms of their understand- ing of course content and their participation in class discussion. Besides, the study suggests that the videos promote the learning efficiency of the students. The present qualitative find-

ings concurred with the previous quantitative findings, suggesting the value of using virtual teaching and learning to promote psychosocial competence in university students. 1. Introduction The use of online teaching mode in higher education institutions has grown rapidly in recent years. Before the COVID-19 pandemic, more than 5.5 million students in the U.S. were enrolled in one or more postsecondary-level courses through online channels [1]. Researchers highlighted that studies on video-based learning has increased significantly after 2006 because of the increased use of videos in higher education sector, both as a supplementary materials for face-to-face lectures or as the main way to deliver online education content [2, 3]. During the early phrase of the pandemic, videos played a key role in sustaining teaching and learning in higher education [4]. For example, in Beijing Normal University, 3238 out of the 4036 courses in February 2020 were offered online [5]. With the increased use of digital device and various online pedagogies (e.g., videos and virtual reality), students may benefit from flexible online PLOS ONE PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 1 / 28 a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 OPEN ACCESS Citation: Shek DTL, Wong T, Li X, Yu L (2023) Use of instructional videos in leadership education in higher education under COVID-19: A qualitative study. PLoS ONE 18(9): e0291861. https://doi.org/ 10.1371/journal.pone.0291861 Editor: Ehsan Namaziandost, Ahvaz Jundishapur

University: Ahvaz Jondishapour University of Medical Sciences, ISLAMIC REPUBLIC OF IRAN Received: June 14, 2023 Accepted: September 6, 2023 Published: September 21, 2023 Peer Review History: PLOS recognizes the benefits of transparency in the peer review process; therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. The editorial history of this article is available here: https://doi.org/10.1371/journal.pone.0291861 Copyright: © 2023 Shek et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability Statement: The data supporting the findings of this study are available on request from the corresponding author, Prof. Daniel Shek (email address: daniel.shek@polyu.edu.hk), Chair

teaching mode and technology-supported active learning. Moreover, researchers argued that the post-pandemic pedagogy could harmoniously integrate technology and physical tools because of technology’s high penetration among Generation Z students, its capability in enhancing teaching and learning effectiveness, and its potential of preparing subjects for unknown future challenges [6, 7]. Online education could be delivered through synchronous (i.e., real-time lecture through the internet) and asynchronous (i.e., pre-recorded lectures) teaching and learning modes. Also, blended or hybrid learning (i.e., combination of online education and traditional class- room teaching) has been shown to optimize students’ online learning experience under the pandemic [8, 9]. Additionally, massive open online courses (MOOCs), which are non-credit online courses for learners not enrolled in higher education institutions, were offered by pri- vate companies (e.g., Udacity and Coursera) and universities (e.g., Stanford University) to mil- lions of learners worldwide [10]. Video-based learning is defined as the learning through the use of videos, which is com- prised of visual and audio cues to present information for learners [11]. It is recognized as a powerful resource for online teaching and learning [12]. Under the pandemic, the use of video-based learning in online education also increased. It was found that, when the lockdown measures were in place during the pandemic, there was a significant increase in the number

and diversity of didactic resources (i.e., video and audio files) in flipped classroom sessions [13]. Video-based learning could bring various learning benefits to students, including enhancing learning outcome, providing flexibility to learn at their own pace, and improving participation in online discussion [11]. In addition, videos could be combined with various teaching methods, including mobile-assisted learning and flipped classroom approach, to opti- mize students’ learning process. For example, exploring the use of videos and texts in mobile learning, Reychav and Wu [14] and Wang [15] found that videos could improve students’ individual learning outcome. Combining with the flipped classroom approach, Wikandari et al. [16] found that animated videos could significantly enhance university students’ ability in comprehension and application. Obviously, there is a need to ask whether virtual teaching and learning tools (e.g., videos) can promote the mental health of students, which is an impor- tant pillar in public health. Under the pandemic, there was an increased use of emergency remote teaching because school lockdown policies were implemented in many countries [17–20]. However, in online learning, students may feel isolated, bored, anxious or frustrated. The lack of instant feedback from teachers may decrease their motivation [21–23]. In hybrid learning, students may face difficulties in self-regulation [24]. Moreover, online learning may pose public health risk for adolescents. Studies suggested that prolonged exposure to digital devices may lead to mental

health problems in learners. Stressors related to quarantine and lockdown plus mental health problems from prolonged screen time may result in burnout and exhaustion in learners [25]. Therefore, it is important to design curriculum materials that could motivate students to learn and reduce their risk of developing mental health problems in an online learning environment. Studies showed that instructional videos could enhance students’ interest in the course content and their learning motivation [26, 27]. Based on the cognitive theory of multimedia learning, Chen [28] found that videos provided self-controlled learning environment for students, so they could easily locate the content they want. The concise text and images in videos could also reinforce students’ learning. Moreover, the audio and visual stimuli in videos could enhance learners’ learning effectiveness and reduce their extraneous cognitive load when processing information. Video was also found to be a more effective learning tool than the sole presenta- tion of text [29–31]. In the post-pandemic period, researchers viewed instructional videos as valuable original online learning resources for higher education courses [32]. PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 2 / 28 Professor of Applied Social Sciences OR Professor Elsie Yan (elsie.yan@polyu.edu.hk), Chairperson of the Departmental Research Committee overseeing research ethics approval, The Hong Kong

Polytechnic University. The data are not publicly available due to serious ethical concerns regarding the publication of focus group interview transcripts in a public repository. Specifically, when the data were collected, participants only gave consent for the findings to be published anonymously for educational and research purposes. Making the data public would violate this consent. Funding: This project was financially supported by the University Grants Committee on the development of virtual teaching and learning materials (3.54.xx.89PZ), and Dr. Daniel Shek Matching Fund from PolyU (1.54.xx.52UK). The preparation of this paper was supported by the Li and Fung Endowed Professorship for Dr. Daniel Shek at The Hong Kong Polytechnic Universit. Competing interests: The authors have declared that no competing interests exist.

Since 2012/13 academic year, we have implemented a leadership course entitled “Tomor- row’s Leaders” for undergraduate students at a Hong Kong university. The positive youth development (PYD) approach was used in the development of this course, which emphasizes fostering of adolescents’ cognitive, emotional and social competences to ensure successful childhood-to-adulthood advancement [33, 34]. The “Tomorrow’s Leaders” course aimed at enhancing university students’ intra- and interpersonal competences to equip them to meet the changing needs for success in the 21st century [35]. Moreover, this course was intended to promote university students’ leadership qualities, so they could be leaders for themselves (i.e., self-leadership) and lead others within their social circles [36]. In the “Tomorrow’s Leaders” course, we have developed videos based on various PYD attributes, including self-leadership, cognitive competence, social-emotional competence, resilience, moral competence, spiritual- ity, law-abiding leadership, cultural competence, effective communication, and team building. Empirical studies argued that leadership qualities, including self-leadership, psychosocial com- petencies, interpersonal relationships, problem solving, conflict resolution, and stress manage- ment, played a key role in strengthening children’s and adolescents’ mental health [37–39]. Besides, some leadership qualities such as resilience and psychosocial competence can be regarded as integral components of mental health. Because of the close connection between

adolescents’ leadership qualities and their mental health, this study could provide implications for public health. Our previous evaluation study of this course showed that students had signif- icant positive changes in a number of PYD attributes after taking the course [40], and high sat- isfaction with the course content, course design, teaching methods, and teaching and learning effectiveness [41, 42]. Under the pandemic, the delivery of this course was changed from face- to-face mode to online teaching or HyFlex learning (i.e., a flexible combination of online and face-to-face teaching) [43, 44]. In view of the benefits of instructional videos, a research project was implemented to develop virtual teaching and learning materials for the “Tomorrow’s Leaders” course, with 15 theory-related videos and 9 scenario-based videos produced to facili- tate students’ learning in this course. The details of the videos can be seen in Shek et al. [45]. After the development of the 15 theory-related videos and the 9 scenario-based videos, we evaluated the quality of the videos by employing the subjective outcome evaluation method via the client satisfaction approach. Our first study was conducted in Semester 2 of 2021/22 aca- demic year, in which the course was delivered in an online synchronous mode. Two subjective outcome evaluation scales (SOES) were developed to assess students’ views of the theory- related videos and the scenario-based videos, respectively. More than 90% of the students per-

ceived that the videos could promote their interest, understanding and reflection of the subject matter, and were beneficial to their development [46]. Furthermore, we replicated the study in Semester 1 of 2022/23 academic year in order to analyze students’ perceptions of the 24 videos under face-to-face teaching mode. Using the same measures, the second study found that stu- dents’ evaluation of the video quality and benefits received from watching the videos were sig- nificantly improved [47]. Although the two subjective outcome evaluation studies showed that students had highly positive appraisal of the 24 videos, there was a need to use qualitative evaluation strategies to further understand students’ perceptions of and experience in watching the videos in this course. Qualitative research examines the nature of human experience and the meaning of cer- tain phenomena to individuals, so that individuals’ interpretation and explanation of the “what”, “how” and “why” of a phenomenon could be explored [48]. For education research, it was stressed that qualitative research could comprehensively explain human beings’ psycho- logical dimensions and the nature of their complex behaviors [49]. Moreover, focus group interview is an effective qualitative method to examine a particular issue in which a group of participants have experienced together. It is suitable when researchers seek to obtain a range of PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education

PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 3 / 28

ideas or feelings from participants about certain issue, or examine the factors that affect indi- viduals’ viewpoints or behavior [50]. Previous studies have used focus group studies to evaluate the impacts of videos on students’ online learning [51, 52]. 1.1. Research questions In two previous quantitative studies, we examined students’ perceptions of the theory-related and case-based videos’ content, implementation quality and effectiveness [43, 44]. Generally speaking, the findings of these two studies are very positive. To understand the experience of the students, we conducted the present qualitative study using the focus group method to answer the following three research questions: 1. What are students’ perceptions of the pre-lesson video-watching arrangement? 2. What are students’ perceptions of the quality (i.e., video content and video design) of the theory-related videos and case/scenario-based videos? 3. What are students’ perceptions of the benefits gained from watching the theory-related vid- eos and case/scenario-based videos? 2. Materials and methods The present study used a focus group method to evaluate the perceived quality and effective- ness of 24 instructional videos in a leadership course in a Hong Kong university. Data were collected in Semester 2 of 2021/22 academic year, during which students attended the lectures through an online synchronous teaching mode. Students were asked to watch two to three the- ory-related videos and scenario-based videos before attending the 2-hour online synchronous

lecture (i.e., flipped classroom approach) every week. To gain participation points, students needed to complete the blended learning exercise and have reflections related to the videos before lecture each week. In total, 10 lectures that taught different leadership qualities, includ- ing intrapersonal development (i.e., mental health) and interpersonal development, applied this video watching arrangement. Focus group interviews were conducted at the end of the semester when students had completed all the lectures in the course. 2.1. Research design In terms of the philosophical orientation of this study, a general qualitative research orienta- tion was employed [53, 54]. The general elements of qualitative studies emphasize naturalistic inquiry, personal experience of research participants, empathic neutrality of the researchers, and an inductive analysis approach. Additionally, qualitative studies could examine individual experience or describe common meaning of several individuals’ lived experiences to develop composite description of the essence of participants’ experience [55, 56]. In this study, we focused on developing patterns of meaning from students’ subjective understanding of their experience. The design of interview questions was mainly broad and general, so students were able to express their own understanding of an experience. Besides, focus group interviews allowed meaning to be constructed through students’ interactions with others. We also upheld principles proposed by Shek, Tang and Han [54], including highlighting the philosophical ori-

entation of the study, detailed description of different parameters of the study, building up consensus and triangulation of findings based on different methods. PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 4 / 28

2.2. Participants and procedure In Semester 2 of the 2021/22 academic year, the “Tomorrow’s Leaders” course was delivered to 1,308 Year 1 undergraduate students at a Hong Kong university. These students majored in different subjects under seven schools or faculties at the university, including School of Design, School of Fashion and Textiles, School of Hotel and Tourism Management, Faculty of Con- struction and Environment, Faculty of Engineering, Faculty of Health and Social Sciences, and Faculty of Science. At the end of the semester, the teachers of the “Tomorrow’s Leaders” course purposefully sampled students to participate in the focus group interviews. The sampled par- ticipants were homogeneous because all students were Year 1 students at the same university. This sampling strategy fits the research purpose because the study aimed at understanding the common experience of students rather than comparing the difference between them. Financial incentive was also given to encourage students’ participation. At the end, 53 students agreed to join the interviews with 48 of them successfully participated in a voluntary manner. In the current study, research participants’ anonymity was maintained, meaning that stu- dents’ demographic information was not collected. Maintaining research participants’ ano- nymity would enable them to disclose more information or opinion during the interview [57, 58]. The sampled participants were all year 1 students in the case university. The general stu-

group interview was conducted on 23 May 2022, while the last interview was completed on 17 June 2022. The interview length ranged from 1 hour 34 minutes to 3 hour 1 minute, with the first interview conducted in English and the rest delivered in Chinese. Table 1 lists the number Table 1. Duration of the interview, number of participants, and language used in each focus group interview. Focus Group Number Group 1 Group 2 Group 3 Group 4 Duration of the interview 1:43:25 2:51:35 2:00:22 3:01:18 Number of Participants 7 9 6 8 Language Used English Chinese Chinese Chinese Focus Group Number Group 5 Group 6 Group 7 Group 8 Duration of the interview 2:31:18 2:37:42 1:34:45 1:44:08 Number of Participants 5 6 3 4 Language Used Chinese Chinese Chinese Chinese https://doi.org/10.1371/journal.pone.0291861.t001 PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 5 / 28

of participants, the duration, and the language used in each focus group interview. Besides, each focus group interview had two moderators, with the first moderator hosting the interview and the second moderator asking follow-up questions and taking notes. All moderators of the interviews have rich experience in qualitative research. 2.3. Instruments Qualitative research focuses on investigating “what” the participants have gone through and “how” they have experienced it [53, 56]. Following this principle, a semi-structured interview guide was developed for the focus group interview (Table 2). The interview guide was divided into five sections: 1) arrangement of watching these videos, 2) general learning experiences, 3) theory-related videos, 4) scenario videos, and 5) conclusion. Each section contains three to seven interview questions. In semi-structured interviews, the use of probing strategy and more open-ended questions by interviewers enable the understanding of more complex issues [68]. Moreover, open-ended questions are preferred in qualitative research because they allow par- ticipants to provide more description of their experience, which could lead to richer textual and structural description of participants’ experience in the final report [56]. 2.4. Data analyses According to Polkinghorne [69], the accuracy of transcription is one of the criteria for judging the quality of a qualitative study. Therefore, the 8 focus group interviews were verbatim tran- scribed, and then double-checked by an experienced researcher. Qualitative research does not

only describe what participants have experienced, but its data analysis focuses heavily on researchers’ interpretation [56]. Therefore, the transcripts were coded using line-by-line cod- ing, which refers to the coding of each line of text in the transcripts. It enables researcher to develop codes that reflect the experience of interviewees rather than the theoretical beliefs of the researcher [70]. To ensure that the data analysis was systematic and verifiable, line-by-line coding followed by constant comparison of the codes between focus groups were conducted [50]. The line-by- line coding identified 92 first order codes. At the secondary coding stage, insignificant codes were disregarded and similar codes were grouped under 22 second order codes, which could be combined into six themes. Besides, the frequency of each first order code was calculated. Sim [71] noted that group dynamics of each focus group would affect the occurrence of the first order codes. However, comparing the difference in topics discussed in each group, only a few topics were more frequently deliberated in some groups, which did not significantly affect the interpretation of the interview data. So, the frequencies of the first order codes could serve as a reference to the strength of participants’ opinions. Validity in qualitative research determines the accuracy, trustworthiness and credibility of the account or perceived realities of the researchers and research participants [72]. It focuses on the researchers’ care in the analysis procedures. Reliability assesses the agreement on codes

by multiple coders. Researchers stress that qualitative research emphasizes more on validity, while reliability only plays a minor role [73]. The concept of trustworthiness, containing four aspects (credibility, transferability, dependability and confirmability), was also used by researchers to understand validity and reliability in qualitative research [72]. Additionally, researchers supported the importance of applying various methodological strategies to demon- strate the qualitative rigor [74]. Methodological strategies commonly applied in qualitative researchers include audit trail, member checking, peer debriefing, and rich description [74, 75]. In the current study, different strategies were applied in the research design and method- ology to ensure that findings of the research were valid and reliable. First, audio recording the PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 6 / 28

Table 2. Semi-structured interview guide. 1. The arrangement of watching these videos 1.1 Have you encountered any difficulties in watching these video clips? If yes, what are they? How do you overcome these difficulties? Can you provide some examples? 1.2 How much time did you spend in watching the video clips on average every week? Usually, how many times did you watch the video clips? 1.2.1 Do you watch them before or after the on-site class? If you like some videos, did you watch them again after attending the lecture? 1.2.2 Did your teacher play the video again in the face-to-face/online class? If yes, any benefits for your learning? Do you enjoy watching these videos again? 1.3 How do you perceive this arrangement (i.e., you are expected to watch the videos before attending the lecture)? What are the benefits and shortcomings of watching these video clips before lectures? Does watching the videos increase your study load? Do you enjoy watching these videos? Any suggestions on this arrangement? 1.4 Can watching the videos reinforce your participation in the face-to-face/online real-time class and facilitate your understanding of teachers’ instruction? 2. General learning experiences 2.1 These video clips aim to provide some background information to facilitate your understanding of related theories. Do you think the video clips can facilitate your understanding of the theories? 2.1.1 If yes, how do the video clips help you understand the theories and support you to become more engaged in the class?

2.1.2 If not, why not? How to improve (e.g., the quality of the videos, the use of the videos)? Any suggestions? 2.2 In general, what do you think about the quality of the video clips (e.g., content, audio, subtitles, number of video clips, duration of the video clips, digital effects)? 2.3 For each lecture (i.e., Lecture 2 to 11), we prepare one or more than one theory-related videos and case scenario videos. Do you think the theory-related videos and the scenario videos are closely related? 2.3.1 If yes, any impressive examples/cases among the 10 lectures? 2.3.2 If not, how to strengthen the links between the two kinds of videos? Or which kind of videos can be removed? 3. Theory-related videos 3.1 Do you think the content of the theory-related videos is clear enough for self-learning? Is it difficult for your self- learning? 3.2 Do you think the theory-related videos are closely related to the relevant lecture? Can the theory-related videos help you better understand the theories? Any impressive examples/cases among the 10 lectures? 3.3 What aspects/components of the theory-related videos can benefit your learning (e.g., model explanations)? 3.4 Are these theory-related videos helpful to your learning? Can you recall the two most impressive videos among the 15 theory-related videos? Why do you like these two videos most? What are the strengths of these two videos? 3.5 Please share one or two theory-related videos that need to be revised/improved. Why are they not helpful for you? How could we improve them? 4. Scenario videos

4.1 Do you think the content of the scenario videos is clear enough for self-learning? 4.2 Do you think that the scenario videos are closely related to the relevant lecture? Can the scenario videos help you better apply the theory to your daily life? Any impressive examples/cases among the 10 lectures? 4.3 What aspects/components in the scenario videos can benefit your learning (e.g., apply the theory in practice, very related to my life, very practical)? 4.4 Are these scenario videos helpful to your learning? Can you recall the two most impressive videos among the 9 scenario videos? Why do you like these two videos most? What are the strengths of these two videos? 4.5 Please share one or two scenario videos that need to be revised/improved. Why are they not helpful for you? How could we improve them? 5. Conclusion 5.1 Overall speaking, are you satisfied with the video clips used in this course? If the full mark is 10, ranging from 1 to 10, how much will you give? 5.2 To benefit your learning and strengthen your understanding of specific topics, do you have any suggestions to improve the video clips? 5.3 What else would you like to share? https://doi.org/10.1371/journal.pone.0291861.t002 PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 7 / 28

interviews allowed researchers to repeatedly check whether the emerging themes truly repre- sented participants’ accounts [61]. Second, rich verbatim extracts enabled readers to judge the truthfulness of the final themes [75, 76]. Both of these strategies were used in this study to ensure that the findings represented the data collected, thus enhancing the validity of the research. Third, emerging themes were discussed transparently among the researchers in this project to ensure that biases in the research method were minimized, which could enhance the reliability of this study [61, 72]. Fourth, inter-rater reliability was conducted to verify the strength of the findings [54, 77]. 20 randomly selected narratives from the focus group inter- views were presented to two independent researchers and they were asked to re-code these narratives. The result showed that the two researchers agreed on 18 out of 20 codes. This showed a high level of inter-rater reliability in the analyses (inter-rater reliability = 18/ 20 = 0.9). These two strategies ensured that researchers arrived at similar codes or themes in the data analysis, thus enhancing the reliability of the current research. Fifth, triangulation was a typical strategy to enhance the validity of research findings [73, 75]. The triangulation of the findings in the current study and findings from our previous quantitative study is presented in the discussion section of this paper. 3. Results This section presents and analyzes the first and second order codes under six categories of

responses (i.e., themes). The six themes are: 1) video arrangement, 2) video design, 3) video content, 4) videos’ benefits to the students’ pre-lesson self-learning, 5) videos’ benefits to the students’ learning of course content, and 6) videos’ benefits to the students’ class participation. These themes are presented below. 3.1. Video arrangement In terms of the video arrangement, this study examined three aspects: 1) positive effect on stu- dents’ learning, 2) connection between the videos and the lectures, and 3) drawback to stu- dents’ learning. First, for the positive effect of the video arrangement, students expressed that the pre-lesson video watching arrangement allowed students to pace their learning according to their own needs and gain more flexibility (N = 6). With the addition of the videos and the blended learn- ing exercise, lecture time was reduced from 3 hours to 2 hours, which enhanced their concen- tration in lectures (N = 9). As elaborated by a student: “This course used to have 3 hours lectures. However, with the addition of the e-learning modules, including watching the videos and answering [the blended learning] questions before lectures, lecture time is reduced to 2 hours. This is beneficial to my concentration in lectures. My concentration is better in 2-hour lectures. Without these videos, we would have 3-hour lectures. Even though we could take a break in the 3-hour lecture, the learning outcome would not be very good.” (Group 8 Line 68–70) This arrangement also allowed students to gain better understanding of the lecture content

and engage more in class discussion (N = 19). As stated by one student: “I think the arrangement is reasonable. The videos are like pre-lesson preparation materials. There are many theories introduced in the videos. If we can’t digest it in class, we can’t dis- cuss it on the spot. Watching these videos before class gives us time to digest, so we can bet- ter answer questions about this topic in class discussion.” (Group 6 Line 130–132) PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 8 / 28

Watching the theory-related videos before class allowed more time for teachers to share personal experience or conduct class discussion during lectures, which was more interesting and beneficial to some students (N = 2). Besides, many of the students thought that the arrangement of video watching and blended learning exercise was fair because it counted towards their participation grade (N = 18), which forced them to do the exercise, reflect on their daily issues, and relate the theories to the cases (N = 6). Second, in terms of the connection between the theory-related videos and the lectures, stu- dents believed that they were closely related because the teachers recapped the theories (N = 16), or discussed related personal experience and real life examples during lectures (N = 21). As elaborated by a student: “I think the teacher’s explanations in the lectures and the theory-related videos are very rel- evant, because the teacher would use some daily examples to explain the theories, so they would not be so difficult to understand. For example, there is a more abstract topic called spirituality, which I think is difficult to understand. However, after we have self-studied the videos, the teacher would further explain how we could exercise this ability in our daily lives, such as cultivating more interests. It is related to what is mentioned in the videos. The videos also mention how we could exercise this ability, and there are some examples to explain it.” (Group 6 Line 357–361) Besides, in terms of the connection between the case-based videos and the lectures, the stu-

dents thought that they were closely related because some teachers discussed the cases during lectures, while other teachers shared personal experience that were related to the videos (N = 20). Sometimes, teachers would share students’ blended learning answers, or explain dif- ficult concepts in the videos (N = 16). As one student remarked: “At the end of the [case-based] videos, there are some blended learning questions which require us to express our opinions on the online learning platform. Our teacher did not only share the views of the students, but also shared his own views. This has triggered my curiosity to read other students’ responses [on the online learning platform]. So I think this motivates my study, and encourages my participation in class.” (Group 7 Line 123–125) These class discussions engaged students and increased students’ understanding of the lec- ture topics. Teachers’ interpretation of the videos and the extra examples offered other per- spectives to students and enhanced their understanding. Third, the video arrangement has some drawbacks. Some students thought that the arrangement exceeded their expectation (N = 1) or the workload was too big when they had midterm exams or assignments for other courses (N = 11). It was also stressful for them to understand so many concepts before attending lectures. Some students preferred watching the videos and completing the blended learning exercise after lectures because they would have better understanding of the topics after teachers’ explanation (N = 4). As pointed out by a student:

“I think it would be better for us to watch the videos and finish the related tasks after lec- ture. Before lecture, many students would forget to watch the videos. After lecture, the teacher has explained the concepts and elaborated using his/her life experience, so we would have more understanding of the concepts. When we watch the videos after lecture, it can deepen our understanding of the concepts. So I think it would be better for us to watch the videos and complete related tasks after lecture.” (Group 5 Line131-133) PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 9 / 28

Students also expressed that they were not able to ask teachers questions immediately after watching the videos. In addition, some students stated that they may not pay full attention in lectures after understanding the course content from the videos before lectures (N = 2). 3.2. Video design In terms of video design, this study investigated two aspects: 1) video quality and features that contributed to students’ learning, and 2) general suggestions for improvement. First, the majority of the students thought that the quality of the videos was good (N = 21). Students thought that the combination of the theory-related videos and the case-based videos provided a variety of video format, which could enhance their interest in the videos (N = 3). For case-based videos, students believed that the videos with real persons acting out the sce- nario (i.e., role-play videos) had good production. The topics of these videos were also related to students’ lives (N = 3). As elaborated by a student: “I also think the quality is good. At least, the production has thought about the things that are closely related to [students’] daily life, especially those about the scenario-based videos. Some of the scenario-based videos are about group projects, some are about arguments between family members, etc. These topics are closely related to our daily lives.” (Group 2 Line 379–381) For theory-related videos, students perceived that various features of theory-related videos had contributed to their learning. Students enjoyed the animation (N = 3), and the interesting

animation helped their understanding (N = 10). Besides, the theory-related videos were very structured and explained the theories clearly. Model explanation was helpful to students’ learn- ing (N = 16). The arrows, list and organization in the theory-related videos made the theories and the causal relationships easy to understand (N = 7). As stated by a student: “I think there are some definitions and models in the clip, which can help me understand the topic better. The animation and narration are very clear . . . step-by-step . . . what the first step is, and what the second step is. The videos let me clearly know how to proceed, the sequence and the like. They allow me to understand the topics more clearly, with guidance, and in a systematic way.” (Group 2 Line 532–535) Besides, the combination of visual and audio stimulation could improve students’ concen- tration (N = 12). The analysis and discussion of examples in the theory-related videos and the blended learning exercise facilitated students’ self-reflection and the development of critical thinking skills (N = 6). In the theory-related videos, the use of discussion or interactive mode for theory learning was more interesting for students (N = 3). However, in some of the theory- related videos, the script of the narration directly replicated the visual content of the videos, and students thought that this presentation style was boring (N = 3). Second, in terms of ways to improve the video quality, students suggested improving the

audio quality of the videos (N = 10). Moreover, students believed that the length of each video should be shortened (e.g. within 5 or 10 minutes) (N = 3), and they preferred the videos to have a faster pace (N = 2). For the case-based videos, students would like to have more varia- tion of video format, so they could have stronger impression of the videos. Besides, students preferred story telling or role-play video production because these presentation styles were more interesting and engaging for them (N = 14). PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 10 / 28

3.3. Video content In terms of video content, this study investigated three aspects: 1) theory-related videos’ con- tent, 2) case-based videos’ content, 3) connection between theory-related videos and case- based videos, and 4) general suggestions for improvement. First, in terms of theory-related videos’ content, students encountered some difficulties. A number of students expressed that the difficult technical terms or English vocabularies in the videos hindered their understanding of video content (N = 15). As elaborated by one student: “My opinion is the same as that of the other students. First, the quality of the videos is very good, but what needs to be improved is that many videos have many definitions from dif- ferent scholars. It would be better to add some daily examples. We checked the definitions on Google ourselves, and they are from different scholars. We can find all the definitions, but it is the definitions that bothered us because the wordings of the definitions are mostly vague and abstract. If the videos could use more conversational English and daily life exam- ples, it will be easier for us to understand.” (Group 6 Line 32–36) Besides, students believed that some theory-related video content was too abstract (N = 3), and suggested adding more discussion of daily life examples in the theory-related videos (N = 2). Some students found that there were too many concepts in the theory-related videos, and recommended adding some guiding questions to help students’ self-reflection. Also, stu-

dents wanted to know the importance of each lecture topic to them personally and how the positive youth development (PYD) attributes would affect their lives. Second, for case-based videos, various video features contributed to students’ learning. The case-based videos were related to students’ daily lives, so students could understand better and enjoy the videos (N = 7). As elaborated by a student: “I think the advantages of the case-based videos are that they are quite interesting, and they use more realistic situations. And another advantage is that I could feel the diligent produc- tion and a lot of effort was used, so I take the video watching very seriously.” (Group 7 Line 367–368) Moreover, students could relate the cases to social issues. The case-based videos provided various perspectives and students could learn the ways they could solve similar problems in the future (N = 14). However, students thought that there should be more case-based videos to explain more complicated theories or offer more elaboration and application of the theories in the scenario videos (N = 6). It was also believed that some case-based videos did not provide solutions and did not analyze different factors, which decreased the videos’ benefits to stu- dents’ learning (N = 3). Besides, some stories in case-based videos were too extreme (N = 3). Students also wanted the case-based videos to highlight the key points and provide some guided reflection on the topic. For cases that needed discussion, students wanted to watch

cases that did not have conclusions, so there would be more room for discussion. Third, this study investigated students’ perceptions of the connection between the theory- related videos and case-based videos. The majority of the students agreed that the theory- related videos were related to the case-based videos (N = 24). The cases could explain the theo- ries, and students could apply the theories into the cases. Moreover, the blended learning ques- tions required students to use the theories to answer questions related to the scenarios, which further strengthened the connection between the two types of videos (N = 3). As elaborated by one student: PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 11 / 28

“I think those scenarios could echo the theories. One of the students mentioned just now the video about whether the husband should steal medicine for his wife’s illness. In fact, the [blended learning] questions after the videos asked us to use the theory to answer. When I answer the questions, I would say from different levels whether the husband’s behavior is in line with. . .that is, it may be unreasonable on the legal level, but if it is on a higher level of morality, it may be more reasonable. That is, we can use the theory to analyze the case, instead of only using general terms to discuss the matter.” (Group 4 Line 604–609) However, a few students thought that the connection between these two kinds of videos was not strong enough (N = 7). Besides, students suggested adding more real life application of the theories (N = 1) or simplifying the theory-related videos (N = 3) in order to strengthen the link between the theory-related videos and the case-based videos. Fourth, for ways to improve the general video quality, students suggested adding translation of English vocabularies in the subtitles (N = 1). Moreover, students thought that the format or topics of some videos were quite outdated, so they suggested adding examples that were more related to the recent trend or news (N = 3) or discuss more social issues in order to strengthen their understanding of related topics (N = 1). The blended learning questions could be revised to strengthen their connection with the videos (N = 2). 3.4. Videos’ benefits to the students’ pre-lesson self-learning

In terms of videos’ benefits to the students’ pre-lesson self-learning, this study investigated two aspects: 1) theory-related videos’ contribution, and 2) case-based videos’ benefits. First, the majority of the students thought that the theory-related videos were good enough for pre-lesson self-learning (N = 28) because there were clear explanation of theories and the examples in the videos could explain the application of theories. As explained by a student: “I think the theory-related videos are easy to understand and suitable for self-study. Although sometimes I don’t understand the content after watching the video once, I would get the concept after watching it two to three times, and the explanation is very clear. There are also examples, which can help my understanding. The theories are simplified for us, and the PowerPoint files summarized the theories. Compared to reading academic journals or only attending the lectures, these videos are definitely better.” (Group 2 Line 466–469) Also, students perceived the theory-related videos as effective learning tool and they were helpful for their term paper writing (N = 8). Besides, with the videos, students were able to pace learning according to their own needs and gain flexibility (N = 6). As elaborated by a student: “I think the theory-related videos are easy to understand, and I didn’t encounter any diffi- culties during the learning process. Especially during self-study, we can choose different ways to watch the videos. We can choose to watch the same video again. We can fast pace

it, or we can pause it for a while at some point. Because videos have such flexibility, I think they are suitable for self-study.” (Group 8 Line 211–213) Besides, the theory-related videos could provide an overall mind map or guidelines (N = 5), and the pointers, highlighter and diagrams could explain the causal relationship in the theo- ries, which were helpful to students’ self-learning (N = 4). However, some students thought that the theory-related videos were not clear enough for self-learning because the English PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 12 / 28

vocabularies were difficult to understand (N = 7), while a student believed that the general duration of the theory-related videos was too long (N = 1). Second, in terms of the case-based videos’ contribution to students’ pre-lesson self-learning, the majority of the students thought that the case-based videos were suitable for pre-lecture self-study (N = 34) because they were related to students’ lives, which can stimulate their reflection. Besides, the case-based videos matched the theory-related videos and demonstrated the concepts. As stated by a student: “I think these case scenario videos are suitable for self-study. Compared with the theory- related videos, they are easier to understand, because they are not just explanations in words, but they are some daily life examples to explain the theories concretely. These are supplements based on the theories, which are more suitable for us in pre-lecture prepara- tion.” (Group 2 Line 635–637) The videos also used a casual and friendly tone to present the stories, which could engage students. As elaborated by a student: “Most of the scenario videos are interesting to me. For example, in one video, a Youtuber walks in [the university campus] and interviews different people. It is quite fun. It uses a very casual and friendly way to present, so it is suitable for us to watch.” (Group 7 Line 278–280) However, a small portion of students (N = 5) thought that while scenario videos were strict forward and easy to understand, they cannot be used to explain complicated concepts and may not motivate students to reflect.

3.5. Videos’ benefits to the students’ learning of course content In terms of videos’ benefits to the students’ learning of course content, this study investigated three aspects: 1) videos’ general benefits, 2) theory-related videos’ benefits, and 3) case-based videos’ benefits. First, the majority of the students thought that the videos could facilitate their understand- ing of lecture content (N = 27) and related theories (N = 26). A student explained as follows: “It is a good starting point to use short videos to preview these concepts. I think this course has a lot of concepts or a lot of theories, so these videos can give us a preliminary under- standing of these theories, then we can have more in-depth elaboration and application of these theories in class. I think this step-by-step approach is pretty good.” (Group 4 Line 425–428) Moreover, the videos allowed students to learn the course content with a self-discipline atti- tude, learn to manage their time (N = 5), and pace their learning according to own needs. The videos also increased students’ learning flexibility and helped their assignment and term paper writing (N = 10). Besides, the combination of the theory-related videos and the scenario-based videos enabled students to understand both the theories and their application to daily life (N = 3). As elaborated by a student: “I think the videos can improve my understanding of related theories. Each topic’s videos are divided into theory and cases. I think the combination is very good. After watching the