Use of instructional videos in leadership education in higher education under COVID-19: A qualitative study

theory-related videos, I am already familiar with those theories. So, when I watch the PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 13 / 28

scenario-based videos, I know how to apply the theories to real life examples. In this way, I can have a deeper and better understanding.” (Group 2 Line 302–307) The majority of the students rated the videos 7 points (N = 10), 7 to 8 points (N = 7) or 8 points (N = 18) out of 10 points. This showed that students were satisfied with the quality and benefits received from watching the 24 videos. Second, the theory-related videos contributed to students’ learning through clear explana- tion of theories, which helped students to understand the content without prior knowledge. The animation, layout and video cutting also made the theory-related videos more interesting. For this topic, students in the focus group interviews were asked to give examples of videos that had significantly contributed to their learning. As elaborated by a student: “I think the video on Richardson’s Model I just mentioned is good. If we don’t have this video, I will look at this model for two hours but don’t know what it is explaining. I think it’s not just about these two videos, but all the videos are very clear. Even about concepts or models I haven’t seen before, I can understand them from the videos.” (Group 8 Line 276– 278) Moreover, some theory-related videos’ content was related to students’ daily lives, so they could stimulate students’ self-reflection. The skills introduced in the videos were useful to stu- dents. For example, one student was particularly impressed by the theory-related videos for Lecture 10 repair strategies and Lecture 5 resilience. As the student explained:

“I like the theory-related videos for Lecture 10 repair strategies and Lecture 5 resilience. Repair strategies are problems we often encounter, such as how to solve problems when we quarrel with friends, however, I never thought about it. After watching the video, I would think about whether I can solve the problem according to the theory. In addition, because I didn’t have a deep understanding of resilience at first, after watching the videos, I would know more about resilience. The content made me understand how we can cultivate this ability.” (Group 6 Line 428–431) Besides, in theory-related videos, there were concrete examples to explain the theories and illustrate the strategies for tackling different problems. Third, students further elaborated on the case-based videos’ benefits to their learning. The case-based videos applied theories to some real-life cases or social issue, so students could understand the concepts very easily. For this topic, students in the focus group interviews were also asked to give examples of videos that had significantly contributed to their learning. As explained by one student, “There are two types of case-based videos. As mentioned just now, the first one stimulates daily situations. The other one, just like the case-based video on resilience, it can strengthen the reliability of the model [or theory]. It is not only the model talking about the concept, but the case-based video presents an example of how the protective factor can help a person to be more resistant to stress. This is more convincing, which impresses me even more. For

example, there is a case-based video on misinformation, which is about the MMR vaccine injection. That video can help me apply the model to daily news. I usually only watch the news, and that is it. However, from the video, I learn that we could apply some of the things we have learned. So, I could gain another perspective.” (Group 2 Line 745–748) PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 14 / 28

Also, the case-based videos were related to students’ daily lives, so they could learn applica- tion of the theories in their daily lives. They could have self-reflection after watching the case- based videos, which was helpful for them to gain more positive mindset. As shared by a student: “I think the case-based videos gave me examples for reference on how to deal with the same situation, and how to deal with some people. They allow me to think more, and I would be aware of similar problems in my daily life, such as interpersonal skills and handling some situations. I could look at it from a third-person perspective. I would be calmer and think about how I should deal with it, or how I could ask for help from someone. I will know how to deal with similar situation. Sometimes when things happen to me, I will be very con- fused. However, from a third-person perspective, I will know how to solve it more clearly. I will have a more diverse perspective. Some of the videos are interviews, and some of them are positive or negative. I could think about the issues from different angles.” (Group 3 Line 345–350) Additionally, the stories of some of the case-based videos were entertaining and surprising, so they could engage students to have reflection even after the lectures. Some of the case-based videos were produced by real-life filming, while others were produced by animation. Some stu- dents thought that the animation in case-based videos were interesting and some video for- mats (e.g., interview) were new to students. The background music also allowed students to

gain knowledge in a relax manner. 3.6. Videos’ benefits to the students’ class participation In this study, many students perceived that both the theory-related videos and case-based vid- eos could enhance their class participation (N = 16). Because they already knew the course content from the videos, they had time to understand and think about the topics before lec- tures, then they could share more during class discussion. As stated by a student: “After gaining some impression of the theories before class, I would have my own under- standing. During the lecture, the teacher would give us some new ideas, which can back up the class discussion. Then, I can share more at this point, which can enhance the class dis- cussion.” (Group 8 Line 95–96) Moreover, after watching the videos, students were able to answer teachers’ questions (N = 16). The videos could also be used for class discussion, which increased students’ motiva- tion (N = 7). Also, students were able to gain more interest in the lecture topics after knowing the content from the videos (N = 20). As discussed by a student: “For class participation, I also think it is helpful, because after watching the videos on the topic, I can have more interest, which will make me particularly interested in those lectures and want to go to class.” (Group 2 Line 305–307) However, some students thought that the videos did not affect their participation in class because they believed that class participation depended on teachers’ leadership, lecture topics,

class atmosphere, class activities, other students’ participation, or the personality and interest of students (N = 8). Some students stated that when they already knew the course content from the videos, they would be less concentrated in class discussion (N = 5). PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 15 / 28

4. Discussion There is a growing interest and efforts in developing instructional videos for higher education students, particularly with the influence of the pandemic. The current research project is inno- vative in a number of areas. First, as the videos were offered to a large number of students (more than 2,000 students annually), development of effective instructional videos could bene- fit a large group of students every year. Second, both theory-related videos and scenario-based videos were developed, which was not common in Chinese and Western contexts. The combi- nation of flipped classroom approach and video-watching was also rare in the Chinese context. Third, the findings in this qualitative study complement the quantitative findings obtained in our previous two studies. The quantitative studies provided support for the benefits of the 24 videos [46, 47], while this qualitative study enabled us to further understand how the videos had generated benefits for students. In this study, the major findings were divided into six themes. First, in terms of video arrangement, students believed that the pre-lesson video watching arrangement enabled them to learn more flexibly, pace learning according to their own needs, and concentrate more in lectures. These results support other studies, which revealed that pre-lesson video watching was associated with better learning enthusiasm and academic results in undergraduate stu- dents [78, 79]. Besides, studies found that lecture videos enabled students to control their

learning pace, which enhanced their satisfaction with the lectures and improved their aca- demic results [80, 81]. This study also found that the teachers would recap the videos or discuss the blended learning exercise during lectures, which further reinforced the connection between the videos and the lecture. Studies revealed that the implementation of quizzes under flipped classroom arrangement, which was similar to the blended learning exercise in this study, could act as incentives for students to complete pre-class self-learning [82, 83] and could improve the effectiveness of the flipped classroom arrangement. The current study con- tributed to the literature by demonstrating how pre-lesson videos could be connected to lec- tures through class discussion. On the other hand, the current study also found that some students preferred watching the videos and completing the blended learning exercises after lec- tures. In this way, they would have better understanding of the topics after teachers’ explana- tion in lectures. Literature also compared the learning effectiveness of pre- and post-lecture learning activities. Analyzing the effectiveness of different teaching and learning activities in an undergraduate course, Inglesi-Lotz et al. [84] found that 16.44% of the students thought that the pre-lecture exercises enabled them to have better understanding of the lecturers, while 25.84% of the students perceived that completing the post-lecture exercises facilitated them to revisit the lecture content and gain more understanding. On the other hand, Liang [85]

revealed that pre- and post-lecture quizzes enhanced students’ learning through different ways. The pre-lecture quiz motivated students’ to reflect on their prior knowledge and arose their interest in the course content, while the post-lecture quiz increased students’ awareness of their learning process and strengthened their metacognitive activities. These studies showed that pre- and post-lecture learning activities could be implemented to serve different learning objectives. Second, in terms of video design, this study found that various features of the theory-related videos had contributed to students’ learning. Students stated that the arrows, list and organiza- tion in the theory-related videos made the theories easy to understand. It was also found that combining visual stimulation and voice-over narration could improve students’ concentration in the videos. These findings add to the literature on the multimedia learning theories, which examined how different video features could contribute to effective learning [32, 86–88]. The current research findings supported the “signaling” and “advance organizer” concepts in PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 16 / 28

multimedia learning theories [86, 89], which stressed that highlighting key points and using flow charts or tables for presentations reduced learners’ extraneous cognitive load, thus improved their learning efficiency. Also, the “modality” principle was supported, which stated that presenting visual illustrations with voiceover narration could improve learning efficiency [32, 90]. For the features of case-based videos, some students in this study favored animation in the case-based videos because they were interesting to them, while some students preferred the use of role-play video production because it was more engaging. Studies also revealed that voice recordings in animated videos could bring cases to life [91], while stories with real-life filming could add a sense of realism and transfer textbook knowledge to practical settings for students [92]. Moreover, this study found that students perceived that the combination of the- ory-related videos and case-based videos offered a variety of video format, which enhanced their interest. In the literature, studies examined college students’ and teachers’ preferences for different video presentation format. It was found that videos on hands-on demonstration were most welcomed by college teachers [93], while videos that demonstrated instructors’ presence were helpful to university students’ learning [32, 94]. The current study adds to the literature by illustrating how theory-related videos and case-based videos could engage students’ interest.

Third, for video content, in this study, students expressed that the difficult English vocabu- laries in the theory-related videos may bring additional challenge for them. In the literature, the majority of studies on video-based teaching and learning were conducted in the Western context, so this study showed how language could create a barrier for learners in the Chinese context. The “subtitle” principle proposed by Mayer et al. [95] also highlighted that having video subtitles was beneficial for students with diverse cultural or language background. Besides, this study found that the use of more realistic situations in case-based videos could arouse students’ interest. In the literature, a study revealed that cases could link content with action, and could relate students’ learning with others’ experience [96]. Another study examin- ing the use of videos to simulate clinical scenarios in nursing education also found that video simulation could effectively enhance students’ critical thinking skills and leadership competen- cies, suggesting the usefulness of videos in helping nursing students to transfer textbook knowledge to professional practice [97]. Fourth, this study examined how the videos benefited students’ pre-lesson self-learning. For theory-related videos, students believed that the videos enabled them to learn the theories in a self-learning mode because of the clear explanation and examples presented in the videos. For scenario-based videos, most of the students believed that they were able to self-learn the

videos before lectures because the cases were related to their daily lives. In the literature, a study showed that flipped classroom approach combined with video-based learning materials were key to students’ self-learning [15]. In practical dental training, a study also found that both two-dimensional videos and 360˚ virtual reality training enabled students’ easy access and portability during online education and were effective for students’ self-learning [98]. The current research adds to the literature by explaining how theory-related videos and case-based videos could benefit students’ pre-lesson self-learning. Fifth, this study investigated the videos’ benefits to students’ learning of course content. The videos enhanced students’ self-discipline attitude and time-management. Watching both theory-related videos and scenario-based videos allowed students to learn the theories and their application. Also, this study showed that both theory-related and case-based videos could engage students in self-reflection. In the literature, studies demonstrated that animated videos on microscopic mechanisms explained the dynamic biological knowledge more clearly [99], and the use of case-based videos enabled students to experience scenarios with real patients even in a classroom setting [100]. Studies also showed that videos, especially case-based videos, PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 17 / 28

could facilitate students’ meaningful learning and enhance their higher order thinking skills [101, 102]. A study on college-level sustainability courses revealed that working on “real world problem” could strengthen students’ learning attitude, including self-regulation and self- responsibility [103]. Sixth, for students’ class participation, this study found that the videos enabled students to learn and digest course content before class, so they would gain more interest in the topics and could answer teachers’ questions during lectures. Students also highlighted that using the vid- eos to induce class discussion could motivate them to participate in the discussion. Investigat- ing the combination of animated videos and flipped classroom approach in a college-level biochemistry course, Wikandari et al. [16] argued that videos enabled students to learn at their own pace before attending lectures, while the flipped classroom approach allowed the imple- mentation of interactive in-class problem solving activities. Besides, Danker [104] supported that the combination of flipped classroom and inquiry-based activities in lectures could engage students, enhance their curiosity and improve their higher-order thinking skills. The objective of the “Tomorrow’s Leaders” course was to enhance college students’ leader- ship qualities through strengthening their positive youth development (PYD) attributes, which are important qualities of positive mental health. We have developed videos based on various

PYD attributes, including self-leadership, cognitive competence, social-emotional competence, resilience, moral competence, spirituality, law-abiding leadership, cultural competence, effec- tive communication, and team building. Our previous quantitative studies showed that the 24 videos were able to improve college students’ learning and implementation of the leadership concepts in their daily lives [46, 47]. The current study used the qualitative research approach through the student focus group interviews. The findings showed that students had positive perceptions of the video content, video design and video arrangement. The results also revealed the videos were able to strengthen their pre-lesson self-learning, learning of course content and class participation, which may eventually contribute to the psychosocial well- being of the students. Studies have shown the positive relationship between adolescents’ leadership qualities and their mental health [37–39], so the production of these videos has public health implications. Furthermore, studies showed that mental health problems during adolescence, including depression, anxiety, personality disorders, eating disorder and substance abuse, would persist into adulthood [105, 106]. Besides, young people with mental disorders were likely to fail at key life events that happened during adolescence, including academic pursuit, job-hunting, friendships and romantic relationships establishment [107]. These young people had greater risks of having low social and economic status during adulthood [108, 109]. From the public

health perspective, early interventions of adolescents’ mental health problems could prevent the progression of primary and comorbid mental disorders into adulthood [107]. Therefore, the current project on developing educational videos for positive youth development could enhance adolescents’ mental health and have implications for public health. 4.1. Triangulation of research findings Triangulation could take four forms: comparing different data sources, involving multiple investigators, testing competing hypotheses, or including results from different research meth- ods [75, 110]. In this study, the triangulation of methods was applied. The goal of this form of triangulation analysis is to look for convergence of results from different methods. When the results corroborate with each other, the validity of the findings is established [111]. Researchers supported the use of triangulation of different methods, including quantitative and qualitative approaches, to strengthen the validity of a study [53]. However, the implementation of PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 18 / 28

triangulation or mixed method requires the epistemological clarity of the different research methods [112]. Our quantitative studies adopted the subjective evaluation strategy through the client satisfaction approach, while the present study employed the qualitative paradigm focus- ing on participants’ subjective experience. Both types of research approaches aim at investigat- ing participants’ perceptions of their experience, so the findings of our quantitative and qualitative studies could be triangulated. In the two quantitative studies we conducted previously, the questions in the survey inquired students’ perceptions of the theory-related and case-based videos’ content, imple- mentation quality and effectiveness of the videos in enhancing their PYD attributes [46, 47]. For the 15 theory-related videos and 9 case-based videos, all of the items received a high mean scores and positive response rates, showing that students had high satisfaction with the video content, quality of video delivery and the video effectiveness. In this focus group study, it was found that the majority of the students were satisfied with the video arrangement, content and design. Students also agreed that the videos could facilitate their pre-lesson self-learning, enhance their understanding of course content, and increase their participation in class discus- sion. Table 3 presents a side-by-side comparison of the findings from our first quantitative study and the findings from the present study. Both studies analyzed the same batch of stu-

dents in Semester 2 of 2021/22 academic year. The comparison showed that the findings from the two studies were highly consistent. For example, in our quantitative evaluation study, one of the items “the animated video is well designed” received positive response rates from 94.69% to 98.20% across the 15 videos. Similarly, in our qualitative study, within the “video design” theme, the majority of the students thought that the quality of the videos (both theory- related and case-based videos) was good (N = 21). On the other hand, for the case-based vid- eos, one of the items in the quantitative study “the video-based activity has increased my inter- action with the teacher” received positive response rates from 86.32% to 96.39% across the nine videos. Comparably, in the current qualitative study, within the “videos’ benefits to the students’ class participation” theme, it was found that the students were able to answer teach- ers’ questions after watching the videos (N = 16). These comparisons showed that the quantita- tive research findings supported the focus group study results, demonstrating that students had high satisfaction with and significant benefits received from watching the videos. 4.2. Limitations Limitation of this focus group study should be noted. First, purposeful sampling was adopted in this study and a homogeneous group of participants was sampled. To further analyze the difference in perceptions between students based on their demographic variables, stratified purposeful sampling could be applied in future studies. Second, because of social distancing

measures, the focus group interviews were conducted online, so physical features of group dynamics could not be observed. Richer analysis of group dynamics could be achieved if focus group interviews could be conducted in a face-to-face setting. 4.3. Implications for teaching practice This study demonstrated the teaching and learning effectiveness of educational videos under a blended learning and flipped classroom arrangement. Education practitioners could consider piloting similar practice in their respective institutions in the future. These educational videos were very valuable under the COVID-19 pandemic [4, 113]. And they could be further used in other situations that require online education (e.g., courses for students in remote areas) [114, 115]. However, it should be noted that the effectiveness of educational videos may be different PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 19 / 28

Table 3. Side-by-side comparison of quantitative and qualitative research findings. Quantitative study Qualitative study Questionnaire items for the 15 animated videos on theories/concepts Positive response rates Corresponding themes in the current focus group study Corresponding findings 1. The animated video is well designed. 98.20% to 94.69% Video design The majority of the students thought that the quality of the videos (both animated and case-based videos) was good (N = 21). 2. The animated video is relevant to the lecture topic. 98.91% to 94.62% Video arrangement In terms of the connection between the theory-related videos and the lectures, students believed that they were closely related because the teachers recapped the theories (N = 16), or discussed related personal experience and real-life examples during lectures (N = 21). 3. The animated video is easy to understand. 98.51% to 95.12% Video design The theory-related videos were very structured and explained the theories clearly. Model explanation was helpful to students’ learning (N = 16). 4. The animated video is interesting. 96.65% to 90.81% Video design Students enjoyed the animation (N = 3), and the interesting animation helped their understanding (N = 10). 5. The teacher used the animated video effectively. 98.60% to 95.03% Video arrangement In terms of the connection between the theory-related videos and the lectures, students believed that they were closely related because the teachers recapped the theories (N = 16), or discussed related personal

experience and real life examples during lectures (N = 21). 6. The animated video has enabled me to understand related concepts. 98.71% to 94.91% Videos’ benefits to the students’ learning of course content The theory-related videos contributed to students’ learning through clear explanation of theories, which helped students to understand the content without prior knowledge. 7. The animated video has deepened my understanding of the topic. 98.43% to 94.42% Videos’ benefits to the students’ learning of course content The majority of the students thought that the videos (both animated and case-based videos) could facilitate their understanding of lecture content (N = 27) and related theories (N = 26). 8. The animated video has enhanced my interests in the topic. 96.64% to 90.62% Videos’ benefits to the students’ learning of course content The animation, layout and video cutting made the theory-related videos more interesting to students. 9. The animated video has helped me to reflect on the topic. 97.75% to 92.46% Videos’ benefits to the students’ learning of course content Some theory-related videos’ content was related to students’ daily lives, so they could stimulate students’ self-reflection. 10. I prefer more such animated video in the teaching and learning of this subject. 97.57% to 94.11% n/a n/a 11. The animated video can benefit my development. 97.94% to 93.58% Videos’ benefits to the students’ learning of course content The skills introduced in the videos were useful to students. 12. Overall speaking, I am satisfied with the animated video. 98.52% to 94.42%

Videos’ benefits to the students’ learning of course content The majority of the students rated the videos (both animated and case- based videos) 7 points (N = 10), 7 to 8 points (N = 7) or 8 points (N = 18) out of 10 points. Questionnaire items for the nine case- based videos Positive response rates Corresponding themes in this focus group study Corresponding findings 1. The video-based activity is well designed. 98.04% to 95.09% Video design The majority of the students thought that the quality of the videos (both animated and case-based videos) was good (N = 21). 2. The video-based activity is relevant to the topic. 98.97% to 97.74% Video arrangement In terms of the connection between the case-based videos and the lectures, the students thought that they were closely related because some teachers discussed the cases during lectures, while other teachers shared personal experience that were related to the videos (N = 20). 3. The video is easy to understand. 98.62% to 97.17% Video content The case-based videos were related to students’ daily lives, so students could understand better and enjoy the videos (N = 7). 4. The content of the video is interesting. 97.02% to 92.48% Video content The case-based videos were related to students’ daily lives, so students could understand better and enjoy the videos (N = 7). 5. The content of the video is stimulating. 98.02% to 96.05% Video content The case-based videos provided various perspectives and students could learn the ways they could solve similar problems in the future (N = 14).

6. The video-based activity has increased my interaction with the teacher. 96.39% to 86.32% Videos’ benefits to the students’ class participation After watching the videos, students were able to answer teachers’ questions (N = 16). 7. The video-based activity has increased my interaction with classmates. 94.74% to 82.60% Videos’ benefits to the students’ class participation Many students perceived that both the theory-related videos and case- based videos could enhance their class participation (N = 16). (Continued) PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 20 / 28

across disciplines [116]. Hence, education practitioners should adjust the video arrangement based on their situations and the needs of their students. Moreover, this study demonstrated that various video design features could enhance univer- sity students’ learning effectiveness, which supported the multimedia learning theories. The mul- timedia learning theories stated that the use of flow charts and tables and the highlighting of key points in multimedia reduced the extraneous cognitive load on learners, which could improve their learning efficiency [86, 89]. Also, the presentation of visual illustrations along with voiceover narrations could increase the learning efficiency of students [32, 90]. These findings not only sup- ported the multimedia learning theories, but also provided design reference for education practi- tioners to develop university-level educational videos in the future. Interested parties could refer to the articles by Castillo et al. [117] and Brame [118] for comprehensive guidelines on the con- tent design and the production of educational videos in a non-studio setting. 4.4. Recommendations for further research The current research was conducted under the COVID-19 pandemic, when universities were forced to adopt synchronous or asynchronous online teaching to sustain teaching and learning amid the pandemic. Research participants in the current study needed to attend online syn- chronous lectures and watched the videos in a flipped classroom arrangement. Future research

was recommended to be conducted in face-to-face or hybrid teaching context [119, 120], so the benefits of educational videos under different teaching and learning contexts could be examined. Moreover, the current study found that different video design features could enhance university students’ learning effectiveness, supporting the multimedia learning theo- ries. Because the application of multimedia learning theories in designing university-level Table 3. (Continued) Quantitative study Qualitative study Questionnaire items for the 15 animated videos on theories/concepts Positive response rates Corresponding themes in the current focus group study Corresponding findings 8. The teacher has used the video effectively. 98.52% to 96.98% Video arrangement In terms of the connection between the case-based videos and the lectures, the students thought that they were closely related because some teachers discussed the cases during lectures, while other teachers shared personal experience that were related to the videos (N = 20). 9. I have been very engaged in the video- based activity. 97.87% to 92.61% Videos’ benefits to the students’ class participation The videos could also be used for class discussion, which increased students’ motivation (N = 7). 10. The video-based activity has deepened my understanding of the topic. 98.32% to 96.87% Videos’ benefits to the students’ learning of course content The case-based videos applied theories to some real-life cases or social issue, so students could understand the concepts very easily. 11. The video-based activity has enabled

me to apply the theories in real life cases. 97.67% to 93.60% Videos’ benefits to the students’ learning of course content The case-based videos were related to students’ daily lives, so they could learn application of the theories in their daily lives. 12. The video-based activity has enhanced my interests in the topic. 97.21% to 92.84% Videos’ benefits to the students’ class participation Students were able to gain more interest in the lecture topics after knowing the content from the videos (N = 20). 13. The video has helped me reflect on the subject matters. 98.04% to 96.07% Videos’ benefits to the students’ learning of course content Students could have self-reflection after watching the case-based videos, which was helpful for them to gain more positive mindset. 14. I prefer more such video-based activities in the teaching and learning of this subject. 97.75% to 94.08% n/a n/a Questionnaire Items on the Effectiveness of the Lecture Positive Response rates Corresponding themes in this focus group study Corresponding Findings 1. Overall speaking, the lecture has enriched my development. 98.78% to 92.77% n/a n/a 2. On the whole, I am very satisfied with this lecture. 98.15% to 92.71% n/a n/a https://doi.org/10.1371/journal.pone.0291861.t003 PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 21 / 28

educational videos was relatively scarce when compared to studies in secondary education context [121, 122], further research should be conducted to strengthen our understanding on this topic. Qualitative research employs a naturalistic approach to understand phenomenon in its nat- ural or context-specific settings [53], so it allows researchers to examine deeper understanding of a phenomenon [123]. The current study demonstrated how the videos benefited students’ pre-lesson self-learning and learning of course content, and enhanced their class participation. More qualitative research or mixed method studies could be implemented in the future, so we could accumulate evidence on how educational videos benefit students’ learning, thus contrib- uting to evidence-based teaching practice [124]. 5. Conclusions This study contributes to the literature by demonstrating students’ positive perceptions of the video arrangement, design and content, and their subjective benefits from watching the 24 instructional videos in a leadership course covering psychosocial competence in a Hong Kong university. First, the study found that the pre-lesson video watching arrangement allowed stu- dents to have flexible self-learning, better understanding of lecture content, and more discus- sion during lectures. The pre-lesson blended learning exercise also engaged students and improved their understanding of lecture topics. Second, students perceived that various video features had contributed to their learning, including the pointers and lists in theory-related

videos and the real-life filming in case-based videos. Third, in terms of video content, similari- ties between the content and students’ daily lives or the social issues could improve students’ learning effectiveness. Fourth, the clear explanation of theories in the theory-related videos and the case-based videos’ down to earth content enabled students’ to self-study the videos before lecture. Fifth, most students believed that the videos could facilitate their understanding of course content. Besides, students perceived that the two kinds of videos complement each other, with the theories explaining the cases and the cases demonstrating the application of theories. Sixth, students perceived that the videos could enhance their class participation because they could have more interest in the lecture topics and answer teachers’ in-class ques- tions after gaining understanding of the course content from the pre-lesson videos. Supporting information S1 Dataset. (DOCX) Author Contributions Conceptualization: Daniel T. L. Shek, Lu Yu. Data curation: Daniel T. L. Shek, Tingyin Wong, Xiang Li. Funding acquisition: Daniel T. L. Shek. Methodology: Daniel T. L. Shek. Project administration: Daniel T. L. Shek, Xiang Li, Lu Yu. Supervision: Daniel T. L. Shek, Xiang Li, Lu Yu. Writing – original draft: Daniel T. L. Shek, Tingyin Wong. Writing – review & editing: Daniel T. L. Shek, Xiang Li. PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 22 / 28

References 1. Dumford AD, Miller AL. Online learning in higher education: Exploring advantages and disadvantages for engagement. J Comput High Educ. 2018; 30:452–465. https://doi.org/10.1007/s12528-018-9179-z 2. Giannakos MN. Exploring the video-based learning research: A review of the literature. Br J Educ Technol. 2013; 44(6):E191–E195. https://doi.org/10.1111/bjet.12070 3. Poquet O, Lim L, Mirriahi N, Dawson S. Video and learning: A systematic review (2007–2017). LAK’18: International Conference on Learning Analytics and Knowledge; 2018 March 7–9; Sydney, NSW, Australia. New York, NY, U.S.A.: ACM; 2018. https://doi.org/10.1145/3170358.3170376 4. Reimers F, Schleicher A, Saavedra J, Tuominen S. Supporting the continuation of teaching and learn- ing during the COVID-19 Pandemic. Organisation for Economic Co-Operation and Development, Paris, 2020. https://www.oecd.org/education/Supporting-the-continuation-of-teaching-and-learning- during-the-COVID-19-pandemic.pdf (accessed 2023-06-01). 5. Zhu X, Liu J. Education in and after Covid-19: Immediate responses and long-term visions. Postdigit Sci Educ. 2020; 2:695–699. https://doi.org/10.1007/s42438-020-00126-3 6. Edwards AAH. From novel to necessary: COVID-19 and the Kairos of bringing technology into the communication class-room. In: Valenzano JM III, editor. Post-pandemic pedagogy: A paradigm shift. Lanham, MD, U.S.A.: Lexington Books; 2023. p. 37–56. 7. Rapanta C, Botturi L, Goodyear P, Guàrdia L, Koole M. Balancing technology, pedagogy and the new

normal: Post-pandemic challenges for higher education. Postdigit Sci Educ. 2021; 3:715–742. https:// doi.org/10.1007/s42438-021-00249-1 8. Rodriguez-Paz MX, Gonzalez-Mendivil JA, Zarate-Garcia JA, Zamora-Hernandez I, Nolazco-Flores JA. A hybrid teaching model for engineering courses suitable for pandemic conditions. IEEE Revista Iberoamericana De Tecnologı´as Del Aprendizaje. 2021; 16(3):267–275. https://doi.org/10.1109/RITA. 2021.3122893 9. Singh J, Steele K, Singh L. Combining the best of online and face-to-face learning: Hybrid and blended learning approach for COVID-19, post vaccine, & post-pandemic world. Journal of Educational Tech- nology Systems. 2021; 50(2):140–171. https://doi.org/10.1177/00472395211047865 10. Nduagbo KC. Online education past, current, and future. In: Association Management, Resources Information, editor. Research anthology on remote teaching and learning and the future of online edu- cation. Hershey, PA, U.S.A.: IGI Global; 2023. p. 1–16. https://doi.org/10.4018/978-1-6684-7540-9 11. Sablić M, Mirosavljević A, Sˇ kugor A. Video-based learning (VBL)—past, present and future: An over- view of the research published from 2008 to 2019. Tech Know Learn. 2021; 26:1061–1077. https://doi. org/10.1007/s10758-020-09455-5 12. Yousef AMF, Chatti MA, Schroeder U. The state of video-based learning: A review and future perspec- tives. Int J Adv Life Sci. 2014; 6(3&4):122–135. 13. Collado-Valero J, Rodrı´guez-Infante G, Romero-Gonza´ lez M, Gamboa-Ternero S, Navarro-Soria I,

Lavigne-Cerva´n R. Flipped classroom: Active methodology for sustainable learning in higher educa- tion during social distancing due to COVID-19. Sustainability. 2021; 13(10):5336. https://doi.org/10. 3390/su13105336 14. Reychav I, Wu D. Mobile collaborative learning: The role of individual learning in groups through text and video content delivery in tablets. Comput Human Behav. 2015; 50:520–534. https://doi.org/10. 1016/j.chb.2015.04.019 15. Wang Y-H. Could a mobile-assisted learning system support flipped classrooms for classical Chinese learning?: Mobile-assisted learning system. J Comput Assist Learn. 2016; 32(5):391–415. https://doi. org/10.1111/jcal.12141 16. Wikandari R, Putro AW, Suroto DA, Purwandari FA, Setyaningsih W. Combining a flipped learning approach and an animated video to improve first-year undergraduate students’ understanding of elec- tron transport chains in a biochemistry course. J Chem Educ. 2021; 98:2236–2242. https://doi.org/10. 1021/acs.jchemed.0c01477 17. Shek DTL. COVID-19 and quality of life: Twelve reflections. Appl Res Qual Life. 2021; 16:1–11. https://doi.org/10.1007/s11482-020-09898-z PMID: 33425064 18. Shek DTL, Leung JTY, Tan L. Social policies and theories on quality of life under COVID-19: In search of the missing links. Appl Res Qual Life. 2023. https://doi.org/10.1007/s11482-023-10147-2 PMID: 36855587 19. Shek DTL, Li X, Yu L, Lin L, Chen Y. Evaluation of electronic service-learning (e-service-learning) proj- ects in main-land China under COVID-19. Appl Res Qual Life. 2022; 17:3175–3198. https://doi.org/10.

1007/s11482-022-10058-8 PMID: 35600112 PLOS ONE Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 23 / 28

20. Summers R, Higson H, Moores E. The impact of disadvantage on higher education engagement dur- ing different delivery modes: A pre- versus peri-pandemic comparison of learning analytics data. Assess Eval High Educ. 2023; 48(1):56–66. https://doi.org/10.1080/02602938.2021.2024793 21. Bhowmik S, Bhattacharya MD. Factors influencing online learning in higher education in the emer- gency shifts of COVID 19. Online J Distance Educ Elearn. 2021; 9(1):74–83. 22. Dhawan S. Online learning: A panacea in the time of COVID-19 crisis. Journal of Educational Technol- ogy Systems. 2020; 49(1):5–22. https://doi.org/10.1177/0047239520934018 23. Simamora RM. The challenges of online learning during the COVID-19 pandemic: An essay analysis of performing arts education students. Stud Learn Teach. 2020; 1(2):86–103. https://doi.org/10. 46627/silet.v1i2.38 24. Rasheed RA, Kamsin A, Abdullah NA. Challenges in the online component of blended learning: A sys- tematic review. Comput Educ. 2020; 144:103701. https://doi.org/10.1016/j.compedu.2019.103701 25. Mheidly N, Fares MY, Fares J. Coping with stress and burnout associated with telecommunication and online learning. Front Public Health. 2020; 8:574969. https://doi.org/10.3389/fpubh.2020.574969 PMID: 33262967 26. Donkor F. Assessment of learner acceptance and satisfaction with video-based instructional materials for teaching practical skills at a distance. IRRODL. 2011; 12(5):74–92. https://doi.org/10.19173/irrodl. v12i5.953 27. Zhang D, Zhou L, Briggs RO, Nunamaker JF. Instructional video in e-learning: Assessing the impact of

35. Yu L, Lin L, Shek DTL, Chai W. Students’ perceived attributes and benefits of a leadership course: Subjective outcome evaluation. Res Soc Work Pract. 2022; 32(2):199–214. https://doi.org/10.1177/ 10497315211042823 36. Shek DTL. Development of a positive youth development subject in a university context in Hong Kong. Int J Disabil Hum Dev. 2012; 11:173–179. https://doi.org/10.1515/ijdhd-2012-0036 37. Ando M. An intervention program focused on self-understanding and interpersonal interactions to pre- vent psychosocial distress among Japanese university students. J Adolesc. 2011; 34(5):929–940. https://doi.org/10.1016/j.adolescence.2010.12.003 PMID: 21251704 38. Lamboy B, Beck F, Tessier D, Williamson M-O, Fre´ry N, Turgon R, et al. The key role of psychosocial competencies in evidence-based youth mental health promotion: Academic support in consolidating a national strategy in France. Int J Environ Res Public Health. 2022; 19(24):16641. https://doi.org/10. 3390/ijerph192416641 PMID: 36554520 39. Phillips TK, Rose AL, Strickland ML. The mediating role of self-leadership in the link between racial identity attitudes, mental health outcomes, and race related stress. International Journal of Systemic Therapy. 2022; 33(3):175–195. https://doi.org/10.1080/2692398X.2022.2084316 40. Li X, Shek DTL. Objective outcome evaluation of a leadership course utilising the positive youth devel- opment approach in Hong Kong. Assess Eval High Educ. 2020; 45(5):741–757. https://doi.org/10. 1080/02602938.2019.1696944 PLOS ONE

Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 24 / 28

41. Shek DTL, Yu L, Wu FKY, Chai WY. General university requirements at Hong Kong Polytechnic Uni- versity: Evaluation findings based on student focus groups. Assess Eval High Educ. 2015; 40 (8):1017–1031. https://doi.org/10.1080/02602938.2014.960362 42. Shek DTL, Yu L, Chi X. Focus group evaluation of teachers’ views on a new general education pro- gram in Hong Kong. International Journal of Adolescent Medicine and Health. 2017; 29:67–74. https:// doi.org/10.1515/ijamh-2017-3009 PMID: 27299208 43. Chai W, Li X, Shek DTL. The effectiveness of a leadership subject using a hybrid teaching mode dur- ing the pandemic: Objective outcome and subjective outcome evaluation. IJERPH. 2022; 19 (16):9809. https://doi.org/10.3390/ijerph19169809 PMID: 36011436 44. Shek DTL, Zhu X, Li X, Dou D. Satisfaction with hyflex teaching and law-abiding leadership education in Hong Kong university students under COVID-19. Appl Res Qual Life. 2022; 17:2833–2858. https:// doi.org/10.1007/s11482-022-10040-4 PMID: 35222736 45. Shek DTL, Li X, Wong T, Yu L, Chai W, Cheng N, et al. Development of virtual teaching and learning videos for leadership education for university students in Hong Kong. Int Public Health J. 2022; 14(4): 377–389. 46. Li X, Shek DTL, Wong T, Yu L. Subjective outcome evaluation of instructional videos in leadership education. IJERPH. 2023; 20(1):367. https://doi.org/10.3390/ijerph20010367 PMID: 36612689 47. Shek DTL, Li X, Wong T. Video teaching materials for leadership education: Subjective outcome eval-

uation based on university students in Hong Kong. International Journal of Child and Adolescent Health. Forthcoming 2023. 48. Draper AK. The principles and application of qualitative research. Proc Nutr Soc. 2004; 63:641–646. https://doi.org/10.1079/pns2004397 PMID: 15831137 49. Hara K. Quantitative and qualitative research approaches in education. Education. 1995; 115(3):351– 356. 50. Krueger RA, Casey MA. Focus groups: A practical guide for applied research. 4th ed. Thousand Oaks, CA, U.S.A.: Sage Publications; 2009. 51. Ishak T, Kurniawan R, Zainuddin Z, Keumala CM. The role of pre-class asynchronous online video lec- tures in flipped-class instruction: Identifying students’ perceived need satisfaction. Journal of Peda- gogical Research. 2020; 4(1):1–11. https://doi.org/10.33902/jpr.v4i1.145 52. Scagnoli NI, Choo J, Tian J. Students’ insights on the use of video lectures in online classes: Students’ insights on video lectures. Br J Educ Technol. 2019; 50(1):399–414. https://doi.org/10.1111/bjet. 12572 53. Patton MQ. Qualitative research and evaluation methods. 3rd ed. Thousand Oaks, CA, U.S.A.: Sage Publications; 2002. 54. Shek DTL, Tang VM, Han XY. Evaluation of evaluation studies using qualitative research methods in the social work literature (1990–2003): Evidence that constitutes a wake-up call. Res Soc Work Pract. 2005; 15(3):180–194. https://doi.org/10.1177/1049731504271603 55. Bogdan R, Biklen SK. Qualitative research for education: An introduction to theories and methods. 5th ed. Boston, MA, U.S.A.: Pearson Education; 2007.

56. Creswell JW, Poth CN. Qualitative inquiry and research design: Choosing among five approaches. 4th ed. Thousand Oaks, CA, U.S.A.: Sage Publications; 2018. 57. Joinson AN. Self-disclosure in computer-mediated communication: The role of selfawareness and visual anonymity. Eur J Soc Psychol. 2001; 31:177–192. 58. Opdenakker RJG. Advantages and disadvantages of four interview techniques in qualitative research. Forum Qualitative Sozialforschung = Forum: Qualitative Social Research. 2006; 7(4):Art.11. 59. polyu.edu.hk. [Internet]. Hong Kong: The Hong Kong Polytechnic University; c2022 [cited 2023 Aug 12]. Available from https://www.polyu.edu.hk/en/ipao/facts-and-figures/. 60. Agius SJ. Qualitative research: Its value and applicability. Psychiatrist. 2013; 37:204–206. https://doi. org/10.1192/pb.bp.113.042770 61. Noble H, Smith J. Issues of validity and reliability in qualitative research. Evidence-based Nursing. 2015; 18(2):34–35. https://doi.org/10.1136/eb-2015-102054 PMID: 25653237 62. Long J, Liu TQ, Liao YH, Qi C, He HY, Chen SB, et al. Prevalence and correlates of problematic smart- phone use in a large random sample of Chinese undergraduates. BMC Psychiatry. 2016; 16:408. https://doi.org/10.1186/s12888-016-1083-3 PMID: 27855666 63. Browning MH, Larson LR, Sharaievska I, Rigolon A, McAnirlin O, Mullenbach L, et al. Psychological impacts from COVID-19 among university students: Risk factors across seven states in the United States. PloS one. 2021; 16(1):e0245327. https://doi.org/10.1371/journal.pone.0245327 PMID: 33411812 PLOS ONE

Qualitative evaluation of virtual teaching and learning videos in leadership education PLOS ONE | https://doi.org/10.1371/journal.pone.0291861 September 21, 2023 25 / 28