Best Practices for Online Course Development in Distance University Settings: Insights from Instructors’ Experiences

This research focuses on K Distance University, South Korea’s national open university, established in 1972 as the country’s first higher education institution dedicated to distance learning. K Distance University operates 47 regional campuses and learning centers nationwide, with an annual enrollment of approximately 100,000 students, ranging from teenagers to individuals in their 90s. Over the past 50 years, the university has evolved into a mega-university with more than 700,000 alumni, continuously offering high-quality education designed to support lifelong learning. It stands as a leading example of a successful open university model in both scale and impact.

Twelve courses were identified as exemplary from a pool of over six hundred lectures offered at the institution under study. As outlined in the research design, the selection criteria included course evaluation results, recognition through awards for excellence in online content design, and a thorough review of recorded lectures to identify best practices in online instruction. <Table 1> presents the list of interviewees along with basic information about the selected courses, which were chosen based on the three-tiered evaluation process.

<Table 1>

Participant list and course information

Text mining was conducted to identify the main themes emerging from the interview data, with a focus on noun keywords to derive the characteristics and related concepts of excellent online courses. The process involved three key stages: pre-processing, keyword frequency analysis, and topic modeling, including topic-word network analysis. Initially, the transcribed interview texts were organized in MS Excel, where pre-processing tasks—such as correcting typographical errors and removing stopwords, such as numbers, particles, and pronouns, were carried out. Subsequently, noun keywords were extracted using NetMiner (v 4.4), a specialized tool for network and text analysis. To further refine the data, three types of keyword dictionaries—‘thesaurus,’ ‘defined words,’ and ‘exception list’—were developed. In the thesaurus, terms with differing spacing but identical meanings, such as ‘media lecture’ and ‘media-lecture,’ were standardized, while variations of the term ‘PD,’ such as ‘PDnim,’ ‘PD,’ and ‘Director,’ were consolidated under a single representative form. The defined words dictionary incorporated institution-specific terminology, including terms such as ‘formative evaluation’ and ‘Prime College.’ Lastly, the exception list was used to exclude frequently occurring but irrelevant terms, such as ‘student,’ ‘class,’ and ‘professor,’ along with department and course names that did not contribute to the research findings.

Second, keyword frequency analysis was performed to derive the main keywords and the appearance frequency that appeared in the interview data. The larger the term frequency (here after TF), the more important the keyword, but there is a possibility that a keyword that is meaningless is included. To compensate for this, TF-IDF (Term Frequency-Inverse Document Frequency) analysis, which means the importance of words, can be used (Seo & Kyun, 2021). In this study, there was no significant difference between the values of TF and TF-IDF, so the analysis was conducted focusing on the TF value.

Third, topic modeling was performed to extract major topics. Topic modeling is a method for calculating the main topics latent in a document and extracting meaningful information through stochastic statistical analysis. In this study, we applied a representative algorithm, Latent Dirichlet Allocation (LDA) (Blei, Ng, & Jordan, 2003). This is a Bayesian approach (Blei, 2012) that infers to what probability the keyword of the original data is distributed through the results of the observed data. In other words, based on the distribution of keywords allocated to individual documents, the topic that occupies the largest proportion is designated as the representative topic of the relevant document (Seo & Kyun, 2021). Meanwhile, before LDA analysis, the optimal number of topics k was derived using FindTopicsNumber in the R ldatuning package (Arun et al., 2010; Cao et al., 2009; Deveaud, SanJuan, & Bellot, 2014; Griffiths & Steyvers, 2004). FindTopicsNumber has been widely utilized to estimate the optimal number of topics by comparing indices from Arun et al.(2010), Cao et al.(2009), Deveaud, SanJuan, & Bellot(2014), and Griffiths & Steyvers(2004) (Baek, 2020). Based on this approach, we identified a total of six optimal topics, considering factors such as topic overlap, word similarity within each topic, and the appropriateness of topic labels. The final topic names and key keywords were determined through discussions among a research team composed of four PhDs in education. Lastly, the detailed keywords for each topic derived from the topic modeling were visualized using a network map. In the network map, each keyword is displayed in a radial network connected by links, enabling easier and more structured identification of the relationships between various topic-specific keywords present in the document (Lee & Eom, 2023; Paranyushkin, 2010).

A qualitative analysis was conducted to examine the key characteristics and critical elements necessary for the successful implementation of best practices in online course delivery within a distance university setting. Two distinct types of data were collected for this study. The first type consisted of interview transcripts with instructors, supplemented by questionnaire data aimed at gaining deeper insights into the processes of course development and administration. The second type included field notes documenting the instructors’ reactions and gestures, as well as the researchers’ observations. The interviews were transcribed using Naver ClovaNote.

The collected data were analyzed using the constant comparative method, involving continuous comparative review through collaborative discussions among the research team. The iterative comparative analysis followed the stages of open coding, categorization, and category validation(Yoo et al., 2018), and was employed as the methodological framework for analyzing the experiences and expertise of instructors involved in the development and implementation of best practice-based courses. During the open coding stage, interview transcripts and field notes were consolidated into an Excel file, where the development and operational processes of exemplary courses were thoroughly reviewed. Key phrases that best represented the content were systematically coded. In the categorization stage, thematic elements identified through open coding were compared to analyze similarities and differences. Similar themes were grouped into higher-order categories, and appropriate category labels were assigned. During the category validation stage, the categorized data were cross-referenced with the original data to verify their appropriateness, ensure accurate representation of the content, and identify any necessary additions or modifications. In conclusion, this study explored the experiences of instructors who developed and operated best practice-based online courses. The research examined individual instructors’ efforts and expertise throughout the course development and operation processes, and conducted a qualitative analysis to identify the primary characteristics and key elements underpinning the successful implementation of exemplary course practices. To ensure the validity and reliability of the research findings, the results were validated through peer review by two education experts with doctoral degrees, thus ensuring the rigor and credibility of the study’s conclusions.

<Table 2>

Interview protocol

The first analytical theme revealed in the interviews was related to sharing the strategic ways of course design and development. Interviewees emphasized that they try to consider learners’ characteristics and the need to improve their motivation for learnings. Professor C highlighted that a “Customer-oriented perspective” is necessary for designing the course:

From a customer-oriented point of view, the concept of the quality of such products might not be satisfied enough. So no matter how hard I tried, and even if I felt that I emphasized enough content, that might be from the perspective of the producer⋯As it turns out, students may not understand the contents of the lecture. So I thought it was important to always think from the perspective of the customers··· (Professor C)

To understand who the learners are and what characteristics they have, instructors will not only meet them in person through offline attendance classes in the past, but also identify their age, employment, and related fieldwork. Professor H stated that he once redesigned the whole course after incorporating students’ comments and opinions.

I used to put all the theory and practicum courses in one week. But then, I was told that my words were too fast and hard to understand especially in the second half of my lectures. Maybe it was because I wanted to cover all content and practicum in such a short period. I thought that I should divide the theory and practicum classes into different weeks. Students’ feedback seemed to be positive after reorganizing the lecture in that way. (Professor H)

However, instructors who participated in the interview said that it is not easy to meet students’ needs because of the diverse characteristics of students and different levels of academic abilities. Many of the instructors who were interviewed were trying to develop their courses including various course activities and offer additional support to stimulate learners’ motivation. Professor F mentioned his strategic know-how as follows:

I try to ask questions to students at the end of the lectures like⋯after listening to this lecture today, let’s think about this and that⋯ It could be some wrapping up questions or just raising a problem..Because we used to offer the lecture and end it without providing students the chance to think and review the contents. Students listen to the lecture for 45 to 50 minutes and go away. (Professor F)

Unlike other universities, K Distance University offers textbooks that are usually written by their affiliated professors and a large portion of the lectures are primarily based on those textbooks. Lectures are usually developed after the textbooks are developed in the order of the “pre-textbook development and post-filming system.” The instructors said that they always think about how much and how to reflect the contents of the textbook in the lecture, and that although each instructor has different strategies, they try not to simply summarize the contents of the textbook.

On the other hand, it is difficult to decide how much the contents of the textbook should be reflected in the lecture, especially when the writers of the textbook and instructors are different. This is because the important points of the lectures may vary by instructor. In this case, Professor E revealed that he intensively includes content that he considers important in the lecture.

Rather than being completely bound by the textbook, I sometimes want to emphasize other contents that are not covered in the textbook. That’s why as a professor, I focus a lot on what I want to highlight in each lecture, not just limiting the contents based on the textbook. (Professor E)

Participants also revealed that they are concerned about how to organize lecture videos when designing lectures. To help learners better understand the contents of the lecture, instructors use various materials and formats when organizing the lectures. Instructors sometimes use orientation videos in the first week of the course introducing the entire subject. Some of them provide animations when explaining important concepts or theories so that students could be more interested and focused during class. Professor I and Professor G mentioned their experience as follows:

Almost all of my subjects had orientation sessions, so I told the students in advance about the overall evaluation, questions, and so on, and that’s how I started. (Professor I)

Even though the textbooks and lectures have changed, we keep creating animation to provide lectures using various formats⋯ Students used to say that this is the most difficult subject that they ever had, but I recently I rarely hear that. (Professor G)

In addition, the instructors who participated in the interview thought about how to connect the contents of the test and textbook to the lecture video, and designed the course in consideration of the functions and roles of the subject in the entire curriculum of the department.

Filming lectures is a special and unique characteristic of lectures provided by open and distance universities such as K Distance University. Since lectures have to be put into film and recorded, it can be seen that unlike face-to-face lectures conducted in real-time, the instructor's efforts for filming are required. Professors and instructors who offer online courses at K Distance University have different experiences compared to regular universities when developing lectures and may need their own know-how and strategies in the process of course production and filming.

Participants confirmed that it was necessary to prepare thoroughly for the lecture before recording. Professor K highlights his experience as follows:

When I’m filming, I think media lectures need to be prepared in advance. Sometimes I can’t think of words. You can’t stutter at times like that. So I tend to prepare a lot. You have to prepare these things well and put them on a tight schedule in 45 minutes. (Professor K)

Some of the instructors who participated in the study shared the know-how that they prepared scripts in advance of filming.

I wrote a script after a while. To give a lecture on what I thought, I wrote a script and read it like this. Not to make a mistake. (Professor J)

When I first came here··· I don’t know if it’s meaningful to talk about the past, but since I recorded it, I wrote down all the manuscripts during the first lecture. So I used to write down everything and read it. (Professor I)

I prepare scripts for graduate school classes. If you have to explain the topic very clearly, preparing a script before filming helps a lot. (Professor K)

Most of the instructors who participated in the study were able to monitor and control the situation when recording the lectures. Professor D revealed that the more experience he had in filming, the more know-how he accumulated at the filming site, and he became more comfortable in filming than before.

With my experience, my ability to control the situation of the lecture has become much better. ⋯My ability to deal with certain situations during recording improved when I got used to them. As it got better, I was so obsessed with delivering something in the past that I was in a hurry to deliver it, and I think my ability to be more controlled and comfortable has improved···In the past, I would have been busy delivering the contents without thinking about it, but now I have gained the ability to control and manage the situation. (Professor D)

In the case of Professor E, he imagined that the students were there with him just like he was offering lectures in face-to-face settings.

I did the same thing as offline lectures. Of course, when I give offline lectures, there are a lot of on-site questions and answers, but in fact, during filming, students are not with you. But sometimes I ask questions to students imagining students are just sitting there, which helps me a lot. (Professor E)

It was confirmed that the instructors who participated in the study were paying attention to using appropriate voice tone, speed of speech, and gestures when filming the lecture.

My voice tone is kind of low. Since one of my weaknesses is that my pronunciation is not clear, Producers adjust the tone to an appropriate level, but I keep trying to pronounce it clearly⋯So when I give lectures, I try to make sure that I don’t have any bad habits. (Professor D)

Collaboration with the producer is essential in the process of lecture production. Interviewees shared various cases about collaborating with producers to create a better quality of lectures. For example, instructors introduced that there were cases in which various production ideas proposed by the producers were applied to lectures and received a lot of favorable responses from students.

There was a producer who made one of the orientation videos really professionally. You know the first lecture is very important. The producer made the orientation video and I thought it was professional. Students also liked it and said it was like watching a movie. That was a long time ago so maybe students may not have seen much of that in other classes. I mean, it was so new and I loved it. (Professor C)

I think it was really nice that the producers gave me a variety of ideas when I was planning a lecture. I think it was really good that they helped me develop the lecture relevant to the subject since there aren’t that many things we can do in the video. It’s not a lot, but it was nice to talk about things that we could do within that range··· (Professor G)

The instructors who participated in the study mentioned that it may be necessary to respect professionalism and listen to each other’s opinions in the collaboration process. Professor B believes that instructors and producers can demonstrate professional skills in each area and supplement each other’s deficiencies to help create more excellent lectures.

I think that most of the instructors are not very interested in offering ideas in the filming process. But in fact, I had an experience collaborating with the producer very well since I tried to be very actively involved in sharing ideas which create diversity here. (Professor B)

The last theme that converges on implementing effective online lectures relates to a process of course management and operation. Many of the instructors considered offline classes important in running the lecture, and it was confirmed that they were making various efforts to design the offline class effectively. For example, rather than using the offline class as an explanation or question-and-answer time for media lectures, instructors used it as an opportunity to provide additional necessary materials to students or deepen their understanding of the contents of media lectures. Professor E says that he aims to operate offline classes that do not repeat the existing media lectures.

As for the attendance class, I think it’s right not to repeat this if the instructor who runs the lecture conducts the attendance class. Actually, during the attendance class, I didn’t want to do the same thing that the professor who recorded the lecture. It’s the same content, but it’s different, so I think the lecture will come out differently even if it’s the same content. (Professor E)

Evaluation in the operation of lectures can be said to be the area that both instructors and students are most interested in. Professors who responded to the interview said they were thinking about the scope of the final exam, evaluation methods, and evaluation criteria. They often said that the final exam was based on textbooks considering students’ various circumstances.

When the exams were asked from outside of the textbook and say that I did additional content in the lecture, there are a lot of complaints about the questions. That’s why I do my questions thoroughly based on textbooks. When some questions were based on the lecture itself, students complain a lot. (Professor D)

Interactions with students were also one of the important issues in running lectures in open university settings. Based on interviews, it is obvious that instructors interact with students in various ways and listen to their reactions. Students often post their thoughts on the contents of the lecture or ask questions after watching the media lecture posted by the instructor. Professor I who interacts with students using Google Group said that students’ responses are increasing over time.

I’m also doing Google Groups. The reason I’m doing that is that in our school system, for example, if you want to deliver something to students, it’s completely one-sided to upload it as a notification. But when using Google groups, it’s sent to all the students who take that subject. After a year or two, the students didn’t reply much at first, but now they answer more frequently than before. I get a lot of emails. (Professor I)

Instructors are trying to improve lectures by reflecting on the feedback they receive from students regarding them. When producing media lectures, the students’ feedback was often immediately reflected as it was sent to the producers and used to check the contents of the lecture.

When students mentioned that there might be some mistakes in the lecture, I immediately check it right away. I sent it to the producer saying that I heard there was an error, so I ask them to fix that. (Professor G)

I have the plan to reorganize the subject this time, so I am trying to create it by reflecting the opinions of the students. I think the strength of our school’s lecture production system is that we can collect students’ opinions and reflect them in the next phase of production. So, next year, I plan to produce my lectures with these aspects in mind. (Professor H)

According to the results of analyzing the experience of instructors in the process of course management and operation, instructors used to think of offline classes and evaluation as important areas. In addition, it was revealed that the lecture management platform, U-KNOU Campus, had some limitations interaction functions with students, but it was trying to interact with students in a way that instructors prefer and reflect students’ feedback on lectures in lecture improvement.