Student Performance in an OnLine General College Chemistry Course

INTRODUCTION

Overview

Distance education occurs when distance and/or time constraints prevent the student and instructor from meeting face-to-face in a classroom setting. Distance education is an alternative for the student who does not have ready access to a college or university, who has restricted hours for course participation, or who simply dislikes the school environment. Education at a distance often involves more student learner dedication and initiative than traditional courses (1). Distance education that utilizes the World Wide Web (WWW) makes course access available anywhere in the world using a personal computer with Internet access. Asynchronous bulletin boards allow students and faculty to participate in discussion at any time and to view all previous archived conversations (2). Chat rooms facilitate synchronous discussion of concepts with peers and the instructor. Finally, media and simulation capabilities available over the WWW provide unique opportunities to enhance the student's learning experience.  

The nontraditional student population is growing across the country at both community colleges and four-year universities. Today's student population is older, is often married or divorced, often has child care issues, commutes instead of living on-campus, is employed (often full-time), and is interested in a variety of degree options. Many corporations are encouraging their employees to further their education because they have realized that education is the key to a motivated and loyal workforce (3). The natural market for distance education lies in continuing and professional education for working adults. Almost half of all people enrolled in higher education in the United States are in part-time classes or training programs. This is one reason why more and more prestigious universities in the U.S. are now offering continuing education programs and it suggests that, in the future, the most successful higher education programs will be those that successfully accommodate the growing number of nontraditional students.

The following needs are associated with nontraditional students: differences in learning styles and preferences regarding technology and its relation to education. The current diversity in student populations, student needs, instructor expertise and experience, and available technology will necessitate a shift away from the traditional modes of either all local course delivery or all distance course delivery. A hybrid approach to education that will blend on-campus education with distance education is needed to meet the needs of this growing nontraditional student population.

Statement of the Problem:

Offering a distance learning experience in chemistry that includes a laboratory component poses some unique challenges. Chemistry is an experimental science and most of the concepts in chemistry are formulated from experiments and observations made in the laboratory. Therefore, the single most challenging aspect of offering a general chemistry course at a distance is providing a meaningful laboratory experience for the distance students.

Several attempts have been made to solve the dilemma for distance chemistry students. Some universities and colleges have not required a laboratory component for their distance education courses. Courses of this type are of limited benefit to distance students, since most science disciplines that require chemistry such as biology, medicine, engineering and environmental sciences generally require the laboratory. Moreover, exposure to experiential science is an integral part of most liberal arts education programs, and it plays an even greater role in technical and vocational educational programs. In order for students to gain an education anytime and anywhere in any chosen discipline, they will need exposure to at least one science course with a pedagogically significant laboratory component. Distance education offerings in chemistry will probably never rival those of other disciplines, but the core course listings must increase so that the alternative of distance education for other disciplines can continue to grow.

Some institutions have continued to require the distance students to come on-campus once a week to participate in a traditional laboratory experience. The on-campus lab offers the convenience of providing all the needed chemicals and equipment as well as face-to-face interaction with a lab instructor. Weekly meetings also provide continuity and regular deadlines for distance learning students who might otherwise lack the discipline to keep up with the course material. However, this approach may not be realistic for the student who has chosen to take the distance class because he/she is unable to travel to campus regularly (5).

Institutions have also relied on home experiments by mailing chemistry kits to their distance students. The choice of chemicals for the home kits has been quite limited because of the safety and disposal issues that this approach presents. Another problem with home-based experiments is the small number of quantitative experiments that have been available; thus, the focus has primarily been on qualitative analyses resulting in an inaccurate picture of the relationship that exists between chemistry concepts and experimental data. Home-based lab experiments can also be problematic for science majors who must gain experience performing experiments in a laboratory setting with standard chemicals, nationally approved procedures, and modern equipment.

Another approach to solving the challenge of providing a distance learning laboratory experience has been to provide students with simulated experiments that contain computer-generated and/or pre-obtained data. Simulations can provide the quantitative aspects of laboratory experimentation to some extent, but they do not allow for human error and are limited to an observational paradigm. Most simulations provide little, if any, opportunity for variations (6).

Limitations of This Study:

In the project described in this paper, the various approaches outlined above were adapted to provide a distance learning chemistry course designed to take advantage of the unique learning environment provided by community colleges. Students received lecture materials over the World Wide Web (WWW) and were required to meet on-campus once a week to perform a kitchen chemistry laboratory. During this period, homework problems were discussed, quizzes were given, labs were performed, and the lab data were analyzed.

In this study, the effectiveness of the hybrid approach was assessed by comparing the performance of the community college students to university students on a common final exam. The final exam is attached as Appendix A. The results demonstrate that the hybrid model of web-delivered lecture materials and weekly lab meetings provides an effective alternative to the traditional approach for community colleges.

EXPERIMENTAL METHOD

Research Objectives:

The primary objective of this study was to compare the performance of community college students in a distance learning general chemistry course to university students in a traditional on-campus general chemistry course. Final exam scores were used as a basis of comparison. A secondary objective of this study was to identify similarities and differences in student performance on specific categories of the exam questions. The categorization of the final exam questions is explained in further detail below. Finally, this study was designed to determine if there are identifiable factors that are significant predictors of final exam performance.

The null hypothesis for this study is that there is no statistically significant difference in student performance for community college students enrolled in an online general college chemistry course when compared to university students enrolled in a traditional on-campus general college chemistry course. The null hypothesis would be rejected if calculated p values were less than a = 0.05, i.e. the null hypothesis would be rejected if there is less than a five percent chance that differences in final exam performance for the two groups occurred by chance only.

Data Analysis:

Data analysis was performed using analysis of variance (ANOVA), Pearson Product Moment correlation, and multiple linear regression.

Experimental Groups:

The traditional on-campus class, Chm 101 Sec 03, was taught during the fall 1998 semester at the University of North Carolina at Wilmington (UNCW), a midsize university located in Wilmington, North Carolina. The instructor, who has more than twenty years of teaching experience, also developed the web-based lecture material for the distance education class. The web-based lecture material is a modified version of his lecture class notes which were also delivered by computer presentation in the traditional lecture setting. The class met twice a week, on Tuesdays and Thursdays, for one and one-quarter hours each, and had a separate three-hour weekly lab component. One hundred seventeen students completed the final exam for this course. These students were a traditional class for a four-year university. Most were 18-19 years old, single, and either weren't working or were employed on a part-time basis only. The course syllabus is provided as Appendix B.

The laboratory component of this course was a traditional chemistry lab employing laboratory-grade chemicals and modern laboratory equipment. Since the laboratory component of this course was a separate course, there was no attempt to coordinate the lectures and the labs. The lessons and concepts learned in the lecture part of the course were not explicitly integrated into the laboratory part of the course. The laboratory component was supervised by a UNCW graduate teaching assistant (TA), not by the lecture course instructor. Lab quizzes and exams did not include content from the lecture portion of the course nor did lecture quizzes and exams include content from the laboratory portion of the course. The students received a separate grade for each portion of the course - lecture and laboratory.

The online distance education class, Chm 151 Sec INT, was taught at Cape Fear Community College (CFCC) during the fall semester of 2000 and during the spring semester of 2001. CFCC is a large state community college also located in Wilmington, North Carolina. This course was the first class for which the instructor had sole responsibility, although she had eight years of experience as the lab instructor/technician for the Chemical Technology program at CFCC. The online lecture materials were an updated version of the face-to-face lecture class notes from the 1998 fall semester course at UNCW. A total of 25 students completed the final exam for these two semesters. While this was a somewhat typical class for the community college, most of these distance education students were demographically different from the students in the traditional class at UNCW. The average age of the students in this class was 24.6 years old, about five years older than the university students. The age range was from 17 to 54 years old with most of the students being in their early twenties. Twenty-four percent of these students were married or divorced and also had children. Eighty percent of these students were employed (44% full-time and 36% part-time). See Table 1 below for the CFCC student data breakdown. The common course syllabus for these classes is attached as Appendix C. Students had access to the course syllabus, calendar, practice quizzes and exams, and could email the instructor directly from the course homepage. A snapshot of the course homepage is attached as Appendix D.

The calendar was designed to help the students stay on-track in the course. The spring 2001 calendar is attached as Appendix E. The calendar consisted of suggested lessons, activities, readings from the text, and homework problems and was believed to be in the best interest of the average community college student based on the instructor's knowledge of students' study habits. The students were expected to work on the lessons for a minimum of one hour three times per week. A sample lesson (Lesson 3: What Chemists Measure) is attached as Appendix F.

The distance class met on-campus every Saturday morning for three hours. This on-campus period was designated as the laboratory component of the course and the kitchen chemistry labs were performed on-campus with the assistance of the instructor. The laboratory component was traditional in the sense that it was on-campus and an instructor was present. The kitchen chemistry labs that were designed to be done in the student's home were performed as a part of the Anytime Anywhere Chemistry Experience (AACE) project to see if a totally distant online general chemistry course is practical, but the answer to that question is not being addressed in this study. These labs utilized chemicals (for example baking soda, vinegar and cooking oil) and equipment (meat thermometer, styrofoam cups and aluminum muffin tins) that could be purchased at a local grocery, drug or hardware store. The evaluators of the AACE course concluded that the AACE course met the objective to provide learning opportunities that are equivalent to or surpass the traditional introductory chemistry course (7). The checklist used by the evaluator is attached as Appendix G. Although the students were allowed to work on most of the lab experiments in groups of two students, each student was responsible for his/her own lab report which included an objective, the data usually compiled in an Excel spreadsheet, and a results/discussion paragraph. Lab reports were due the Wednesday after they were performed and most students sent their reports via email to the instructor. Students were allowed to work on their lab reports in a nearby classroom that was equipped with computers during the laboratory period. Many students were able to complete and turn their lab reports in to the instructor the same day.

The labs were designed to link directly with the lecture material. For example, the burning calories lab demonstrated that the energy stored in foods and commonly referred to on labels as calories is the energy released during chemical reactions. Although most students reported that the energy calculated from the temperature rise of a can of water heated by burning a pecan accounted for only 50 - 75% of the calories reported on the pecan label, they were able to conceptualize the concept that stored chemical energy was converted to thermal energy in the combustion reaction. They also appreciated that the discrepancy was due to the loss of heat to the surroundings using the kitchen method of experimentation. Kitchen labs were designed to be as quantitative as possible. In the density lab, students measured the density of Crisco vegetable oil and Karo syrup and compared their values to those reported at the manufacturer's websites. Most students reported densities within five percent of the manufacturer's value.

In addition to the lab experience, the instructor utilized this weekly three-hour period to review homework assignments, discuss important concepts, and administer a quiz covering the previous weeks' lessons. Examples of the quizzes are attached as Appendix H. As mentioned above, online quizzes and practice tests were also available to the students on the website. At the completion of these web-based quizzes and tests, the students could review the correct answers along with an explanation if they had not marked the correct response. The lecture and laboratory were treated as one integrated course at the community college with the laboratory component accounting for thirty percent of the student's final grade.

Since the community college instructor met with the students on a weekly basis, the instructor knew each student and was personally involved with each student. The instructor was available to the students during the week during on-campus office hours and via email. Many of the students were also taking other on-campus courses and would drop by to see the instructor during the week to ask questions regarding the lecture and lab material. A few of the students were not on-campus during the week due to their work schedules, but they would email the instructor during the week with various questions.

The same final exam was administered to the fall 2000 and spring 2001 class at CFCC as was administered to the fall 1998 class at UNCW. The questions on the final exam were categorized as one of four types: (1) algorithm - category A, (2) conceptual - category C, (3) definition, rule or memorization - category D and (4) quantitative - category Q. This distinction of question categories has been used in other studies (8). Category A (the algorithm questions) includes those questions that involve a step-by-step procedure. Category C (the conceptual questions) includes those questions that involve interpreting graphs and pictorial representations, as well as those that require the student to apply reasoning based on previously developed ideas. Category D (the definition, rule or memorization questions) includes those questions that require a definition, a rule or the memorization of some bit of information. Category Q (the quantitative questions) includes those questions that require calculations or that involve the manipulation of an equation. The questions were categorized by three UNCW chemistry students, two graduate students and one undergraduate student. The three students independently rated each of the final exam questions. The interrater agreement for the question categories was 0.63.

RESULTS

As discussed above, 117 students completed the traditional course at UNCW and a total of 25 students completed the online course at CFCC. Because the sample sizes were relatively small for CFCC (nine students in the fall 2000 semester and sixteen students in the spring 2001 semester) and the courses were taught by the same instructor using the same hybrid design and course materials, the CFCC student data were combined. The instrument used to measure student performance was the final exam (students in both courses took the same final exam). Because combining the CFCC student data from the fall semester of 2000 with that of the spring semester of 2001 lowered the average final exam score, combining these two groups of students did not bias the results reported in this study. The null hypothesis for this study postulated that there would not be a statistically significant difference in student performance in the hybrid general college chemistry course offered at the community college when compared with a the traditional on-campus course offered at the university. Our results indicate that the null hypothesis should be rejected.

Statistical Analysis:

The final exam mean score for each class is shown in Figure 1. The 1998 UNCW traditional class had a mean score of 61.98, while the mean score for the combined CFCC student population was 75.84, see Figure 2. A t-test, two sample assuming equal variances, was run for the UNCW and combined CFCC final exam scores. As shown in Table 2, this test indicated that the difference in mean scores is statistically significant, p < 0.001, between the two groups and indicated that the online students outperformed the traditional students.

To further analyze the differences between the traditional and online student populations, a Two-Factor Analysis of Variance (ANOVA) with replication was done using Excel. Because this statistical test required that the size of the two populations be the same, 25 UNCW students were randomly selected from the traditional population using random numbers. The 72 questions on the final exam were divided into four categories, A, C, D, and Q, as described above and the percent (%) correct for each question type was determined for each student.

This provided 100 measurements for each population, traditional vs online, of 25 students. The results are shown in Table 3. For each question type, the average score was higher for the online students at CFCC than for the traditional students at UNCW. The sample p-value was < 0.001, indicating a statistically significant difference between the two student populations. However, the columns p-value of 0.632 indicates that there was not a statistically significant difference across the question types. This finding may be due in part to the low interrater agreement (0.63) which indicates poor discrimination among the question categories.

DISCUSSION

The results of these analyses indicate that the students in the general chemistry course offered at CFCC outperformed their UNCW counterparts on a common final exam. This result indicates that a hybrid model for offering general chemistry can provide a meaningful alternative to traditional methods. Thus it is important to determine the factors that contributed to the success of the community college students.

As discussed above, the three-hour laboratory period was utilized for more than doing the kitchen chemistry labs. Because the instructor anticipated that the online students would need some structure to keep them on-track and up-to-date with the lessons, a review of some of the previous weeks' homework assignments was done during the lab period. Quizzes designed to test the students' comprehension of the important concepts covered in the previous weeks' lessons were given almost every week. The quizzes are attached as Appendix H. The students in the fall semester of 2000 reported to an independent evaluator during a focus group at the end of the course that they did not like the weekly quizzes. This dislike stemmed from the requirement to attend the weekly meetings in order to take the quizzes. The students stated that they had taken the online course specifically to have flexibility in their schedule and the weekly quizzes reduced that flexibility.

 Other factors that might predict student performance on the final exam were their overall grade point average (GPA) and their performance on the laboratory experiments. To assess the importance of each of these factors, a multiple linear regression analysis was performed. As shown in Table 4, this analysis indicates that the quiz scores (p-value of 0.016) were the only statistically significant predictor of final exam performance and suggests that the quizzes contributed to the success of the students on the final exam. It is possible that the weekly quizzes helped emphasize to the students that it was important for them to stay up-to-date with the lessons.

The online courses at CFCC were small classes with nine and sixteen students respectively, but the traditional UNCW course consisted of 117 students. The CFCC instructor knew every student by name, while the UNCW instructor knew the names of very few of his students. Many of the CFCC students came by the instructor's office during regularly scheduled office hours to ask questions, but it was a rather rare occasion for a UNCW student to see the instructor during office hours. The CFCC students also utilized email to contact the instructor on a regular basis. The CFCC instructor made a conscious effort to reach out to any student who appeared to be lagging behind the class. While instructor-to-student interaction cannot be measured mathematically, it is commonly believed that small classes with a high ratio of instructor-to-student interaction greatly influence student success in any course. We believe that this mentoring relationship between the instructor and the students played an important role in the success of the students in the online course.

The students in the online course were expected to utilize the lecture materials on the web. These materials contained an outline and explanation of the important concepts for each lesson, as well as animations and videos. Although the students reported on an end of course survey administered by the instructor that they studied only a few hours per week, they apparently were more actively engaged in learning than their university counterparts. Many traditional students mistakenly believe that simply attending class on-campus equates to some type of learning.

The CFCC instructor reported that 100% of the students attended the final exam review. The review was held during a regularly scheduled laboratory period, the Saturday immediately prior to the final exam. The UNCW instructor estimates that only 25% of the traditional students attended his review session which was held the morning that the final exam was given.

The online CFCC students were required to attend a minimum of 80% of the Saturday laboratory periods in order to complete the course. The traditional UNCW students had no attendance requirement and the instructor estimated that only 75% of the students regularly attended the lectures. The traditional UNCW students had access to the course lecture notes on the WWW, but no means of measuring how often the students utilized these notes was available for analysis.

The distance education students at CFCC were a nontraditional group of students. The community college students were older (CFCC average age was 24.6 years and UNCW average age was 18-19 years) and more mature than their university counterparts. Only 76% of the community college students were single whereas more than 90% of the university students were single. Twenty-four percent of the community college students also had one or more children. The community college students commuted to campus, but many of the university students lived on-campus. While many of the university students were employed at least on a part-time basis, 80% of the community college students were employed (44% were full-time and 36% were part-time). Although being employed full-time and having a family <span style='color:black'>can impose some difficult challenges, our observations indicate that</span>, in general, the nontraditional community college students were much more mature and disciplined than their university counterparts. We believe that the results of this study indicate that most nontraditional students will be well-suited to learning via distance education courses.

IMPLICATIONS AND FUTURE RESEACH

The results and observations of this study have practical significance for instructors of general college chemistry. This hybrid model can provide a meaningful alternative to the traditional course. It is hoped that more community colleges will take advantage of this alternative approach to offering general chemistry to better service the growing nontraditional student population across the country.

The success of this hybrid model provides a baseline against which other models can be tested. Future models might allow the students to perform the kitchen labs at home. Although the results of this study seem to indicate that the instructor should meet with the students on a regular basis, future models might reduce that number of lab meetings. Internet relay chat (IRC) rooms and web bulletin boards could be utilized to stay in touch with the students. More online lab quizzes which are electronically sent to the instructor for grading might also be incorporated into future models.

This study has evaluated one hybrid model for an online general chemistry course that was shown to be highly effective when compared to a traditional on-campus general chemistry course.

LITERATURE CITED

1. Epstein, Margarete Distance Education 1999, 4, 1-5.
2. Paulisse, William J.; Polik, William F, J. Chem. Educ. 1999, 76, 704-708.
3. Dessy, Ray Analytical Chemistry News & Features 1999, 209A-211A
4. Confessore, Nicholas New Republic 1999, 14, 26.
5. Greenhowe, Thomas J.; Burke, K.A., J. Chem. Educ. 1998, 75, 1308-1312.
6. Kimbrough, Doris Anytime Anywhere Chemistry Experience Project, 1999.
7. Heath, Barbara; Rogers, L. H.; Reeves, J.H. The Process of Evaluating an Online Course with a Laboratory Component, 2001.
8. Bullock, Martin L. Jr. A Comparison of Instructional Methods for Introductory College Chemistry, 1996.

APPENDICES

1. Appendix A: Final Exam (Word 2000 Document, 196 kB)
2. Appendix B: UNCW Chm 101, Sec 03, Fall 1998 Syllabus (Word 2000 Document, 38 kB).
3. Appendix C: CFCC Chm 151, Sec INT, Fall 2000 & Spring 2001 Syllabus (Word 2000 Document, 55 kB).
4. Appendix D: Homepage for Chm 151, Sec INT (Word 2000 Document, 46 kB).
5. Appendix E: Chm 151 Course Calendar (Word 2000 Document, 67 kB).
6. Appendix F: Chm 151 Course Calendar (Word 2000 Document, 42 kB).
7. Appendix G: TICC Checklist (Word 2000 Document, 38 kB).
8. Appendix H: Chm 151 Sec INT Quizzes (Word 2000 Document, 89 kB).