A Study of Persistence in Learning Chemistry Through Technology Applications

INTRODUCTION

Several online websites are available to help students learn chemistry and practice techniques important to success. Older sites use simple drill and practice. More sophisticated sites have developed methods for context-sensitive help and hints. Some sites are static and aligned with particular textbooks or teaching programs. Other sites are more flexible and allow a range of choices in topics, level of questions, and extent of coverage. Most allow for flexible assignments, some collect and display performance data. Some sites include instructional material and other enhancements.

Since the overall point of these websites is simply to assign and enforce doing homework, is there an advantage to using an online site compared with the more traditional paper and pencil worksheet of end-of-chapter problems? Worksheets are flexible, can be prepared with a selected level of questions or problems, contain whatever number of items is deemed appropriate, and the textbook provides random access, context-sensitive help. Unfortunately, as most instructors know, worksheets also give students opportunities for collaboration which may or may not be beneficial. In simple terms: students can cheat. Further, instructors may put in a great deal of effort to assess the work, and students do not always pay attention to the corrections.

This paper will describe the use of the Mastering Chemistry online program used in a high school chemistry class. It will describe the apparent advantages of the online system compared with traditional teaching and show that the online system appears to increase learning, and retention, and performance.

A DESCRIPTION OF MASTERING CHEMISTRY - SUMMARY

Mastering Chemistry is an online learning system incorporating a homework/quiz system, animated tutorials and explorations, and online access to written guided instructional assignments (with answers). The quizzes feature immediate feedback, context-sensitive "ChemHelp" with text describing the concepts and sample problems, and extensive scoring and reporting. The homework/quiz system includes more than 55 units covering more than 175 topics. Teachers have flexibility to assign entire units or individual topics. The system, developed by Dr. Patrick Wegner, (California State University, Fullerton) can be accessed at http://titanium.fullerton.edu/mcweb. Teachers may use the system without creating a class using the link at http://treefrog.fullerton.edu.

Detailed Description of Mastering Chemistry

Sample screenshots

THE RESEARCH METHOD - SUMMARY

Students in the first-year chemistry classes at Pacifica High School took an 128 question assessment of visualization and proportional reasoning skills in chemistry (the "OCCA") at the beginning of September 2003 (pretest 1 or "pre1"). After the first semester ended (February 2004), they took the OCCA again as a post test (post test 1, or "post1"). One teacher used the MCWeb system in place of many worksheets and textbook assignments. The other two teachers used traditional instruction and homework assignments. All teachers followed the same course of study, used similar labs, and gave similar (sometimes the same) tests. The students in the traditional classes are referred to as the "control." The students using MCWeb are referred to as the "MCWeb" group.

Advanced Placement chemistry at Pacifica is a second year class. It attracted 46 students, of these 24 were in the MCWeb group, and 22 in the control. They all took the OCCA as pretest 2 ("pre2") on the first day of class (September 2004). They took it once more at the end of the year (June 2005) as post test 2 ("post2"). Instruction in the AP class included MCWeb for all students. To assure that students in both groups had similar ability, test score percentiles on the California Achievement Test (sixth edition) for Mathematics were compared. The only significant difference found was that the MCWeb group had (apparently) lower mathematics ability.

Scores and gains on the OCCA were compared between groups, with adjustments for math ability. Both groups had similar scores on pretest 1. The MCWeb group outperformed the control group on post1 and pre2 during the differentiated instruction. Both groups had similar performance on post2, following the use of MCWeb by both groups.

Detailed Description of Research Methods

RESULTS AND DISCUSSION - SUMMARY

Mean Scores and Gains on the Assessment Tools
PRE and POST scores adjusted by ANCOVA for differential math ability

FIGURE 1

Link to Histogram of Mean Scores

Link to Histogram of Gains

As Figure 1 shows, the MCWeb group has lower mathematical ability than the control group. This is significant, since 50% of the OCCA test used to compare visualization and proportional reasoning skills focuses on the mathematics used in chemistry. As expected, the MCWeb group did score lower on the first OCCA pretest, but the difference when corrected for math ability was not significant, and amounted to less than one question out of 18. Although the pots 1 scores are also not significantly different, the performance is now reversed - the MCWeb group average score is now 1.1 questions better. Overall, the control group gained only 1.5 questions (8%) compared with a gain more than twice as large (19%) for the MCWeb study group. This difference is statistically significant.

Both groups had 12 additional weeks of chemistry instruction before summer break. Neither group had any assigned study or chemistry practice during the 11 weeks of vacation. Teachers know from experience that students forget what they have learned over the vacation period, so a decline in scores would not be a surprise. The additional instruction and practice accounts for the fact that both groups of students showed a gain in score after the time off. However, the MCWeb group again showed nearly twice the gain of the control group. While the gains, since the are small, are not significantly different, the mean scores now favor the MCWeb group by 1.5 questions and this is a significant difference.

Both groups achieved similar scores after a year using MCWeb in the AP Chemistry class.

FIGURE 2

A comparison of students with negative gain (regression) shows that far fewer MCWeb students "forgot" what they learned compared with the control. A relatively large number of control students decreased performance during the time of instruction compared with only a single student in the MCWeb group. Many students regressed following the break, as expected. But the number of control students regressing was nearly twice as large as in the MCWeb group and the regression was nearly 50% larger than that from students using MCWeb.

Detailed Description of Results and Discussion

CONCLUSIONS

The results of the series of OCCA tests suggest that an online program (like MCWeb) which has immediate feedback, and enforces completion of assignments (particularly when combined with cooperative learning techniques) is more effective in teaching the important skills of visualization and proportional reasoning in a chemistry context than are traditional homework. Further, students who have used MCWeb have greater retention of material after 13 weeks without instruction or practice than their counterparts who did not use the program. Although the MCWeb students had lower mathematics ability than the control, they showed greater gains in performance (which presumably means learning) and surpassed their more able counterparts in the mathematics-related proportional reasoning skills. Therefore, the author concludes that an online system such as MCWeb, which enforces doing assignments and achieving correct answers, particularly when combined with cooperative learning techniques, is more beneficial that the usual types of homework.

ACKNOWLEDGMENTS

I would like to express my appreciation to Dr. Wegner, the developer of the MCWeb program. His enthusiasm, support, and advice on using MCWeb have revolutionized my teaching methods. Dr. Barbara Gonzales's help in preparing and analyzing the data have been invaluable. Dr. William Sandoval, University of California, Los Angeles, provided much useful insight and discussion of results. The ANCOVA analysis was done using the applets created by Dr. Richard Lowry, Vassar College and was accessed at his website, http://faculty.vassar.edu/lowry/VassarStats.html (accessed during July 2005).

NOTES (for DETAIL sections)

(1) The test was originally designed to be given via the Internet and was called the "Online Chemistry Concept Assessment." When it was found that there was too much variability in access, administration supervision, and lack of effort by students the test was converted to a more standard multiple choice format. The reference to "OCCA" as the test designation remains.

(2) A refereed journal article by Dr. Barbara Gonzales, California State University of Fullerton, will be published this fall with details on the validation study.

(3) Gonzales, Barbara, Report the MCWeb Advisory Committee, July 2003.

Copyright David Paul Licata, 2005. All rights reserved.