BestChoice, a Model for Interactive Web-Based Teaching

BestChoice is an open-access interactive web site (www.bestchoice.che.auckland.ac.nz) that was developed initially to support learning in large first year Chemistry classes at The University of Auckland. More recently BestChoice has been expanded to provide learning opportunities for a wider ranger of users. The model underpinning BestChoice learning activities is simulation of the interchange of a student with an experienced teacher. Thus student responses on BestChoice question pages generate instant assessment and feedback. BestChoice is innovative in its emphasis on teaching both concepts and problem-solving strategies by guiding students in ways that promote their understanding.
 

The discussion below considers the 

• design of BestChoice
• features of BestChoice
• evaluation of students' perception of BestChoice
• incorporation of BestChoice into first year university and high school courses
• evaluation of student usage of BestChoice

Design considerations

Teaching Chemistry to large classes at first year university is largely transmission of information to students who adopt a passive role. However, for quality learning to occur, the student must assume an active role and engage with the content through problem-solving. Often the subject expert is not readily accessible to offer guidance to the student during the learning process. This may cause

• uncertain learners to become discouraged and to conclude that chemistry is "too hard".
• confident learners to solve problems algorithmically without developing full understanding.

BestChoice activities have been designed to support the learning process by acting as a bridge between "hearing about it in lectures" and "doing it yourself". The authors' aim was

• to construct activities that students like to use and which help them to learn
• to develop methods for tracking how students use the activities in order to detect how better to support learning

The process by which we have achieved these objectives is described below.
 

Establishing a connection between web-based learning activities and other study resources

Traditionally, students study using print materials. They solve problems using pen and paper. Many students prefer to have model answers readily accessible.

If BestChoice activities are to be valued as learning tools, students should be able to relate these activities to other study tools. The authors have established this connection by

• constructing BestChoice pages that are visually appealing and use conventional symbolism in display of mathematical and chemical concepts
• developing activities that help students learn how to solve problems that appear on written assessments
• giving users ready access to correct answers and to background material required.

Overcoming some of the limitations of teaching over the web

Reading from the screen is more difficult than reading from a printed page.

Waiting for downloads or for the computer to respond is frustrating.

Common web browser answer-input devices (text-boxes, dropdown lists) do not support subscripts and superscripts. 

Constructing BestChoice modules to simulate a one-on-one tutorial

An experienced teacher would, in a one-on-one tutorial, mix discussion of concepts with questions to probe the student's level of understanding. Concepts are presented as Review Pages in BestChoice. These are typically brief and highlight only major principles.
 

Most modules begin with a Review Page. Concepts are then developed systematically through the

• questions on Question Pages that follow each Review Page.
• feedback displayed when users answer questions on Question Pages.The feedback either relates to the previous Review page by reiterating principles or presages the next Review Page by applying and extending the principles.

Thus, during completion of a BestChoice module, learning is scaffolded by the constant feedback and guidance provided by the system. Some students struggle to understand what the question requires or may not be able to enter the correct answer. These BestChoice users have the option for the system to show them the correct answer so that they can overcome this barrier and move on to complete the problem.
 

Connecting the BestChoice activities with the student's course of study

Teachers choose course resources to help students study. In order for BestChoice learning activities to be accepted as a course resource, we constructed groupings of modules for target groups both in first year Chemistry courses at The University of Auckland and in Chemistry classes at New Zealand high schools. We also constructed a General course (aimed at General Chemistry courses) and an Organic Chemistry course for users outside the target groups. 
 

Implementation of BestChoice

The BestChoice web site went live in 2002, with a total of 1900 pages in 60 modules available. The content was taken from a DOS version of BestChoice that had been developed by one of us. 90% of the pages were Question Pages; thus the emphasis was, as intended, on users entering responses that generate feedback. 

The use of BestChoice was recommended to students in two first-year courses at The University of Auckland in Semester 2, 2002. An end-of-semester survey in one of these courses endorsed the approach taken in BestChoice modules. The survey asked "What feature of BestChoice did you like best?". The two most common responses to the question were "combination of questions and review pages" and "feedback".

While we had put in place the structure described above and had some validation of the BestChoice approach to supporting learning, BestChoice 2002 was limited to one answer per question page; thus multistep problems extended over several pages. Furthermore, the lack of control over the positioning of the answer fields and their associated feedback was a severe hindrance in authoring. 
 

BestChoice for the web 2003-2006

Both the BestChoice content and the BestChoice system have been upgraded and expanded on a continual basis during the last three years. BestChoice currently has 2500 different pages in more than 100 modules with 6000 possibilities for interaction that result in instructive feedback. A variety of new features have been developed so that the author can

• construct Question Pages having any number of answer fields.
• control the positioning of the answer fields and their associated feedback on Question Pages.
• use a variety of types of answer fields on the same Question Page.
• use correct answers to one part of a Question Page to trigger the appearance of a second part.

The BestChoice Question Pages accessed using the links below show that we have made significant progress toward mimicking stepwise problem-solving for a variety of problem types.
 

Evaluation of BestChoice using an on-line survey

We wanted our student users to indicate what they liked about BestChoice and how BestChoice could be improved. Therefore we incorporated on-line evaluation mechanisms into theBestChoice system, in order to get a continual feedback from our users. 

In BestChoice 2002 students could enter feedback using a text form accessed by clicking a Comments button that appears at the top of every page. Very few comments were received. As a consequence, for 2003 we added the survey below on the last page of each module. This allows the user to enter comments and to rate the modules on a six-point scale. Each user could enter one rating and one comment per module. 
 

This on-line survey has been a very rich source of student comment, most of which is pertinent to teaching and learning. 2100 complimentary comments were entered during 2003-2005. A few unedited examples are given below. 
 

The comments entered during 2003-2005 have been assigned to categories. Three of these categories (compliment, suggestion and criticism) are relevant to teaching and learning. 78% of comments have been assigned to one of these. 22% of comments either do not pertain to BestChoice or highlight typographical errors and bugs that have been fixed. 

Many of the comments relevant to teaching and learning have multiple threads. Any comment that includes a compliment is assigned to compliment. Critical comments that include a suggestion are assigned to suggestion. The table shows the distribution of comments in these three categories. The trend is toward a lower percentage in the criticism category and higher percentages in the compliment and suggestion categories as a result of enhancement both of the system and the content.
 

Comments entered in on-line survey
Year	Total	Compliment		Suggestion		Criticism
2005	1173	68%		15%		17%
2004	1134	67%		14%		19%
2003	401	64%		12%		24%

These data show that the learners perceive that BestChoice is beneficial to them and helps them to learn. Furthermore users suggest ways in which BestChoice can be improved (even through critical comments). One advantage of a web-based system is that problems identified can be rectified and the revised version made available within a short time frame.

The module ratings (out of 6) indicate that most students find BestChoice modules helpful 
Students indicate how helpful they have found a module by choosing one of the six radio buttons on the end-of-module survey form. These range from not at all (1) to fantastic (6). The average response rate to this part of the survey over all modules and all cohorts of users is 30%. As shown below, the radio buttons corresponding to 4,5 and 6 ratings are much more commonly chosen than those corresponding to 1,2 and 3 ratings. The most frequent choice in each year is fantastic (6).
 

Using BestChoice as part of course materials

In 2003, use of BestChoice was compulsory in two first year Chemistry courses at The University of Auckland. In 2006 use of BestChoice is compulsory in five first year Chemistry courses. In total this involves approximately 2000 users, some of whom are enrolled in more than one course. 

Although one of the authors has had extensive experience teaching first year Chemistry, neither of us is currently lecturing in any of the first year Chemistry courses. Therefore, support of course coordinators and teachers is crucial. These academic staff

• introduce their class to BestChoice, providing details of the registration process.
• promote BestChoice activities as learning experiences.
• encourage students to use BestChoice to support their learning during the entire course. One lecturer has recently referenced relevant BestChoice modules in the course notes.

What form does the compulsory usage take?
BestChoice modules are currently used as assignments in four courses. 

BestChoice modules are used in one course as both pre-lab activities and assignments

BestChoice sessions in a computer lab were also a feature of our four day catch-up course in 2006. 

BestChoice for New Zealand high school students

New Zealand high school students use BestChoice on a voluntary basis. High schools in New Zealand have computer labs, and students are often introduced to BestChoice through their teacher booking a period in the school computer lab. Teachers are made aware of BestChoice nationally through an electronic newsletter circulated by a high school teacher. 
 

Evaluating BestChoice through usage data

BestChoice collects a variety of data for each user. This includes demographic details (user-entered), registration date, number of logins, and a count of pages on which answers have been entered as well as how many attempts were required to get the correct answer and the time interval over which the question was answered. 

The table below shows that there has been a steady increase in the number of registrations per year over 2003-2005. The increase in 2005 is largely due to New Zealand high school students.

There has also been growth in the number of active users. These users have answered questions on more than 20 pages. In 2003 an active user entered at least 20 answers. In 2004-2005 an active user will have have entered at least 100 answers. There is a correlation between the percentage increase in either logins or registrations over an entire year and the percentage increase in active usage.
 

Active usage data highlights the importance of the connection to the user's course of study

Users are asked on registration to identify themselves with one of the institutions shown in the table below. The table also shows the number of active users associated with each institution for 2005. 
 

The largest groups of active users are those for whom specific courses have been created. The difference between the two large cohorts is that while use of BestChoice is compulsory for most university users, New Zealand high school students use BestChoice on a voluntary basis. 
 

The timing of logins gives evidence for how these two cohorts use BestChoice

The graph below shows the number of logins by students at The University of Auckland for each month during 2005. This indicates that usage of BestChoice during the semester (March-June and then July - November) is fairly consistent and that usage drops during the inter-semester period. Comparison with data from 2003 and 2004 shows that, due to the support of the course coordinators, students in each subsequent year use BestChoice both earlier in the semester and more consistently throughout the semester. 
 

The graph above for New Zealand high school students logins shows that their highest usage is at the end of the year. Thus this cohort regards BestChoice primarily as an exam preparation tool. This perception may, however, be changing because the percentage of logins to BestChoice during the period February through September is higher for 2005 (53%) than for 2004 (41%).
 

In conclusion

What have we found?

The BestChoice project has shown that web-based activities can facilitate learning by requiring the user to take an active role, with the system then responding to the user's input. The guidance offered by BestChoice makes successful completion of the problem more accessible for more learners who, over time, should gain the skills to answer the questions without guidance. Consider the unedited comments. 
 

BestChoice activities also provide capable learners with an enhanced learning experience as insights are revealed during problem-solving and appear in the form of feedback just as the user has entered the answer that they thought was correct. 

The BestChoice project has also shown that users can inform the design of systems to support their learning. The extent and quality of feedback entered by users has been overwhelming.
 

How does BestChoice compare to other learning tools?

A paper survey asking "What feature of the course most helps you to learn?" has been circulated in some courses where use of BestChoice is compulsory.
 

The student responses indicate that they view BestChoice as an effective learning tool that complements traditional learning tools. BestChoice is not, and was never intended to be, a replacement for these. 

It is a challenge for lecturers to accommodate the variety of learning styles and academic backgrounds in the large classes at first year university level. Web-based activities like those inBestChoice can be an addition to the suite of tools available to help meet this challenge. Unlike the most other learning tools, use of BestChoice by students generates data that can provide information on how to support learning more effectively and efficiently. 
 

Where to now?

The most frequent suggestion entered in our survey is "more questions". New Question Pages are currently being added on a continual basis. An important aspect of the writing of these pages is that new ways being explored to present content interactively, including working toward enabling the user to choose the extent of guidance offered on a BestChoice page.

The discussion above is a broad analysis of data pertinent to student perception and usage. It is now appropriate to extend these investigations by considering student response data relevant to individual modules and questions to determine whether areas of difficulty for students can be identified. Once barriers to learning have been identified, the focus would be on improving support for learning in these areas. 

In conclusion, it is evident that a large number of users believe that BestChoice is very effective in helping them learn Chemistry. Furthermore, once a system like BestChoice is in place, it thrives on student usage because usage creates data that highlights where the system needs improving. 

BestChoice (www.che.auckland.ac.nz/bestchoice) is open access. Use Demo mode (no registration, no marks stored) or register. In either case, we would appreciate your feedback.