Impact of a simulation on student understanding of acids and bases

Introduction

An in-depth understanding of chemistry requires that students understand several fundamental concepts. It also requires students to (a) be able to see the relationships between various concepts, b) have an ability to connect one or more concepts and c) apply a coherent, constructive, relational understanding to further make sense of the ideas presented in specific subdiscipline for example inorganic or organic chemistry or in the interdisciplinary areas such as biology or biochemistry.  Among several concepts covered in the first-year college-chemistry courses, acid-base chemistry is considered to be a foundational concept (1-3). Acid base chemistry is important to understand and explain the structure and behavior of molecules. Despite the importance of acid-base chemistry, students in college chemistry courses struggle with understanding and representation of acids and bases. This paper is focused on the student-conceptions of acid-base chemistry.
Background:
There have been detailed studies in chemistry education literature on how students develop their understanding of acid base concepts. Some studies have also shown that students have identifiable misconceptions in acid base chemistry. Part of the development of these misconceptions comes from student prior experiences and inadequate instruction on the topic (4-5).

Students also struggle with understanding how to connect the microscopic, macroscopic and symbolic representations of concepts in chemistry. A major part of instruction in acid base chemistry revolves around translating microscopic concepts, such as ionization and dissociation, to macroscopic concepts. It is difficult for students to envision properties of ionization and/ dissociation in aqueous (or other) media and to apply these behaviors to make sense of the real-world examples such as the functioning of batteries and the phenomena of corrosion. Students struggle to apply and subsequently transfer their understanding to explain a concept or solve advanced conceptual problems due to gaps in their understanding. The ability to draw connections between microscopic, macroscopic and symbolic representations is limited and rote memorization and definitions do not really help for building connections among concepts (6,7).

Specially, when students have misconceptions in their mental models or representations, they are more likely to retain this misconceptions or incorrect conception concerning the content. To address this, researchers have conducted studies to understand how students develop mental models in acid base chemistry and how to improve them (8,9). Early interventions on poorly formed mental models as well as the encouragement of alternative conceptions has shown to marginally improve student’s awareness of misconceptions but have shown little improvement in removing these misconceptions (10)

          
Visualization tools such as simulations have shown promise in impacting student understanding in chemistry. The use of interactive simulations in chemistry has seen a rapid surge in the past few years. Simulations are digital environments that allow users to interact with models of various concepts and processes. A simulation uses a mathematical or a logical model to illustrate real world phenomenon with a goal of providing opportunity to students to understand the underlying concept or scientific model (11, 12).

Prior researchers have found simulations to be effective in helping students visualize motion of particles, and in improving problem solving and thinking skills by complementing the sensory experiences of learners. Simulations been found effective in engaging students, improving academic performance, and to an extent student’s representational competence. Simulations allows students to explore phenomena and their representations while manipulating variables. Simulations provide a certain degree of control to learners through their interactive features such as play and pause buttons, adjusting variables such as mass, volume etc. using buttons and sliders, and stop, review, and exit functions (13-21).  

For example, a study by Moore et. al. demonstrated the impact of simulation-based learning on student understanding of molecular polarity (22). The students participating in the study found that engaging in the simulation helped provide a clearer understanding of molecular polarity because of the representation of electronegativity in the simulation. By being able to engage with and manipulate variables presented in a simulated model, students are able to make some connections and translations between the microscopic and macroscopic representations of a concept or a process.

Despite all of these benefits, there are few studies in which simulations are used for the instruction of acid-base chemistry (23-25). With the success seen in other areas of literature the lack of evidence on the impact of simulations on student understanding of acids and bases specifically from students from two unrelated institutions with substantial background in chemistry was also noticed.  The study reported in this paper was conducted to address if simulations can aid student conceptual understanding and their representation of acids and bases specifically for students who have completed general and organic chemistry courses in college. Further, most studies involve students from a single institution or institutions within the same region or country (26-30). In this study student participants were from an institution in USA and India. Following research questions were addressed in this study focused on SBL of acid-base chemistry.

·      Does simulations-based learning impact student conceptual understanding of acids-and bases?

·      Do simulations impact student representations of the aqueous solutions of acids and bases?

Methodology
Qualitative research methods involving surveys and qualitative interviews were used to answer the research questions. Semi-structured interviews and survey data was collected pre- and post-simulation. Data from the interviews was coded for thematic analysis and survey was analyzed to see the patterns in student understanding of acids and bases.

About study participant’s and demographics:

Demographic surveys were taken of each student to better understand their background in chemistry. Among those who participated 37% were third year students, 37% were fourth year students and 15% where graduate students. All of the students participating in the study were either pursing or had completed a degree program in natural sciences, and had also completed general and organic chemistry courses at their respective institutions.

The study participants were recruited from two completely different institutions from different countries. The participants were purposefully sampled based on a similar background and years of experience of learning chemistry. Participating students were selected from a mid-sized institution in Western India. Participants from USA were from a mid-sized upper midwestern institution. There were 10 students recruited to participate in this study from both the institutions to have equal representation from the Indian and US counterpart (total N=20). Students from the Indian institution had about 6 years of school chemistry courses (2 years of basic middle school and 4 years of high school) and further 2-3 years of college chemistry. Students from the US-based institution had at least two years of high school chemistry and 3 years of college credits in chemistry (including general, organic chemistry).

Students in the US institution were all enrolled in pre-medical preparation course and met weekly with their instructor to prepare for MCATs specifically for chemistry portion of the MCAT.  Though the student participants in this study were from completely unrelated institutions in all respects, both groups had only experienced traditional instruction in chemistry courses and had no exposure to simulation-based learning. Further, students had learned about acid-base chemistry in their high school and college years and this was important to establish a baseline for student prior knowledge during the qualitative data collected via interviews. Data collection is discussed in the following section.

Data collection and analysis:

At the beginning of the study, students’ where given a demographic survey and ASCI (31)to assess their attitudes towards the subject of chemistry (ASCI is Attitude towards the subject of chemistry inventory). The purpose of using the ASCI was to gather information about students’ general attitudes about the subject of chemistry prior to their experience with the simulation hence the ACSI survey was conducted only once for each participant.

The ASCI is a semantic differential where students position themselves on a seven-point Likert scale between two polar objectives in reference to how they feel about “chemistry” which measures their attitude for the subject. In ASCI, the adjectives and choices are placed on the same line. To avoid a bias and to help respondents to think carefully, some adjective pairs are categorized with the “positive” adjective on the left and some on the right side of the line.  Inventory adjectives are positioned at the end of each line, and the word “middle” is labelled above the number 4. Inventory instructions are brief and understandable for students to complete the survey within a few minutes.

The ASCI survey has 20 items that relate to five factors of Interest and Utility, Anxiety, and Intellectual accessibility, Fear and Emotional Satisfaction to measure student attitudes. In order to evaluate the interest and utility, the adjective pairs in survey include worthwhile-useless, worthless-beneficial, good-bad, interesting-dull, and exciting-boring. For measuring the anxiety, the adjective pairs of tense-relaxed, work-play, scary-fun, insecure-secure, and disgusting-attractive are used. The adjective pairs for intellectual accessibility are complicated-simple, confusing-clear, easy-hard, challenging-unchallenging, and comprehensive-incomprehensible. For fear of the subject safe-dangerous are used as the adjective pairs and the adjective pairs for emotional satisfaction are pleasant-unpleasant, comfortable-uncomfortable, chaotic-organized, and satisfying-frustrating. All survey results were calculated and averaged using the methodology reported in the literature on ASCI. Assessing student attitudes provided greater clarity on how students viewed the subject of chemistry based on the years of chemistry they had in high school and college.

Qualitative Interviews

After completing the ASCI, students participated in the pre-simulation interview during the first meeting. These interviews were about 30-45 minutes long during which students answered specific questions about their understanding of acids and bases. Students were also asked a set of specific questions concerning their previous experience in chemistry, their confidence with chemistry, and specific content questions on acid-base chemistry. The content questions for acid-base chemistry where narrowed to information covered in an introductory chemistry course; acid-base definition (Arrhenius, Bronsted-Lowry, Lewis), pH calculation, physical properties, and chemical properties. During the interview the students were also asked to draw their models of acids and basis and explain what would happen when an a strong and a weak acid/base is added to water.  Further, students were allowed to select their own examples of acids and bases to draw and explain their representation of a acids and bases.

After the interview the participants were introduced to the PhET simulations on acids and bases. They were provided the tutorial, following which each participant was instructed on acid-base chemistry using the simulation. The participants used simulation in a guided-manner and explored various components of the simulation during the instruction. The instruction using the simulation lasted from 2-3 hours.

The PhET simulation software used for this study on simulation-based instruction was developed by the PhET group at the University of Colorado, Boulder. The simulation allowed students to observe solutions containing either acids or bases with pH strips, pH indicators and electrodes attached to a light bulb as seen in Figure 1. PhET simulations showed substantial promise as the simulation to be used in the study because of its convenience and simplicity to integrate it during one-to-one interactions with students to instruct them on acid-base chemistry via the simulation.

Figure 1. PhET acid-base simulation

After receiving the instruction students were for invited for a second round of interviews a week later (post-simulation) and similar sets of questions focused on acid and bases were asked as were during the pre-interview. Students were also asked questions regarding their experience with the simulation for learning about acids and bases. During the post-interview students were again asked to draw their model of acids and bases in order to evaluate shifts in student understanding of the concepts.

The pre- and post-simulation interview data was transcribed, read and coded. Several codes were developed based on the initial readings of the transcripts and iterative analysis. These codes were finally reduced to 29 codes that were used during the analysis of qualitative interview data. The code definitions are provided in Appendix 1. These codes were used for coding student-responses during the pre- and post-interviews following the simulation-based instruction.

To ascertain interrater agreement, 12 anonymized transcripts that included both pre- and post-interviews from the student participants were provided to an expert researcher with the list of codes. Codes list including code definitions are presented in the Appendix 1. All student identities were removed for these transcripts. The researchers met with each other over a period of 6 weeks to discuss the qualitative coding for the 12 transcripts and also to establish the degree of agreement and consistency of the coding process. Based on the coding by the two researchers the interrater agreement was found to be 90% which is considerably strong for a qualitative study. Also, the student-drawn pre- and post-simulation models were coded and compared to textbook models of acids and bases.

Results and Discussion

In this study on the impact of the simulation-based learning on student understanding three tools were used were used for data collection: general student survey for demographic information, ASCI, and pre- and post-qualitative interviews. The ASCI survey data was analyzed in order to determine student attitude for chemistry. The scale on the horizontal axis of figure 2 is based on the scaling as described in the Bauer study for the analysis of ASCI (31).

Figure 2. Averages of student responses to ASCI

Based on student response to the survey, for interest and utility, the students favored more positive descriptions of chemistry (i.e., worthwhile, interesting, exciting) compared to other sections of the survey. This can be indicative of the students having a more positive outlook on the applications of chemistry. However, student responses for anxiety and intellectual ability favored negative or neutral descriptions of chemistry (i.e., hard, confusing, scary, disgusting, tense) compared to other sections of the survey.

Based on the ASCI responses, it appears that students generally have negative outlook on the learning process as it relates to chemistry. It is important to note that each student who participated in the study was mainly taught using traditional approaches of lecture and verification-based laboratory activities. Students had no prior exposure to using simulations for any of the topics and further each participant had years of chemistry exposure in high school and college courses. The ASCI results provide a snapshot of student participants in this study. Students have interest in chemistry, and they find it to be intellectually important. However, even after years of spending time in the classroom and laboratory in school and college, students feel anxious and have a fear of the subject.

The ASCI does not measure student confidence however the presence of fear, anxiety, and neutral to negative responses for the intellectual accessibility indicates that student do not feel connected with chemistry, and it remains intellectually abstract. It is difficult to pin-point exact cause of this apathy from this group of students. Though it is possible that students have not really experienced deep-learning that comes with constructivist teaching that could have helped them to build connections and lead to positive attitudes among students on all the factors measured by the ASCI.

The results from the analysis of student qualitative interviews are presented in Table 1 which shows code frequencies for pre-simulation and post-simulation interviews. During pre- and post-simulation interviews students were asked a series of questions related to four conceptual ideas related to acids and bases: strong and weak acid-base chemistry, acid-base reactions, equilibrium reactions, and the pH. The students’ pre- and post-simulation interview data was coded for the level of detail of their response to the acids and bases as well as the student perception on the use of the simulations.

Table 1: Code frequencies showing conceptual understanding, student confidence and view of simulation-based instruction (Shaded Green: conceptual understanding codes – general and detailed understanding; definition-based explanation; shaded orange - incorrect conceptual understanding or no response; shaded blue – personal traits - student confidence; brown-misconceptions related to science and logic-based misconceptions; light grey-simulation related)

Codes and their definitions are reported in table 2 in the appendix. It is important to note the definition of general, detailed, and incorrect content that were used to evaluate conceptual understanding of acids and bases. General content is any description a student gave that would be definition heavy, detailed content is any description a student gave where they correctly answered information as well as gave a relevant example in their response, and incorrect content is any description a student gave that inaccurately identifies or describes the content.

Based on code frequencies, the number of codes for GSA increased from 53 to 58. Likewise, there was a positive shift in the frequency of GWA and GWB, GpH and GEC. Also, the detailed content improved for strong acids, weak acids, equilibrium and pH. This shows that students not only provided a detailed response, but they also shared relevant examples along with the description of strong acid, weak acids, and acid-base equilibrium.

Based on changes in code-frequencies, the student understanding of strong and weak acids and bases, pH and acid-base equilibrium shows an improvement from pre- to post simulation instruction. Further, students displayed a positive shift in their confidence of their understanding of acids and bases, the frequencies for weak confidence as a personal trait decreased from pre-to post interview whereas strong confidence showed an increase.  It appears that students had a more positive view of simulations post-instruction for learning chemistry. As mentioned before these students had never been instructed with simulation. Based on the pre-interview responses it appears that students have a sense of what a simulation is, however whether a simulation can help them learn acids and bases becomes clear with post-instruction. The positive responses of students seem to be indicative of this shift among students regarding the usefulness of simulations for learning and visualizing a concept.

In general, looking at the code-frequencies that relate to student understanding of acids and bases, these display an improvement in student conceptual understanding. From these code frequencies it is observed that the students had some general misconception based on their foundational understanding of acids and bases for both strong and weak acids and bases.

After going through the simulation-based instruction, students were able to address some of these misconceptions in their responses during the post-simulation interview. For example, student 3 in their pre-simulation interview showed misunderstanding of weak acid and strong acid dissociation. This student would describe in their explanations that the solution would dilute the acid rather than allow it to ionize. After receiving simulation-based instruction, the student was able to observe that the ionic dissociation of the acid particles did not mean there was a decrease in acidic concentration.

During both the pre- and post-simulation interviews students were instructed to create representations based on their descriptions of the acids and bases. Students were also asked what would happen when an acid or a base interacts with water. The representation that students presented through their drawings and symbols (equations) were compared to the textbook representations of acids and bases. Reason being that the textbook models were clear, well-developed, peer-reviewed, and used as a standard reference for a concise presentation of information on acids and bases.

Changes in student models of acids and bases were positive from pre to post-simulation instruction. As figure 3 and 4 depict, student representations became closer to the representations presented in the text and also in the simulation. Students were able to articulate their understanding of acids and bases as an equation and also depict what would happen if a few molecules of an acid or a base were added to water. For example, students showed that incase of strong base there would be a complete dissociation of the base as compared to a weak base where some of the base molecules would remain undissociated and were able to explain what would happen to a light bulb connected to a battery, with two electrodes dipped in such as solution of an acid or a base.

Figure 3. Comparison of student’s pre-simulation (a) and post-simulation (b) model with textbook model for acids and bases (Acids, Bases, and Salts. In Science Textbook for Class X, National Council of Educational Research and Training: 2006, 2006; p 282.

Figure 4. Comparison of student’s pre-simulation (a) and post-simulation (b) model with textbook model for equilibrium chemistry (Equilibrium. In Textbook for Class XI: Chemistry Part I National Council of Educational Research and Training: New Delhi 2005; p 259.)

Limitation

Part of the limitation of this study is the frequency in which students were able to engage with the simulation-based instruction of acids and bases. Students were purposefully sampled based on their background in general and organic chemistry courses and no prior use of simulations in chemistry. However, student exposure to the use of simulation was limited to one complete session (equivalent to a laboratory duration of 2-3 hours). Based on the results and the promise of simulations in addressing student conceptions, it seems that a more diverse and expanded demographic of students needs to be studied over a longer duration to have a lasting impact on student-understanding. In addition to qualitative interviews, quantitative data on student problem-solving will provide additional evidence on the effectiveness of simulations in student conceptual understanding and student representations of acids and bases.

Conclusion

The study is unique with respect to student participants and the misconceptions/ gaps in student knowledge of acids and bases which seem to transcend the geographical boundaries when students have similar prior exposure to chemistry.  However, using simulations can alleviate some of these student misconceptions and improve student conceptual understanding and representations.

A single session involving the use of the PhET acid base simulation during one-to-one instruction was helpful to draw student attention to their understanding of acids and bases and to address some misconceptions and gaps in their understanding, however much work remains to be done in this area. None of the participants in the study had a completely clear conception of the acids and bases and how these interact in aqueous solutions. However, the misconceptions that were addressed were enough to show positive trajectory for a more longitudinal study. Students who struggled with understanding acid-base chemistry on the microscopic level benefited from seeing a visualization of how the particles interacted in a solution with one another.

Student acid-base representations became more consistent with both the simulation and also textbook representations after an exposure to simulation-based learning. Students showed a clear knowledge of the content in their post-simulation interviews, however the impact varied depending on the severity of the misconceptions that a student had prior to the study. The study shows the potential of simulations in impacting student understanding of acids and bases for a select group of students who had been educated through their entire student life in different countries. Yet student struggles and misconceptions of acids and bases were much alike. Further studies are needed to add evidence to the effectiveness of long-term impact of simulations on student-conceptual understanding, specifically for students in different countries who have no exposure to simulation-based instruction.

Acknowledgments

The authors acknowledge the support of principal and vice-principal of D.G. Ruparel college and also the Homi-Bhabha Center for Science Education, Tata Institute of Fundamental Research for this study.

References

1.     Murphy, K.;  Holme, T.;  Zenisky, A.;  Caruthers, H.; Knaus, K., Building the ACS exams anchoring concept content map for undergraduate chemistry. Journal of Chemical Education 2012, 89 (6), 715-720.

2.     Holme, T.;  Luxford, C.; Murphy, K., Updating the general chemistry anchoring concepts content map. Journal of Chemical Education 2015, 92 (6), 1115-1116.

3.     Raker, J. R.;  Reisner, B. A.;  Smith, S. R.;  Stewart, J. L.;  Crane, J. L.;  Pesterfield, L.; Sobel, S. G., Foundation Coursework in Undergraduate Inorganic Chemistry: Results from a National Survey of Inorganic Chemistry Faculty. Journal of Chemical Education 2015, 92 (6), 973-979.

4.     Talanquer, V., Commonsense Chemistry: A Model for Understanding Students' Alternative Conceptions. Journal of Chemical Education 2006, 83 (5), 811.

5.     Tsaparlis, G., Acid-base equilibria, Part I: Upper secondary students’ misconceptions and difficulties. The Chemical Educator 2004, 9, 122-131.

6.     Nakhleh, M. B., Student's Models of Matter in the Context of Acid-Base Chemistry. Journal of Chemical Education 1994, 71 (6), 495.

7.     Nakhleh, M. B., Why some students don't learn chemistry: Chemical misconceptions. Journal of Chemical Education 1992, 69 (3), 191.

8.     Muhamad Damanhuri, M. I.;  Treagust, D.;  Won, M.; Chandrasegaran, A. L., High School Students’ Understanding of Acid-Base Concepts: An Ongoing Challenge for Teachers. The International Journal of Environmental and Science Education 2016, 11, 9-27.

9.     Smith, K. J.; Metz, P. A., Evaluating Student Understanding of Solution Chemistry through Microscopic Representations. Journal of Chemical Education 1996, 73 (3), 233.

10.  Rappoport, L. T.; Ashkenazi, G., Connecting Levels of Representation: Emergent versus submergent perspective. International Journal of Science Education 2008, 30 (12), 1585-1603.

11.  Gupta, T., Promoting mathematical reasoning and problem solving through inquiry-based relevance focused computer simulations: a stoichiometry lab. Chemistry Teacher International 2019, https://doi.org/10.1515/cti-2018-0008    

12.  Suits, J. P.; Srisawasdi, N., Use of an Interactive Computer-Simulated Experiment To Enhance Students’ Mental Models of Hydrogen Bonding Phenomena. In Pedagogic Roles of Animations and Simulations in Chemistry Courses, American Chemical Society: 2013; Vol. 1142, pp 241-271.

13.  Stieff, M.; Wilensky, U., Connected Chemistry—Incorporating Interactive Simulations into the Chemistry Classroom. Journal of Science Education and Technology 2003, 12 (3), 285-302.

14.  Plass, J. L.;  Homer, B. D.; Hayward, E. O., Design factors for educationally effective animations and simulations. Journal of Computing in Higher Education 2009, 21 (1), 31-61.

15.  Falvo, D. A.;  Urban, M. J.; Suits, J. P., Exploring the Impact of and Perceptions about Interactive, Self-Explaining Environments in Molecular-Level Animations. Center for Educational Policy Studies Journal 2011, 1 (4), 45-61

16.  Falvo, D. A., Animations and simulations for teaching and learning molecular chemistry. International Journal of Technology in Teaching and Learning 2008, 68-77

17.  Liu, H.-C.;  Andre, T.; Greenbowe, T., The Impact of Learner’s Prior Knowledge on Their Use of Chemistry Computer Simulations: A Case Study. Journal of Science Education and Technology 2008, 17 (5), 466-482.

18.  Lancaster, K.;  Moore, E. B.;  Parson, R.; Perkins, K. K., Insights from Using PhET’s Design Principles for Interactive Chemistry Simulations. In Pedagogic Roles of Animations and Simulations in Chemistry Courses, American Chemical Society: 2013; Vol. 1142, pp 97-126.

19.  Kelly, R. M.;  Barrera, J. H.; Mohamed, S. C., An Analysis of Undergraduate General Chemistry Students’ Misconceptions of the Submicroscopic Level of Precipitation Reactions. Journal of Chemical Education 2010, 87 (1), 113-118.

20.  Kearney, M., Classroom Use of Multimedia-Supported Predict–Observe–Explain Tasks in a Social Constructivist Learning Environment. Research in Science Education 2004, 34 (4), 427-453.

21.  Finkelstein, N. D.;  Adams, W. K.;  Keller, C. J.;  Kohl, P. B.;  Perkins, K. K.;  Podolefsky, N. S.;  Reid, S.; LeMaster, R., When learning about the real world is better done virtually: A study of substituting computer simulations for laboratory equipment. Physical Review Special Topics - Physics Education Research 2005, 1 (1), 010103.

22.  Moore, E. B.;  Chamberlain, J. M.;  Parson, R.; Perkins, K. K., PhET Interactive Simulations: Transformative Tools for Teaching Chemistry. Journal of Chemical Education 2014, 91 (8), 1191-1197.

23.  Watson, S.W.; Dubrovsky, A. V.; Peters, M. L.; Increasing chemistry students' knowledge, confidence and conceptual understanding of pH using a collaborative computer simulation. Chemistry Educ. Res, & Practice, 2020, DOI: 10.1039/c9pr00235a.

24.  Cook, M. P., Visual representations in science education: The influence of prior knowledge and cognitive load theory on instructional design principles. Science Education 2006, 90 (6), 1073-1091.

25.  Clark, T. M.; Chamberlain, J. M., Use of a PhET Interactive Simulation in General Chemistry Laboratory: Models of the Hydrogen Atom. Journal of Chemical Education 2014, 91 (8), 1198-1202.

26.  Çetingül, P. İ.; Geban, Ö., Understanding of Acid-Base Concept by Using Conceptual Change Approach. H. U. Journal of Education 2005, 29, 69-74.

27.  Bretz, S. L.; McClary, L., Students’ Understandings of Acid Strength: How Meaningful Is Reliability When Measuring Alternative Conceptions? Journal of Chemical Education 2015, 92 (2), 212-219.

28.  Boo, H.-K.; Watson, J. R., Progression in high school students’ (aged 16–18) conceptualizations about chemical reactions in solution. Science Education 2001, 85 (5), 568-585.

29.  Czysz, K.;  Schroeder, L.; Clark, G. A., Making Acids and Bases MORE Basic: Supporting Students’ Conceptualization of Acid–Base Chemistry through a Laboratory Exercise That Connects Molecular-Level Representations to Symbolic Representations and Experimentally Derived Evidence. Journal of Chemical Education 2020, 97 (2), 484-489.

30.  Cooper, M. M.;  Kouyoumdjian, H.; Underwood, S. M., Investigating Students’ Reasoning about Acid–Base Reactions. Journal of Chemical Education 2016, 93 (10), 1703-1712.

31.  Bauer, C. F., Attitude toward Chemistry: A Semantic Differential Instrument for Assessing Curriculum Impacts. Journal of Chemical Education 2008, 85 (10), 1440.

Appendix