Assessment for Active Learning

When faculty get together to discuss assessment too often the conversation is restricted to the evaluation of students. In a national survey of chemistry faculty, 90% reported that the motivations for assessment were external relating to accreditation and certification.¹ Only 7% believed that these assessment efforts were of value. Assessment in its various forms is much broader and can have a much broader context. Frequently faculty and departments are required to report on assessments of student learning at both course and program levels. At the same time classroom assessment is used for a variety of diagnostic and evaluative purposes. Here it will be proposed that through the use of hierarchically structured learning objectives, assessment can be aligned and integrated to serve all these needs. Moreover, informing students of these learning objectives at every stage of their learning engages them and allows them to better monitor their learning progress. Through the development of hierarchical aligned learning objectives there is a shift from a focus on assessment OF learning to one of assessment FOR learning. The core principles of assessment for learning include: (1) clearly communicating measurable learning objectives prior to instruction and (2) providing meaningful feedback that guides teaching and learning.

More meaningful assessments require a focus on objectives and outcomes. Often this process begins when a gap is noted between what faculty expect students to learn and what they do in fact learn. In many cases faculty expectations are implicit and are never clearly articulated or communicated. Instructors may not have clearly thought out exactly what it is they want students to be able to do as a result of instruction. Learning objectives are a clear statement about what students should be able to do as a result of instruction. Having a clear set of learning objective for a course, unit or lesson allows one to better plan instruction. Learning objectives not only set a framework for instructional design, they also guide assessment.

Any effective assessment plan must begin with a clear articulation of what faculty expect of students. Learning objectives need to follow the S.M.A.R.T framework in that they must be specific, measureable, achievable, relevant and completed within a specified timeframe. In addition the number of learning objectives should be limited in number. Given that one of the goals of this form of structured hierarchical assessment is to focus teaching and learning, too many learning objectives will undermine this intent.

So where does one begin the process to develop aligned, hierarchical assessment? Begin at the top with the end in mind. For most faculty this means beginning with course level learning objectives. In developing these consider what underlying concepts connect the content in the course. If a student were asked to explain content six months after completing the course what enduring concepts should be included in the explanation. Course level learning objectives should be:

• Central – What are the key concepts that thread through the entire class (no more than 7) (e.g. stereochemistry, electronic structure, functional groups, thermodynamics, kinetics, mechanism)

• Leveraged – What are the concepts that connect multiple units within the class (e.g. Acids and bases, nucleophilicity, electrophilicity…)

• Enduring – What concepts in this class will students need in future classes or career applications

The ideal number of course level learning objectives is somewhere between 7 and 10 and certainly not more than 15. For some state institutions course level learning objectives are mandated. Even if this is the case, going through the process of developing course level learning objectives forces a valuable reflection on the content and scope of a course. Once course level learning objectives are set these are broken down to unit level and lesson level objectives.

At each stage learning objectives provide a measurable learning target and thus guide instruction and focus learning. The gold standard for the development of measureable learning objectives is Bloom’s taxonomy.³ To be of maximum utility as a guide for instruction learning objectives should reflect how they will be measured. The goals must communicate both what is to be measured and the cognitive level at which the measurement will be structured. Bloom’s taxonomy provides a framework for accomplishing both these tasks. While it is certainly desirable for students to know or understand some fact or concept, understanding and knowledge are not directly measurable. Bloom’s taxonomy provides a mechanism for structuring learning objectives such that a measureable specified level of cognitive complexity is specified.

An example is provided below

• Course level: Analyze reaction mechanisms in terms of energetics, electron flow, reaction kinetics, and thermodynamics
• Unit Level: Distinguish between kinetic and thermodynamic control in electrophilic addition reactions
• Lesson Level: Predict the major product of an electrophilic addition to conjugated dienes

Learning objectives once established should be introduced early; preferably in the syllabus.⁴ The syllabus introduces the course structure to students. It typically contains a list of course graded course requirements but rarely how these requirements will be assessed. A large body of research suggests that assessment information can significantly improve learning.⁵ It provides benchmark against which students can measure their own progress. In so doing students improve their ability to regulate their own learning process.

The ability to regulate one’s own learning process or metacognition has been shown to be an important factor in learning and problem solving. The metacognitive skills of novices are generally rather weak. Students do not accurately assess what they do and do not know.⁶   In providing measurable benchmarks in a syllabus, prior to a lesson and in the form of a test blueprint prior to a test can significantly improve metacognitive skills. This process works best when learning objectives are reinforced often (before each lesson), recycled and integrated into new material, and used in the construction of test blueprints.

A test blueprint identifies the learning objectives that will be assessed, the level of cognitive complexity and the weight given to each objective.

The test blueprint should be specified before the test is written. The objectives assessed and the weights given to each objective should reflect actual classroom instruction in terms of content, cognitive complexity and emphasis. In the absence of a test blueprint, the development of an examination is often haphazard. Some learning objectives are over-emphasized and others neglected. Questions are selected based on cleverness or ease of writing rather than as an accurate reflection of learning goals. A test blueprint communicates to students the performance goals they are expected to achieve.

Most importantly the test blueprint should convey to students the behaviors that they are expected to achieve on the examination. The blueprint should be provided to students at least one week prior to the examination. Too often students believe that re-reading a text or going over notes is an effective way to study for an examination. By specifying performance measures for an examination, a blueprint can guide students to a more effective study patterns.  A test blueprint is not a study guide. It does much more than inform students of the content to be covered. Rather it is a set of performance goals against which students can benchmark their own learning.

After exams are graded students can compare their expected achievement on specified learning goals with their actual level of attainment. This feedback is helpful in focusing further study and in improving metacognitive skills. Cumulative data on class level achievement on each specified performance goals on the blueprint is also shared with students. This lets students see how they are doing relative to others on specific performance goals. In addition when overall class achievement on a learning goal is low this may be related to instruction or a misconception that has become entrenched. It can be very useful to discuss this with a class as it may lead to instructional improvements. One possible improvement might be the introduction of formative assessments that catch identified misconceptions. 

Classroom assessment can be broadly categorized as formative or summative. Summative assessments are typically exams given after completion of instruction, In contrast formative assessment is often integrated with instruction with the purpose of providing feedback to both instructor and student on progress towards a learning goal. Formative assessments frequently target common difficulties or misconceptions.  The purpose is to bring these into the open and correct them during instruction. Thus when a summative assessment reveals class level low achievement on a learning objective, the instructor may want to develop formative assessments on this objective to be used in future instruction. Clickers, warm up exercises, quizzes and class discussions are common forms of formative assessment.

In summary setting up aligned hierarchical learning objectives can provide data to improve both student learning and classroom instruction. Assessment data form measures that are aligned with hierarchical learning objectives further provides information that may be required for external reporting related to accreditation or certification. The aggregate data collected from examinations that were designed from blueprints provides information on class level achievement of learning objective. Setting up this scheme is not an easy task but the investment has substantial payoffs.

The process assures that there is alignment from course level objectives all the way to classroom assessments. Assessments designed with hierarchical learning objectives in mind better reflect instructor expectations and classroom instruction. Rich feedback is provided that can improve both teaching and learning. Communication to accreditation and certification agencies is facilitated.

References

1. Emenike, M.E.; Schoeder, J.; Murphy, K.; Holme, T. J. Chem. Educ. 2013, 90, 561-567.
2. Black, P.; Harrison, C; Lee, C.; Marshall, B.; Wiliam, D. Phi Delta Kappan  2004, 86 (1) 8-21.
3. Bloom’s Taxonomy taken from: http://www.celt.iastate.edu/teaching-resources/effective-practice/revisedblooms-taxonomy/
4. Ludwig, M. A.; Bentz, A.E.; Fynewever, H. Journal of College Science Teaching 2011,  40 (4) 20-23.
5. Black, P.; William, D. Phi Delta Kappan 1998 Phi Delta Kappan 80 (2) 139-148.
6. McGuire, S. Y. (2015) Teach Students How to Learn: Strategies You Can Incorporate Into Any Course to Improve Student Metacognition, Study Skills, and Motivation Stylus Publishing,  Sterling,  Virginia