An adapted process to develop customized assessments to measure the impact of biology lessons

Abstract When instructors, curriculum developers, or education researchers implement an innovative lesson or develop a shareable lesson plan, they likely want to know if the lesson is effective at facilitating student learning. Published assessments, which have undergone a substantial development and optimization process, are commonly used to evaluate the impact of curricular and pedagogical innovations. However, there are some cases when using an existing concept assessment does not align with the goals of the person administering the assessment. Concept assessments usually cover a broad content domain spanning entire units, courses, or majors. If an instructor, curriculum developer, or researcher intends to assess learning associated with a discrete time period (e.g., a lesson), a concept assessment may be too broad and thus lack alignment validity. Consequently, they may decide to develop their own assessment. They can accomplish this by writing original questions, using question banks, using questions provided in open educational resources, or a combination of these approaches. Alternatively, they may pursue a time- and resource-intensive comprehensive assessment development process, typically involving several steps to accrue validity evidence. There are currently no guidelines for scaling the process for contexts in which existing concept assessments are misaligned and a comprehensive assessment development process is impractical. Here, we describe an adapted assessment development process to generate lesson-customized assessments. We started with five published biology lessons, covering a broad range of biology topics, and created a lesson-customized assessment for each. Our assessment development process consisted of identifying lesson learning objectives, drafting an initial assessment, soliciting expert feedback and student interviews, and conducting several cycles of review and revision. We provide suggestions to help with practicality and make the process more accessible. Though we developed customized assessments for published biology lessons, our approach is broadly applicable to other cases where customized assessments may be beneficial.