Evidence-Centered Assessment Design (ECD)
and Design Patterns at AERA 2011
One symposium and one professional development course at AERA 2011 were devoted to the use of ECD Design Patterns to support assessment design and development activities across different contexts.
Symposium: How Design Patterns Address Three Pressing Challenges in Large-Scale Assessment
This symposium describes an assessment design methodology, design patterns, for improving the validity of inferences about students from performances in complex large-scale assessments. The presentations describe the framework of evidence-centered assessment design (ECD) and focus on the layer called domain modeling, where design patterns are used. Design patterns are a conceptual tool to guide assessment designers through the web of design decisions inherent in developing assessment tasks. Findings focus on the use of design patterns to address three challenges in large-scale assessment: (1) hard-to-assess standards in the domain of interest (e.g., inquiry science skills), (2) accessibility of tasks for diverse learners with widely varying perceptual and expressive capabilities, and (3) efficient and valid design and development of complex tasks.
- The Critical Role of Design Patterns in Large-Scale Assessment (Mislevy, Haertel, Liu)
- Design Patterns to Support Storyboards and Scenario-Based, Innovative Item Types (Snow, Fullkerson, Nichols, Feng)
- How Design Patterns Integrate Universal Design for Learning (UDL) Into Assessments for Students With Disabilities
(Murray, Cheng; Cameto, Haertel, DeBarger, Morrison)
- Design Patterns Based on a Cognitive Perspective on Learning (Yarnall)
Professional Development Course: Designing Scenario-based Assessment Items Using an Evidence-Centered Design Framework
(Snow, Mislevy, Haertel, Feng, Fulkerson, Nichols)
The course introduces design patterns under the evidence-centered design framework as a high level tool to guide the
design and development of scenario-based, innovative assessment items. The course includes lecture, discussion, and
group activities. Participants will use design patterns to: (1) select a scenario topic; (2) identify the focal knowledge,
skills and abilities to be assessed; (3) conceptualize sequential scenes and "item ideas"; and (4) identify sources of
construct irrelevant variance that may contribute to students' poor performance and ways to reduce the variance through task design.
- Presentation Slide Deck
- Resource Materials: