School Improvement in Maryland

The Scoring Process

  

All assessment items go through an elaborate process of content, style, and language revision. Revision of items is necessary:

  • to verify content,
  • to improve the Item Stem and/or the Answer Choices,
  • to make the item clear and concise,
  • to ensure that it appropriately addresses the indicator that it has been designed to measure, and
  • to make the item as accessible as possible to all students.

To begin the process, items are reviewed and refined several times by staff from MSDE and ETS, the HSA test development contractor. Next, each item is examined by an independent Item Review Team made up of content experts who have not been part of the development process. Included in this group are biology teachers, coordinators and supervisors of science, representatives from higher education, and teachers of English as a Second Language (ESOL) and Special Education who have experience in teaching biology. Because they have not been involved in the initial development process, these content experts bring a fresh perspective that greatly improves the quality of the items. Their objective eyes detect the subtler elements of content and style that previous reviewers (those more intimately involved in development of the items) did not see. Finally, items are put through a comprehensive Sensitivity Review to detect any form of bias that was inadvertently overlooked during the other reviews. Each of these steps is required before items are approved for use in field tests. After field testing, items may be further revised to reflect changes suggested by scorers and others who analyze the field test data.

MSDE has worked extensively with Special Education and ESOL teachers to develop a set of “Language Guidelines for Biology”. These guidelines have helped to minimize problematic or awkward language constructions that make items less accessible to special education and ESOL students. The guidelines are employed from the start of item development; however, their greatest benefit comes into play when the items are reviewed by the Item Review Team. It is at this stage of the process that accessibility of the items by various student populations becomes of primary concern.