The Maryland School Assessment (MSA) is a test of reading, math, and science achievement. This test provides educators, parents, and the public valuable information about student, school, school system, and state performance.
Before the state could hold schools accountable for student proficiency on the new state assessment, proficiency had to be defined. Setting proficiency standards for the Maryland School Assessment (MSA) involved determining proficient and advanced performance on the tests.
The Performance Standard determines where the cut score is for students considered to be performing at the proficient and advanced levels. All other students will be considered to be performing at the basic level.
A large number of educators and stakeholders were involved in setting standards through a structured process. Listen to Gary Health describe: Why did we need to set new standards? What was the standards setting process? Who participated in the standards setting process? How did the process work? Who reviewed the work? Who approved the standards?
School systems nominated teachers, principals, and school system staff with subject-matter and grade-level expertise to serve on 8 standard-setting groups, alongside representatives of various education organizations (e.g., Maryland PTA, MSTA). The reading MSA required four groups one each for grades 3, 5, 8, and 10. The math MSA required four more.
Standard-setting requires several rounds of discussion and voting to establish two cut scores for the MSA (one between basic and proficient and one between proficient and advanced).
During the first round, the 8 groups broke into smaller groups and took the test. Given an ordered-item booklet a booklet in which test items are arranged from those that most students answer correctly to those that fewer and fewer students do each member put a bookmark on the test item he or she considered the dividing line between basic and proficient performance. (That is, if students can answer that item correctly, as well as all those that came before it, they are proficient in the subject.) Each member placed another bookmark on the test item he or she thought divided proficient and advanced performance1 (see Figure 1). Each item correlates to a scale score, so that by choosing an item, the members actually chose a cut score. Members discussed their selected scores and established the median.
Members voted again for a cut score, again reviewed their votes, and established a new median. They also examined impact data, which is the percentage of students (disaggregated by race/ethnicity and special services received 2 and reported by state and school system) that will make the proficient and advanced cuts (or pass/not pass cut) given the selected scores.
During Round 3, the small groups converged into the original 8. Members voted and discussed yet again, before logging a final vote (a total of four votes for each member). The groups sent their final median cut scores for proficient and advanced performance to a Psychometric Council.
The Psychometric Council reviewed the work of all groups, made sure that quality controls were followed, ensured the standard-setting process was technically sound, and forwarded the recommendations and comments to the Review & Articulation Committee.
Review & Articulation Committee
The Review & Articulation Committee reviewed the work of all groups and the Psychometric Council, ensured that rigor was equivalent across grades and subjects, reviewed the articulation among grades and subjects, and forwarded the recommendations to Dr. Grasmick.
Dr. Grasmick reviewed the work of the groups, the Psychometric Council, and the Review & Articulation Committee and made a final recommendation to the State Board of Education.
State Board of Education
The State Board set MSA cut scores during its July meeting.