Step 1: Analyze the Data
A Principal’s Role:
Analyzing your data is a process you will want to involve your entire staff. There are a number of variables that will help you determine the best strategy for your school including the size of staff, organization of teams, availability of computers with Internet, and the amount of staff meeting time.
The critical piece is that you model the importance of data analysis and that you involve (mandate) all staff in the process. The odds of teachers making the instructional changes needed for improved student achievement are much greater when the data and what it tells them about current achievement.
A process recommended for data analysis of the State Accountability and MSA or HSA data is that you assign teams of teachers (grade level, interdisciplinary, department, or random) the detailed analysis of one of the testing areas using the graphed data in the “Analyzing Your State Assessments Data” section of the web site and the worksheet in the same section. A middle school principal might assign the interdisciplinary teams to analyze the reading scores whereas the math team would analyze the math scores, and starting in 2008, the science team would analyze the science scores.
The assignment could be given at a staff meeting where the principal introduced some key results of MSA using the web site to show selected graphs or using transparencies printed out from the web site. This overview should focus on both areas of strength and need and disaggregated as well as aggregated data. Low performing schools may need to dig a little deeper for areas to celebrate. You might also print out the Data Analysis worksheet for this staff meeting so that you can discuss the team data analysis assignment more specifically.
The teams would be expected to complete their data analysis at a regularly scheduled team time (if there is a regularly scheduled time). They would be expected to report back their findings at an upcoming staff meeting so that all staff would get a complete picture of the student performance at their school.
Because HSA is given at the completion of each of the assessed courses, the teachers of those courses will have a particular interest in those data. Principals may want to assign those teachers to analyze and report back to the leadership team or the school improvement team the results along with identified areas of strengths and needs and with a list of questions raised by the data.
Once the data has been studied and strengths and weaknesses identified, the school improvement team needs to prioritize the needs as indicated by the data. Though a good instructional program will address all of the appropriate content standards, a good school improvement plan will focus a school-wide effort on one or two priority instructional needs.
Often your data is very clear about your areas of need. Sometimes the data indicates too many areas of need to be able to focus on in one year. In the latter case, you will need to involve staff in identifying which areas should be focused on in the coming year. Teachers are responsible for teaching the district curriculum which, hopefully, has been aligned with the state content standards. In addition to their curriculum, however, teachers will be expected to support and accommodate the school-wide objectives in their classrooms, where appropriate. It must also be understood that a significant amount of energy will be exerted in the monitoring of the school improvement plan and the capacity building of staff to implement the school improvement strategies.
- Data Analysis Worksheet
- Analyzing your State Accountability Data
- Analyzing your MSA Data
- Analyzing your HSA Data
Step 2: Clarify the Problem
A Principal’s Role:
Clarifying why your data results are what they are will increase the odds that the strategies you select to improve student performance will produce the results you want.
To lead your school through this process, you need to work with the entire staff to address the following questions:
- What is the problem?
- What evidence do you have that this is the problem?
- Which of these problems do we have control over?
- Which of the remaining problems have the greatest impact on our performance?
Because you will want to get the input of all your staff, as well as other representative stakeholders, on the question of why your data is strong or weak in a certain area, you will want to identify appropriate times to solicit feedback from a variety of groups.
A principal might introduce the fishbone exercise at a staff meeting and then assign grade-level, interdisciplinary, or department teams to complete the worksheet. They can then report back their consensus on the high impact root causes or contributing factors at the following staff meeting. Though you could also make the assignment directly to the team or department leaders, introducing the exercise at a staff meeting communicates a clear message to all staff that this is an important exercise. The principal might also facilitate the fishbone exercise with the PTA Board or a selected group of students, parents, and community members.
You will also want to emphasize to your staff that the identification of the problem for your data being low in a specified area will drive the identification of the strategies needed to improve the results in your new school improvement plan. The more accurate your identification of the problem, the more likely you will choose strategies that result in improved performance.
Once you have identified a few root causes or contributing factors that have a high impact on your data results and are in your control, you need to remind staff that these are still hypotheses and need to be tested. You could have requested evidence that this was a root cause during the process of filtering the causes for control and high impact. But because this process is time consuming, I would recommend not testing the hypotheses until you have reached consensus on a few high impact causes.
Schools with lower performance will focus on revamping their instructional program whereas schools with higher performance will focus on fine-tuning their instructional program. Schools with the lowest performance results will be looking for broader instructional program issues. For instance, they may be exploring whether there is a curriculum, whether it is being followed, whether teachers understand how to teach the curriculum, assess student progress, and use a variety of strategies to work with a wide diversity of learning styles and needs. Schools with higher performance results will focus more sharply on identifying their problem and honing in on what specific instructional topics they need to improve their instructional performance. When student performance scores are flat over the last several years, you may need to remind staff that
“if you continue to do what you have been doing in about the same way that you have been doing it, then you can expect to get the same results you have been getting.”
They will need to try something different if they want their scores to be different. This is particularly true for schools whose scores have been flat for the last six years.
The following questions form the basis for an exploration of why students aren’t performing well in a testing area:
- Do all staff understand what students are asked to know and do on the state assessments?
- Do all staff understand how student performance is scored on the state assessments and what proficient and advanced student responses looks like?
- Do all staff know how to teach the content standards?
- Do all staff know how to assess the content standards?
- Are all staff monitoring progress of individual students on these standards?
- Do all staff provide interventions for students not demonstrating attainment of an indicator/objective?
The more staff and other stakeholders focus on contributing factors in their instructional program, the more control they will have in addressing the problem, and, consequently, the more impact their strategies will have on producing better results.
One way to check on your group’s process is to have members from another group (another school or another team within your school) complete a peer review. The peer review worksheet provides a structure for hearing a group’s consensus on the high impact root causes or contributing factors for why their data looks like it does and reviewing the consensus decisions based on a series of questions:
- Are the root causes they have identified really in the school’s control? Can staff do something to change that cause?
- Are the root causes really high impact? Are they likely to result in improved performance if you try to address them?
- Are the root causes stated in specific enough terms so that you know how to address them?
- Are there root causes that might be more likely to have greater impact on improved performance in the area identified?
The peer reviewers will also have to ask clarification questions like “Why did you decide…?” or “What led you to …?” as well as questions that expand their thinking like “Have you thought about…?” or “Did you explore…?”.
- Fishbone: MSA and Our Instructional Program
- Fishbone: HSA and Our Instructional Program
- Hypothesis Testing
- Collecting Additional Data
- Peer Review of Root Causes
- Illustrative Questions to Clarify Why Your Data Looks Like it Does
- Collecting Data about Your Instructional Program
Step 3: Create the Action Plan
A Principal’s Role:
By the time your staff have completed the data analysis and problem clarification, you probably have a clear understanding of what your objective needs to be. The greater challenge in this step is deciding what evidence you will accept that it has been met and how you will monitor progress toward that objective. Many schools use the percent of students scoring satisfactory on a section of the MSA test as evidence. That makes your end of year evidence easy since that data is already collected and disseminated to you. In that case your greater challenge is to decide how to monitor progress. What data do you (or could you) collect every two or three weeks that would indicate that you are making sufficient progress toward your end goal/objective?
This might be a good time to open the discussion about how teachers track the progress of each of their students toward the attainment of proficiency on content standards. If teachers do not monitor individual student progress (or if they don't know how to monitor for progress) then they are unable to provide appropriate interventions or change their instructional delivery to ensure greater student understanding and progress. They may need to re-look at the tests they give and rethink the kind of assessments that would provide them the information they need to determine that all students are making progress toward school objectives.
When selecting the best strategies for your school improvement plan, you should base your selection on the data analysis and problem clarification outcomes.
Step 4: Implement the Action Plan
A Principal’s Role
Once the action plan has been created, your school improvement team and you need to make sure the implementation plan is doable. This would include making sure that the implementation calendar does not overload staff or students during any one month and that your priorities are clear in how you are asking staff to use their time.
Now you will need to communicate the plan and your expectations for what role each staff member plays in the implementation of the action plan. If you have engaged your staff throughout the process, then the communication will only be an affirmation of what they already know. If you have not done that, then you will need to attend to their buy-in as well as communicate your expectations.
You may have already anticipated some capacity building needs of staff to be able to implement the action plan. Consequently, your staff development plan needs to address the greatest needs of staff to implement the strategies you have chosen to be able to improve student achievement. Since we already have learned that quality professional development is ongoing, then you may want to address some of these capacity building needs through team meeting time across the year. This allows for job-embedded “just in time” learning. The collaborative examination of student work provides and excellent opportunity to build capacity during the process. For example, when teachers are discussing what makes a proficient response on an assignment or assessment, they often self-diagnose that they did not fully understand the intent of an indicator or objective. Therefore they will be more receptive to a content expert providing some additional support at that time. In fact, the process of collaborative examinations of student work follows the professional learning community model which in turn is another model for building capacity.
Step 5: Monitor Progress
A Principal’s Role:
In most organizations, what gets monitored gets done. Even clearly stated curricular goals will lose their potential to drive the efforts of a school if no effort is made to collect and analyze accurate information about student achievement that is reflective of those goals.
Staff learn what you value by observing what you pay attention to. When you devote considerable time and effort to the continual assessment of a particular condition or outcome, it notifies all members that the condition or outcome is considered important. Conversely, inattention to monitoring a particular factor in a school indicates that it is less than essential, regardless of how often its importance is verbalized. Your role, then, will be to spend a good deal of time on the monitoring and data analysis processes. Use staff meeting time to highlight what your data is telling you about progress and use team time to model for staff the questions you need them to address in their monitoring progress activities. When everyone is clear that you place a high value on staff collecting and analyzing monitoring data (including the regular examination of student work), then staff will be clear that you are very serious about your expectations for them. You will also want to celebrate those staff who are modeling good data-driven decision making in their classrooms and showcase those strategies in staff meetings.
Another way you demonstrate your value on monitoring is to regularly communicate student achievement progress in newsletters and other correspondence to staff, parents, and other stakeholders. You might also track progress on a entryway bulletin board. You should not miss a communication opportunity to reinforce that your top priority is improved student achievement.
Step 6: Act on Needed Changes
Once you have engaged your staff in the regular analysis of their monitoring data and a discussion of their findings from regular examinations of student work, you will be able to assess whether your school is on target to meet your achievement goal or not. If the rate of progress (or collection of data) does not reassure you that your school is on track to hit your target, then you need to do some additional problem solving to determine what the problem is and how you might pick up speed in improving student achievement. Any decisions you make will need to be data-driven. You may need to reassess whether you now have in place the five key processes described in the problem clarification action. All of them need to be in place for you to hit your target. Staff may need some re-direction in how they are spending their time. They will be much more willing to buy-in to needed changes if they have also been engaged in the data analysis. Do not let too much time slip by before you recognize you need a change in strategy. Since data collection and analysis is the key to making this decision, then you also need to avoid letting too much time slip by before you have collected and analyzed your monitoring data.