Sometime in the next two years your school district will formally convene the District’s PERA Joint Committee to discuss and agree upon the various elements required by law for performance based teacher evaluations. Administrators should be careful to not expand the duties, authority, and jurisdiction of the PERA Joint Committee beyond the law. Please check with your legal counsel before discussing teacher evaluation issues such as how to determine the relative importance of the various components within the framework or how to determine summative ratings.

Management retains certain rights under this law and you need to be familiar with these rights and exercise them accordingly. For example, there will be an equal number of representatives on the PERA Joint Committee and the union has no right to have input into who the management members of this committee are. You could obviously talk about this and agree on the members but this is a management decision to appoint the management members.

The evaluation result or rating is a management right. Be careful of any language related to an appeal process to the teacher rating. Again, consult with your legal counsel prior to making this decision.

The persons who will do the evaluating are a management right as is the training of these individuals. The PERA Joint Committee should not be discussing or agreeing to language that would limit this right. For example, the committee should not be discussing continued training and inter-rater reliability of the raters.

When I train school administrators on using the Danielson Frameworks for Teaching (FFT) I am often asked how to score (rate) the professional practice of teaching. A good place to start this discussion is the ISBE model process for summative rating. This model system has been created and published already and is on the ISBE/PEAC website at http://www.isbe.state.il.us/PEAC/default.htm The following is information copied from the website:

Subject

This document provides a description of excellent, proficient, needs improvement and unsatisfactory levels of teacher practice.

Model System Component

The Performance Evaluation Reform Act (PERA) requires that teachers “be rated using the 4 categories of ‘excellent’, ‘proficient’, ‘needs improvement’, or ‘unsatisfactory’” (PERA Sec. 2-3.25[b]).

Explanation

 In the Model Teacher Evaluation System, districts will provide teachers with an overall teacher practice rating and overall student growth rating. The overall teacher practice rating and the overall student growth rating will be combined to create an overall summative rating using the decision matrix (see Table 1).

Table 1. Overall Summative Rating Decision Matrix

For the Model Teacher Evaluation System, Excellent, Proficient, Needs Improvement and Unsatisfactory performance will be defined as below.

• Excellent: Teacher receives a rating of excellent in both practice and student growth; or is proficient in practice and excellent in student growth; or is excellent in practice and proficient in student growth.

• Proficient: Teacher receives a rating of proficient in both practice and student growth; or is needs improvement in practice and excellent in student growth; or is excellent in practice and needs improvement in student growth; or is needs improvement in practice and proficient in student growth; or is proficient in practice and needs improvement in student growth. A teacher also may be proficient if rated excellent in practice and unsatisfactory in growth or unsatisfactory in practice and excellent in growth and after additional evidence is collected, those ratings are confirmed as being accurate.

• Needs improvement: Teacher receives a rating of needs improvement in both practice and student growth; or is unsatisfactory in practice and proficient in student growth; or is proficient in practice and unsatisfactory in student growth; or is unsatisfactory in practice and needs improvement in student growth; or is needs improvement in practice and unsatisfactory in student growth.

• Unsatisfactory: Teacher receives a rating of unsatisfactory in practice and student growth.

Frequently Asked Question

1. Is our district required to adopt these definitions?

The only districts that are required to follow these steps are those implementing the Model Teacher Evaluation System, which occurs when the Joint Committee cannot come to a decision on this aspect of student growth within 180 days of the first meeting of the Joint Committee. See PERA Section 24A-4(b) for more information on the Model Teacher Evaluation System and deadlines for a Joint Committee. These definitions are not appropriate for districts not implementing the overall decision matrix included above as Table 1.

The information above does not give direction on how to actually calculate the Overall Teacher Practice Rating. Thus, this will become a decision that the Joint Committee in your school district will have to make.

Charlotte Danielson designed the Frameworks for Teaching to be used for professional development of teachers, not for rating of teachers. I do not believe that Danielson has ever put specific percentages on the various components. I have heard Danielson state and I have read in her books that she believes that Domain 3, Instruction, is the heart of the frameworks. She goes on to state that 3c, Engaged Learning, is the heart of the heart of the frameworks. Thus, I interpret this to mean that Domain 3 is most important with 3c being the component within Domain 3 with the most importance.

Danielson goes on to say that 2a, Respect & Rapport, is vital to the student learning and I interpret this to be the most important in Domain 2 with Domain 2 being second in overall importance.

Domains 3 and 4 follow domains 1 and 2 in importance. Component 1e, Designing Coherent Instruction, is the most important component in Domain 1 and Danielson states is the heart of planning and instruction. Component 4e, Reflecting on Teaching, is vitally important because of Danielson’s basic belief that “the learner has to be intellectually involved in order to learn.” This applies to teachers as well as students.

My personal interpretation of the weights of the various components is in the following breakdown. (Remember, these are my interpretations of the Danielson Frameworks for Teaching.) The Joint Committee could also decide to use words to calculate the Overall Teacher Practice Rating.

Domain 1: Planning and Preparation;

1a=5%, 1b=5%, 1c=3%, 1d=3%, 1e=3%, and 1f=3%.

Domain 2: The Classroom Environment;

2a=5%, 2b=3%, 2c=3%, 2d=3%, 2e=2%.

Domain 3: Instruction;

3a=6%, 3b=10%, 3c=15%, 3d=6%, 3e=2%.

Domain 4: Professional Responsibilities

4a=7%, 4b=2%, 4c=4%, 4d=4%, 4e=4%, 4f 2%.

For what it is worth, this would be my summary score range:

80 to 100% Excellent

65 to 79% Proficient

50 to 64% Needs Improvement

Below 50% Unsatisfactory

In my opinion there is much confusion over the use of the words “model” and “default” in the State of Illinois new Performance Based Teacher Evaluation System.

Most school districts (the top performing 80%) will not have its Performance Based Teacher Evaluation system in place until the start of the 2016 school year. The lowest 20% districts will need the system in place by 2015 and some Title I grant receiving districts will be implementing sooner.

ISBE has declared that the Danielson Frameworks is the state model for practice based teacher evaluation. It is not the “default model,” it is a “model.” The following sentence was copied from the ISBE Website “The classroom observation portion of the state-designed optional evaluation system is being adapted from Charlotte Danielson’s Framework for Professional Practice, widely used for evaluations by districts across the country.”

School district joint committees are free to develop their own model but if they do they need to make sure they meet the requirements of the TITLE 23:  EDUCATION AND CULTURAL RESOURCES SUBTITLE A:  EDUCATION CHAPTER I:  STATE BOARD OF EDUCATIONSUBCHAPTER b:  PERSONNEL PART 50 EVALUATION OF CERTIFIED EMPLOYEES UNDER ARTICLE 24A OF THE SCHOOL CODE. SUBPART A:  GENERAL REQUIREMENTS. Several important concepts in the Part 50 rules that the Joint Committee will need to consider are the following:

• “The evaluation plan shall contain a rubric to be used in rating professional practice that aligns to the instructional framework.”
• “Instructional framework developed or adopted by the school district that is based upon research regarding effective instruction; addresses at least planning, instructional delivery, and classroom management; and aligns to the Illinois Professional Teaching Standards.”
• “The evaluation plan shall consider the teacher’s attendance and competency in the subject matter taught, as well as specify the teacher’s strengths and weaknesses and the reasons for identifying the areas as such.”
• The district shall quantify the relative importance of each portion of the framework to the final professional practice rating.

The state model plan that becomes the “default plan” is referred to in this following sentence from the Part 50 Rules: “In situations where a joint committee cannot reach agreement on one or more aspects of student growth within the timeline established under Section 24A-4(b) of the School Code, the school district shall adopt the State model plan contained in Subpart C of this Part with respect to those aspects of student growth upon which no agreement was reached.”

The PEAC is in the process of determining the “State Model Plan” for student growth. Updated information on this process can be accessed at the ISBE website http://www.isbe.net/PEAC/default.htm

In a previous blog article I summarized the findings of the MET Study Ensuring Fair and Reliable Measures of Effective Teaching. The MET project researched what measures could be used to identify effective teaching fairly and reliably. In the initial MET report in 2010 the researchers found that “a well-designed student perception survey can provide reliable feedback on aspects of teaching practices that are predictive of student learning.” The 2012 study Gathering Feedback for Teaching “presented similar results for classroom observations.”

Illinois districts have for the first time (outside of the City of Chicago that has been giving this student survey for many years) given students a survey titled “Illinois 5Essentials Survey.” This 5Essentials survey was given to students in 6^th through 12^th grades.  According to research completed on the 5Essentials Survey “Research shows that schools strong on the five essentials are ten times more likely to improve student learning than schools weak on the five essentials.” The five essentials are the following:

• Effective leaders: The principal works with teachers to implement a clear and strategic vision for school success.
• Collaborative teachers: The staff is committed to the school, receives strong professional development, and works together to improve the school.
• Involved families: The entire school staff builds strong relationships with families and communities to support learning.
• Supportive environment: The school is safe and orderly. Teachers have high expectations for students. Students are supported by their teachers and peers.
• Ambitious instruction: Classes are academically demanding and engage students by emphasizing the application of knowledge.

The MET Project used the Tripod Survey. The Tripod survey identifies seven constructs—the 7 Cs—that are core to a student’s experience in his or her classroom. They are 1) care; 2) control; 3) clarify; 4) challenge; 5) captivate; 6) confer and 7) consolidate.

The study concluded, “The average student knows effective teaching when he or she experiences it.”

The MET Project recommends a score weighting of either 1) 33% for Student surveys, 33% for Observations and 33% for Achievement gains on state tests or 2) 50% on Observations, 25% for Student surveys and 25% for Achievement gains on state tests to achieve the highest reliability for teacher evaluation.

It will be interesting to see if the State of Illinois will take these MET Research findings into account when deciding on the final default model for Illinois Performance Based Teacher Evaluation. This is a topic Illinois school districts should be discussing with its Teacher Evaluation Joint Committee members in an informal setting prior to the formal announcement of the Joint Committee work.

The Measures of Effective Teaching (MET) Project has three major findings; 1) student achievement gains are the direct result of more effective instruction; 2) trained evaluators using a research based evaluation instrument (Danielson Frameworks for Teaching) does result in proper summative practice teacher evaluation; and 3) student perception surveys are effective in determining effective teaching.

The most important finding of the (MET) is that research validated the theory that teaching ability affects student achievement taking into account that students have various learning characteristics. The student academic achievement scores of the most effective teachers were significantly higher than the student scores of the least effective teachers. In addition, the student scores of the least effective teachers actually decreased.

This study showed that more effective teachers are truly better than other teachers at improving student learning, not that they simply have better students. The study collected data and produced estimates of teaching effectiveness for each teacher. The study then adjusted the estimates to account for student differences in prior test scores, demographics, and other traits. The study then randomly assigned a classroom of students to each participating teacher.

The adjusted measures did identify teachers who produced higher (and lower) average student achievement gains following random assignment. The data showed that we can identify groups of teachers who are more effective in helping students learn. Moreover, the magnitude of the achievement gains that teachers generated was consistent with expectations.

The more effective teachers not only caused students to perform better on state tests, but they also caused students to score higher on other, more cognitively challenging assessments in math and English.

The study pointed out that “Teaching is too complex for any single measure of performance to capture it accurately. Identifying great teachers requires multiple measures.” The MET project identified three composites that reliably predicted student academic growth. They were 1) state assessments, 2) classroom observations, and 3) student surveys.

The project went on to say that significantly placing a lot of weight on one measure such as state assessments will lead to unintended consequences. In the case of weighting state assessments as more than 50% weight, results in teachers teaching to the assessment and leaving out of the curriculum of other important education goals such as higher order thinking and creativity. The study concluded “Heavily weighting a single measure may incentivize teachers to focus too narrowly on a single aspect of effective teaching and neglect its other important aspects….If the goal is for students to meet a broader set of learning objectives than are measured by a state’s tests, then too-heavily weighting that test could make it harder to identify teachers who are producing other valued outcomes.”

The project studied four ways to weight the measures. They are the following:

1. Model 1 places 81% of the weight on the achievement gains on state tests; 17% on student surveys and 2% on observations.
2. Model 2 places 50% of the weights on achievement gains on state tests; 25% on student surveys and 25% on observations.
3. Model 3 places 33% of the weights on achievement gains on state tests; 33% on student surveys and 33% on observations.
4. Model 4 places 25% of the weights on achievement gains on state tests; 25% on student surveys and 50% on observations.

Model 1 with the highest percentage (81%) on achievement gains on state tests resulted in the highest correlation with state tests gains. However, it also had the lowest correlation with higher-order tests. Model 3 with an equal 33% weight on each measure produced the highest correlation, .76, to reliability reflecting the year-to-year stability of teacher’s results. Model 4 had a .75 correlation to year-to-year stability of teacher’s results based on 50% on observations, 25% on achievement tests and 25% on student surveys.

Other important findings from this project are the following: 1) the student data that is being used needs to be accurate; 2) classroom observers not only need training on the instrument they need continual scoring practice with master observers; 3) observations should be done by more than one observer; and 4) multiple years of student achievement gains, observations, and student surveys should be used if they are available.

The following table illustrates various methods for districts to consider when observing teachers. The MET project states “Adding lessons and observers increases the reliability of classroom observations. In our estimates, if a teacher’s results are based on two lessons, having the second lesson scored by a second observer can boost reliability significantly.”

Another significant finding was “Our analysis from Hillsborough County showed observations based on the first 15 minutes of lessons were about 60 percent as reliable as full lesson observations, while requiring one-third as much observer time.”

Reliability measures for observations are listed below from highest to lowest:

1. .72 reliability- Two lessons observed by own administrator; one lesson observed by peer observer; three 15-minute lessons observed by three different observers
2. .69 reliability- Two lessons observed by own administrator; one lesson observed by one peer observer; one lesson observed by different peer observer
3. .67 - One lesson observed by own administrator; three 15-minute lessons observed by three different observers
4. .67 – One lesson observed by own administrator; one lesson observed by peer observer
5. .66 – Two lessons observed by own administrator; two lessons observed by peer observer
6. .58 – Two lessons observed by own administrator
7. .51 – One lesson observed by own administrator

A Peoria County judge threw out a lawsuit Friday filed by two former Peoria School District teachers who claimed they were terminated improperly after receiving poor evaluations. This case contested the new Illinois law that places teachers in one of four groups based on their teacher performance evaluation ratings. Teachers rated as Unsatisfactory are placed in Group 1, teachers rated Needs Improvement are placed in Group 2, teachers rated Proficient in Group 3 and teachers rated Excellent in Group 4. Teachers in Groups 1 and 2 are not considered for reemployment if the district has a Reduction in Force. Teachers are laid off first from Group 1 then progressing to Group 4.

The Peoria teachers sued because they said the district did not in fact Reduce in Force any teachers even though several teachers were given RIF notices in the spring. The judge ruled “Chief Judge Michael Brandt ruled in a two-page decision the district acted properly when it pink-slipped Eymarde Lawler and Michelle Frakes at the end of the school year.” Brandt held that “the uncertainty of enrollment and state funding were valid reasons for a reduction-in-force at the end of a school year.”

The newspaper article in the Peoria Journal Star stated “Gil Feldman of Chicago, the teachers’ attorney, claimed at a hearing earlier this month a ruling in favor of the district would render seniority rights meaningless.”

I have contemplated writing this article about the use of student growth metrics as a measure to evaluate teachers for several months. I am still in the process of forming my final decision but I do know that my own interpretation of using student growth has drastically changed in the two plus years I have been a member of the Performance Evaluation Advisory Council (PEAC).

At the least I would suggest that there are major flaws in using student growth scores for teacher evaluation. However, I do believe we need to look at and consider student growth for both teacher and principal evaluation purposes. If I had the power to make a state level recommendation (which I obviously do not) I would recommend that Illinois use student growth scores in the area of math and reading only and slowly incorporate the score into the teacher evaluation summative rating. An example of this would be the following, use valid and reliable (Type I Assessments) for all teachers who teach reading and math and only count that score for 10% of the teacher summative rating the first two years, 20% the next two years and 30% the next two years and after six years form another committee to determine if the scores are accurately predicting teacher performance. If the scores were determined to not be accurate they would no longer be used for teacher evaluation purposes.

I tweeted a reference to a very good article on the use of student growth for teacher evaluation titled “Who will be held responsible when state officials are factually wrong? On statistics & Teacher Evaluation.” You can read this article here and draw your own conclusion.