"Measuring" What We Do: The Changing Landscape of Teacher ...
Measuring Up: Effective Strategies for Teacher Evaluation Patricia A. Popp, Ph.D. State Coordinator, Project HOPE-VA Clinical Associate Professor [email protected] Xianxuan Xu, Ph.D. Post-Doctoral Research Associate [email protected] School of Education The College of William and Mary Teacher Evaluation in an Era of Accountability Currently, ESEA flexibility has been granted to 34 states and the District of Columbia. As part of the flexibility requirements, the states were required to establish new teacher evaluation systems that factor in student
achievement progress for statewide implementation by the end of the 2013-2014 school year. All states that received Race to the Top funding are undertaking substantial reforms with teacher evaluation. The quality of an education system cannot exceed the quality of its teachers (Barber & Mourshed, 2008). Share of At-Risk Students on PISA (Not reaching PISA baselines): Reading 25 21 20 15 11 10 5
0 10 8 5 8 Share of At-Risk Students on PISA (Not reaching PISA baselines): Math 25 21 20 15 10 5
0 11 10 8 5 8 Percentage of Countrys Students in PISA Top Performing Groups: Reading 19 20 16 12
8 4 0 13 15 16 13 10 Percentage of Countrys Students in PISA Top Performing Groups: Math 60 50
50 36 40 30 20 10 0 18 21 26 10 What do we do about it?
How Long Do Students Attend School? Days in the Academic Year Country Canada Finland Singapore Shanghai South Korea United States Days in an Academic Year Average: 188 187 200 180 204 180
How Long Do Students Attend Schools? Minutes in the Day Location Canada Finland Shanghai Singapore South Korea United States Minutes in a School Day 304 240 390 330 264 402
How Much Do We Spend? Location Canada Finland Shanghai Singapore South Korea United States Annual Expenditures Per Pupil 8,045 7,216 N/A N/A 6,663
10,259 Student/Teacher Ratio Country Average Student/Teacher Ratio Canada Finland Shanghai Singapore South Korea United States 25 19 39
35 36 24 Source: Available at worldbank.org. Which factor is a strong predictor of student achievement gains? Class size Classroom heterogeneity School resource differences Its the teacher. Sources: Wright, Horn, & Sanders, 1997; Hanushek, Kain, & Rivkin, 1998, Influences on Student Achievement:
Explained Variance Source: Hattie, J. Teachers make a difference: What is the research evidence. Retrieved 20Nov08 from http://acer.edu.au/documents Dallas Research: Teacher Quality 4th gr. Math Achievement 100 80 60 40 1st grade 0 4th grade
20 Highly Effective Ineffective Dallas, Texas data: 2800-3200 students per cohort Comparison of 3 highly effective & 3 ineffective teachers (Jordan, Mendro, & Weerasinghe, 1997) Dallas Research: Teacher Quality 4th gr. Reading Achievement 70 60 50 40 30 20
1st G... 0 4th ... 10 Highly Effective Ineffective Time in School Year Needed to Achieve the Same Amount of Learning 75th Percentile Teacher 25th Percentile Teacher 0
1/4 1/2 3/4 1 Years Needed Leigh, Economics of Education Review (2010) Time in School Needed to Achieve the Same Amount of Learning Time in School Year Needed to Achieve the Same Amount of Learning 90th Percentile Teacher
10th Percentile Teacher 0 1/ 4 1/ 2 3/ 4 Years Needed Source: Leigh, A. (n.d.). Estimating teacher effectiveness from two-year changes in students test scores. Retrieved May 22, 2007, from http://econrsss.anu.edu.au/~aleigh/. 1
Sequence of Effective Teachers Low Low Low High High High 52-54 percentile points difference
Sanders & Rivers (1996) Sequence of Effective Teachers Low Low High High High High 13 percentile points
difference Sanders & Rivers (1996) Revised Teacher Evaluation System in Virginia: An Overview Primary Purposes of the Evaluation System Improve student achievement through the quality of instruction by assuring accountability for classroom performance Contribute to the successful achievement of the goals and objectives defined in a school divisions educational plans Provide a basis for instructional improvement through productive teacher appraisal and professional growth
Share responsibility for evaluation between the teacher and the evaluation team in a collaborative process that promotes self-growth, instructional effectiveness, and improvement of overall job performance A Flawed System Problem No. 1: Observation equals evaluation Problem No. 2: Likely to rely on intuition, not evidence, to make judgments about teacher performance Problem No. 3: One size fits all Problem No. 4: Dont communicate Problem No. 5: Fragmented evaluation process Problem No. 6: Irrelevant evaluation Problem No. 7: One-point rating scales Problem No. 8: No impact evaluation Question 1
What is the basis of the teachers evaluation? Main Components Performance Standard Standard 2: Instructional Planning The teacher plans using the Virginia Standards of Learning, the schools curriculum, effective strategies, resources, and data to meet the learning needs of all students. Sample Performance Indicators Examples may include, but are not limited to: The teacher: 2.1 2.2 2.3
2.4 2.5 Performance Indicators Uses student learning data to guide planning. Plans time realistically for pacing, content mastery, and transitions. Plans for differentiated instruction. Aligns lesson objectives to the schools curriculum and student learning needs. Develops appropriate long- and short-range plans, and adapts plans when needed. Exemplary In addition to meeting the standard, the teacher actively seeks and uses alternative data and resources and consistently differentiates plans to meet
the needs of all students. Proficient Proficient is the expected level of performance. The teacher plans using the Virginia Standards of Learning, the schools curriculum, effective strategies, resources, and data to meet the needs of all students. Developing/ Needs Improvement The teacher inconsistently uses the schools curriculum, effective strategies, resources, and data in
planning to meet the needs of all students. Performance Appraisal Rubric Unacceptable The teacher does not plan, or plans without adequately using the schools curriculum, effective strategies, resources, and data. Performance Standards Teacher Performance Standard 1: Professional Knowledge
The teacher demonstrates an understanding of the curriculum, subject content, and the developmental needs of students by providing relevant learning experiences. Teacher Performance Standard 2: Instructional Planning The teacher plans using the Virginia Standards of Learning, the schools curriculum, effective strategies, resources, and data to meet the needs of all students. Teacher Performance Standard 3: Instructional Delivery The teacher effectively engages students in learning by using a variety
of instructional strategies in order to meet individual learning needs. Teacher Performance Standard 4: Assessment of and for Student Learning The teacher systematically gathers, analyzes, and uses all relevant data to measure student academic progress, guide instructional content and delivery methods, and provide timely feedback to both students and parents throughout the school year. Teacher Performance Standard 5: Learning Environment The teacher uses resources, routines, and procedures to provide a
respectful, positive, safe, studentcentered environment that is conducive to learning. Teacher Performance Standard 6: Professionalism The teacher maintains a commitment to professional ethics, communicates effectively, and takes responsibility for and participates in professional growth that results in enhanced student learning. Teacher Performance Standard 7: Student Academic Progress The work of the teacher results in acceptable, measurable, and appropriate student academic progress.
Question 2 How will teacher performance be documented? Multiple Data Sources Data Source Evaluator Informal Observations Formal Observations
Selects/develops Measures of Academic Progress Percentage of Evaluation based on Student Growth Percentiles (SGPs) Percentage of Evaluation based on Other Growth Measures Teachers of reading and mathematics for whom SGPs are available
20 20 Teachers who support instruction in reading and mathematics for whom SGPs are available No more than 20 20 to 40 N/A 40
Teachers Teachers who have no direct or indirect role in teaching reading or mathematics in grades where SGPs are Student Achievement Goal Setting Step 1: Determine needs Step 2: Create specific learning
goals based on preassessment Step 3: Create and implement teaching and learning strategies Step 4: Monitor student progress through ongoing formative assessment
Step 5: Determine whether the students achieved the goal What are the Purposes of Student Achievement Goal Setting? Focus on student results Explicitly connect teaching and learning Improve instructional practices and teacher performance Tool for school improvement Question 3 How will teacher
performance be rated? Evaluations Interim Evaluation Used to document evidence of meeting standards Does NOT include rating of performance Summative Evaluation Comes at end of evaluation cycle - One year for probationary teachers - Three years for continuing contract teachers Assessment of performance quality - Four point rating scale - Performance rubric for every standard Evaluating Performance Category
Description Definition Exemplary The teacher maintains performance, accomplishments, and behaviors that consistently and considerably surpass the established standard. Exceptional Performance Proficient The teacher meets the standard in a manner that is consistent with the schools mission and goals.
Effective Performance Developing/ Needs Improvement The teacher often performs below the established standard or in a manner that is inconsistent with the schools missions and goals. Below Acceptable Performance Unacceptable The teacher consistently performs below the established standards or in a manner that is inconsistent with
the schools missions and goals. Ineffective Performance Sustains high performance over period of time Behaviors have strong positive impact on learners and school climate Serves as role model to others Meets the requirements contained in job description as expressed in evaluation criteria Behaviors have positive impact on learners and school climate Willing to learn and apply new skills Requires support in meeting the standards Results in less than quality work performance Leads to areas for teacher improvement being jointly identified and planned between teacher and evaluator Does not meet requirements contained in job
description as expressed in evaluation criteria Results in minimal student learning May result in employee not being recommended for continued employment Sample Performance Appraisal Rubric Standard I: Professional Knowledge The teacher demonstrates an understanding of the curriculum, subject content, and the developmental needs of students by providing relevant learning experiences. Proficient Proficient is the expected level of performance. In addition to meeting The teacher the standard, the teacher demonstrates an
consistently understanding of the demonstrates extensive curriculum, subject knowledge of the subject content, and the matter and continually developmental needs of enriches the curriculum. students by providing relevant learning experiences. Exemplary Developing/Needs Improvement The teacher inconsistently demonstrates understanding of the curriculum, content, and
student development or lacks fluidity in using the knowledge in practice. Unacceptable The teacher bases instruction on material that is inaccurate or outof-date and/or inadequately addresses the developmental needs of students. Summative Rating Summative ratings should apply the rating for each of the seven performance expectations, with the most significant weight given to Standard 7 - Student Academic Progress. Weight each of the first six standards equally at
10 percent each Weight Standard 7 Student Academic Progress at 40 percent Lessons Learned Most practitioners believe new models provide specific measures of teacher effectiveness that are useful for distinguishing effective from less effective teachers. Exemplary In addition to meeting the standard, the teacher actively seeks and uses alternative data and resources and consistently differentiates plans to meet the needs of all students. Proficient
Proficient is the expected level of performance. Developing/Needs Improvement The teacher plans using the states standards, the schools curriculum, effective strategies, resources, and data to meet the needs of all students. The teacher inconsistently uses the schools curriculum, effective strategies, resources, and data in planning to meet the needs of all students.
Unacceptable The teacher does not plan, or plans without adequately using the schools curriculum, effective strategies, resources, and data. Lessons Learned Most practitioners believe that new models have the potential to improve teaching and learning by providing useful feedback that can be used to diagnose and guide teacher improvement. Formal Observation PostConference Teacher Self-Assessment Student Learning Objectives Student Surveys
Lessons Learned With more rigorous assessment to identify problems and recognize excellence, investments in teacher development can be better related to school and division goals for improvement. Evaluatio n Strengths and Weaknesses Identified Targeted Professional Developme nt
Lessons Learned Practitioners appreciate the value in using multiple data sources to provide evidence of performance standards. Observations Teacher Performance Standards Documentati on Logs Student Learning Objectives Student Surveys
Lessons Learned Practitioners believe that new models set up realistic expectations for teacher performance, and they reflect the most important elements of effective teaching. Professional Knowledge Instructional Delivery Learning Environment Instructional Planning Assessment of/for Learning
Professionalism Student Progress Lessons Learned The evaluation framework is valid in terms that the process standard ratings of teacher have a moderate ability to predict the student academic progress. In addition, there is a significant correlation between each of the six process standards and student academic progress. Lessons Learned New models can be time-consuming to implement. Practitioners mistrust the validity of student progress models as a measure of student growth. Teachers believe that the nature, quality, and
credibility of the evaluation process vary depending on the qualifications of the evaluators. Concluding Thoughts on Transforming Teacher Evaluation State Convene stakeholders across the state to design, implement, and improve evaluation system. Develop validated and reliable evaluation measures. Provide incentives and on-going support to school divisions. Make knowledge of new developments in teacher evaluation part of leadership and teacher preparation programs. Make evaluation count. Concluding Thoughts on Transforming Teacher Evaluation (Continued)
Schools and School Divisions/Districts Clear expectations of the WHAT will be evaluated. Clear communication of the evaluative criteria for effective performance. Instruments and procedures for teachers to provide evidence from multiple sources of the HOW of their performance. Increase the use of evaluation results to inform professional development.
Health Assessment Across the Lifespan NRS 102 Slide 27-* Seated together Slide 27-* Standing Slide 27-* Seated, changed into gown Slide 27-* Supine Slide 27-* Patient Sits up and Dangles Legs Slide 27-* Patient Stands up Slide 27-* Male Genitalia...
"The Intolerable Acts"- Text p. 169-172. What rights were threatened by the Intolerable Acts (Coercive Acts)? In response to the Boston Tea Party the British Parliament passed the "Intolerable . Acts" in 1774. They . closed the port of Boston,...
DAGMAR Approach. DAGMAR: An Approach to setting Advertising Objectives. DAGMAR stands for "Defining Advertising Goal For Measuring Advertising Results". This approach was given by Russel Colay in 1961. According to Russel Colay "The main objective of advertising is to communicate...
What is MI? MI is a collaborative, goal oriented style of communication with particular attention to the language of change. It is designed to strengthen personal motivation for and commitment to a specific goal by eliciting and exploring the person's...
Overview of Mathematics Standards Development. Texas: 2010 - 2014 Jane F. Schielack. Texas A&M University. Jane F. Schielack is an Associate Dean for Assessment and PreK-12 Education and a Professor in the Department of Mathematics at Texas A&M University.
Message Strategy for Community Education and Engagement. Goal: Learn strategies to communicate prevention messages openly and honestly with diverse community audiences. April 5, 2017. Katie Titterton. communications consulting for Vermont nonprofits. c. learspotvt.com