Florida Catholic Schools Assessment Program TerraNova and InView Score Interpretation 1 Understanding and Using Test Data The Purpose of Standardized Tests Evaluation of Student Progress Administrative Planning and Direction Needs Assessment Instructional Program Planning

Curriculum Analysis Assessment should promote learning, not simply measure it. Jay McTighe Types of Information Provided by Achievement Tests TerraNova Scores: Accurate Norm-Referenced Information Relevant Criterion-Referenced Information Standards-Based Performance Information Inview Scores: Cognitive Scores Norm-Referenced Test Scores Norm-referenced Test questions are primarily selected based on their ability to distinguish specific performance levels matching the Bell Curve The test is given to a nationally representative group of students of all performance levels Norms provide a way to understand how a students test performance

ranks/compares to similar students nationwide Common Score Types Grade Equivalent (GE) Scale Score (SS) National Percentile (NP) Normal Curve Equivalent (NCE) National Stanine Scale Scores (SS)

Designed to measure student achievement from elementary school through high school Expected to increase with grade level TerraNova content forms a developmental continuum through which students move as they develop new capabilities Scale score shows where the students are on this continuum 3 4 K 1 2 492 535 571 606

Math Scale Scores 627 5 645 National Percentile Scores (NP) Represents the percentage of students in the national norm group whose score was at or below a students score A student whose NP is 65 performed better than 65% of those students who took the test when it was normed Important Reminders: NP scores range from 1 to 99 The national norm of NP is 50; i.e., 50% of students scored at or below a given students score NP scores are NOT percent correct scores NP scores SHOULD NOT be added or subtracted because they are not an equal-interval scale score National Percentile Scale 50 NP

Scale NP is where th national middle is Half the Nation is below 50 Half the Nation is above 50 Normal Curve Equivalent (NCE) Scores NCEs were developed to evaluate student growth NCE scores indicate the relative ranking in comparison to the national group used to establish norm scores on a test NCE scores: denote relative ranking (1-99) have an average or norm of 50 can be confused with NP scores

are on an equal interval scale (equal distance between score points) can be compared and averaged for group comparison or measure growth/gain Normal Curve Equivalent (NCE) Scores NCE scores are the best tool to use for: Principals showing grade to grade level growth School comparisons Growth across time NCE scores show aggregate gain scores from different programs (even those using different standardized tests) Statistically Significant Growth scores: Students (11 NCE points) Class (7 NCE points) School (5 NCE points) System wide/Diocesan wide (4 NCE points) NP of the Mean NCE This score allows us to compare groups(e.g., Mrs. Smith can see her students performance in relation to all schools within the Diocese or nationally.) DRC uses the following two step process

1. Calculates the average NCE score of the students in Mrs. Smiths 5th grade class 2. NCE scores are ranked using the NP score to rank the average (NCE) scores with other 5th grades National Stanines Standard scores based on a scale of nine equal units that range from a high of 9 to a low of 1 Stanines 1-3: Below Average Stanines 4-6: Average Stanines 7-9: Above Average Comparing the Scales Q2 Q3 Focus on Q3 First NP = 50-75

Q1 Q4 Harder to grow How are Data Related? Q1 = NP 1-25 1. Stanine National Percentile Rank Range Description 9 Highest level

8 High level 7 Well above average Q4 Percent of students in the nation. 2. Q2 = NP 26-50 3. Q3 = NP 51-75 4. Q4 = NP 76-99 96 - 99 4%

89 - 95 7% 77 - 88 12% Q3 National Quartile Distribution 6

Slightly above average 60 - 76 17% 5 Average 40 - 59 20% Q2 4

Slightly below average 23 - 39 17% 3 Well below average 11 - 22 12% 2 Low

1 Very Low Q1 4 - 10 7% 1-3 4% What Does Average Growth Look Like? If a student is in the 50th National

Percentile each year he is still growing! K 492 1 2 3 4 5 535 571 606 627

645 Norm-Referenced Data Grade Equivalent Scores (GE) Created to provide meaningful interpretation First Number represents the grade Second number represents the month 5.3=Fifth-grade, third month Does not reflect student progression in grade GE scores are rank-order scores only Advantages Easy to understand Disadvantages How would you interpret a GE score of 4.8 obtained by a second grade student on a second grade reading test?

Easily misunderstood and frequently misinterpreted. Types of Information Provided by Achievement Tests TerraNova Scores: Accurate Norm-Referenced Information Relevant Criterion-Referenced Information Standards-Based Performance Information Inview Scores: Cognitive Scores Criterion-Referenced Testing Describes the students acquisition of specific skills and areas of knowledge. TerraNova is organized in terms of commonly taught curriculum objectives Scores based on content knowledge of specific sub-skills Score Type: Objective Performance Index (OPI)

Mastery Scores OPI An Objective Performance Index (OPI) is an estimate of the number of items the student could be expected to answer correctly if he/she had taken 100 items measuring that objective. Scores based on content knowledge of specific sub-skills Diagnoses strengths/weaknesses High Mastery (Strong grasp of the concept) Moderate Mastery (Moderate grasp of concept; not yet at grade level expectations) Low Mastery (Weak grasp of the objective) Individual Profile Report At a Glance How Strong, How Weak Weaker Performance

Stronger Performance Mastery bubbles are toward the right side of the graphic. MMR bubbles are somewhere in the middle of the graphic. Low mastery (white) bubbles are to the left side of the graphic. Confidence Band Confidence Band takes into consideration error of measurement Compensates for variations in performance ... if the student were to take the test multiple times If the student took the test again, it is highly likely that his/her score would be somewhere on this line. The shorter the line, the more confident we are in the score accurately reflecting student performance. The wide line may indicate that the student is inconsistently selecting the correct responses.

Differentiated Instruction Online Reporting Easily groups students by their performance on objectives Analysis Activity Strength = 75% or more of students at Mastery The great majority of students grasp this objective Last years instruction effective for most students A good base for this years curriculum Challenge = 50%-74% of students at Mastery Good news: majority of students grasp this objective Bad news: it may not be a strong base for new curriculum Critical Need = < 50% of students at Mastery Majority of students not at grade level in this area

Review last years curriculum-instruction in this area 23 Types of Information Provided by Achievement Tests TerraNova Scores: Accurate Norm-Referenced Information Relevant Criterion-Referenced Information Standards-Based Performance Information Inview Scores: Cognitive Scores Performance Level Information Pathway to Proficiency Level 1: Starting Out (Grades 1-2) Step 1 (Grades 3-12) Level 2: Progressing Level 3: Nearing Proficiency Level 4: Proficient

Level 5: Advanced Performance Level Groupings: Grades 1-2 (Primary) Grades 3-5 (Elementary) Grades 6-8 (Middle School) Grades 9-12 (High School) Scores based on the demonstration of ability compared to a set of standards (Standard Setting: Bookmarking Procedure) OPI Mastery Types of Information Provided by

Achievement Tests TerraNova Scores: Accurate Norm-Referenced Information Relevant Criterion-Referenced Information Standards-Based Performance Information Inview Scores: Cognitive Scores Inview Testing (Cognitive Abilities) Based on students performance on InView Age-dependent standardized score Students grade levels are not considered CSI considers overall cognitive ability No subscores per se Mean (average) score for the group Range identifies high/low score in the group CSI scale Mean = 100 Standard Deviation = 16 Average Range = 84 to 116

Guide to Interpreting the Cognitive Skills Index (CSI) Do not treat CSI as an IQ score True full scale IQ scores are based on individual tests with multiple subtests administered by a highly qualified person Relate to the Anticipated Achievement scores Are students working within/above/below their CSI? Can be useful in identifying students who need more support Using Inview Anticipated Achievement Scores Look for a difference between the Obtained Normal Curve and the Anticipated Normal Curve If Positive, students overachieved If Negative, students underachieved Obtained Anticipated Difference = 75

= 70 = +5 Instructional Strategy is working for this type of learner Obtained Anticipated Difference = 75 = 80 =-5 Look at what type of learner this student is: Verbal? Nonverbal? National Percentile by Age / National Percentile by Grade

HOME REPORT What data do we get from TerraNova 3 and InView? Score Anticipate Type Measure Growth Scale Score (SS) 2. Norm Curve Equivalent (NCE) 3. National Stanine (NS) 1. Student Rank

National Percentile Rank (NP) 2. National Percentile Grade/Age (NPG, NPA) 3. Grade Equivalent (GE) 4. Quartiles 5. Local Quartiles 1. Median National Percentile (MDNP) 2. NP of the Mean Norm Curve Equivalent (NP of the Mean NCE) 1. Summary Grade, Class, School

Mean Scale Score (MSS) 2. Mean NCE (MNCE) 1. Mastery Achievement Anticipated Scale Score 2. Anticipated NCE 3. Anticipated NP 4. Anticipated GE 5. Anticipated NS 1. 1. Objective Performance Index (OPI) 2. Mastery

Category Measuring Growth Score Type Definition Purpose Notes 1. Equal Interval 0-999 Measure Growth over time 2. All students have one 2. Used as basis for all Norm Referenced scores (NCE, NP, GE) 3. SS can be averaged 1. SS Scale Score

1. Equal Interval 1-99 Measure Growth over time 2. All students have one 2. Compare performance between 3. NCE can be averaged subtests on same test and converted into 3. Comparing growth between stanines or percentiles tests from different publishers 1. NCE Norm Curve Equivalent Stanine National Stanine

Measures growth 2. Less precise as SS or NCE 1. Equal interval 1-9 2. Convert to NCE for more precise growth 1. Ranking Students Score Type Definition Purpose Notes Scale is bell curve not equal interval 2. Cannot compare NP across subject areas

3. It is not % correct 1. NP National Percentile Rank NP of the MNCE National Percentile of the Mean Norm Curve Equivalent MDNP Median National Percentile

Quartile National Quartile 1. Ranks student performance against the national norm (nation). 1. Shows where the average student is ranked 1. Shows where my middle student is

1. Shows where students are ranked vs. national distribution of 25% in each Quartile Q1 = NP 1-25 2. Q2 = NP 26-50 3. Q3 = NP 51-75 4. Q4 = NP 76-99 1. Why we are here THANK YOU 35