DIAGNOSTIC TESTING A TALE OF TWO INITIATIVES Using

DIAGNOSTIC TESTING  A TALE OF TWO INITIATIVES Using

DIAGNOSTIC TESTING A TALE OF TWO INITIATIVES Using diagnostic tests to assist in (i) widening access and (ii) post hoc curriculum design NAN YELD The Alternative Admissions Project The Exit-level Testing Project To identify talented students whose school results would not qualify them for entry To provide useful information for curriculum design To place these students into appropriate programmes of study To explore the effectiveness (for entry level curriculum design) of diagnostic information on graduating students To deliver meaningful diagnostic information for curriculum design To assess the suitability of using existing entrance

selection/diagnostic tests in this attempt Selected TIMSS 1999 performance indicators Benchmark Description of benchmark % students reaching benchmark Highest performing group (e.g. Singapore, Korea, Japan) Lowest performing group (South Africa, Philippines, and Morocco)1 Top 10% Students can organise information, make generalisations, and explain solution strategies in non-routine problem solving situations. 33 % 1%

Upper quarter Students can apply their understanding and knowledge in a wide variety of relatively complex situations. 66 % 1% Median Students can apply basic Mathematical knowledge in straightforward situations. 90 % 10 % Lower quarter Students can do basic computations with whole numbers. 99 % 31 % Note 1: South Africa scored the lowest of this group

. Percentage of students scoring below benchmark 69 % 1% South African School- Leaving Cohort 2001 (estimate) failed 19% rest of age cohort 51% certificate 22% endorsement 8% Percentage candidates in aggregate categories School-leaving examination data 2001 - black candidates obtaining exemptions 80 70 60 50 40 30 20

23,430 10 0 7,260 2,310 40-50 50-60 60 Examination results in aggregate categories Some factors impacting on test development Issues to be considered Consequences for test development Affordability and feasibility Pen and paper mark-ability format issues Impact on system (e.g. importance of not undermining the Sen. Certificate) Tests not in school subject areas

Performance on SC remains important Impact on individuals (importance of not creating another barrier to access) Writing of tests voluntary Candidates assessed on EITHER SC results OR project test results The tests need to do something different (avoid predicting the past) Develop and implement innovative ways of getting at underlying ability - scaffolding The tests must not The tests need to be able to In a higher education context, students are required to: Make meaning from what they read; Understand and interpret conceptual and metaphorical language; Identify and track academic argument; Follow discourse structure in text; Make inferences about and extrapolate on what they read;

Demonstrate familiarity with and understanding of the conventions of visual literacies, such as reading and interpreting graphs, flow-charts and diagrams; and Cope with basic numeracy demands. Getting to grips with these requirements is seriously impeded by approaches to texts and epistemic practice such as: A tendency towards verbatim reproduction or plagiarism in essays A tendency to describe rather than analyse, and to offer tautologies in place of justification A tendency to focus on examples (tokens) rather than on principles (types), and the relation between them A tendency to write from a highly subjective viewpoint without depersonalising A tendency to be prescriptive or normative when asked to be analytic (Slonimsky and Shalem 2005) Assessing the effectiveness of the scaffolding approac Did the approach provide a greater range of scores (spread), so that capable students could be more clearly differentiated from weaker students? Did the approach increase the predictive validity of the test (that is, did the test correctly distinguish between weaker and stronger students)? Did the approach improve (raise) the level of stronger students scores? ELPT and ALPT Scores 600 No. of Candidates

500 400 300 ELPT ALPT 200 100 0 10 20 30 40 50 60 Test Results (% ) 70 80 90

100 Task Preparation Study Question 6.1: In note form (i.e. not full sentences), list the main points the author makes about the Japanese educational system. For this exercise, do not include comparisons or references to the American system. Question 6.2: Drawing on the text of the whole article, but using your own words, summarise the points the author makes about the Japanese system of education. Note: your summary should not include comparisons or references to the American system. Points made in notes and carried over to summary Top 30% of candidates (n=114) Bottom 30% of candidates (n=114) .62 - 0.35 Semester- specific hazard rates for the Admissions Project group (511) and the control group (563) 0.18 0.16

hazard rate 0.14 0.12 Admissions Group 0.1 Control group 0.08 0.06 0.04 0.02 0 6 12 18 24 semesters 30 36 42 ' Challenge levels' of papers % items at challenge levels

70 60 50 40 30 20 10 0 Level 1 Level 2 Level 3 I tem challenge levels 1992 JMB 2001 1st National Paper 2003 2nd National Paper ASSUMPTIONS UNDERLYING EXIT-LEVEL TESTING PROJECT 1 By and large, and with many notable exceptions, academics . do not take seriously/ are not well informed about / are not very interested in . information about the learning needs of incoming students. 2 Academics are more likely to take information about the learning needs of their students seriously if these needs are demonstrably still present in the students they are about to graduate. 3 In order to demonstrate these needs, a valid and

credible (and feasible) instrument is needed. 1. By and large, academics . do not take seriously/ are not well informed about / are not very interested in . information about the learning needs of incoming students. Why might this be so? They believe it would be lowering standards to pander to needs? They believe its the responsibility of the schooling sector to prepare students, not theirs? They assume that massification will entail higher failure rates, its only natural? They are really only interested in students who will be majoring in their disciplines, and so its the survivors from the first-year shakedown that they are focused on? Because doing so requires a shift from an inward looking, introjected orientation, to an outward looking, projective orientation (Barnett 2000), and a willingness to negotiate curricula across departmental boundaries? The outsider status of those providing the information? DESCRIPTIVE STATISTICS Entering Group N = 95 Mean Standard Error Median Mode Standard Deviation Minimum Maximum Range Graduating group

N = 67 36.9 1.3 45.1 1.2 35.4 39.9 46.1 39.3 13.0 11.8 9.5 27.0 71.9 60.1 65.2 38.2 Quintile boundaries: Entering and graduating ' disadvantaged schools' groups 60 Percentage 50 40

Entering 30 Graduating 20 10 0 1 (top) 2 3 4 Quintile lower boundaries 5 (bottom) Skill cluster descriptions (examples) Skill Assessed Explanation of Skill Area Extrapolation, application and inferencing Students capacities to draw conclusions and apply insights, either on the basis of what is stated in texts or is implied by these texts.

Understanding the communicative function of sentences Students abilities to see how parts of sentences / discourse define other parts; or are examples of ideas; or are supports for arguments; or attempts to persuade Separating the essential from the nonessential Students capacities to see main ideas and supporting detail; statements and examples; facts and opinions; propositions and their arguments; being able to classify, categorise and label Understanding basic numerical concepts Students abilities to make numerical estimations; comparisons; calculate percentages and fractions; make chronological references and sequence events / processes; do Cliff, A., Hanslo, M. and Visser, A-J. (2003). Faculty of Engineering and the Built Environment: AARP basic computations tests and first-year students academic performance for the 2002/3 cycles. Unpublished report:

University of Cape Town, December 2003. Skills clusters: graduating group ' disadvantaged Performance (%) on skills clusters schools' 70.0 60.0 50.0 40.0 30.0 20.0 10.0 0.0 l s g % al al nt es al ia rg te or ns re ab ab ic t l c

u c e t an a n c h c o i n o l O n a s i t n L e p r o i o a t o t e T

e V G V V on e se ta la ce to ch e ap nt m i c s C e e e i c r e u e E c R st M

S oi Vo N ce Vo e M xt oi e i V n E V o n T w V ic w n wn o O w n O V O w O O wn O

Skills clusters REFERENCES Barnett, R. (2000). Supercomplexity and the curriculum. Studies in Higher Education, 25, 255-2665. Cliff, A., Hanslo, M. and Visser, A-J. (2003). Faculty of Engineering and the Built Environment: AARP tests and first-year students academic performance for the 2002/3 cycles. Unpublished report: University of Cape Town, December 2003. Slonimsky, L. and Shalem, Y. (2004). Pedagogic responsiveness for academic depth. In Griesel, H. (Ed.) (2004). Curriculum responsiveness case studies in higher education. South African Universities ViceChancellors Association: Pretoria, 81 - 101.

Recently Viewed Presentations

  • Maricopa County Assessor

    Maricopa County Assessor

    A.R.S. ยง 12-172: Small Claims Value Limit Increase to $2,000,000. If the case is based on the income approach to value, the taxpayer shall comply with section 42-16052 no later than ten business days before the hearing.
  • Procurement Shared Service Center September 21, 2017 Carmen

    Procurement Shared Service Center September 21, 2017 Carmen

    - Replaces their hosted catalog. Along with a comprehensive product selection to match the new OSU contract, this supplier website includes a quote pick-up feature called "Open Bids" located under the My Account menu on the landing page. Users can...
  • Peer-to-Peer Networks

    Peer-to-Peer Networks

    Gnutella protocol overview Connecting to the Gnutella network bootstrap using GWebCache system and locally cached hostlist Ping/Pong messages are exchanged with potential neighbors Searching on the network Query messages are flooded on the network QueryHit messages are received (back-propagated ...
  • Tumor marker and Screening DR YOUSEFIY www.zohrehyousefi.com 1

    Tumor marker and Screening DR YOUSEFIY www.zohrehyousefi.com 1

    Macrophage-colony stimulating factor. appears to be a marker of high specificity, with levels correlating to stage. When combined with other markers, it may have a role in early detection. M-CSF may also be sensitive and specific for malignant germ-cell tumors...
  • MSU College of Education Power Point Template

    MSU College of Education Power Point Template

    Participants will leave this session more confident about using their school's assessment plan for data-based literacy instruction. The discussion will revolve around minimizing testing time and maximizing ongoingsupport of language and literacy instruction. ... Pro tip: When you take data...
  • The Ebook Title Author or Company YOUR LOGO

    The Ebook Title Author or Company YOUR LOGO

    Ferferunt et fugitibus quiam que nat. Igentotat. Tat persped moluptat iligend andicipsa dis ut quae plignihilit, sus mo beribe- rum as velectate core rerati adiati adit la comnitatur, odicimod molupta tem- porem renetus qui ut lab idelit harunt lab ius...
  • Benefits of Early Nutrition Intervention with a TBI

    Benefits of Early Nutrition Intervention with a TBI

    Positively correlated to high blood pressure, HTN, metabolic syndrome, inflammation. Swiss Mammography Cohort study: a high processed meat consumption was associated with statistically significant increased risk of stroke and cardiovascular disease. Vitamin A . The Kuopio Heart Disease Factor Study
  • Strategic Analysis

    Strategic Analysis

    PepsiCo Related DiversificationGlobal Divisions, 2016. PepsiCo. CEO: IndraNooyi. Exec. VPs: Franchise, Legal, HR, CFO, Operations, R&D, Communications