Using VALUE Rubrics to Assess Almost Any Program Outcome

Using VALUE Rubrics to Assess Almost Any Program Outcome

LETS GET IT STARTED! USING VALUE RUBRICS TO ASSESS ALMOST ANY PROGRAM OUTCOME WORKSHOP OUTCOMES As a result of this workshop you will be able to: Describe how VALUE rubrics can be used for program assessment Create a signature or key assignment that is aligned with program learning outcomes Collect student work from multiple course sections and/or instructors Use results to help students improve their skills and content knowledge THE ASSESSMENT CYCLE Today!

WHAT ARE VALUE RUBRICS? AAC&U (Association of American Colleges and Universities)s VALUE (Valid Assessment of Learning in Undergraduate Education) project A component of the LEAP (Liberal Education and Americas Promise) initiative Goals: Develop shared understanding of student learning outcomes Promote authentic assessment of student work (vs. standardized tests) Teams of faculty and academic affairs professionals synthesized rubrics into 15 areas of learning. AAC&US 15 VALUE RUBRICS Intellectual and Practical Skills: Inquiry and analysis Critical thinking Creative thinking

Written communication Oral communication Quantitative literacy Information literacy Teamwork Problem solving Personal and Social Responsibility: Civic knowledge and engagementlocal and global Intercultural knowledge and competence Ethical reasoning and action Foundations and skills for lifelong learning Integrative and applied learning POSSIBLE USES OF VALUE RUBRICS Course evaluation Models for rubrics faculty can use to score of class assignments Student reflection Program evaluation Help specify department learning outcomes

Models for rubrics for program-level evaluation projects LEVEL OF PERFORMANCE DESCRIPTORS Capstone- culminating level of achievement expected for baccalaureate degree Milestones- progressively more sophisticated performance Not intended for 1 = freshmen, 2 = sophomore, or 4 = A, 3 = B, etc. MODIFYING VALUE RUBRICS Meant to be modified! Add more specific criteria based on your program or assignment Add new dimensions to

reflect issues important to your program SALT LAKE COMMUNITY COLLEGE Require instructors to use of signature assignments to assess their general education outcomes, but allow faculty freedom in creating the assignments. A few general requirements for the assignments: address at least two learning outcomes include student reflection demonstrate a real world, not theoretical, application of disciplinary knowledge A mathematics instructor created a signature assignment where students acted as potential car buyers and calculated how different interest rates affect the amount of money spent. Learning outcomes- quantitative literacy and written communication Students reflect on how this activity can be applied in other classes or real world scenarios.

ORAL COMMUNICATION SCORES: PSYCHOLOGY (N = 23) Proficiency Score Organization Language Delivery Supporting Material Central Message 3.75-4.0 0 (0%) 3 (13%)

1 (4%) 2 (9%) 1 (4%) 3.0-3.5 15 (65%) 14 (61%) 10 (44%) 15 (65%) 18 (78%) 2.0-2.75

8 (35%) 6 (26%) 9 (39%) 6 (26%) 4 (17%) 1.0-1.75 0 (0%) 0 (0%) 3 (13%) 0 (0%) 0 (0%)

Note. Scoring was as follows: 1 = Benchmark (Does not Meet Competency), 2 = Milestone (Minimal Competency), 3 = Milestone (Meets Competency), 4 = Capstone (Exceeds Competency). What trends do you notice? What questions are left unanswered? How could we collect more useful data? ACTIVITY #1: CHOOSING AN OUTCOME How do these align with your Program Learning Outcomes? Which learning outcomes are emphasized by your program? What are your strengths and weaknesses? Which seems like a priority for assessment? WHERE IS THE OUTCOME TAUGHT IN YOUR CURRICULUM? 1500

PLO1 2000 I I PLO3 I 3080 3100 D D 3230

D D PLO5 I D D PLO6 I D D 4110 4120

4250 4650 D D M M D D I 3220

D D PLO7 I 3040 D PLO2 PLO4 3020 M D

D D D D M M M D M I = Introduced; D = Developed/Reinforced; M = Mastered WHAT IS A SIGNATURE ASSIGNMENT? Embedded in a course

Used for course grade and program assessment Aligned with Program Learning Outcomes Collaboratively designed by faculty Meaningful and integrative Why? Allows a program to assess learning across course sections or instructors Creates consistency Assignment 3 (to be assessed with the VALUE rubrics Written Communication and Intercultural Understanding) Instructions: Please take approximately 1 2 hours to complete this assignment. The essay topic is designed to give you an opportunity to demonstrate your ability to write clearly and effectively. It will also allow you to display your knowledge of psychological diversity. Perfection is not expected, but you

should try to produce the best essay possible in the time allotted. Your essay should be about 2 pages in length (4-5 paragraphs). Type this assignment and then upload the digital file to the course Moodle site. You do not have to cite sources in this essay, but please include specific terms and concepts from your psychology classes. Prompt: Think about a group of people who are very different from you. These could be individuals from a different culture or perhaps members of a social group that hold views you disagree with. Briefly describe the groups characteristics and how individuals in this group are different from you. Then, describe 1-3 concepts that youve learned from this class (or other psychology classes youve taken) that could be used to change or improve the way you interact with members of ACTIVITY #2: SIGNATURE ASSIGNMENT What courses would use this assignment? Describe instructions to students, providing explicit guidelines on:

Learning outcomes and goals How to complete the assignment Length and time required Sources needed Evaluation criteria GATHER AND EVALUATE Student Student Assignments Student Assignments Student Assignments Student Assignments Assignments Course #1 Instructor

Student Student Assignments Student Assignments Student Assignments Student Assignments Assignments Grade to student s Course #2 Instructor Team of Faculty Score Assignments

or Student Student Assignments Student Assignments Student Assignments Student Assignments Assignments Grade to student s Course Instructor(s) Score

Assignments Assessment Coordinator or Committee Compile Results Program Faculty Reflect on Results Course #3 Instructor Grade to student s SCORING ASSIGNMENTS: RUBRIC CALIBRATION Hold a calibration session with all instructors or faculty scorers. Begin with a close reading of the rubric and identify areas of discussion. Faculty should come to an agreement on interpretation of language in

rubric. Faculty are given an example of student work to score. Discuss scores row by row. Faculty provide rationale for their scores and try to reach consensus. Goal is to identify two scores around with the majority cluster. Repeat with more examples of student work (high, low, medium) ACTIVITY #3: ASSESSMENT PLAN What assignment or activity will you use? How will you score student achievement? What classes would you target for sampling and when? Which faculty will be responsible for coordinating data collection? Data analysis? How will you analyze the results? Will you disaggregate results in some way? How will results be shared, discussed, and used to make changes? When will the PLO be assessed again?

DOS AND DONTS OF DATA COLLECTION AND ANALYSIS DO Form a department assessment committee charged with regularly collecting and disseminating data Ask for faculty volunteers Give faculty early notice regarding assessment plans Disaggregate results across time, populations, and outcomes Protect the confidentiality and anonymity of students and faculty by examining results at the group level DONT

Wait until the last minute Pressure faculty to comply with assessment activities Use assessment results to call attention to, judge, or punish individual faculty or students Expect perfection Collect more data than you can use USING RESULTS TO CREATE A CULTURE OF EVIDENCE Use results: To examine skill development across the curriculum To examine curriculum content coverage and areas for program modification To improve instruction and introduce new pedagogies Contact CETL for resources and support

To improve and refine your assessment process/methods NEXT STEPS What have you learned today that you want to share with others in your department? Write down 1-3 you can do this semester to keep your assessment momentum going? RESOURCES AND CREDITS Using the VALUE Rubrics for Improvement of Learning and Authentic Assessment by Rhodes & Finley (2013) Association of American Colleges and Universities Using Signature Assignments for Program-Level Assessment Presentation Slides by University of Hawaii, Manoa University of Texas signature assignments webpage: https://ugs.utexas.edu/sig/plan/samples/writing-model4

Recently Viewed Presentations

  • ARES - Warstones

    ARES - Warstones

    Symbols and Sacred Animals. These are Ares' symbols a spear, helmet, dog, chariot, boar, vulture and a flaming torch. Ares has a sacred animal which is a dog and his bird was a vulture.. Did you know that Ares had...
  • EMPC Final Intern Presentation

    EMPC Final Intern Presentation

    EMPC Final Intern Presentation. Caitlynn Roy. EMPC Projects Execution Intern. 7 August 2014. A summer internship is an opportunity to step out of the classroom and apply our knowledge in a real world environment.
  • Weathering of western Greenland: Influences on oceanic fluxes ...

    Weathering of western Greenland: Influences on oceanic fluxes ...

    Radiogenic isotope suite: Expand 87Sr/86Sr dataset. Complete 143Nd/144Nd analyses . Inverse models: Within main channels in each watershed. Questions? Acknowledgments. This material is based upon work supported by the National Science Foundation Graduate Research Fellowship under Grant No. DGE ...
  • Americas 600-1300 Aztec Poem Describing the God of

    Americas 600-1300 Aztec Poem Describing the God of

    We make him laugh, He Mocks us Mexica 1168- Group of people moved from Aztlan believed to be somewhere in southern California to Central Mexico Problem with Toltec in Teotihuacan Story of Mexican Flag- Forced to live in snake infested...
  • BASIX Risks faced by Rural Households and Private

    BASIX Risks faced by Rural Households and Private

    Title: Slide 1 Last modified by: IFPRI Document presentation format: On-screen Show Other titles: Times New Roman Arial Unicode MS Wingdings Tahoma Arial Ricepaper Microsoft PowerPoint Presentation Risks faced by Rural Households and Private Sector Insurance Initiatives in India A...
  • test the system against user & system requirements - The Tilly

    test the system against user & system requirements - The Tilly

    white box testing. White-box testing (also known as clear box testing, glass box testing, transparent box testing, and structural testing) _____ and uses that knowledge as part of the testing process. If, for example, exception is thrown under certain conditions,...
  • Transportation Funding Deal Explained

    Transportation Funding Deal Explained

    Repeal the taxes and fees associated with SB 1. ... Express the agency's views about the effect of the measure on the agency and its programs, provided the agency is exceedingly careful not to advocate for or against the measure's...
  • Ethics in Healthcare

    Ethics in Healthcare

    The following is a series of vignettes for discussion of ethical dilemmas in healthcare from the television show Nurse Jackie. Disclaimer! Yes, this show is rated R. All of your students should be at least 18 years old. It has...