Implementing ROMA at the Local CAA Workshop Using the Performance Management Framework ROMA Modules
The ROMA Next Generation Training Series Learning Objectives Understand basic principles of a results orientation and performance management Clarify how ROMA fits into the Performance Management Framework Acknowledge that we start with assessment Understand your Organizational Culture
Identify current ROMA implementation Identify elements of an Impact Pathway Plan Prepare to create a Local Theory of Change Create action steps in your pathway toward impact Prepare for monitoring progress and identifying success SETTING THE CONTEXT Part I
CSBG Performance Management Framework Involves an inclusive, multi-pronged approach to the purposes of the CSBG act Provides accountability at 3 levels (local, state, federal) of the CSBG network Ensures efficiency and effectiveness Demonstrates stronger results for
individuals, families and communities with low-income nationwide 5 CSBG Performance Management Framework (PMF): Implementation 6 What is ROMA?
Results Oriented Management and Accountability Results Oriented basic assumption that your agency services and strategies are provided to produce results (change). Management the system for producing and monitoring services and strategies Accountability the process for knowing, documenting and sharing the impact of your efforts
Elements of ROMA Organizational Standard 4.3 The organizations Community Action plan and strategic plan document the continuous use of the full Results Oriented Management and Accountability (ROMA) cycle or comparable system (assessment, planning, implementation, achievement of results, and evaluation).
Organizational Standards Consumer Input and Involvement Community Engagement Board Governance Human Resource Management
Financial Operations and Oversight ROMA Cycle Community Assessment Strategic Planning Reporting Data and Analysis
Organizational Leadership Implementation of Services and Strategies ROMA Next Generation Enhancements to ROMA
NASCSP | Performance Management Framework 11 What is a Theory of Change? A Theory of Change (TOC) is a conceptual road map for how an organization expects to achieve its intended impact. While similar to a logic model (which has
detailed information about needs, activities, inputs, outputs, outcomes, and indicators), a Theory of Change demonstrates the big picture about how all of these components work together. A Theory of Change Is Like a Logic Model It includes the core principles of the network, identification of strategies and services, and how
the network will convert their organizational and financial resources into the desired results. Often a graphic presentation of a mission statement Why We Need It *Make a complicated system easier to understand.
*Unify the network. *Articulate core principles. *Identify national goals. NASCSP |2017 ROMA ICEP The National Community
Action Network Theory Of Change: * Describes * Portrays * Identifies * Demonstrates * Connects
Community Action and CSBG PERFORMANCE MANAGEMENT How well does the Network operate? What difference does the Network make?
Local Organizational standards State and Federal Accountability Measures Results Oriented Management and Accountability System
Individual and Family National Performance Indicators Community National Performance Indicators 17 Applying the National TOC to Your Agency Not all local CAAs are the same
Each CAA will use the Performance Management Framework in ways that match their identification of local community needs and resources At the local level you will want to demonstrate How well does the AGENCY operate? What difference does the AGENCY make? Consider What Results? What stands in the way of a results orientation? A historical focus on what we do, not what we
accomplish Confusion between services and results. If we give out food boxes and people receive food boxes, the receipt of the service is sometimes considered to be an outcome. It is not an outcome. It is receipt of a service. Lack of follow up lack of resources for follow up, lack of measurement plans
Two Different Approaches Self Sufficiency Improved Quality Of Life Improved Living Conditions More Basic Services
Asset Accumulation OR Jobs Education & Training Basic Services
Advocacy Basic Services Health Care Prevention/ Treatment Supporting Results for Individuals and Families
ACTIVITY: What are the service -> outcome connections? Think about the services your agency provides and the strategies you have been involved in. Identify the outputs and the associated outcomes: PROGRAM NAME: Outputs Outcomes
Think About It! Considering your own agency Mission and the services and strategies you provide can be a window into what you are communicating to the community. Are these really the assumptions of your agency? Your staff? Your board? Do your mission and services help to identify the big goals your agency has?
Local Theory of Change What does your agency believe it should be doing to address issues related to poverty in your community? The answer to this question will guide the selection of outcomes and actions that will accomplish those outcomes. Local Theory of Change
Starts with acknowledging specific assumptions about: the causes and conditions of poverty in your local community the agencys role in addressing the issues related to poverty. Program or Agency Outcomes? Program Outcomes Identified in Silos
Integrated View of Agency Outcomes Integration of Levels of Work Trying Something New Think about how we have always done the work of the agency What is real, what is true, what is good, what has been done?
Think about the outcomes (changes) that you have achieved in the past. Think about WHAT ELSE your agency could achieve if you were not focused on the provision of services (but rather on change). The ideas that are generated by this kind of thinking will influence your creation of your local Theory of Change Assessment: Where are you now?
Part II Implementing ROMA Organizational Culture If we dont know where we started, we cannot identify what has changed.. If we cant identify the change, we cant acknowledge success!
Using a Checklist for Implementation aka ROMA Audit Helps you identify: What are you doing now that would demonstrate each phase of the ROMA Cycle What could be done differently to include the ROMA element
WHEN COMPLETING THE CHECKLIST, CONSIDER: --What can you identify right now for each phase of the cycle? --What do you need to find out more about? --Who do you need to talk to (to find out what is happening in other
departments)? In each phase of the ROMA Cycle, consider :
Who is involved and what do they do? Is their work done at program or agency level? What is expected to be achieved in each phase for the agency? For the specific program? What (if anything) is required by other funding sources? How does the agency collect, aggregate and analyze the data in each phase? Is it different in different departments? How does it come together for the agency? How will you know if the activities and processes are
being done well? Are they successful? What makes the process useful? How is the data/information from each phase used at the program or agency level? What would happen if this part of the cycle was left out? What if it was improved? Case Study Debbie Markman Resource Development Director
Economic Security Corporation of Southwest Area, Joplin, MO How do you know? To find out who is doing ROMA it is important to talk to peers and colleagues in the agency.
ESCs Weatherization ROMA Audit- Assessment ESCs Weatherization ROMA Audit - Evaluation Considering Each Phase
Mission and Local TOC Assessment Planning Implementing Reporting Analysis
Reassessment Mission and Local TOC What do you know about your Mission statement? has been reviewed within the past 3 years Is used to guide agency decisions and actions. Meets Organizational Standard 4.1 If the agency has a Local Theory of Change, it
is used in decision making, communications, etc. Assessment of Needs and Resources The Community Needs Assessment includes a variety of data from reliable sources Qualitative, Quantitative, Customer Satisfaction, Resources, Agency Report CNA identifies population in need Is there a clear process for making meaning out of all the
assessment data Identifies family, community, agency levels of need Who is involved in the analysis of data? How are priorities identified? Agency meets Org Standards 3.1- 3.5 CATEGORY THREE: Community Assessment Board involvement is documented Planning
There is an agency-wide Strategic Plan The Community Action Plan is a part of the Strategic Plan Focused on issues related to families and communities of low income Identifies how agency funds are used to meet broad agency goals (not just program activities in silos) Plans connect directly to Community Needs Assessment Identifies both outputs and outcomes Identifies how success will be measured Tools, indicators, etc.
Agency meets Org Standards 6.1- 6.5 CATEGORY SIX: Strategic Planning Board involvement is documented Implementing Services and Strategies Understand the Human Resource policies and procedures related to staff qualifications, training, supervision, monitoring, etc. Staff have access to resources for improvement of knowledge and skills CATEGORY SEVEN: Human Resource Management
Fiscal policies and procedures are in place to assure funds are spent appropriately to support achievement of outcomes CATEGORY EIGHT: Financial Operations and Oversight There is process in place to secure Customer Satisfaction feedback All level of staff understand the target population to be served and the expectations related to service:
How many people will be served? Who are they? What service do they get? What changes? Community strategies are understood, discussed and monitored throughout the agency staff and board.
Observation of Results and Reporting Do all staff in the agency understand the system for data collection and aggregation into reports? Are reports shared with program staff? Shared across programs (agency wide reporting practices)? What is included in the reports? Demographics of the population served Service provided Outcomes achieved
What info is provided to the Board? Do they review reports? Is CSBG Annual Report data submitted to the State Office, assuring accuracy and timeliness of submission. Agency meets Org Standards 9.1, 9.2, 9.4 Considering Your Reports Quality of the data provided
Presentation of the data in the report Usefulness of the data to your specific job duties; usefulness to the decision makers Performance Focus Analysis of the Data/Evaluation How is data/reports used? Compare actual results with performance targets/projected success rates Compare the demographics of the population served with the population identified to be in need in the CNA.
Consider the connections among people, services, and outcomes. Identify those who have received single services, multiple services, or bundled services and determine the difference (if any) in their outcomes. Are questions asked that will need additional assessment data? Does the Board receive recommendations based on analysis of data. Are recommendations and decisions based on data? Organizational Standard 9.3 Does the agency take action based on analysis of data?
Adjust performance goals as needed as evidenced by actual performance. Continue strategies that achieve desired performance goals. Revise or discontinue strategies that do not achieve desired performance. Reassessment, Planning Updates Based on analysis and recommendations, what additional assessment data would be useful for agency decision making? Collect appropriate data elements to refine assessment profile.
Update plans based on new data. Consider if the data collection and analysis processes are working or if they need to be modified. Establishing a Profile Considering your Organizations Culture HOW DOES WORK GET DONE?
Work gets done Drucker cautions us: Work doesnt get done by a magnificent statement of policy. Work is only done when it is done. Done by people. By people who are properly informed, assigned and equipped. Afterword, Five Most Important Questions
Organizational Culture Why is organizational culture so important? Culture shapes behaviors and help individuals understand the organization. Culture is often so strong and so powerful that when there is a discrepancy or inconsistency between the current culture and the objectives of change, the culture will win.
The Culture In An Organization Includes beliefs about What the world is like (what is reality?) How to respond to social and material environments The appropriate way for people to interact with each other Such things as chain of command and other management structures Culture Is Reflected in
Way work is organized How technologies are used How staff responds to customers The understanding of conditions related to poverty Health/wellness (including mental health), disabilities education, motivation, etc.. . Organizational culture
is a pattern of shared basic assumptions that the group learned as it solved its problems of external adaptation and internal integration. These patterns have worked well enough to be considered validand therefore, taught to new members as the correct way to perceive, think, and feel in relation to those problems. - Edward Schein Organization Culture
https://www.youtube.com/watch?v=y-PvBo75PDo Organizational Consider The formal/overt messages your agency uses in describing your services and goals The informal or hidden aspects, attitudes, feelings under the surface
One tool: Assessing the Organizational Culture Look Around What do you see? What does that tell you? Internal Communications Communications with Others
Chain of Command Decision Making Values of the Organization Staff Qualifications and New Staff Orientation Look around
What do you see at the entrance? In work spaces? How are people dressed? Does this differ in the various areas of the building? How much interaction is there? Who is talking to whom? Where do people congregate? How is the furniture arranged?
What is the noise level? Is everything quiet? Or loud? Are people talking? Can you hear conversations? Or just general voices? Another Set of Tools: Are You Results Oriented? Two surveys: one that staff responds to about their own activities and one in which they consider the agencys
perspective Are you results oriented? -- AGENCY SURVEY Read the statements below. They are "paired" to represent either a service orientation or a results orientation. Pick either statement a or b in each pair. 1 2
a. Our agency publicity highlights what we do (the services we provide). b. Our agency publicity highlights what was accomplished by the families and communities we serve and does not separate these accomplishments by "department" or "division" of the agency. a. Our agency policies and procedures are compliance driven. b. Our agency policies and procedures are results driven. 3
a. Our agency's strategic plan identifies what each department (or each funding source) will do. b. Our agency's strategic plan includes clearly defined outcomes that will be achieved through the work of the entire agency, not just by a department. Select the one that is MOST like your agency a.
b. Are you results oriented? -- STAFF SURVEY 1 2 3
Read the statements below. They are "paired" to represent either a service orientation or a results orientation. Pick either statement a or b in each pair. a. When asked about my job, I primarily talk about what services I do. b. When asked about my job, I primarily talk about what changed and what was accomplished by the families and communities I serve. a. I I think about my work in terms of applying the rules of funding
sources or of the program. b. I think about my work in term of helping families and communities change for the better. a. I think of myself as a staff member of a particular department or program of our agency. b. I think of myself as a staff member of the agency, who is working to improve the lives of people with low income. Select the one that is MOST like your agency
a. b. CSBG Performance Management Framework (PMF): Implementation 69 From ROMA Audit to
Impact Pathways Plan Part III What action will you take? Identifying a Path Help agencies (using their NCRPs) make explicit their personal and agency-identified expectations related to improving implementation of the full ROMA cycle in the Community Action Network. Help agencies use ROMA to improve performance
and visibility. Some barriers: o o o Have fear and anxiety about approaching agency change. Dont have a map (need the steps showing how we could complete something). Need to understand both quantitative and qualitative sides of the work
We are introducing two terms to this process: Scale UP and Scale OUT Scaling Up involves building a ROMA Culture in agencies and in the network that produce a supportive environment to embrace a results orientation and a performance management framework. Scaling Out is a horizontal spread of knowledge
and principles from peer to peer throughout the entire national community action network. Scale Up: Recognizing a ROMA Culture Support Organizational Culture that is Results Oriented Embrace a unifying Local Theory of Change Connect with the National Theory of Change
Why do efforts to initiate organizational change fail? No matter the change, no matter the organization, there is one constant that largely determines success or failure it is the role and importance of organizational culture. Culture shapes behaviors and help individuals understand the organization. Culture is often so strong and so powerful that
when there is a discrepancy or inconsistency between the current culture and the objectives of change, the culture will win. Moving to a Results Oriented Culture Implementing the concepts of Performance management What we hear
ROMA isnt a part of the broad agency culture. It may be something CSBG programs talk about, but its not recognized across the agency. There is limited buy in from executive leadership Is it often noted that leadership
sees that things are working just fine now so why change? ROMA is just for CSBG so it doesnt impact the rest of the agency Change to: "there's an opportunity to shift our org culture to one
of results-orientation "we have yet to prove the value of ROMA to the leadership" we are able to use ROMA resources to improve the whole agency, not just CSBG funded projects" I am very familiar with Leadership that wont buy in to ROMA. Because,
for years, that was me. BUY-IN OR NO BUY-IN AT THE TOP ? Who is the Champion? The person at the top of the agency (ED/CEO) Someone from agency leadership (upper management) who has authority Anyone else who feels passionate about the anti-poverty work the agency is doing
to move individuals and families to economic stability and to support the creation of healthy communities Building Blocks of a ROMA Culture 1. Recognition of a Results Orientation 2. Broad exposure to Introduction to ROMA 3. Executive Staff and Board acceptance and participation in ROMA and Performance Management
4. Trained staff in trusted position 5. Employing a shared language 6. Includes the whole agency (not just CSBG) 7. Incorporated into existing regular activities #1. Recognition of Results Orientation and Performance Management The agency demonstrates Having a focus on results not services
Acceptance that agency performance must be identified and improved Understanding of ROMA principles Value of Performance Management systems (ROMA) as a tool Being Results Oriented Means Being Data Centric Both long term strategic and day-to-day operational decisions should be based on facts that can be
sourced back to reliable and accessible data. Is the agency oriented to apply data as a source of actionable insight in support of advocacy, marketing, engagement of stakeholders and the broader focus of the organization? Does the organization value decisions that are supported by verifiable data that is accurate, reliable, timely and complete? #2. Broad Exposure to Basic ROMA and
Performance Management Principles Assure shared understanding across the whole organization Schedule and conduct Introduction to ROMA training for all departments Directors, Managers, Supervisors Board Members Direct service staff Repeat or expand training as needed to reinforce and refresh knowledge
New staff and new board members Staff who have a change in work duties; Board members with new responsibilities. #3. Executive Staff and Board Acceptance and Participation ED and Board demonstrates Results Orientation and Performance Management principles Organization activities are linked to National Goals Focus is on outcomes not just outputs
Actual performance is compared to projections ROMA language is part of every Board agenda Frequent reference to the ROMA cycle in reports and discussions Assure Executive and Board have training on ROMA #4. Trained staff in a trusted position Organization has invested in staff member(s)
becoming Certified ROMA Professional (NCRP) NCRP is valued and trusted Organization trusts in the expertise that training imparts to that staff person NCRP has assigned duties that include review and input on all aspects of agency planning and implementation of ROMA. The agency has staff with expertise, experience, and skills to get the most value out of their data with an emphasis on data analytics.
#5. Employing a Shared Language Become familiar with the terminology of Performance Management, Results Oriented Management and Accountability, Continuous Quality Improvement Use terminology consistently (we suggest adapting ROMA terminology) Use acronyms carefully
#6. Include the Whole Agency All program areas are Results-oriented ROMA thinking is not limited to CSBG ROMA Cycle is foundational to organization strategic planning. ROMA cycle is applicable to all activities in the agency Identify results and report performance throughout all programs. Create a Local Theory of Change
#7. Incorporate ROMA in regular Activities Use the Checklist to map out current practices for all areas of the ROMA Cycle in all program (department) areas. Adapt Monthly Program Reports Assure all program reports include outcome language Reference National Goals and activities reported
by National Performance Indicators (NPIs) How to Change the Organizational Culture First, assess the current culture. Then, understand what the changed culture would look like. Make a plan for change Implement, reassess, analyze, plan for next steps Scale Out:
Everyone Has a Role in ROMA Implementation Support Effective Action Scale Out Spread knowledge and principles from peer to peer to create a cohesive body of support tied together by common understanding and the desire to act.
Some Things That Have Worked Talk to your peers Engage a planning partner Organize a group of interested people who can discuss and consider actions related to improving agency efficiency and effectiveness. Create stories and speculation about the expected impact Promote Knowledge Transfer
Sharing ROMA Consider how you can share key elements of ROMA with peers and colleagues in the agency. New employees Management Teams Board Members Connect with people who are champions in the agency.
Positional and Personal Power There are two kinds of power one that comes from your position in the agency (because of your job, you have authority to make decisions) and the other that comes from your interest and passion about a subject (your personal power) Recognize the importance of influence (instead of leadership or authority) Improve your communication skills:
conflict management, relationship and trust building reflection, powerful questions, empathy, curiosity Act from humility Join your knowledge with the knowledge of others Where are you in the agency? Thinking about your role Is ROMA valued in the organization?
Do you have tasks and responsibilities related to ROMA implementation? How do other professionals/staff in your setting view work related to ROMA? Are you valued for your work with ROMA? How do you view your ROMA work? What do you think your contribution is to your agency? What could it be?
Find The Influencers Sub Groups Can Be Important -Identify the IMPACT -Understand the PATHWAY needed to get to the impact -Create action steps in your PLAN
CREATE A PLAN What would be easy? What is one thing you identified as a barrier that might be easy to change? Are there some people/departments that are interested in change, but just dont know how to move forward? If they are already moving, it could be easy to
help them along! What is critical? Is there something that is not happening, or something that is not happening well? Something that is an Organizational Standard that is unmet? Something that you feel you can influence? Recognizing what you can
and cant do (right now) Start with something you CAN change Build a win and then you will have success stories to build on Is there something happening that is an excellent example of ROMA principles? Something that is already working? How can you promote and celebrate it? Establish a Pathway forward.
What do you need for change? Quality Improvement ROMA skills Ability to
interact Software Hardware time Example: We do a good job of reporting on outputs but not with reporting on outcomes. How is the reporting on outputs structured?
Who does it? How often? Measurement tools? How is it collected? What is the related outcome? (or outcomes) How would you know if that was achieved? Who would do data collection? How would they measure? How often? Identify the point where the system breaks down or could break down What is the barrier (time, customer is not available for follow up, tool doesnt provide measure, etc.)?
What has to happen first to make an impact? What will that open the door for? (and so on..) Establish a pathway to support the establishment of the new action steps. Identify resources you will need (time, people, material, training, etc.) Time frame for you to be engaged in the pathway. Pathway for collecting output data Appears no
discussion about overall family goals is happening Supe rvis not t or and co alk a uns
have bout wha elor do happ t mig famil ened to t ht ies se h rved e No plan for follow-up to find out results
Only servi c no e data is o repo utc expe cted ome data rted
to be i repor s ted Some Ideas of Areas to Pick for the Plan
. . . . . .
Blank Flow Chart After you have taken action, you need to know what happened OBSERVE AND MEASURE RESULTS Include Indicators in your Plan Your Impact Pathways Plan will not only include steps and time frame, but will also
include indicators of success. How will you know you are making progress? What do you expect to see change that you can measure? Use a Scorecard to Follow Progress Some General Trends to Monitor These are things the network is talking about: Using ROMA cycle to consider performance,
improvement and results of new efforts Identifying measurement tools to track impact at both family and community levels Strengthen data collection and analysis (in addition to just using data for reporting) Using data to inform decision making Closing Thoughts What else do you need to know?
Will my organization do all of this? Considering how much we are already doing, what needs to happen to improve? Who will take the lead? Who is responsible? So what can YOU do? What is the first thing you will do? Will you use the ROMA checklist to determine your agencys current practices?
Other steps? Implementing ROMA ROMA Next Generation Training Series https://communityactionpartnership.com/publication_toolkit/ roma-next-generation-resource-guide On-Demand eCourses Community Action Academy
NEW! Implementing ROMA Access Community Action Academy (FREE!) Create Account/Login ROMA Training ROMA Next Generation Training Series https://moodle.communityactionpartnership.com Associated
artnership.com/course/index. php?categoryid=23 Contacts Courtney Kohler, MPA, CCAP, NCRT Senior Associate, T/TA Community Action Partnership National Office [email protected] Barbara Mooney, Ed. D., NCRP
Director Association of Nationally Certified ROMA Professionals [email protected] Carey Gibson, MSW, NCRP Project Manager National Peer to Peer NCRI Project [email protected] This presentation was created by the National Association of Community Action Agencies Community Action Partnership, in the performance of the U.S. Department of Health and Human Services, Administration for Children and Families, Office
of Community Services Grant Number 90ET0465. Any opinion, findings, and conclusions, or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the U.S. Department of Health and Human Services, Administration for Children and Families .
(TDWG call), possible joint call w/ CGSC . early Aug. Tribal contacts list - format & arrange for state planning use via RHPWG. OGWG - webinar possibility? Remaining EI's from contacted tribes- share with OGWG. Fire and Smoke Work Group.
Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann Publishers, San Francisco, CA, USA. 1999 [Kohonen (1982)] Teuvo Kohonen. Self-organized formation of topologically correct feature maps. Biol. Cybernetics, volume 43, 59-62 [Kohonen (1995)] Teuvo Kohonen.
No mobile code. No knowledge of tariff policy or structure. No need for smartness. ... Shared key K. Can verify . correctness. of computation without access to secret readings! Decrypted. ... Private billing, and other uses of data are possible....
Only national security ranked higher among overall concerns, according to a Reuters/Ipsos poll conducted during the 2016 presidential campaign. 92% of all Americans, regardless of political affiliation, think that all Americans should have the right to affordable health care.
Prof. Eileen Baldry — UNSW. Prof. Leanne Dowse — UNSW. Dr Jesse Cale — UNSW. Prof. Joseph Graffam — Deakin University . Prof. Jane McGillivray — Deakin University . A/Prof. Margaret Giles — Edith Cowan University. Prof. Andrew Day —...
Compound Sentences ~ A Glance at Grammar. Definition. A Compound Sentence is a sentence that joins two independent clauses together with a coordinating conjunction or semicolon. ... Fixing Choppy Sentences. Ex) The cat was happy. Ex) He slept underneath the...
Ready to download the document? Go ahead and hit continue!