|
| |
REP YEAR 3: AUGUST 1, 1998 – May 31,
199 EXECUTIVE SUMMARY By Dr. Anita Baker, Evaluation Partner
The Rochester Effectiveness Partnership (REP) began in 1996 as a two-year pilot, participatory evaluation project that brought together funders, evaluators, non-profit human service agencies, and other organizations seeking to determine and improve the effectiveness of their work. In the first year (1998-1999), of a two-year Phase 2, REP continued to focus on its primary goal to educate its partners about effectiveness and evaluation. Additionally, the partners extended project learning further in their organizations, and began to address institutionalization issues.
Year 3 Outcomes
In addition to fully incorporating six new provider partners and five new funding partners, specific project outcomes were accomplished. While there were some lingering and new challenges (e.g., the need for training for provider partner CEOs), overall, the benchmarks established during the pilot were met or exceeded during the first post-pilot year. Specifically:
 | All Class II provider partners, like their Class I peers, demonstrated their understanding of the training content by conducting teachbacks and mini-studies, designing comprehensive program evaluations, and applying what they learned to their regular work. They also shared their new knowledge with the other partners through a teachback session and an extended governance meeting. |
| The Effectiveness Study Group (ESG) for funders continued to meet regularly and took on cross-partner and community-wide projects such as revision of the Common Application and Report Forms (CAF/cRF), development of the Rochester Area logic model, and provision of training to the community on use of these. |
| As in the pilot, most partners, including those new to REP, worked together more, and all indicated they had increased their understanding of and respect for each other’s work. |
| Participatory impacts, such as enhanced organizational capacity and extension of REP practices beyond the individual trainees, were maintained by those original partners who remained active in REP. |
The clearest finding from analysis of Year 3 REP evaluation data, is that despite project modifications and some staff turnover among participating agencies, REP continues to engage partners and deliver effective training and participatory impacts. So far, REP outcomes are both sustained and replicable.
Immediate Challenges and Key Questions for REP’s Future
Initial Phase 2 outcomes indicate that REP continues to be a viable strategy for internal capacity building at both the agency level and community-wide. The future of REP, however, is still to be determined. The governance team began planning for REP’s post-Phase 2 future early in 1999. This resulted in the decision to maintain REP beyond its second phase, and to continue clarifying future directions. It also resulted in the identification of three immediate challenges and several key questions that will direct the governance team in the planning beyond Phase 2. Immediate challenges include the need to:
| clarify impact (quantify progress where appropriate), measure it and report it; |
| involve additional evaluators in REP to incorporate additional insights/expertise and ensure feasibility of continuing REP; |
| explore incorporation of external evaluation of REP efforts to augment current self-assessment. |
Key questions regarding the future include:
| How is the community different because of REP? For example, staff, agencies, boards, funders, clients? |
| Is it possible to sustain the learnings from REP so they become part of organizational and community culture? Program evaluation should be part of the infrastructure of all organizations. |
| What is the critical mass for the community in terms of organizations that have been trained by REP? How will we know if/when we have enough of them? How do organizations trained by REP act? What are our standards of desirability? |
| Can components of REP be institutionalized? |
| How do we continue to foster the safe harbor for learning that REP has created? |
| How can we expand REP’s interconnections with the broader evaluation community? |
These questions will guide ongoing efforts to plan for the conclusion of Phase 2 (through December of 2000) and beyond.
Next Steps for Phase 2
| Involve CEO and staff of 6 new provider agencies in REP, while continuing work with both Class II and Class I alumni provider partners (including a tutorial for additional staff from Class I). |
| Continue Effectiveness Study Group for funders and help participants continue to work collaboratively on such projects as review of actual logic models and new CAFs, and meeting community needs for gathering and disseminating research and evaluation findings. |
| Establish and work with five interest-area resource groups among the partners to effectively complete Phase 2 efforts. The groups would address fund development (to determine financial support for post-Phase 2 work), evaluation (to develop evaluation plans including an external component for Phase 2), conference planning (for June 2000), diversity (to determine how we can increase diversity on the Governance Team and throughout REP operations), and communications (to develop promotional materials). |
| Follow-up on the usefulness of the Directory of Upstate New York Evaluation Professionals. |
| Link other evaluation professionals with REP to assist with ongoing training and to help guide future development. |
| Develop and conduct training for provider partner CEOs so that they can better support their trainees and extension of REP within their organizations. |
| Explore the value and feasibility of conducting training for other evaluation professionals on the REP approach to participatory evaluation. |
| Give presentations at national and regional conferences, explore related efforts outside the Rochester community, and meet with other groups interested in developing activities related to REP. |
Click here if you would like to receive a more detailed Year 3 Report.
|
