To What End? The Importance of Outcomes and Performance

Recently, a good friend, the new head of a k-12 private school, called for advice. Like most organizations, the school has challenges, and board members have recently asked for more information on how the school is performing. Working to establish a greater discipline and transparency in how the educational and administrative aspects of the school are managed, he wisely reached out to one of the professional associations of private schools for its template “scorecard.” When he asked for my feedback, I suggested two issues: 1) if he used the scorecard with no adaptation, he would have “a report for report’s sake”—an approach that inevitably does not work—versus sharing the information he should be using to manage the school; and 2) the scorecard vividly illustrated the dissonance between social outcomes and operational performance (little on student outcomes and much on operations) that characterizes such reporting in the nonprofit sector and long term would not be the answer they sought.

The good news for my friend is that his board’s request provides the opportunity to move beyond this kind of operational reporting—standard financials and information that lets you "check the boxes"—to a system that provides more relevant information, connecting the organizational achievements with the ultimate goals for students. Information like this helps the board and leadership make good decisions based on real data. More »

To What End? The Importance of Outcomes and Performance

Recently, a good friend, the new head of a k-12 private school, called for advice. Like most organizations, the school has challenges, and board members have recently asked for more information on how the school is performing. Working to establish a greater discipline and transparency in how the educational and administrative aspects of the school are managed, he wisely reached out to one of the professional associations of private schools for its template “scorecard.” When he asked for my feedback, I suggested two issues: 1) if he used the scorecard with no adaptation, he would have “a report for report’s sake”—an approach that inevitably does not work—versus sharing the information he should be using to manage the school; and 2) the scorecard vividly illustrated the dissonance between social outcomes and operational performance (little on student outcomes and much on operations) that characterizes such reporting in the nonprofit sector and long term would not be the answer they sought.

The good news for my friend is that his board’s request provides the opportunity to move beyond this kind of operational reporting—standard financials and information that lets you "check the boxes"—to a system that provides more relevant information, connecting the organizational achievements with the ultimate goals for students. Information like this helps the board and leadership make good decisions based on real data.

Our conversation reminded me that leaders struggle again and again to respond to public and private funders seeking greater accountability, while trying to move from their own intuitive styles of management to one more disciplined, results-focused, and information-based. All of this is made more difficult by the sea of social complexity in which nonprofits exist, where they do not have the same types of clear outcomes and metrics, e.g., profit/loss, earnings per share, etc., their business counterparts have.

One of the unfortunate results of the increased focus on accountability, social return on investment, and the professional management of nonprofits is the conflict it can create. Many believe that being mission-driven and performance-driven are mutually exclusive, yet our experience shows that this need not be the case. In fact, I challenge the naysayers on both sides of this debate: The leaders who are the most mission-driven should be the strongest advocates to ensure they are focused and achieving the social outcomes they claim for those they serve, resulting in the broader societal impact they have labored so hard to advance.

The Problem
In our work with Venture Philanthropy Partners, we’ve found tension and confusion in how the nonprofit field approaches the definition, execution, assessment, and use of social outcomes and operational performance. Ideally, a nonprofit should be keenly focused on achieving social outcomes for those served and, thus, managed accordingly. Yet, there is great ambiguity and lack of definition for social outcomes and operational performance, how clear and sharp focus on execution to achieve them takes place, how they are assessed, and, ultimately, how this information is used. And, there is a lack of experience and educational support to help nonprofit leaders—and their boards—learn to define outcomes and performance, manage the organization to their achievement, and, in turn, assess how they are doing in using this type of information as an integral part of how the organization is managed.

The following examples illustrate some of the frustration and confusion.

▪ In the late 1990s, I was involved in a national initiative to make a difference in improving the lives of children. In one of the planning sessions, the discussion was singularly focused on increasing the volume of volunteerism. New to this topic (and clearly naive), I asked, “To what end?” The silence in the room of experts was deafening. Most there just assumed that if someone volunteered, that would inherently be good. And, I certainly wasn’t questioning the “goodness” of volunteering. I was, however, seeking to tie the act of volunteering (a means) to making a difference in the life of a child (an end, or outcome). I think, at that point, I was considered an interloper who clearly didn’t understand the value of the larger vision they were defining. Years later, my question remains unanswered.

▪ In many cases, there’s a gap between research that shows what works with children and organizations implementing those research-based approaches. Despite strong, empirical evidence that shows the importance of young people having positive relationships with caring adults, few nonprofits working with children measure the amount of time that young people actually spend with adults with whom they are matched and how long such matches last. I have, however, seen RFID tracking systems that count the number of youth at the site—providing little to suggest what they do or how they are served when they show up. Although most would not see these relationships as outcomes per se, I believe they are when one considers that as a consequence of these relationships, young persons gain hope about their future and the belief that someone cares about them. Even if not considered an outcome, they are important inputs (strong programmatic performance indicators) and can be managed so that the young people who are intended to benefit from a program actually receive a high enough “dosage” (the Big Brothers Big Sisters mentoring evaluation suggests this should be a matter of four hours a week for at least a year) that lasts long enough so that the children will actually achieve measurable benefits (outcomes) with real social consequences, e.g., better school attendance, improved relationships with their families, less alcohol abuse, etc.

▪ One of the region’s brightest and most enlightened business and civic leaders I’ve had the pleasure to know was refreshingly candid about the difficulty of determining outcomes and appropriate indicators. As the board member of a faith-based community organization, which has done yeoman’s work for many years, he said, “We track the number of volunteers.” And, as before, I said, “But that doesn’t tell you about what you’re doing to improve the lives of young people.” To which he responded, “I know, but that’s all we’ve got.”

A Different Leadership Mindset
It won’t surprise you that I believe the solution to this confusion begins with leadership.

Our first challenge as leaders is to define, gain clarity, and understand the social outcomes and operational performance relevant to our mission and organization. Gaining this focus to help guide the organization is, in and of itself, a major accomplishment and mark of progress in the organization’s development and evolution. As an organization gains greater sophistication in its focus on results, the hope is that they will evolve ever more effective means to assess how they are doing in the achievement of their targeted social outcomes and operational performance that, in turn, helps them manage more effectively for greater results for those served.

From my experience in the private sector, I’ve learned that executive leaders have to want to have greater clarity on what it is they seek to achieve and to use information to steer their organization and keep it on course. The most successful are predisposed (and committed) to this personal and organizational focus and accountability. The same mindset is critical in the nonprofit sector. Focusing on actions to achieve targeted outcomes and operational performance goals requires an organizational culture—starting with the head—that wants to think and function with a results orientation, while still being highly mission-driven. McKinsey & Company’s publication, “Effective Capacity Building for Nonprofit Organizations,” highlights the need for an organization to have a strong values-based culture and a strong performance culture. They go on to say that cultivating these cultures “will require a strategic, intellectual, and often difficult process. It requires a substantial commitment from the organization in terms of time and resources, but the payoff can be substantial.”

Leaders must focus on doing “the right things,” which translates to focusing first on the social outcomes that lead to the greatest, most relevant, lasting change in the lives of children and youth. At the same time, they have to focus on the operational performance that enables and prepares their organizations to deliver on their mission. Essential will be a performance culture that develops to use information as a natural part of their organization to know how they are doing, and, being so informed, learn to adapt, improve, and stay on course—on a continuing basis. A question such as "What would you have to observe in two or three years to indicate that you are on track to achieving your desired longer-range goals and outcomes?" helps to focus an organization somewhere between (and beyond) today's activity and organizational measures and the long-term focus of most social outcomes. This mindset or management attitude can’t be delegated and it’s more than “systems.”

Sounds like a mindset everyone would say they want and/or already have? Theoretically, yes, but getting an organization to focus on and execute to social outcomes and operational performance objectives and then to implement an effective performance assessment and management system can be problematic, even evasive, for some organizations. As supported by the McKinsey report, doing so is a significant undertaking requiring time, thought, and expertise.

Lori Kaplan, Executive Director of the Latin American Youth Center, describes their progress: “Since we began our journey in evaluation, I recognized that I had to align the entire organization against this commitment. From the Board, through me, utilizing the Evaluation and Learning Staff, I have fused outcomes and performance evaluation throughout the fabric of the organization. Evaluation is discussed from the time of an initial interview, through new staff orientation and tied to performance appraisal. LAYC offers a monthly prize to the staff or team who best exemplifies our work using the ETO (Efforts to Outcomes) system and evaluation data to guide their work. My role in championing this change is critical to the success of the work. The excitement that I see—when direct line staff view and understand their data, look at their outcomes, and make decisions accordingly—is really fun.”

Rewarding it can be, but even organizations, like the Edna McConnell Clark Foundation, whose work has been deliberately focused on helping its grantees gain outcome clarity and focus and define and measure their outcomes and impacts, have found that these efforts are indeed challenging. And this Foundation’s work is considered by many, me included, to be at the forefront of the social outcomes and assessment movement.

The Formidable Eight
Organizations—regardless of size, mission, or sector—must be clear about what they do for those they serve (mission), how what they do is benefiting those they serve (social outcomes), and how well the organization functions (operational performance). As leaders, board members, or staff, we must ask these eight formidable questions of our organizations and ourselves with rigor and objectivity.

1. What do we claim to do? (What are the social outcomes that we are trying to achieve for the children and youth in our programs?)
2. What are the programs and services we offer and, in theory, how do we believe these actions lead to the achieving the sought after social outcomes? (What empirical research or evidence suggests what works, what doesn’t, how the parts have to fit together, and where might we have gaps? What is our “theory of change”?)
3. How do we know we do what we claim to do? (What are the social outcomes we are actually achieving for these children and how do we and they know the results?)
4. How do we inform those we serve on how they are doing? (What do we do to share social outcomes and indicators that lead to them with those served to allow their understanding of the progress and growth they achieve and the value they can take from this interaction?)
5. How effective and efficient are we? (What does our operational performance tell us about our cost-effectiveness, quality, financial health, organizational capacity, sustainability, and other factors that inform about an organization’s performance and how it is positioned for what lies ahead?)
6. Why is what we do important and relevant? (Are these social outcomes likely to produce long-term, significant change in the children’s lives and, if so, what is the envisioned impact?)
7. What would be the loss if we no longer existed? (Would anyone care, would our work be subsumed by others, or would the absence of what we do be truly missed over time?)
8. To what end? (Do we always ask this overarching question whenever we propose or approve actions?)

I predict that facilitating thoughtful discussions about the answers or views to these questions with your board, management, staff, and those served by your organization will be revealing. Just like a good planning process, major value comes from getting people to think about how their work contributes to the achievements of social outcomes and which are of greatest importance. You’ll be a step ahead of many organizations with a clearer focus of what you are about.

Identify and Describe Your Social Outcomes
In our work in the nonprofit sector, we are increasingly learning that the definitions of outcomes lie, much like beauty, in the eye of the beholder or, perhaps more provocatively, in their vested positions and/or the context of their own experience.

Many nonprofit leaders, especially founders, have a pretty clear understanding of the social outcomes they want to achieve, but these are not always codified and communicated to others. And, in some cases, perception of the social outcomes to be achieved may not have stayed in step with how the organization has grown. For example, a school moving from its original programming for grades 1-8 to the addition of grades 9-12 may find the definition of social outcomes much more complex and quite different from what they had envisioned.

Taking the time to identify and describe the direct and indirect social outcomes for children and youth that your organization seeks to achieve will be a worthwhile effort to help clarify and codify what you do and allow for a more deep-rooted and common understanding among board, management, and staff.

And, if you find that exercise illuminating, then give serious thought to undertaking a more intensive effort that would attempt to codify and understand your organization’s theory of change. Aligning and assessing the identified outcomes with the current programs and services will give you the opportunity to identify gaps, redundancies, or areas needing improvement in your programs and services. Consider, for example, an organization offering a wide range of programs and services for young people. An analysis uncovered that if youth went through the programs and services in the right sequence and with monitoring to ensure completion, the link to a social outcome was clear. Management learned, though, there was no assurance this progression occurred. Result of this finding? The adoption of a “case manager” approach to monitor that youth were going through the programs and services in sequence to assure the social outcome was achieved.

The next step, after you have confirmed that the programs and services (theory of change) do, in fact, lead to the defined social outcomes, is to determine how to assess on a systematic and regular basis that these outcomes are being met and achieved. This could be using tests to gauge reading and math proficiencies in schools and after-school programs, it could be counting the number of immunizations given children in healthcare centers, and so forth. But I urge you to invest the time and engage the appropriate expertise to identify to the most relevant and meaningful social outcomes.

Common Sense Applies
Mission focus, outcomes achievement and assessment, and performance management are absolutely relevant and applicable to any organization, but approaches may differ. More pointedly, common sense must prevail. To ask a nonprofit after-school program serving 30 children, with a staff of one and several volunteers, and an annual budget of $200,000 to implement a rigorous outcome definition and corresponding information system wouldn’t make much sense. At the same time, even this organization needs to stay on course and know that it is making a difference. Should they be challenged to track a single social outcome with an unrelenting focus, be able to report on that outcome regularly, and then use this information to make informed programmatic decisions? Going out on a precarious ledge, should this organization ask if it should remain a free-standing entity or consider exploring a merger or consolidation with a similar or larger organization to gain the scale to adopt a basic approach to social outcomes and operational performance and the capacity to implement a performance management system? Regardless of the decision, the fact that thoughtful discussion takes place and that this leader exhibits a desire to have the right focus and to understand how he or she is doing with their work bodes well and probably puts them in a better position to deal with funders, partners, and other opportunities.

If, however, you are an organization providing youth transition/employment services, with 50 full-time employees reaching 500 youth through an annual budget of $2,900,000, one would hope (and the stakeholders should expect) some level of investment in outcome definition, clear execution, and systematic reporting on social outcomes and operational performance achievement—again, consistent to this size organization. And, once you start eclipsing higher annual budgets and increased complexities, then such focus and assessment are absolutely critical.

Conclusion
Old programmers, like me, are drawn to the elegance of logical sequences. So here’s my take on the progression of what we’ve discussed. Ensure clarity of mission. Project the long-term impact of your actions. Define, focus on, and be diligent in your execution so that those you serve are achieving relevant, high-benefit social outcomes. Establish the systems to inform you on how well you are doing. And, don’t fall prey to the “gods of evaluation” and analytical metrics. Remember raw data is captured, edited, and stored to become information. As the information continues to inform our experience in using and applying it grows, we gain knowledge. And, it is the judgment we apply in using this knowledge that leads to wisdom.

Although it’s difficult, I am confident that a more rational and systematic way to look at outcomes and performance is on its way to becoming more broadly accepted and applied. When all is said and done, our ultimate question is “To what end? How does what we do help improve the lives of children and young people?”

- Mario Morino

VPP Stakeholders Participate in K-12 Education Event

Earlier this month, a number of VPP investors and stakeholders including VPP President and CEO Carol Thompson Cole, VPP Investors Jack Davies, Washington Post Chairman Donald Graham, CityBridge Foundation President Katherine Bradley, Fight for Children Chairman Joe Robert, and VPP Investment Partner Friendship Public Charter School President and CEO Donald Hense spoke at a regional meeting of The Philanthropy Roundtable (TPR).  Co-hosted by The Community Foundation for the National Capital Region and Washington Regional Association of Grantmakers, the day-long session, “The Promise and Peril of DC Public School Reform,” featured DC Public Schools Chancellor Michelle Rhee and explored philanthropy’s role in the school system’s reform efforts.

Cole moderated a panel on Creating a Culture of Results, which featured Prince George's County Public Schools Superintendent Dr. John Deasy, E.L. Haynes Public Charter School Founder Jennifer Niles, and New Leaders for New Schools CEO and Founder Jon Schnur.  The panel discussed how developing an organizational culture focused on results takes time, demands visionary and committed leadership, and requires a team that is dedicated to, and equipped for, this process. The panelists agreed that a culture of results should be data-driven, with a focus on school culture and people, strong systems and processes, and executive accountability.

Another panel, Turning the Corner to Excellence, included Hense, VPP Co-Investor New Schools Venture Fund Partner Jordan Meranus, and Davies, who participated in his role as a Board member of See Forever Foundation/Maya Angelou Public Charter School (also a VPP Investment Partner). Panelists argued for increased support of charter organizations, performance and data management systems, developing strong principals and teachers, and early childhood education.  Panelists also called for an increase in community and political support in the form of advocacy and authorizer-support in replicating high-performing schools and shutting down low performers.

Cole said, “This is an exciting time for education in the District of Columbia. The fact that the philanthropic community is questioning how they can best support Rhee, and not why they should support her, is certainly an indication of great things to come.”

Management Changes

LAYC Welcomes New Chief Operating Officer
Thanks to Cheryl Aguilar, Communications Specialist, for this update.

The Latin American Youth Center has just finished a strategic plan focusing on a new pathway for “disconnected youth.” Responsibility for managing implementation of the strategic plan falls to Sandy Gutierrez, LAYC’s new Chief Operating Officer. In addition, Gutierrez will be responsible for all day-to-day operations at the Latin American Youth Center’s offices in the District of Columbia and Maryland.

“I am honored to be a part of one of the premier youth development agencies in the country,” Gutierrez said. “I am very familiar with the great work the Latin American Youth Center does for the community. I humbly accepted the opportunity to work at LAYC because I believe our youth need opportunities to grow thrive and contribute. Here at LAYC we create and provide them those opportunities.”

With more than 25 years of youth development experience, Gutierrez has successfully developed youth engagement programs in a variety of settings. Before joining LAYC, she served as Chief Programs Officer at the Boys & Girls Clubs of Greater Washington. In this role, she was responsible for overseeing all programs at the clubs in the District of Columbia, Maryland, and Virginia. Prior to moving to the District of Columbia, she served in New York City as the Bronx Regional Director for the Children’s Aid Society, a nationally recognized multi-service agency dedicated to supporting children and families in crisis. She worked at The After School Corporation, Center for the Next America, and CityKids Foundation, and has been a consultant to the New York City Department of Education, Department of Health, and United Way. She has held adjunct positions at Columbia University Teacher’s College where she provided training in mediation and conflict resolution and at Vanderbilt University Peabody School teaching Organizational Development.

“LAYC’s staff is committed to helping youth become successful and happy young adults,” said Lori Kaplan, Executive Director. “Sandy fits right in. She brings passion, energy, commitment, knowledge, and the desire to make a difference. We are fortunate to have Sandy on our team.”

Gutierrez has a BS Degree in Psychology from Pace University and a MS in Public Administration from the College of Human Services in New York City. In addition to her extensive experience in youth development, she is a poet, painter, and performer.

Outcomes

AALEAD Focuses on Measuring Impact
Thanks to Rick Chen, Development Associate, for this update.

AALEAD has always found ways to evolve and improve its programs to fit its population’s changing needs. During the last few years, they have put particular emphasis on the way program outcomes are measured. Since 2003, AALEAD has used the Efforts to Outcomes (ETO) database to track data on program effectiveness. Staff records information on the key goals, indicators, and outcomes for each program and reviews data on a monthly basis to refine and enhance programs. Their next step is to review and implement effective, appropriate evaluation tools, and assessment that fit into the ETO network. Going forward, AALEAD will ensure that new curriculum components will incorporate measurable outcomes before they are launched.

Awards & Recognition

MAPCS–Evans Campus Chosen for EPIC Award
Thanks to Anne Schoolcraft, Development Manager, for this update.

The See Forever Foundation (SFF) is pleased to announce that the Maya Angelou Public Charter School (MAPCS) – Evans Campus has been chosen as one of this year’s Effective Practices Incentive Community (EPIC) National Charter School Consortium award winners. New Leaders for New Schools developed the EPIC initiative to identify and learn from schools with significant student achievement gains and to honor educators in these schools for sharing their knowledge and skills with colleagues nationally. The Evans campus was in pool of 99 charter schools from 18 states and the District of Columbia.

As such, MACPS-Evans Campus is one of only two charter high schools in the District to have shown gains that categorized it as a Silver Gain school (the second highest ranking). Further, the campus is one of only four high schools to have received the Silver Gain Award nationwide. Awards were determined based on state assessment (DC-CAS) test scores comparing gains from MAPCS scholars in the eighth and tenth grades. "This award is a reflection of the hard work and dedication of our staff, teachers, parents, and most importantly, our scholars," according to Thaly Germain, assistant principal of the MAPCS-Evans campus.

In the coming weeks, the SFF/MAPCS community will host a school-wide celebration for students and staff and share best practices with other schools nationally. As noted by SFF/MAPCS Executive Director Cynthia Robbins, “This award is evidence that our approach to education is working, and we look forward to next year’s review as we strive to achieve even higher gains.”

AANN Featured in Washington Business Journal

The African American Nonprofit Network (AANN) was featured in this week’s Washington Business Journal. This new organization, led by former Freddie Mac Foundation President and CEO Maxine Baker, is proactively addressing the issue of the lack of minority representation in the National Capital Region's nonprofit sector. AANN assists nonprofits in identifying experienced African Americans for leadership and board placements by sharing opportunities, building relationships, and offering networking, mentoring, and training programs. AANN was created at the end of 2006 with funding and support from Venture Philanthropy Partners (VPP), which has committed $500,000 in seed money and strategic assistance to launch and grow AANN.

Last month, AANN held an inspirational launch event with more than 80 thought leaders, corporate representatives, and community leaders, including two of the nation’s foremost authorities on African American issues, civil rights leader Dr. Dorothy Height, Chair and President Emerita of the National Council of Negro Women, and Dr. Johnetta Cole, the first woman African American president of Spelman College.

Through its growing network, AANN has assisted in the placement of a number of African Americans into senior leadership and board roles, and is on track for additional placements in the coming months. VPP President and CEO Carol Thompson Cole said, “We believe that VPP’s support of AANN furthers our mission to strengthen nonprofit organizations serving children and families in our region. AANN efforts will help to create a diverse management base giving nonprofits the ability to better understand and address the needs of the multicultural children, youth, and families they serve.”

Nonprofit Roundtable Launches New Leadership Development Program

The Nonprofit Roundtable of Greater Washington has just announced the Future Executive Directors Fellowship, a new leadership development program that will provide intensive peer coaching, mentoring, skills development, and ongoing support to nonprofit professionals who aspire to become executive directors or nonprofit CEOs as the next step in their careers. The Future Executive Directors Fellowship is a nine-month program that begins with an intensive six-month series conducted over 11 days and concludes with three months of ongoing support facilitated by the Roundtable.

In the pilot year, the Roundtable will select up to 25 Fellows. The fellowship class will likely include deputy directors, program managers, and other mid- and senior-level nonprofit professionals from a wide range of nonprofits including direct service, advocacy and grantmaking and will reflect the diversity of the region including, but not limited to: geography, race and ethnicity, sexual orientation, gender, and religion. More information can be found on the Roundtable's website. Applications are due Friday, May 30, 2008.

• Download the complete Future Executive Directors Fellowship Description.
• Review the list of Frequently Asked Questions.
• Complete the Future Executive Directors Fellowship Application.

Applicants are encouraged to email fellowship@nonprofitroundtable.org with additional questions and Roundtable staff will respond.

The Washington Business Journal quoted Roundtable Executive Director Chuck Bean: "We've been watching the research over the last two years ... and we knew it was time that we really needed to do something," said Chuck Bean, executive director of the Nonprofit Roudtable, a network of the region's nonprofit organizations. "If we can accelerate someone entering a nonprofit leadership position and hitting the ground running, that's what this program is for."

Bean said that many current deputy directors or program directors may have one or two really strong skills, like fundraising or advocacy, but lack the time outside of their job responsibilities to add experience in other areas and improve their candidacies for top jobs.

"We want to top them off with the other [skills] and connect them with mentors that can help them," Bean said.