I just returned from a very meaty and reflective meeting on the role of evaluation in philanthropy (NOT as dry a subject as it may sound!).

It was organized by the Evaluation Roundtable, a program of the Foundation Center to improve philanthropy. The Roundtable was held in Baltimore, at the Annie E. Casey Foundation, headquartered there. Thought leaders in the field presented a number of rich discussion topics, including a first look at the history of the uses of evaluation by charitable foundations – a history that goes back only to the 1980’s, in terms of dedicated staff and evaluation units within foundations.

Within this 30-year span, four distinct ‘eras’ were identified – a lot of evolution and change in such a short period of time. The progression through these eras shows how evaluation has moved from being used by funders as simply an after-the-fact research tool to prove what worked and what didn’t within stand-alone projects. It’s current role is more of an ongoing tool for strategy development, learning and to measure the impact of the pro-active work of foundations – often over decades-long programs and initiatives.

The titles given to evaluation staff and units is telling in this regard. In the ‘80’s it was “Research and Evaluation.” Today it’s more often “Organizational Learning and Evaluation.”

There was a lot of discussion at the Roundtable about the natural tensions between evaluation to prove an idea or theory, and evaluation for learning and improving performance over time. Foundation boards, in their important fiduciary and oversight role, want proof of impact and accountability for the use of resources. Program staff tends to seek learning from evaluation in order to be more effective in their work.

The discussion surfaced an important shift in intent for the proactive work of foundations that deeply affects evaluation as well. Foundations are becoming more aware that they cannot, by the sole virtue of their grantmaking, solve entrenched social problems. Rather, the challenge is to better understand the root causes of problems and develop strategic approaches to address these causes. Here, evaluation as a learning tool and a strategy development tool is key, and long-term initiative frameworks must be flexible enough to adjust to learning as it is acquired.

Most pro-active and strategic grantmaking within foundations begins with a Theory of Change. This is a term of art for a framework that offers a rationale for the initiative, identifies the inputs and strategies that will be employed to effect the change that is desired, and a clear understanding of the outcomes that will demonstrate success. Too many Theories of Change have expectations that go beyond the power of a single funder to effect. Too many are fuzzy on the strategies/actions required to make change. And too few have clear outcome measures that will show whether the work is being successful. 

The Roundtable surfaced many, many valuable lessons. I’ll note just two here.

1. A Theory of Change must be:

• Plausible (realistic in their expectations for success)
• Do-able (enough management capacity and strategies that have a good chance of being implemented)
• Knowable (clear and measurable success indicators at specific steps a long the way). You can’t evaluate what you can’t measure.

2. Although funders never intend to do harm with their initiatives, any effort to make change at the systemic level will always involve winners and losers. Funders (or other actors) must decide what constitutes harm and have good reasons for acting to make change that may not be perceived as beneficial to all who are affected.

Finally, to sum up my experience of the Roundtable, I’d like to paraphrase a saying that was coined by the Kentucky Arts Council (Art is not a Thing; it is a Way). Evaluation is not a “thing” you do once when a project is over. It is a way of continuous learning within organizations wishing to do well and do good.