The social sector is abuzz with the bruhaha over the proper role evaluation should play. There are some who argue evaluation is all important, that every intervention is guilty until proven innocent.
Others, like Jonathan Lewis, have taken the mantel of evaluation skeptic, arguing that it is the intervention, not the evaluation, that is paramount.
Personally, I have grown tired of the evaluation debate. The more interesting question, not only to me but to those who (allegedly) benefit from the services of the social sector, is how can evaluation make interventions more effective?
To be clear, this is a different question than the core query driving the debate in philanthropy circles. Those who identify on the philanthropy side (which despite the title of this blog, I do not) think about evaluation as a tool for deciding what to invest in.
While evaluation can, and should, inform investment decisions, the brighter promise of evaluation is in informing the decisions of implementers. In this way, I think about evaluation in two tiers, one that provides shorter term ongoing feedback to interventionists and another that is longer-term, and more exhaustive in scope to prove or disprove impact.
Although this grander, longer term, up or down, live or die evaluation is seen by some as the sole objective of evaluation, the fact is we are a long ways off (if we will ever get there) of having such reliable binary indicators.
What we are equipped to do is develop interim, iterative indicators that can inform implementers of whether their interventions are headed in the right direction, and more importantly, how to adjust accordingly.
But in the current environment of philanthropist driven evaluation furor, imperfect evaluation techniques have been mistaken for reliable crystal balls. This has had a chilling effect on implementing agencies, as daring to use evaluation for tactical purposes creates the political liability that negative results can lead to loss of funding.
In the business world, investors invest in competent people with vision. Those people, the day to day managers, use a myriad of techniques to improve products and raise profits. One such tried and true technique is hiring consultants to evaluate any number of parts of their businesses.
Smart investors do not parachute in, scoop up an isolated internal memo, and use that to decide whether the company is going to hell or not.
Yet that is what we risk doing so long as we position evaluation as a threat to implementing organizations. ultimately, the issue is not how implementing organizations nor philanthropist funders feel about their investments or interventions, all that matters is what happens on behalf of those in need.
The best way to help people better is to foster an environment where seeking information and changing behaviors as a result is rewarded rather than de-funded.