On face, funding agencies providing grants contingent on implementing organizations meeting outcomes objectives seems sensible. After all, the sector is moving toward an era of smarter philanthropy and impact oriented giving, so shouldn't funders demand outcomes for their money?
I have been aware of this problem for some time, but was recently faced with an ethical dilemma when a customer asked me to help them adjust their program offerings to better achieve funder required outcomes.
The organization I was working with provides employment services to low-income individuals. Their grant stipulated that a certain number of program participants had to get placed in jobs in a given period of time. At first glance this seems to be a simple optimization problem where the employment program wants to maximize the number of people placed into employment.
Given this simplistic directive (place as many people as possible into employment), the optimization is actually quite trivial; serve the people most likely to get employment and ignore the hard to serve.
In this case, the hard to serve also tend to be those who need employment assistance the most. People with children might be harder to place into employment, given childcare needs, yet these individuals arguably need employment more than the otherwise equivalent person without dependent children.
Similarly, better educated people are easier to place into jobs than those with less education. But better educated people are more likely to find employment irrespective of the program intervention. This, of course, highlights the difference between outcomes and impact. The grant required improvements in outcomes, that is, the number of people placed into employment. When the focus is on outcomes, simple rationality dictates finding the most well-off persons who still qualify for services, and serving those folks first at the detriment of those who are harder to serve but in greater need.
While serving those who are better off first is rational given the way outcomes thresholds are typically written in grants, it probably does not lead to the desired outcome of the implementing organization (nor the intended outcome of the funder either!).
Hence my firm's ethical dilemma. By wanting our assistance in optimizing outcomes according to my client's funder's guidelines, our client was unwittingly asking us to help them identify and serve only those who didn't need their help that much to begin with. Our mission is to help organizations use metrics to help people better, and in this case, a data oriented approach given a misguided objective would likely lead to under-serving a hurting demographic.
Of course, this mess is nothing new, and is outside the control of implementing organizations. Funders requiring meaningless metrics of implementing organizations is not news. However, as funders try to press their grantees for more results, I am concerned that funders with a poor grasp of the difference between outcomes and impact, and insufficient knowledge to properly operationalize social indicators, will force implementing agencies to act in financially rational ways that end up hurting their target populations.
The answer to this problem is better data literacy in the social sector. My practice to date has focused on the data literacy of implementing agencies, but I'm worried that the zeal for more proof of social impact has underscored an open secret; both our front-line and grant-making institutions have a limited capacity to use data effectively.
And to those who say that some data is better than no data, I would argue that data is more like fire than we tend to realize. Fire has done incredible things for humanity, but those who do not know how to use it are likely to burn themselves.