Last Friday the White House Office of Social Innovation and Civic Partnership hosted a one day seminar on the pay-for success (also known as social impact bond) approach to social sector financing. I’ve been critical of social impact bonds (SIB) in the past, so was interested to watch the White House’s live stream of the event.
The promise of social impact bonds is to provide a new way of financing some social interventions. Panelists at the White House summit were quick to acknowledge that SIBs are not a cure all approach, and that not all interventions are well suited for this type of funding.
My criticism of SIBs in the past has been that the investment approach assumes evaluations can be performed reliably. Obviously, I think evaluation is a pretty thorny issue that we have a long ways to go on. I was happy to see that proponents of SIBs at this event were aware of this problem, and while more optimistic about which programs are easily measured than I am, realized there were limitations.
The example that has been most used in peddling SIBs, and the only actual implantation of the concept I’m aware of, involves an anti-recidivism program in a UK prison. Prisons provide a nice opportunity for evaluators because you can easily collect longitudinal data (prisoners and parolees are already monitored) and you have a logical control group (neighboring prisons).
Other social interventions don’t provide such fertile ground for experimentation. Yet ascertaining the effectiveness of a program is at the heart of the agreement between an investor and the government, which backs the bond.
Indeed, as was discussed at the seminar, the critical negotiation in a SIB is when the investor and government establish the ground-rules for when the government pays out, and when it doesn’t. The investor has an incentive for weaker evidence of success while the government prefers stronger.
While the SIB discussion centers on thresholds like the number of positive outcomes, the discussion got me thinking about another metric of success, the probability that an intervention actually caused a desired outcome.
For example, an employment program might place a hundred people in jobs in a month, but would those people have found jobs anyway? This is the question a control group helps inform. But the control group does not answer this question definitively. Instead the best evidence one can get is that:
- A program exceeds its outcomes threshold
- There is a certain probability that the program deserves credit for this success
Therefore, the question is not just about achieving an outcome objective, but how sure we are that the relationship between the intervention and outcome is not due to random chance. Achieving a high significance threshold further limits the possibilities of SIBs, as in order to ensure fairness to both the bond holders and backers, ample data in a controlled environment is a necessity.
The White House seminar, as well as discussions I’ve had since my last post on this topic, softened my stance on social impact bonds. As Tracy Palandjian, CEO of Social Finance smartly said, social impact bonds are about managing “execution risk”. Where there are opportunities for well controlled experiments, like prisons, SIBs might very well help governments manage that risk.
What SIBs won’t do is drive innovation, nor will they replace all existing forms of social funding. But they don’t need to in order to be a success. If SIBs can help governments improve the performance and financing of a subset of their social investment portfolios, that’d be a significant accomplishment anyone would have a hard time questioning.