Good evaluation habits start early

It is common knowledge that bad habits are hard to break. The longer one lives with a bad habit, the more difficult it is to shake.

The same is true of social sector organizations. Good evaluation requires good habits. But good habits are difficult to form, and even more difficult to introduce into an organizational culture that has grown accustomed to evaluative misbehavior.

So first, what are good evaluation habits?

Broadly speaking, good habits involve

  • Intentional data collection designed to capture client indicators stemming from an organization’s theory of change
  • Regular, incremental program changes based on feedback from client indicators
  • Identifying comparison groups to compare progress against

Many organizations adhere to anything but good evaluation practices. It is especially difficult to get established organizations to change course, like turning around a cargo ship drifting downstream.

I have been hired time and again under the pretense of helping established organizations with their evaluation strategy. However, I have learned over time that while the request for evaluative assistance might be earnest, inevitably the conversation turns from “help us assess our impact” to “show us why we are awesome.”

I certainly understand why organizations devolve from the former to the latter. These established organizations have existing funders that want to see that their efforts have been paying social dividends. Ultimately, evaluators are brought in to prove to funders that money has been well spent rather than to help assess how future monies should be spent.

This makes sense, but it enforces bad habits. With funders breathing down agencies’ necks, agencies are incentivized to look for any positive trend in their data and report it as success, instead of looking for feedback that informs intervention design.

I’m not arguing that good evaluation habits are unachievable in larger organizations. But, instilling good habits in startup organizations is not only easier, it is also the low-hanging fruit in our collective shift toward a more outcomes oriented social sector.

The problem, though, has been that evaluation consultants (I stand amongst the guilty) have focused their efforts on snagging contracts with established organizations rather than working with emerging agencies.

To do our part in helping instill good habits in start-up social sector organizations, my company designed an evaluation consulting package that

  • Offers access to evaluation guidance year-round, as good evaluation strategies develop through time instead of in a single Word document
  • Fits in the budget of emerging organizations

Under the new package, we will work with organizations under three years old helping design and implement evaluation strategies. Because good strategies take time, we have structured our contracts to run annually with quarterly meetings and deliverables, instead of using an hourly rate model.

Each contract is a flat rate of $3,500 so startup agencies can afford the service and are not at risk of ballooning costs.

I love working in the social sector. Having the opportunity to help organizations develop effective evaluation strategies that inform program design is by far the most rewarding experience my team and I can have.

If we want to foster a sector that values the importance of good evaluation, we have to instill those principles in the fabric of the next generation of non-profit organizations, before they get stuck in their ways.