Month: February 2013

People hate standing in line, and just about everyone in the social sector loathes having to decide who should receive a particular intervention and who should not. But service rationing and queuing are facts of life, and taking a sophisticated approach to how we prioritize services can make a substantial difference in a program’s net impact.

I take an operations research approach to program effectiveness. In this framework, each program aims to maximize a social good (or minimize some harm) given a set of constraints. Constraints come in many forms, including financial and time constraints.

Most of the organizations I work with, particularly human service organizations, have to ration their programs. For example, a homeless shelter only has so many beds and might have more demand for beds than supply in any given night. Food, tutoring, vaccinations, and many other types of social interventions have to manage scarcity daily.

So how do we prioritize who gets food, or who has a place to sleep tonight and who doesn’t? Typically I see organizations adopt a first-come first served approach, but this approach doesn’t always generate the best outcomes.

For example, let’s say a homeless shelter has nine beds and ten people need a place to stay, the first nine get in and the tenth person doesn’t. Seems fair, right?

But what if the first nine people are healthy men in their twenties who have been homeless for one night, and the tenth person is an eighty year old chronically homeless man who faces the possibility of death spending one more night on the street?

Assuming this shelter in question prioritizes minimizing life lost, the rational thing to do in this case would be to abandon the first-come first-serve prioritization model and to ensure a bed for the eighty-year old, turning down one of the nine people in their twenties.

This example illustrates that primitive rationing algorithms don’t properly optimize organizations’ intended outcomes, and can have unintended consequences for more vulnerable persons.

A better way is to develop a rationing system that aims to maximize the outcomes most important to the implementing organization. If the organization in our example aims to minimize life lost due to homelessness, then its shelter waitlist should recognize this preference.

Now, the shelter could ensure that it always has beds for the most vulnerable homeless persons by always turning away younger, healthier homeless persons. But this strategy would risk going nights where the shelter has empty beds, which wouldn’t optimally use the shelter’s resources, as the shelter might prefer that a healthy homeless person occupy a shelter bed than no one at all.

So the question then is, how do we make sure we prioritize more vulnerable persons while still trying to minimize the number of empty beds at the end of each evening? If you guessed data, I must be getting predictable.

Specifically, we would use concepts from queuing theory, utilizing historical data on what types of people come during what times of the year to try to build a model that changes prioritization based on the time of year, staffing levels, and perhaps even hour of day (for example, the shelter might have lower entry thresholds later in the evening to ensure maximum occupancy). This is the same type of approach that business like restaurants and airlines use to allocate their scare resources, except instead of maximizing profits we are maximizing social outcomes.

There is a lot of talk, and plenty of nonsense, around how data is supposed to revolutionize the philanthropic sector. I think this focus on revolution is misplaced. Instead, approaches like the one outlined in this post can help us get incrementally more effective. Indeed, the real promise of data is not to show us we have been wrong all along, but instead to provide suggestions as to how we can improve. Strategic service rationing and queuing is a great place to start.

With every organization I work with, I begin the consulting engagement by developing an impact theory. The impact theory is the portion of the theory of change that identifies the causal assumptions of how a set of social interventions is expected to drive particulars social outcomes.

The impact theory is important because it sets the basis for how a program defines success, and how it intends to get there. Used correctly, the impact theory sets boundaries around what data points an organization’s survey instruments should collect.

At Idealistics we use a database system we developed to model an organization’s impact theory. We call this system the Decision Framework, as it models the decision relevant factors an organization faces in trying to maximize social outcomes.

When I put together an impact theory, I spend time speaking with program directors and staff, hearing from them what they believe differentiates their program offerings. I’m most interested in identifying what they believe would be different in the lives of those they serve were their particular program not to exist. In evaluative speak, this is called the counterfactual.

A large service provider I am working with emphasizes the customer service aspect of their work, which includes traditional basic needs human services offerings like clothing, food, and medical services. Many of these services are available in other forms, provided by alternative agencies. However, this organization argues its interventions are unique not just because of the services offered, but the way in which people are treated when receiving assistance.

Customer service

Social interventions tend to be defined as tangible outputs like medical vaccinations, counseling hours, and food baskets. While traditionally we think about interventions as activities or items that are plainly enumerable, can the way an output is delivered be an intervention in itself?

Brands differentiate themselves on service quality in the marketplace all the time, and charge a premium for it. While we all prefer not to be treated like crap, does better customer service drive better social outcomes? The impact of customer service on social outcomes is an empirical question which likely varies depending on what outcomes an organization intends to affect.

If good customer service is identified as a causal element in driving a set of outcomes, customer service needs to be operationalized to something we can measure, so we can evaluate the possible relationship between intended outcomes and quality customer care.

While an organization might assume there is a positive relationship between good customer service and positive social outcomes, the more interesting is how should an organization respond if the relationship between customer service and positive outcomes does not bear out?

The value of not being a jerk

Ideally everyone would be treated well, and no one would act like a jerk while administering social programs.

But what if customer service made no difference in the outcomes an organization was trying to drive, and good customer service came with a cost? If you could only treat half the number of people with good customer service than with crappy service, but otherwise get the same result, which is the optimal choice?

As we move toward a data driven social sector, these are the kinds of questions we will face, especially if the evidence runs contrary to our prior beliefs. It is easy to assume there is value in providing good customer service (which there might be), but we make a lot of assumptions all the time that on face seem intuitive, all to later uncover unintended consequences.

Used properly, evaluative metrics can help guide organizations make more impactful program decisions, but the real test is how we will react when data suggests what we are doing might be wrong.