Impact calls are the future of transparency

Transparency is a building block buzz-word of the social sector. While there seems to be general consensus that transparency is important, the proprietary actions of social sector actors runs contrary to that idealized vision.

Given this imbalanced rhetoric to reality ratio, I was especially intrigued by Guidestar’s new approach to sharing its progress with the public. Guidestar is experimenting with what it calls “impact calls”, quarterly webinars where the nonprofit’s leadership discusses its finances, impact, and strategic road map. The concept of the impact call is modeled on the quarterly earning calls publicly traded companies hold for shareholders.

On May 12 Guidestar held its second quarterly impact call, the first I have had the opportunity to listen in on. The impact call provided a solid overview of the organization’s finances and projected revenues, as well as a short and mid-term strategic road map, although the call was quite a bit lighter on actual impact reporting.

During the call, Guidestar CEO Jacob Harold explained that Guidestar’s impact assessment strategy is still evolving, and that the organization is developing an impact measurement dashboard it may present at the next impact call. The insinuation seems to be that as the impact measurement tool evolves, Guidestar will be better positioned to report its outcomes.

Although I was disappointed not to hear much about Guidestar’s impact on its impact call, I was nonetheless impressed with the concept and even found the sharing of less exhilarating (although more easily enumerated) metrics such as subscribers and web-usage statistics a great step toward real nonprofit transparency.

A criticism of earnings calls is that the quarterly reporting encourages companies to focus on short-term gains at the expense of long term progress. Guidestar CFO James Lum wisely cautioned that while Guidestar is committed to reporting quarterly results, the organization’s focus is on its long-term strategy. I think this is the right sentiment to have, and hope Guidestar doesn’t feel pressured over time to start optimizing for short-term gains to score favorable headlines in philanthropy media at the exense of the big picture.

This should be a trend

The impact call is an obnoxiously obvious idea. Everyone should be doing this, although I’m not sure many organizations will. Kudos to Guidestar for taking this step, I would love to see, at the very least, foundations follow suit.

While it would be great for every nonprofit to host quarterly impact calls, I’m not sure many folks would care to tune in. Guidestar is the right organization to pioneer this approach because many of its constituents are nonprofits themselves, and more likely to consume this type of information. Similarly, foundations (directly) invest in nonprofits, who would not only be interested in hearing more about how foundations think, but could benefit from learning about foundations’ thought-processes, strategic planning, and overall claims of impact.

Transparency is easy when you’re winning. It will be interesting to see if this type of hyper-transparency holds when findings are less than stellar. The Hewlett Foundation has demonstrated a willingness to embrace this type of transparency in their recent decision to discontinue the Nonprofit Marketplace Initiative, which they announced with the explanation that evaluators found ” our grants have not made much of a dent” in the intended outcomes.

Publicizing wins and losses is the future of transparency. Impact calls are a compelling medium to communicate those findings. I look forward to the next Guidestar impact call, especially if the next call has more impact in it.

Using word clouds to select answer options

Selecting the right questions for your survey instruments can be tough. Equally difficult is identifying the right answer options for the questions you ask people. When selecting answer options, ideally you would provide enoughoptions to get some meaningful feedback and variation in responses, but not too many answer options as to overwhelm survey respondents.

Before launching any survey instrument it’s preferable to do what is called a survey pretest. A pretest is where you get a subsample of people who are like your intended survey audience, and you ask them for feedback on each of your survey questions and answer options. However, pretesting isn’t always possible.

I’ve been working with a nonprofit called Team Tassy that provides workforce services to families in Menelas, Haiti. Team Tassy wanted to learn more about the employability of families in its targeted communities by conducting a survey at a free medical clinic day the organization sponsored.

One of the questions on the survey asked what work related skills each of the respondents possessed. The problem was that we didn’t know whether we were providing the right answer options to the skills question.

Ideally we would have pretested the question to get feedback on what types of skills should be included in the answer options. However, pulling together a focus group abroad would have provided logistical challenges, making pretesting less of a viable option.

Since we were not able to pretest the answer options, instead Team Tassy took its best guess as to what the answer options should have been, and provided an option for respondents to fill in any other skills not included as part of the question’s answer options. We planned to use the free-form options to better learn what job skills options should have been included.

Team Tassy collected more than 250 surveys at the medical clinic it sponsored. Given the relatively large number of surveys, reading through each of the free-form answers wasn’t completely practical. Instead, we built a word cloud of the free-form skills options to get a visual idea of what types of skills were most mentioned.

additional_skills-300x266

The word cloud revealed that several individuals reported having merchant and dress making related skills, options that were not included among the original answer options. Going forward, Team Tassy will now include these options on future skills questions.

Word clouds are a pretty low-tech approach to data analysis. But they can be really effective, especially for getting quick feedback on what types of answer options you might included on your surveys.