Thursday, June 27, 2013

Remember that Evidence that Better Charities Have Higher Administrative Expenses? Turns Out the Reverse Holds in the US


In May, the group Giving Evidence released a study showing that high performing charities actually have higher administrative expenses.  This news was welcomed with fanfare by many in the nonprofit sector who believe there is too much pressure to reduce administrative costs.  Since then, it has been cited heavily and even featured as part of the Overhead Myth campaign launched by BBB Wise Giving Alliance, Charity Navigator, and Guidestar.  While I am willing to believe that lower administrative costs are not always better, and the Overhead Myth campaign has some merit, I also worry that the evidence to support the claims is rather weak.  To their credit, the folks at Giving Evidence have made the data publicly available so that these questions can be addressed.

Saturday, June 22, 2013

What Your Nonprofit's Twitter Usage May Reveal About Its Fundraising

When it comes to nonprofit fundraising, does the old adage that silence is golden ring true?  Consider the following mini-experiment:
  1. Take the top 100 charities as ranked by Forbes.
  2. Gather data on their fundraising efficiency from Charity Navigator (this excludes some observations due to missing data).  The fundraising efficiency number reflects how much it costs the organization, on average, to raise $1 of donations.
  3. Gather data on how talkative the charities are, as based on their number of tweets.
  4. Consider if/how the fundraising efficiency of organizations correlates with their talkativeness.
To see the data most clearly, I split the observations into quartiles based on number of tweets, from least talkative to most talkative.  The following chart reports the average fundraising efficiency of each quartile.


Note that the least talkative charities spend, on average, less than 5 cents to raise $1; in contrast, the most talkative charities spend 12 cents to raise $1, quite a stark difference.  Not only that, but the cost gradually increases as you move through the intermediate quartiles as well.  In short, organizations who tweet more actively are, on average, less efficient at fundraising.

Statistical Significance
The first question one must ask is whether this is mere coincidence that arises from a small sample.  Using a regression model, with fundraising efficiency as the dependent variable and number of tweets as the independent variable, reveals a positive relationship with a p-value well below 1%.  This means that if there was truly no relationship between the variables, the probability of seeing such a positive relationship in a sample of this size is less than 1%.  That is, the relationship is clearly statistically significant.


Variations
  • Is this connection due not to talking (tweeting) but more broadly an organization's overall presence on twitter?  After all, as discussed in this blog before, organizations with more followers also turn out to be less efficient fundraisers.  To address this, I used tweets per follower (rather than tweets) as the "talkative" variable, and the positive correlation persists.
  • Is this connection driven by how an organization splits its costs between fundraising and non-fundraising categories?  After all, many complain that different application of accounting rules is what makes some organizations appear more efficient than others.  To address this, I used revenue growth (rather than fundraising efficiency) as the "fundraising" variable, and the connection again persists.
Interpretation
As with any data exercise, it is important to stress that the statistics demonstrate correlation, not causation.  That is, there is no indication that tweeting will itself reduce fundraising efforts.  However, it does indicate that organizations who actively tweet are also organizations that tend to struggle with fundraising efficiently.  Thus, twitter usage may portend difficulties in fundraising, even if it is not a cause of it.

Friday, June 14, 2013

Cherry Picking Impact Measures

Navigating grocery aisles in search of healthy foods has become an exercise in futility.  This is, in part, due to the proliferation of nutritional information.  Food companies have become quite adept at selective disclosure of nutritional information, from sugary cereals prominently labeled "gluten free" to fried foods boasting of "zero trans fats."  These cherry picked disclosures are meaningless at best and misleading at worst.  As the acceptance of impact measures for nonprofits starts to spread, similar concerns are sure to arise.  Will we see nonprofits collecting a variety of impact measures only to cherry pick those that paint them in a favorable light to donors?  Absent standardized (and audited) measures, donors who care about impact should be aware of the inevitable bias in those impact measures prominently displayed on websites and glossy annual reports.  It may pain some to admit it, but these issues may bring many back to the less thrilling (but also more objective and standardized) measures provided by accounting.

Monday, June 10, 2013

Impact Measures are Not a Panacea

Throughout the nonprofit sector, heavy criticism of accounting measures of efficiency and heavy praise for impact measures are familiar refrains.  Accounting measures are not perfect, but I believe that impact measures are not the panacea many people seem to think they will be.  My arguments are laid out in this piece in the Nonprofit Quarterly.