Thursday, June 27, 2013

Remember that Evidence that Better Charities Have Higher Administrative Expenses? Turns Out the Reverse Holds in the US

In May, the group Giving Evidence released a study showing that high performing charities actually have higher administrative expenses.  This news was welcomed with fanfare by many in the nonprofit sector who believe there is too much pressure to reduce administrative costs.  Since then, it has been cited heavily and even featured as part of the Overhead Myth campaign launched by BBB Wise Giving Alliance, Charity Navigator, and Guidestar.  While I am willing to believe that lower administrative costs are not always better, and the Overhead Myth campaign has some merit, I also worry that the evidence to support the claims is rather weak.  To their credit, the folks at Giving Evidence have made the data publicly available so that these questions can be addressed.

Take first the larger of the two data samples, the Givewell ratings from 2008-2009.  This data reveals that the administrative costs of the well-ranked charities is higher, as reported by Giving Evidence:

However, if one restricts attention to charities that focus on the United States (as classified by the Giving Evidence spreadsheet), the reverse conclusion obtains:

Now consider the smaller data sample that has been quoted more extensively, the Givewell ratings from 2010-2011.  Again, the data reported by Giving Evidence shows that the better charities having higher administrative costs:

In this data set, virtually all of the charities reviewed focused on international issues.  However, if one restricts attention to those charities domiciled primarily in the United States, another reversal occurs:

Is my point that the data indicates that lower administrative costs are better?  No, not at all.  Instead, the point is that the data provides little evidence of anything.  (The reaction to it, however, may be good evidence of confirmation bias on the part of nonprofit professionals fed up by pressure to cut administrative costs.)  The comments on this study are quite thoughtful in demonstrating that the data gets nowhere near providing statistically significant evidence.  This lack of significance becomes quite apparent when one considers subsets of the data; as evidenced here, results can be reversed quite easily.  What I suggest one takes from this exercise is only that more data and more rigorous analysis is needed before one can make any conclusion about how administrative costs and nonprofit effectiveness are related.

No comments:

Post a Comment