Winning in sports is pretty straightforward. Every team knows what to aim for, such as winning the World Series in professional baseball. But what does winning look like in the nonprofit sector? Many would say impact, and we would agree. However, there is not currently a standardized way measure impact - so we cannot use it to truly separate the winners from the losers.
We would argue that winning, or creating impact, can be defined and measured by how successful an organization is at reducing (or averting) suffering in the world. When it comes to measuring impact, the challenge we face is that there is no general consensus for how to place value on our ability to reduce (or avert) suffering, and we are therefore unable to measure it.
It is not that we do not place value judgements on suffering as a society, we do. However, these decisions tend to be made more unilaterally and in relative privacy than collectively and in the open. This is in part due to the moral and/or ethical implications that are associated with these value judgements. You and I might have different opinions about who to help when faced with a choice between two individuals suffering from different diseases. So who is right? I imagine that we will both be able to find data that supports a compelling case for our conclusion. Even if we ultimately agree after having a thoughtful debate, does this make our decision correct given all of the alternatives? Is everyone else likely to agree with our conclusion? Should anyone else be allowed to have input in this decision?
As a society, we tend to only take our strategic planning so far when it comes to positively influencing impact. We typically stop at measuring outcomes and we typically only evaluate alternatives within similar cause areas (cancer v. cancer rather than cancer v. education). We generally do not attempt to compare across cause areas because the outcomes are not easily comparable. Using our sports analogy, this would be like trying to determine a single World Series winner for the entire league by only comparing teams within their respective division. Lord knows we would never reach general consensus on that.
It is not practical to expect society to come to a precise determination of value that we can all agree on; this would be like expecting all baseball fans to cheer for the same team. However, it is reasonable to expect that we can establish a set of rules and scoring systems to help rank teams (nonprofits) according to their impact. This doesn’t mean that second and third place teams are bad or unworthy of support, it just means that they may need to adjust their strategy to be the best next year. The same is true for the non-profit sector.
There are already significant resources being used to measure outcomes. Those nonprofits that engage in these activities can typically report the number of people served or patients treated (equivalent to the number of games won) to help convince donors (fans) that their organization is worth supporting. Measuring these things are both important and necessary first steps in the process of measuring impact, but they only represent part of the answer. If measuring outcomes represent 80%, then measuring impact is the final 20%.
Outputs, outcomes, and impact are common terms when it comes to measuring success in the social sector. Measuring outputs are like measuring a player’s stats, they help identify who is playing their position well but they don’t necessarily tell us who won the game. Outcomes can be useful in telling us who won the game, but they won't tell us who the best team in the league is. (In the social sector, trying to compare outcomes from two separate organizations can sometimes feel like trying to compare teams playing two different sports.) Quantify impact is similar to having a World Series winner in baseball in the sense that we are comparing all teams on a relative basis to determine an ultimate winner, and we are ranking the rest of the teams according to their merit.
As we mentioned already, “winning” in the social sector can be determined by quantifying the value that comes from reducing or averting the burdens that cause suffering in the world. By assigning value to these burdens we can create a scoreboard that helps us evaluate the impact of outcomes on a level playing field. Because each burden is assigned a value we would know how much credit nonprofits should receive each time their program successfully resolves or averts the burden, giving us the ability to compare that organization to another even if the burdens are completely different.
We have thought a lot about how to create a scoring system for the social sector given the moral and ethical implications mentioned earlier. Our current approach is not perfect, but we believe it provides a strong foundation to build on, read our previous blog post for a more detailed overview. Generally speaking, we believe that the process of assigning value to burdens should be democratized to reflect society's collective perspective, but a case could be made for mining the data from smaller groups such as experts or stakeholders.
The point is, we believe that we owe it to those we seek to help to optimize impact within the social sector. We believe that one of the best ways to do this is by using better data to keep score and learn from the winners.
What does any of this have to do with baseball?
If you have ever read the book, or seen the movie Moneyball you likely already have a general understanding of how data can be used to win even when the cards seem to be stacked against you. For those unfamiliar Moneyball - it is a story of how the Oakland Athletics used data to make to, then win, the 2002 World Series with a team budget that was among the lowest in the MLB (just one-third of the New York Yankees that same year). This success did not happen by chance, the team was able to capture four American League West titles and make five playoff appearances from 2000-2006.
Data provided what the best coaches and consultants could not - a clear roadmap with tight feedback loops to optimize their strategy quickly and with greater levels of confidence. In the years since, the ‘Moneyball’ approach has been adopted by most winning teams in the MLB. We believe that a similar success story will be written about the social sector once we have, and use, the right data.
A Standard Scoring System for Impact in the Social Sector