Continuation of Buying Vegetarians: Handling Complications
Recently, I’ve been trying to figure out how much it costs to save animals from factory farming. The idea is to get people to become vegetarian by funding advertisements to influence them. Previously I did a simple calculation and came up with two cents to avert a year of suffering in a factory farm. Then I did a more complex calculation and came up with a range between two cents to $65.92 to avert a year of suffering.
Many people are skeptical of these figures. Perhaps surprisingly, so am I. I’m trying to strike a balance between being an advocate of vegan outreach as a very promising path for making the world a better place, while not losing sight of the methodological hurdles that have not yet been met, and open to the possibility that I’m wrong about this.
In this essay, I’m going to discuss those hurdles.
The Quality of the Surveys
The big methodological elephant in the room is that my entire cost estimate depends on having a plausible guess for how likely someone is to change their behavior based on seeing an advertisement.
I feel slightly reassured because:
There are two surveys for two different media, and they both provide estimates of impact that agree with each other.
These estimates also match anecdotes from leafleters about approximately how many people come back and say they went vegetarian because of a pamphlet.
Even if we were to take the simple calculator and drop the “2% chance of getting four years of vegetarianism” assumption down to, say, a pessimistic “0.1% chance of getting one year” conversion rate, the estimate is still not too bad – $0.91 to avert a year of suffering.
More studies are on the way. Nick Cooney is going to do a bunch more to study leaflets, and Xio Kikauka and Joey Savoie have publicly published some survey methodology [Google Docs].
That said, the possibility for desirability bias in the survey is a large concern as long as the surveys continue to be from overt animal welfare groups and continue to clearly state that they’re looking for reductions in meat consumption.
Also, so long as surveys are only given to people that remember the leaflet or advertisement, there will be a strong possibility of response bias, as those who remember the ad are more likely to be the ones who changed their behavior.
Furthermore, there’s a concern that the surveys are just measuring normal drift in vegetarianism, without any changes being attributable to the ads themselves. For example, imagine that every year, 2% of people become vegetarians and 2% quit. Surveying these people at random and not capturing those who quit will end up finding a 2% conversion rate.
How can we address these? I think all three problems can be solved with a decent control group, whether it be a group of people that receive a leaflet not about vegetarianism, or no leaflet at all. Luckily, Kikauka and Savoie’s survey intend to do just that.
Jeff Kaufman has a good proposal for a survey design I’d like to see implemented in this area.
Market Saturation and Diminishing Marginal Returns
Another concern is that there are diminishing marginal returns to these ads. As the critique goes, there are only so many people that will be easily swayed by the advertisement, and once all of them are quickly reached by Facebook ads and pamphlets, things will dry up.
Unlike the others, I don’t think this criticism works well. After all, even if it were true, it still would be worthwhile to take the market as far as it will go, and we can keep monitoring for saturation and find the point where it’s no longer cost-effective.
However, I don’t think the market has been tapped up yet at all. According to Nick Cooney [PDF], there are still many opportunities in foreign markets and outside the young, college kid demographic.
The Conjunction Fallacy
The conjunction fallacy is a classic fallacy that reminds us that no matter what, the chance of event A happening can never be smaller than the chance of event A happening, followed by event B. For example, the probability that Linda is a bank teller will always be larger than (or equal to) the probability that Linda is a bank teller and a feminist.
What does this mean for vegetarian outreach? Well, for the simple calculator, we’re estimating five factors. In the complex calculator, we’re estimating 90 factors. Even if each factor is 99% likely to be correct, the chance that all five are right is 95%, and the chance that all 50 are right is only 60%. If each factor is only 90% likely to be correct, the complex calculator will be right with a probability of 0.5%!
This is a cause for concern, but I don’t think there’s any way around this. It’s just an inherent problem with estimation. Hopefully we’ll be balanced by (1) using the different bounds and (2) hoping underestimates and overestimates will cancel each other out.
Conversion and The 100 Yard Line
Something we should take into account that helps the case for this outreach rather than hurts it is the idea that conversions aren’t binary – someone can be pushed by the ad to be more likely to reduce their meat intake as opposed to fully converted. As Brian Tomasik puts it:
Yes, some of the people we convince were already on the border, but there might be lots of other people who get pushed further along and don’t get all the way to vegism by our influence. If we picture the path to vegism as a 100-yard line, then maybe we push everyone along by 20 yards. 1/5 of people cross the line, and this is what we see, but the other 4/5 get pushed closer too. (Obviously an overly simplistic model, but it illustrates the idea.)
This would be either very difficult or outright impossible to capture in a survey, but is something to take into account.
Three Places I Might Donate Before Donating to Vegan Outreach
When all is said and done, I like the case for funding this outreach. However, I think there are three other possibilities along these lines that I find more promising:
Funding the research of vegan outreach: There needs to be more and higher-quality studies of this before one can feel confident enough in the cost-effectiveness of this outreach. However, initial results are very promising, and the value of information of more studies is therefore very high. Studies can also find ways to advertise more effectively, increasing the impact of each dollar spent. Right now, however, it looks like all ongoing studies are fully funded, but if there were opportunities to fund more, I would jump on it.
Funding Effective Animal Activism: EAA is pushing for more cost-effectiveness in the domain of nonhuman animal welfare and is working to further evaluate what opportunities are the best, Givewell-style. Giving them more money can potentially attract a lot more attention to this outreach, and get it more scrutiny, research, and money down the line.
Funding Centre for Effective Altruism: Overall, it might just be better to get more people involved in the idea of giving effectively, and then getting them interested in vegan outreach, among other things. CEA could do just that.
Vegan outreach is a promising, though not fully studied, method of outreach that deserves both excitement and skepticism. Should one put money into it? Overall, I’d take a guarded approach of putting in just enough money to help the organizations learn, develop better cost-effective measurements and transparency, and become more effective. It shouldn’t be too long before this area will become studied well enough to have good confidence in how things are doing.
More studies should be developed that explore advertising vegetarianism in a wide variety of media in a wide variety of ways, with decent control groups.
I look forward to seeing how this develops. Don’t forget to play around with my calculator.
Edited 18 June to fix two typos.