To examine the effect hunters and jostlers may have on vaccinations, we conducted two pre-registered and monetary incentivized survey experiments — a “main study” and a “follow-up study”.
Data for the main study were collected on the Prolific platform at a time (March 22–24 of 2021) when COVID-19 vaccines were still unavailable to large groups of the adult US population. We recruited a sample of Americans, representative of the population on age, gender and race / ethnicity. The survey experiment was pre-registered in the AEA registry for RCTs as AEARCTR-0007285. Informed consent was obtained from all participants, the experiment was conducted in accordance with relevant guidelines and regulations, and the experimental protocol was approved by the George Mason IRB (# 1724890).
Participants were paid $ 1 for completion, plus any incentives earned as part of the survey. While a total of 1,503 participants answered the survey, we here focus on those (N = 1,117) who were randomized into the following treatments: i) a control treatment (participants received information about the existence of COVID-19 vaccines); ii) a “hunters” treatment (participants were additionally provided brief information that described how vaccine hunters went through great lengths to secure left-over vaccine doses); and iii) a “jostlers” treatment (providing corresponding information about how some privileged people jumped the vaccine line). The remaining 386 of participants of the total sample (N = 1,503) were randomized into a “safe” treatment. The safe treatment emphasized the safety of COVID-19 vaccines and was included to provide a benchmark for the size of any treatment effects in the hunters and the jostlers treatment. The safe treatment did, however, not impact the enthusiasm about the vaccines (most likely because our sample already have very high trust in the safety of the vaccines — more than 85% of participants believed vaccines were safe). Details on the safe treatment, its (null-) results, as well as on our participants’ trust in the vaccines, are provided in the Supplementary Information.
Specifically, participants in the control group were shown information about the existence of COVID-19 vaccines, among other things saying that “[t]he COVID-19 vaccines will decrease your risk of getting COVID-19 and of becoming seriously ill or dying […]. As the COVID-19 vaccines prevent the coronavirus from spreading and replicating, they will also help in preventing additional mutations of the virus ”.
In addition to the information given in the control group, participants in the hunters and jostlers treatments saw information that described the respective phenomenon. We took care to ensure that the information contained language similar to that used in news media reporting, eg, how vaccine supply shortages in the early spring of 2021 fueled the behavior of hunters and jostlers. Participants randomized into the hunters treatment read: “Even though the vaccines have been approved, the supply is still too low to meet the demand. This has led to the global rise of so called ‘vaccine hunters’ […]. The vaccine hunters wait for entire days outside for example grocery store pharmacies in hopes of securing left over vaccine doses (that would otherwise be discarded) at the end of the day. ” Those randomized into the jostlers treatment read: “Even though the vaccines have been approved, the supply is still too low to meet the demand. This has led to a situation where, globally, the wealthy are trying to jump the line to get a COVID-19 vaccine […]. One example of this is the Canadian billionaire Rod Baker who, together with his wife, chartered a private plane and traveled to a remote region in the Yukon to pose as a motel worker in order to feign being eligible for the vaccine. ” Immediately after the treatment information, participants answered a short question about the main message of the paragraph. This was done to identify participants who might not be paying adequate attention, or misunderstanding the text. Such limited attention / misunderstandings were, however, uncommon: only 2.95% of participants provided an incorrect answer and our results are robust to excluding these.
Four outcome measures were assessed immediately after the treatment. First, we asked participants to state their (1) willingness to get vaccinated immediately [VAXTODAY]and (2) in two months [VAX2MONTHS] on a 1–10 scale (from definitely not being willing to definitely being willing). If participants had already received at least their first vaccine dose, these measures assessed their willingness to recommend vaccination to friends and family, using the same scale. Thereafter participants were asked whether they (3) wanted (yes / no) to get a link to general vaccine eligibility and sign-up information (for use to self, or to friends and family) in the post survey confirmation email [VAXINFO].
The last outcome variable, which was only asked of participants who had not yet received at least a first COVID-19 vaccine dose, measured their (4) monetary valuation of a vaccine sign-up service that facilitated access to a COVID-19 vaccine. Specifically, the service provided individualized help with identifying, and signing up for, a COVID-19 vaccine appointment in the participant’s geographical vicinity once the participant became eligible (at the point of data collection, in March 2021, most adults in the US were still not eligible, and many people were eager to get their vaccines as soon as they became eligible). Additional information about the vaccine sign-up service, and how it was made available to participants, is available in the Supplementary Information.
Willingness to pay for the vaccine sign-up service was elicited using a multiple-price-list, MPL28—Participants were presented with a list of eight choice pairs. In each choice pair, participants had to decide between either getting access to the vaccine sign-up service, or instead receiving a monetary prize ($ 2, $ 5, $ 10, $ 25, $ 50, $ 75, $ 100 and $ 200 in the eight choice pairs, respectively ). Participants were informed that 15 survey participants would be randomly selected and that their preferred alternative in a randomly chosen choice pair out of the eight would be implemented (ie they would either receive the monetary prize, or access to the vaccine sign-up service, depending on their chosen alternative in a randomly selected choice pair). Choice pairs were ordered from lowest to highest monetary amount, such that we can use the number of times a participant selected the vaccine sign-up service before switching to the monetary prize as a measure of the willingness to pay for the service [VAXHELP]. The last part of the survey assessed participants’ demographic information.
Of the 15 people randomly selected to receive their preferred alternative in the MPL, 3 participants preferred the vaccine sign-up service in the randomly selected choice pair, and 12 participants were paid a monetary price (which averaged $ 62).
While the sample is, by design, quota representative of the US population on gender, age and race / ethnicity, it is not necessarily representative in other respects. Notably, close to 60 percent (59.4 percent, SE = 1.3) of our sample have completed at least a four year college degree, which is a higher proportion than in the general US population. As education correlates positively with beliefs about vaccines in general being safe1,2,3,4,5,6,7,8,9this might (at least partly) explain the large share of our participants believing that COVID-19 vaccines are safe.
The willingness to get vaccinated is generally high in our sample. In the control treatment, 82.6, SE = 1.9, (84.6, SE = 1.8) percent were more willing than unwilling to receive / recommend the vaccine immediately (in two months). This is at least weakly higher than corresponding shares observed in most studies who estimate willingness to get vaccinated against COVID-19 in the US1,2,3. Of participants in the control treatment, 42 percent (SE = 2.49) stated that they wanted to receive information about eligibility and sign-up for COVID-19 vaccines. The willingness to pay for the vaccine sign-up service is low, however: the average participant in the control group only chooses the vaccine service over the monetary prize in 0.85 (SE = 0.11) of the 8 questions, which indicates a mean WTP of less than $ 2.
We examined the balance of demographic and attitude variables across treatment groups by conducting 36 pairwise t-tests (two-sided) of equality of means. One test was statistically significant (p<0.05): we find that the share of female participants is higher in the hunters than in the control treatment group. While this is not surprising with 36 pairwise tests, we therefore include the variable 'female' as a control variable in our main regression specifications reported in Fig. 1 and Table 1 (excluding this control variable does, however, not impact the results reported, or conclusions drawn).
To better understand the effect on willingness to vaccinate from the hunters and jostlers treatments that we observed in the main study, we next conducted a “follow-up study.” Data collection took place on May 19, 2021, on Prolific, and participants were 800 Americans, distinct from those who responded to the first survey. They were paid $ 1 for completion, plus any incentives earned as part of the study. The experimental survey used in the follow-up study elicited participants’ emotional response to the control and treatment information in the “main study,” as well as their incentivized predictions about the treatment information’s effect on the willingness to get vaccinated. This study was pre-registered in the AEA registry for RCTS as AEARCTR-0007656. Informed consent was obtained from all participants, the experiment was conducted in accordance with relevant guidelines and regulations, and the experimental protocol was approved by the George Mason IRB (# 1756922-1).