Information Delivered by a Chatbot Has a Positive Impact on COVID-19 Vaccines Attitudes and Intentions

"Chatbots could be a powerful resource to fight COVID-19 vaccines hesitancy."

Most countries face the issue of vaccine hesitancy, with sizeable fractions of the public opposing some vaccines. The problem is particularly acute in the case of COVID-19 vaccination. This study introduces and tests a messaging strategy for reassuring vaccine-hesitant individuals: a chatbot that answers questions about COVID-19 vaccines.

Results from previous studies reviewed here show that changing people's minds at scale is a difficult endeavour. A major obstacle for communication campaigns is that, when people encounter a message that aims at changing their minds, they typically generate counter-arguments. If they do not have an interlocutor who can address these counter-arguments (e.g., if they read a leaflet), they are less likely to change their minds. This likely explains why small-group discussion, in which counter-arguments can be addressed in back-and-forth discussion, is more effective at changing people's minds than the simple presentation of arguments.

The interactivity that small-group discussion provides is, however, difficult to scale up. A potential solution is to gather the most common counter-arguments and to offer rebuttals to each of them. Since not every rebuttal is relevant to everyone, chatbots can work as an alternative to long texts presenting every possible argument. When interacting with a chatbot, people select the questions (or counter-arguments) that are most relevant to them and read the corresponding answers, which can then raise further questions and answers.

To test a chatbot on COVID-19 vaccination hesitancy, the researchers identified the most common questions about COVID-19 vaccines by conducting a survey on a representative sample of the French population documenting the reasons why people were willing, or not, to take a COVID-19 vaccine. They also consulted press articles refuting common myths about the COVID-19 vaccines and resources from health institutions. Answers to these common questions were drafted based on a wide variety of publicly available information and checked by several experts on vaccination.

In late December 2020, 701 French participants were randomly assigned to a chatbot condition, in which they had the opportunity to interact with the chatbot for as long as they wanted, or to a control condition, in which they read a brief text (93 words) describing the way vaccines work (Wave 1). (The design is not meant to compare the efficacy of an interactive chatbot compared to a noninteractive chatbot or a long text; rather, it is primarily meant to test the efficacy of a chatbot to inform people about COVID-19 vaccines. Thus, the control condition was meant to control for potential demand biases.) Between 1 and 2 weeks after the experiment (in early January 2021), the researchers surveyed the participants again to measure whether the effect of the chatbot would last over time (Wave 2).

The chatbot was organised as follows. Participants were first asked whether they had any questions about the COVID-19 vaccines, and they were given a choice of six questions to select from (e.g., "Are COVID-19 vaccines safe?") Participants were able to select, at any stage, an option "Why should I trust you?", which informed them of who the researchers are, who funded the research, and what its goals are. Every time participants selected a question, the chatbot offered an answer. Participants could choose between several subquestions that the initial answer might not have addressed. Responses contained hyperlinks to scientific articles, reports from scientific agencies, media articles, and Wikipedia. In total, the chatbot offered 51 questions and answers about the COVID-19 vaccines.

The researchers found that interacting with the chatbot for a few minutes significantly increases people's intentions to get vaccinated and has a positive impact on their attitudes (on all 5 dimensions evaluated) toward COVID-19 vaccination. Namely, before interacting with the chatbot, 145 out of 338 participants had positive attitudes toward the COVID-19 vaccine; after interacting with the chatbot they numbered 199, which corresponds to a 37% increase. Before interacting with the chatbot, 123 out of 338 participants said they did not want to take the COVID-19 vaccine; after interacting with the chatbot they numbered 99, which corresponds to a 20% decrease. This relation held among the third of the participants initially holding the most negative attitudes toward the COVID-19 vaccines. (This finding means that there was no "backfire effect".)

In the chatbot condition, 45% of participants reported having tried to convince other people (typically, between 2 and 5) of their position on the COVID-19 vaccines, with 72% of these participants reported having used information from the chatbot in their attempts to convince others. 38% of the participants reported being willing to share the chatbot in at least one way (social networks, 11%; entourage, 37%; other means, 9%). These results suggest that the chatbot could play a role beyond providing information to those directly exposed to it, as people use and share the chatbot with others.

Participants spent a significant amount of time interacting with the chatbot (between 5 and 12 min for half of the participants), and the more time they spent, the more they changed their minds.

Participants were more likely to report being willing to take the COVID-19 vaccines after the experimental task in the chatbot condition than in the control condition. This relation held among the third of the participants initially least willing to take the COVID-19 vaccines.

However, in Wave 2, there was no significant difference between participants' attitudes in the chatbot and in the control conditions, a pattern that is similar for vaccination intentions. The researchers suggest two interpretations. The first is that the gains in attitudes and intentions after interacting with the chatbot persisted, but that participants in the control condition were also exposed to provaccination information because of an intense media coverage of the COVID-19 vaccination campaign in France. The second interpretation is that participants in the chatbot condition quickly reverted to their original attitudes and intentions, and that those were then buoyed by the media coverage, along with those of the participants in the control condition. The researchers lean toward the first explanation, but the evidence remains unconclusive.

On average, participants deemed the chatbot to be very intuitive, their interaction with the chatbot to be quite pleasant and not very frustrating, and the information presented in the chatbot to be neither too complex nor too simple.

In conclusion, the results "suggest that a properly scripted and regularly updated chatbot could offer a powerful resource to help fight hesitancy toward COVID-19 vaccines. Besides its direct effect on vaccine hesitant individuals, the chatbot could prove invaluable to provaccination individuals, including professionals looking for information to use in interpersonal communication with vaccine hesitant individuals."