Why Does Corrective Information Have a Muted Effect on Immigration Attitudes?

Introduction

The media portrays immigrants as criminals (Farris and Silber Mohamed 2018), welfare abusers (Brown 2016; Yoo 2008), and a general threat to the well-being of the United States (L. R. Chavez 2008; M. Chavez, Whiteford, and Hoewe 2010). These portrayals are common despite research showing that immigrants are less likely to engage in criminal behavior than natives (Butcher and Piehl 2007; Forrester et al. 2019; Orrenius and Zavodny 2019) and are net tax contributors (Blau and Hunt 2019; Clemens 2022; Nowrasteh 2015).

One might reason that if negative immigration attitudes form due to media misinformation, then corrective information should improve immigration attitudes. Curiously, the extant literature finds, with a notable exception (Haaland and Roth 2020), a muted effect of corrective information on immigration attitudes (Jørgensen and Osmundsen 2020; Hopkins, Sides, and Citrin 2019). In general, immigration attitudes appear stable within individuals (Kustov, Laaker, and Reller 2021). While work in this area provides insights as to whether corrective information changes attitudes, it has yet to explain why corrective information has a muted effect on attitudes. Utilizing a preregistered survey experiment, we show that research subjects respond to corrective information about immigrant behavior by accepting corrective information about immigrants, but instead of updating their attitudes, they shift the goalpost of acceptable immigrant behavior to justify their preexisting attitudes. This study contributes to the literature by helping us understand why and how immigration attitudes remain stable.

Our results suggest that although the media may play a role in the initial formation of immigration attitudes, individuals engage in motivated reasoning to conform new information to preexisting attitudes and therefore do not change their long-held beliefs. Motivated reasoning is the process in which “motivation affects reasoning through reliance on a biased set of cognitive processes: strategies for accessing, constructing, and evaluating beliefs” (Kunda 1990). According to the motivated-reasoning literature, subjects arrive at the conclusions that they wish to arrive at but are constrained by whether they can construct reasonable justifications for their conclusions (Harmon, Fisman, and Kamenica 2019). Prior political-behavior research has established a link between cognitive dissonance and political behavior broadly (Mullainathan and Washington 2009).

Information-Provision Experiments

With one exception (Haaland and Roth 2020), most information-provision studies have found a muted effect on immigration attitudes (Hopkins, Sides, and Citrin 2019; Lergetporer, Piopiunik, and Simon 2018; Jørgensen and Osmundsen 2020). Our study differs from prior information-correction research in two important ways. First, we do not require a high level of political numeracy from our respondents. It is well established that Americans have low levels of political numeracy (Carpini and Keeter 1996). Instead, we provide respondents with a baseline statistic and ask them to compare immigrant welfare usage relative to white Americans. This lowers the amount of cognitive load required from respondents. Second, our research seeks to address generalizability issues by focusing on Mexican immigrants—the United States’ largest immigrant group. In this paper, we propose that the key to understanding the muted effect in this literature lies in understanding how corrective information is processed.

Research Design

For our study, we recruited a total of 1,019 research subjects from the United States on Prolific’s online marketplace in January 2022. We subset our analyses to the 962 research subjects who passed our attention checks. Data exclusion of inattentive subjects was preregistered. Research subjects were balanced on partisanship (n = 477 Democrats, n = 446 Republicans) to allow us to explore potential moderation by party identification.

Our study received approval from our institutional review board. Research subjects affirmatively consented to participate in our study. No deception or otherwise problematic research-design elements were present in our study (Desposato 2015). Prolific requires that research subjects receive at least the federal minimum wage for their participation.

Demographic descriptive statistics are included in table 1. The number of observations slightly differs across demographic survey items because respondents skipped some demographic questions. This most notably impacted self-reported party identification. Nonetheless, we see a near-equal split along identification with major parties with 48 percent of the sample identifying as Republican. Measured on ideology (five-point scale), the mean respondent leans slightly center-left. The sample overwhelmingly self-identified as white (77 percent) and college educated (60 percent).

After providing affirmative consent to participate, research subjects self-reported their demographics and news-media consumption. We then elicited research subjects’ prior beliefs about Mexican-immigrant and Mexican American welfare use. Subjects were asked what percentage of these groups they thought were on Supplemental Nutrition Assistance Program (SNAP), more commonly referred to as food stamps, and what percentage they thought should be allowed to be in said welfare program. We operationalized welfare use by asking about SNAP benefits because it is one of the better known federal welfare programs.

Importantly, in contrast to similar studies (Jørgensen and Osmundsen 2020), we provided all research subjects with reference information about how many white Americans (8.60 percent) were on SNAP. This reference information is vital in our experimental design because we are interested in those individuals who overestimate Mexican-immigrant welfare use relative to white-American welfare use. Our experimental design does not presume nor require a high degree of numeracy on the part of Americans. We are not interested in testing whether research subjects have encyclopedic knowledge about welfare usage. What we wish to know is whether subjects believe Mexican immigrants use less or more welfare relative to the dominant group, white Americans.

Eliciting prior beliefs, as opposed to only eliciting beliefs after the experimental condition, provides us with a baseline to detect within-subject change, classify subjects as over- or under estimators, and allow us to conduct moderation analyses by prior beliefs. After eliciting prior beliefs, research subjects were randomized into either the control or treatment group.

In the control groups, respondents were warned that they were about to be asked about Mexican immigrants again but were not otherwise provided corrective information about welfare use.¹Survey item: “You are about to be asked about Mexican immigrants again. Please read everything carefully.” In the experimental condition, respondents were given the same warning that the control group received, in addition to corrective information about welfare use among Mexican immigrants.²Survey item: “According to the United States American Community Survey, a government survey, 15.15 percent of Mexican Immigrants (1st generation) received food stamps/SNAP in the United States in 2019.” The survey was rounded out by asking subjects about their posterior beliefs and immigration attitudes.

The general research design borrows heavily from Jørgensen and Osmundsen (2020) and other similar informational-provision experiments. In addition to utilizing a different research-subject pool (we use US subjects), we differ from earlier studies in two important design elements. First, we ask respondents about both Mexican immigrants and Mexican Americans. This allows us to disentangle whether any observed effects are driven by the intersection between nativity and ethnic out-group status (see online appendix C). Second, we provide respondents with reference information by telling them how much welfare white Americans use. This provides us several benefits, chief among them that less numeracy is required of our research subjects. While Americans can engage in advanced political calculus when properly motivated (Prior and Lupia 2008), their mental capacity is bounded (Jones 1999). For our experimental design to function, we need to know whether subjects believe immigrant welfare use is lower or higher than the dominant group, but we do not require a high degree of political numeracy. The preregistered inference criterion for statistical analyses is p-value < 0.05. Unless otherwise stated, we rely on linear probability models as our primary method of estimation (Angrist and Pischke 2008; Gomila 2020).

Results

First off, did the corrective-information treatment succeed? Did research subjects who were placed in the informational treatment group update their prior beliefs about Mexican-immigrant welfare use when provided corrective information? As shown in figure 1, the answer is yes. Treated research subjects converged toward the correct answer, while respondents in the control group largely remained unchanged. In the control group, the correlation coefficient between prior and posterior beliefs is 0.9364.

Figure 1. Effect of Information Treatment on Posterior Beliefs, Scatterplot

We hypothesized not only that the treatment would change beliefs about immigrant welfare use  but that the effect of the informational treatment would differ by whether individuals under- or overestimate Mexican-immigrant welfare use relative to whites’ welfare use.³Online appendix E discusses the demographic differences between under- and overestimators. It is worth noting that while a larger share of overestimators are Republicans, Democrats (approximately 47 percent) still make up a significant portion. Specifically, we hypothesized:

Underestimators will increase the percentage of Mexican immigrants they believe are on welfare (positive-directional hypothesis).

And:

Overestimators will decrease the percentage of Mexican immigrants they believe are on welfare (negative-directional hypothesis).

To test these hypotheses, we estimate the following linear probability model (Angrist and Pischke 2008; Gomila 2020), subsetted by whether someone is an under- or overestimator:

Posterior Belief Informational Treatment

Results are graphically presented in figure 2. In support of our hypotheses, we find that under- (over)estimators increased (decreased) their belief about the percentage of Mexican immigrants they believed to be utilizing SNAP. The result is statistically significant for both underestimators (p-value < 0.001) and overestimators (p-value < 0.001).

Figure 2. Effect of Information Treatment on Posterior Beliefs by Under- or Overstimation

Note: Intercept suppressed for brevity. Ninety-five percent confidence intervals used.

For the remainder of the analyses, we focus only on overestimators, as we are largely interested in whether corrective information alters their immigration attitudes. In addition to providing results about all overestimators, results are further subsetted by small and big overestimators. This allows us to detect potential heterogeneity by intensity of belief. It is plausible that small overestimators’ immigration attitudes are more malleable than big overestimators’ attitudes because of differences in magnitude. Small (00.01–11.40 percentage point overestimate) and big (11.41+ percentage point overestimate) overestimators are divided at the 50th percentile. This breakdown by overestimation magnitude was not preregistered and should therefore be interpreted as exploratory analysis. Furthermore, given our sample size, we caution against drawing strong conclusions.

Some may be concerned that this breakdown between degrees of overestimation may be driven by partisan differences in overestimation. As discussed in online appendix E, Republicans make up a majority of big overestimators, but Democrats still make up approximately 40 percent of the grouping. Additionally, as noted in the appendixes, results are robust to the inclusion of control variables, including partisan identity and ideology.

We hypothesized that “overestimators provided with corrective information will improve their immigration attitudes.”

We primarily measure immigration attitudes using a survey item asking about respondents’ support for increasing immigration: “Should immigration be kept at its present level, increased, or decreased? [Increase a lot / Increase somewhat / Keep at present level / Decrease somewhat / Decrease a lot / Don’t know/refused.]”

As a secondary measure, we asked respondents about their support for increased government funding for SNAP: “Should government funding for food stamps/SNAP be increased or decreased? [Increase a lot / Increase somewhat / Keep at present level / Decrease somewhat / Decrease a lot / Don’t know/refused.]”

Both outcomes were turned into binary outcomes. Answers are coded as 1 if respondents selected either “Increase a lot” or “Increase somewhat” and 0 otherwise. Using a linear probability model, we estimate the following:

Immigration Attitudes Informational Treatment

Results are presented in figures 3 and 4.

Figure 3. Effect of Information Treatment on Support for Increasing Immigration

Note: Ninety-five percent confidence intervals used.

Figure 4. Effect of Information Treatment on Support for Increasing SNAP Funding

Note: Ninety-five percent confidence intervals used.

In figure 3, among all overestimators, we fail to find evidence that the informational treatment changes support for increasing immigration (p-value = 0.291). We find similar null results for both small (p-value = 0.508) and big overestimators (p-value = 0.450). Corrective information does not appear to substantially improve attitudes toward increasing immigration. Likewise, in figure 4, we find null results when looking at subjects’ support for increasing government funding of SNAP (p-value = 0.741). The effect is muted whether we look at small (p-value = 0.715) or big (p-value = 0.989) overestimators. Again, we fail to find evidence that corrective information changes immigration attitudes. Our results are consistent with prior studies finding a muted effect of corrective information on immigration attitudes.

These muted effects were not unexpected given the existing literature. As mentioned, there is a growing consensus that within-subject immigration attitudes are relatively stable, and our results support this work. In anticipation of the above null results, we hypothesized two non-mutually exclusive explanations for why individuals’ immigration attitudes remain stable despite being provided corrective information.

One possible explanation for our null results is that we provided corrective information primarily about the economic impact of immigration. Immigration attitudes are, however, driven by both economic and cultural concerns (Abramyan and Alexander 2019). It is possible that a substitution effect occurs—that is, after receiving information that addressed their alleged economic concerns, respondents increased their concern about the cultural impact of immigrants. Lergetporer, Piopiunik, and Simon (2018) find some support for a substitution effect. In their study, corrective information about the economic impact of immigrants reduced concerns about the economic threat posed by migrants but did not change overall immigration attitudes. Formally, we state the substitution hypothesis as follows:

Overestimators provided with corrective information will increase their expressed concern about the cultural impact of immigration (positive-directional hypothesis).

A related hypothesis is that respondents will retroactively shift the goalpost of acceptable immigrant behavior to justify their immigration attitudes. In other words, upon learning that only X percent of immigrants use welfare, subjects will retroactively claim that only X−N percent of immigrants should use welfare. Formally we state the shifting-goalpost hypothesis as follows:

Overestimators provided with corrective information will decrease the percentage of migrants they believe should be allowed to receive welfare (negative-directional hypothesis).

Both hypotheses, the substitution effect and shifting goalposts, are instances of motivated reasoning caused by cognitive dissonance. Subjects may have already formed their attitudes toward immigrants and alter the presentation of new information to conform to their preexisting attitudes instead of changing their attitudes in response to updated information. By either changing their cause for concern from economic to cultural reasons (the substitution hypothesis) or shifting the goalpost of acceptable welfare-use behavior (shifting-goalpost hypothesis), subjects are engaging in motivated reasoning to resolve the cognitive dissonance caused by the corrective information. As shown below, we provide suggestive evidence that research subjects are engaging in motivated reasoning.

We provide results for the substitution hypothesis in figure 5. The results were estimated using a linear probability model:

Cultural Concern Informational Treatment

The dependent variable is a binary variable equal to 1 if research subjects indicated being “a lot” or “somewhat” concerned about the cultural impact of immigration. If the substitution hypothesis is correct, we expect that the informational treatment had a positive effect on self-reported concern about the cultural impact of immigrants in figure 5. We fail to find evidence that the informational treatment changed concerns about the cultural impact of immigrants among all overestimators (p-value = 0.830) or when we subset analyses to small (p-value = 1.000) or big overestimators (p-value = 0.862).

Figure 5. Effect of Information Treatment on Cultural Concern about Immigration

Note: Ninety-five percent confidence intervals used.

We present results for the shifting-goalpost hypothesis in figure 6. The results were estimated using an ordinary least squares model:

Acceptable Percent on Welfare Informational Treatment

The dependent variable is continuous from 0 to 100 and indicates what percentage of Mexican immigrants the research subject believes should be allowed on SNAP.⁴Survey item: “8.60 percent of white Americans received food stamps/SNAP in 2019. What percent of Mexican Immigrants (1st generation) and Mexican Americans (2nd+ generation) do you think received food stamps/SNAP in 2019?” If the shifting-goalpost hypothesis is correct, we should find a negative and statistically significant effect for the informational treatment on the perceived percentage of Mexican immigrants who should be allowed on SNAP. That is to say that, among overestimators, subjects who received the informational treatment should reduce the percentage of Mexican immigrants they think should be allowed to access welfare. By shifting the goalpost of acceptable welfare-use behavior, subjects can justify their extant immigration attitudes despite the new information provided to them.

Among all overestimators, we find suggestive evidence, albeit not statistically significant at our preregistered criterion level, that the informational treatment reduces the stated percentage of Mexican immigrants who subjects believe should be on welfare by 2.791 percentage points (p-value = 0.232). This result is driven largely by big overestimators (coefficient = −9.818; p-value = 0.004). As shown in online appendix E, the result is not driven by ideological differences between small and big overestimators. Our results are consistent with Jørgensen and Osmundsen’s (2020) results: research subjects shifted the goalpost to justify their preexisting beliefs.

Figure 6. Effect of Information Treatment on Perceived Percentage of Mexican Immigrants Who Should Be Allowed on SNAP

Discussion

The media regularly portrays immigrant populations in a negative fashion. Migrants are negatively portrayed by the media as criminals, welfare abusers, and general threats. Given the prevalence of misinformation, can corrective information improve the public’s attitudes toward immigrants? The idea is alluring, and several studies have tested the possibility by providing research subjects with a range of corrective information about immigrants. With a notable exception, however, most of these studies, including our own, have found a muted effect.

Why does corrective information have a muted effect? We hypothesized that individuals are engaging in motivated reasoning to address the cognitive dissonance caused by corrective information. The media can play a role in the formation of immigration attitudes, but, after immigration attitudes are formed, any additional information is interpreted to conform with preexisting beliefs. Our study finds suggestive evidence that this is what is occurring. Among our research subjects, especially among those subjects who initially overestimated Mexican-immigrant welfare use by a large amount, we find evidence in support of our shifting-goalpost hypothesis. Subjects exposed to the corrective-information treatment believe that a smaller share of immigrants should be allowed to access welfare services compared to the control group. The big-overestimate control group believes that approximately 43 percent of Mexican immigrants should be allowed to receive SNAP benefits, compared to treated big overestimators, who believe it should be closer to 34 percent.

This finding is important because it means that immigrant attitudes will not change even in the presence of model migrants. Even if immigrants used even less welfare in the future, the relevant public would simply shift the goalpost of acceptable welfare use to argue that migrants are nonetheless welfare abusers. Although our experiment focused on welfare use, it is likely that goalpost shifting would occur in other domains as well, such as immigrant criminal behavior or immigrants’ economic impact. This finding should give pause to campaigns that focus primarily on providing corrective technical information in the hopes of improving attitudes toward immigrants and other marginalized populations. Although corrective information alone appears to have a muted effect on immigration attitudes, there is some hope that corrective information as part of a larger positive messaging campaign can have an effect. Most recently, Bearce and Stallman (2022) found a positive effect of positive messaging emphasizing the economic, cultural, and national security benefits provided by immigrants. It may also be possible that information-provision experiments that focus on novel information, information not regularly discussed by the media, have an effect (Coppock 2023).

Appendix A. Effect of News Consumption on Migrant Perceptions

While the promulgation of misinformation about migrants is an issue across the broader mediascape, conservative media outlets are disproportionately responsible for portraying immigrants negatively. These negative portrayals may lead the public to misperceive the impact immigrants have on society and subsequently form negative attitudes toward immigrants (Fitzgerald, Curtis, and Corliss 2012). Thus, we test whether there is a strong correlation between conservative-media consumption and holding negative immigration attitudes.

Media Use and Immigration Attitudes

Is conservative-media consumption, operationalized as consuming Fox News, associated with the belief that a larger share of Mexican immigrants consume welfare? Our hypothesis is:

Respondents who consume more conservative media believe more Mexican immigrants are on welfare (positive-directional hypothesis).

To test our hypothesis, we regressed the perceived percentage of Mexican immigrants on SNAP using an ordinary least squares model:

Perceived Percent on Welfare Fox News Consumption

The dependent variable is continuous from 0 to 100. The independent variable of interest is Fox News consumption, broken up as a four-point categorical variable: Almost never, Rarely, Occasionally, and Very often. Standard demographic controls are included given the non-experimental nature of this test. The result is presented graphically in appendix A, figure 1.

Figure 1. Perceived Percentage of Mexican Immigrants on SNAP by Fox News Consumption

Note: Ninety-five percent confidence intervals used.

We find that individuals who occasionally (coefficient = 4.331; p-value = 0.016) and very often (coefficient = 5.185; p-value = 0.010) watch Fox News, as opposed to individuals who almost never watch Fox News, believe that a larger share of Mexican immigrants consume welfare. These results are consistent with hypothesis 1: conservative-news-media consumption is positively associated with the perception that a larger share of Mexican immigrants are utilizing welfare programs like SNAP. These results are not from an experiment, so we hesitate to make causal claims based on our results alone, but they are consistent with findings from experimental (Broockman and Kalla n.d.; Kim 2022) and quasi-experimental (Martin and Yurukoglu 2017) studies.

Appendix B. Media-Use Consumption, Collaborative Multiracial Post-Election Survey 2020

As noted in the main text, our primary data are a convenience sample, and we do not claim to make population estimates. Nonetheless, our sample mirrors the general trends in media use as shown in appendix B, figures 1–2. Herein we report on media consumption using a nationally representative survey, the Collaborative Multiracial Post-Election Survey (Barreto et al. 2018). We subset our data to the survey’s white subsample (n = 3,200) and use the survey’s default weights based on the 2019 American Community Survey.

Figure 1. News-Media Consumption, All

Figure 2. News-Media Consumption by Party Identification

Appendix C. Nativity vs. Ethnic Out-group

One of the difficulties in interpreting our results is that Mexican immigrants are out-group members on two key dimensions—nativity and ethnicity. They are respectively non-natives and nonwhites. It is therefore difficult to interpret whether results are driven by either out-group status or the intersection of the two.

It is possible to partly disentangle the two identities by looking at how Mexican immigrants (first generation) are viewed compared to Mexican Americans (second generation or greater). If our results are only being driven by ethnicity, the perception of Mexican immigrants and Mexican Americans should be the same. If, however, any differential exists, that should be suggestive of an independent or intersectional effect of nativity as well since Mexican Americans are ethnic out-group members but are natives. As shown in appendix C, figure 1, a higher percentage of Mexican immigrants are perceived to be on SNAP benefits than Mexican Americans. This difference is statistically different (p-value < 0.01) using a two-sided t-test. This suggests that our results are being driven by both nativity and ethnicity.

Figure 1. Perceived Food-Stamp Use by Nativity Status

Appendix D. Deviations from Preregistered Report

Replication and Transparency Statement: The analyses in this manuscript were preregistered on Open Science Framework. Any deviations in the protocol are listed here. At the time of publication, all data and script files necessary for replication of this study will be made available. In this appendix we report any deviations that occurred between the preregistered analyses and the reported analyses.

For brevity H1 was moved to online appendix A, but, unless otherwise noted, we report the preregistered analyses herein. Fox News consumption in preregistered H1 was meant to be a binary variable equal to 1 if respondents watched Fox News “occasionally” or “very often” and 0 if “rarely” or “almost never.” In the paper, results are presented as “almost never,” “rarely,” “occasionally,” and “very often.” This was done to show how even marginal variation in Fox News consumption had a noticeable relationship with the outcome variable. Substantive results are unchanged if the original binary form of Fox News consumption is used; see appendix D, figure 1 below. Individuals who frequently (“occasionally” or “very often”) consume Fox News believe a larger share of Mexican immigrants are receiving SNAP benefits (p-value = 0.016).

Figure 1. Perceived Percentage of Mexican Immigrants on Food Stamps/SNAP by Fox News Consumption (Binary Consumption Variable)

Note: Ninety-five percent confidence intervals used.

Starting with H3 onward, the paper presents select results for all overestimators and for those who are big or small overestimators. Small (00.01–11.40 percentage point overestimate) and big (11.41+ percentage point overestimate) overestimators are divided by the 50th percentile. As noted in the main text, these results should be interpreted as exploratory.

H4b had a significant typographical error: crucial words were missing in the preregistered version. H4b in the preregistered analyses reads: “Over estimators provided with corrective information will decrease. Negative directional hypothesis.”

Hypotheses H4b in the final manuscript reads, with the added text bolder: “Overestimators provided with corrective information will decrease the percentage of migrants they believe should be allowed to receive welfare (negative-directional hypothesis).”

Appendix E. Under- and Overestimators and Small vs. Big Overestimators

In online appendix E, tables 1–2 we report the demographic characteristics of under- vs. overestimators and small vs. big overestimators. We find that the largest differences between these different categories are in partisan identification and ideology. Most overestimators are Republicans (approximately 53 percent), and this majority is larger among big overestimators (60 percent). We must, however, emphasize that in all categories Democrats still make a significant portion of the sample.

The main results of the paper are robust to the inclusion of control variables such as partisan identity and ideology (see appendix E, table 3). Big overestimators who receive the treatment (Treatment × Big Overestimators) substantially lower the stated percentage of Mexican immigrants they claim should be allowed on SNAP, regardless of the exclusion (p-value = 0.004) or inclusion (p-value = 0.004) of controls.

Download Appendix F in the pdf.

References