A reexamination of the neurorealism effect: the role of context

Popescu, Maria; Thompson, R. Bruce; Gayton, William; Markowski, Vincent

doi:10.22323/2.15060201

Neuroscience research has garnered much public attention in recent years. Racine, Bar-Ilan and Illes [ 2005 ] drew attention to the difficulty of trying to convey scientific information to the general public. Specifically, they were interested in the degree to which the print press conveyed the strengths and weaknesses of functional magnetic resonance imaging (fMRI) research. They introduced the term neurorealism to describe the uncritical way in which the public can interpret the results of fMRI research. The authors suggested that the root of this uncritical consumption are related to the perception of visual proof of brain activity included in such imagery.

In similar research, Weisberg et al. [ 2007 ] proposed that articles dealing with neuroscience explanations of human behavior were especially alluring to the general public. They hypothesized that people would uncritically accept explanations of behavior that contained neuroscience information, even in those cases where the information was irrelevant. Their results indicated that undergraduate participants in both naïve and neuroscience groups judged explanations with irrelevant neuroscience information as more satisfying and believable than explanations without such visual information.

Further studies, conducted by McCabe and Castel [ 2008 ] attempted to determine whether the mere presence of brain images, separate from technical graphics in general, was what led people to perceive neuroscience information uncritically. In their first experiment they reported that a fictional news article including a brain image received higher ratings of scientific reasoning than articles containing a bar graph or no image. Their second experiment was carried out to determine if the results of the first study could be attributed to the visual complexity of brain images in comparison to that of bar graphs. Such visual complexity, rather than the images themselves, could in fact be responsible for the higher ratings of scientific credibility. Visual complexity effects were controlled via a topographical map of the brain employed in conjunction with the article in one group. The other experimental group received the brain image condition used by the first study. Results revealed that the brain image condition still received significantly higher ratings, thereby suggesting that it was not visual complexity, but the image itself that was responsible for rating differences. The third experiment by McCabe and Castel [ 2008 ] used a factual article from the BBC website in attempt to generalize results of their previous two studies to non-academic contexts. In contrast to participants in the no image group, the brain image group was significantly more likely to agree with the conclusion of the article. The authors concluded that images themselves could impart scientific credibility to brain imaging as a research technique. Weisberg et al. [ 2007 ], and later Hook and Farah [ 2013 ], described this conclusion as the “seductive allure” hypothesis.

Research consistent with the above findings, suggesting that neuroimagery has influence, include those of Ikeda et al. [ 2013 ] and Schweitzer, Baker and Risko [ 2013 ]. Ikeda et al. [ 2013 ] investigated the impact of neuroimagery on metacomprehension judgments (participants’ subjective judgments of their own comprehension) and actual comprehension scores of neuroscience research. They found that the presence of brain images resulted in inflated metacomprehension estimates with no difference in actual comprehension. This study appears to be the only attempt to date to distinguish between understanding and belief within the context of neuroimagery related bias.

Schweitzer, Baker and Risko [ 2013 ] attempted to replicate the McCabe and Castel’s [ 2008 ] findings. Only one of five experiments showed a neuroimagery bias. Importantly, this one study reporting an effect used a repeated-measures design, with the following conditions: information containing weak arguments; a lack of neuroscience descriptors; and a realistic, 3-D color neuroimage used by Keehner, Mayberry and Fischer [ 2011 ]. These conditions were seen as fertile grounds for a neuroimagery bias to occur. A significant neuroimagery effect was found under these conditions. The authors suggest that in order for the neuroimagery effect to be demonstrated, participants need an experimental procedure that allows several points of reference, as found in a repeated-measures design. If significant results are partially an artifact of experimental design, then between-subjects designs may create conditions that require absolute, rather than relative judgments, and therefore may explain a more ephemeral neuroimagery bias.

Research inconsistent with a neuroimagery bias includes Gruber and Dickerson [ 2012 ], who attempted to replicate the McCabe and Castel [ 2008 ] “mere presence” results and found no significant differences between the no-image condition and the other three image conditions (fMRI, artistic rendering of brain, or sci-fi image of brain). In addition, the authors recommend that the persuasive power of article content should be examined simultaneously with the influence of brain images in print media. This would allow researchers to determine how the text of an article interacts with the image in determining the readers’ level of comprehension or evaluation of the article’s message. [Struthers et al., 2013 ] partially addressed this suggestion in their attempt to determine if the neuroimagery bias could be extended into topics of a religious nature. However, no impact of neuroimagery was found on any of the three dependent measures in this context.

Michael et al. [ 2013 ] reported that when an attempt to build on the McCabe and Castel [ 2008 ] findings failed, they conducted a meta-analysis that included 10 experiments involving over 2000 participants. They found that a brain image had little or no influence on participants’ beliefs of whether brain imaging could be used to detect states of mind. Similarly, Hook and Farah [ 2013 ] attempted to replicate and expand on the “seeing is believing” hypothesis reported on by McCabe and Castel [ 2008 ]. Their results showed no significant effect for image condition on the degree of agreement or disagreement with the conclusions reached in the articles.

It would appear that further study is needed in order to clarify the results described above. At this point the “seductive allure” hypothesis seems to have a volatile, alternating nature, possibly associated with experimental design, wherein some studies can document an effect and others cannot. Certainly, recent results suggest that the influence of brain images on ratings of fMRI research may have been overestimated. Given the inconsistent findings, research ought to turn from the question of whether the “seductive allure” hypothesis valid, to research that attempts to examine the specific factors or contexts that may influence potential neuroimagery effects. In additional to the issue of experimental design, one context variable that may influence the impact of neuroimagery (its “slant”) has to do with the claims made in the article, particularly whether the article is advocating for the efficacy and value of fMRI imaging, or conversely, is presenting a skeptical, critical view of fMRI use.

Following up on the recommendations of Gruber and Dickerson [ 2012 ] and Schweitzer, Baker and Risko [ 2013 ] we examined whether author viewpoint difference — a positive versus negative slant in the evaluation of the technology — may interact with the presence of neuroimagery and influence ratings of believability. The aforementioned disparity in the literature may partially stem from repeated-measures versus between-subjects designs. Therefore we hypothesized that our between-subjects design, in line with Schweitzer, Baker and Risko [ 2013 ] would not result in a main effect for image presence, but may reveal a main effect for article slan — positive or negative that may interact with the presence of a neuroimage. We approached this question by presenting participants with an article about the ability of fMRI to detect states of mind — either with an fMRI or without, and with either a clearly positive author evaluation or a clearly negative author evaluation. Drawing from examples cited in Schweitzer, Baker and Risko [ 2013 ] each participant’s article was accompanied by a set of five possible states of mind that could conceivably be detected by fMRI (Lying, buying motivation, romantic feelings; being seriously mistaken, and feeling satisfied by altruism). As with most previously cited research, in the present study we are formally testing for effects on individuals’ beliefs.

There is no extant literature on the neuroimagery bias phenomenon that has also addressed possible age effects or sex differences. Moreover, the literature specifically addressing the potentially biasing effects of neuroscience imaging goes back less than ten years, making trends in popular interpretation difficult to evaluate. Age differences may reveal important cohort effects relating to the critical consumption of neuroscience literature. However, given the absence of any published data, the direction of a cohort effect is not obvious — older individuals, educated in pre-neuroimaging culture, may experience less of the “seductive allure”. Conversely, less exposure in general to advanced imaging technology may cause the opposite to be the case — greater exposure among younger individuals may make them more tech-savvy and perhaps more guarded about the efficacy of neuroimaging compared to older individuals. Age may also play a role as a predictor of intellectual maturity and critical thinking overall. For both reasons, participants’ age was included in our investigation as a control variable. There are also no reported effects on how neuroimaging may affect males and females. We recorded the sex of participants and included it in our analyses.

1 Method

1.1 Design

Our design used a $2 \times 2 \times 2$ factorial Analysis of Covariance (ANCOVA) to test for main effects and interactions associated with presence/absence of fMRI image, article slant (negative or positive) and sex (Male/Female). Thus the four possible manipulated conditions across male and female participants are as follows: image present/negative slant; image present/positive slant; no image/negative slant; and no image/positive slant. ANCOVA was used to partial out the potential effects of participant age. Each participant was randomly assigned to one of the four manipulated conditions and read a short article that reported the use of fMRI. Our resulting dependent measure reflected the degree to which participants agreed with the conclusions of the article.

1.2 Participants

The sample was composed of 112 undergraduate students (22 males, 90 females; mean age = 22.5 years, $S D = 5.9$ years, range = 18–61), of mixed socio-demographic backgrounds, that reflected the regional population of southern Maine, U.S.A., as well as the typical sex ratio in an undergraduate psychology degree program. All participants were offered extra credit in as an incentive for participating in the study. This information was detailed in the informed consent documentation.

1.3 Procedure

Following the informed consent process, and randomized assignment to one of the four manipulated conditions, each participant was presented with the test materials described below. Participants rated the degree to which they agreed/disagreed with each article’s discussion of fMRI. The scale ranged from 1: strongly agree to 7: strongly disagree, with the midpoint falling upon undecided/neutral.

1.4 Stimulus Materials

Test materials consisted of a lay neuroscience article with the magazine’s name omitted to prevent response bias due to each publication’s reputation. In addition, the authors’ names and university affiliations were fictionalized. The positive condition of the article only included text that reported ethical, efficacious and practical potential for the use of fMRI within medicine, forensics, and human resources. The negative condition of the article omitted the positive aspects of fMRI, and focused only its ethical problems, technical limitations and shortcomings in applicability. Each of these two conditions either contained an fMRI image or did not. All four conditions included the brief descriptions of five possible uses of fMRI (Lying, buying motivation, romantic feelings; being seriously mistaken, and feeling satisfied by altruism) in order to provide a diverse set of mental-state domains that fMRI could plausibly detect. This procedure allowed us to determine the degree to which participants’ beliefs were influenced by the presence or absence of images as well examine the influence of article slant on beliefs.

2 Results

Our first hypothesis, that our between-subjects design would not result in a significant main effect for the presence or absence of an fMRI image, was confirmed ( $F (111) = 1.70$ ; $p = . 20$ ; $η^{2} = . 02$ ). Those participants who read the article that included an fMRI image, regardless of sex ( $M = 3.48$ , $S D = . 59$ ), had a mean aggregated response regarding persuasiveness that did not differ significantly from the mean of those not receiving an image ( $M = 3.62$ , $S D = . 60$ ). Moreover, the low effect size did not suggest that a larger $N$ with greater statistical power would have result in a significant effect.

A significant main effect for article slant (negative versus positive) was observed, whereby aggregated scores (controlling for age) resulted in significantly different responses ( $F (112) = 6.85$ ; $p < . 01$ ; $η^{2} = . 09$ ). The trend, though with a weak effect size, was for participants to agree with the arguments of the article more ( $M = 3.70$ , $S D = . 61$ ) when it described neuroscience findings in a positive light compared to those receiving articles written in a negative light ( $M = 3.38$ , $S D = . 55$ ), regardless of whether the article contained an fMRI image No main effect of sex was found ( $F (111) = . 38$ ; $p = . 54$ ; $η^{2} = . 004$ ). Males’ mean response ( $M = 3.65$ , $S D = . 66$ was very similar to female responses ( $M = 3.55$ , $S D = . 59$ ).

Potential interactions were explored among each of the possible combinations of conditions. No interactions were found between Image (presence/absence) and slant (negative/positive) ( $F (111) = 1.89$ ; $p = . 17$ ; $η^{2} = . 02$ ); between image and sex ( $F (111) = . 31$ ; $p = . 58$ ; $η^{2} = . 003$ ); or between slant and sex ( $F (111) = 3.12$ ; $p = . 08$ ; $η^{2} = . 03$ ) Lastly, a three-way interaction (image $\times$ slant $\times$ sex) was not found ( $F (111) = 2.34$ ; $p = . 13$ ; $η^{2} = . 02$ ) Age was entered into the main effect and interaction models as a covariate, but the resulting correlations were not significant predictors of participants’ beliefs.

3 Discussion

The results of this study are consistent with recent findings that no broad evidence was found for the influence of neuroimagery on participants’ beliefs in the power of fMRI to detect mental states. The results reflect a lack of influence even when the images are presented in a context emphasizing the strengths of fMRI technology. Although null results cannot prove there is no neurorealism effect, our results strongly support the possibility that article context (slant), either negative or positive, is a key variable, rather than the present/absence of an image. In particular we found a broad effect of readers’ judgments about credibility being associated simply with the author’s positive slant. Thus, lay readers of neuroscience information may be more influenced by the tenor of a claim, which could interact with pre-existing biases in favor of high-tech science. This finding opens up the possibility of exploring links to research that has explored how author’s or speaker’s intentional manipulation of specific facets of language (i.e. rhetorical devices) may shape consumers’ believes about a technology such as brain imaging.

A second explanation for the recent lack of support for the influence of neuroimagery deals with the fact that most of the research reporting null findings has used between-subjects designs (i.e. independent, comparative samples). As Michael et al. [ 2013 ] and Schweitzer, Baker and Risko [ 2013 ] have pointed out, the studies reporting an effect have used repeated-measure designs primarily (i.e. comparison within subjects’ responses). These authors suggest that in order for the neuroimagery effect to be demonstrated, participants need an experimental procedure that allows several points of reference, as found in a repeated-measures design. Our study, which utilized independent groups for each of the four conditions, mainly disconfirmed a “ neurorealism ” effect, when judgments are formed without the explicit opportunity for comparison. Although between-subject designs have the potential to introduce confounds related to subject differences across groups, the counter argument is that within-subject designs are inherently less ecologically valid. This is because lay readers of science in everyday life are highly unlikely to be afforded multiple versions of a neuroscience article (image/no image; positive/negative) and be asked to judge the validity of each one in one sitting. Therefore we argue that the “neurorealism” effect is weak at best, and highly context sensitive. This is not to discount the value of within-subjects designs for this area of study, since it is often the only way to conduct studies that result in nuanced differences in how stimuli affect individuals’ perceptions.

In summary, our findings are consistent with recent research suggesting that the biasing influence of neuroimagery may have been overestimated. It should be pointed out, however, that there is recent research that continues to argue for an effect of neuroimagery [e.g. Ikeda et al., 2013 ; Keehner, Mayberry and Fischer, 2011 ]. As Schweitzer, Baker and Risko [ 2013 ] have suggested, the failure of recent research to find evidence of “neurorealism” may be a function of the increased exposure to neuroimagery since the original research between 2005 and 2008. It may be that the tendency of neuroimagery to influence judgments has decreased as a result of this exposure. However, prior to the 2005 Racine et al. study this is difficult to document. It is also possible that critiques of fMRI research have filtered down to the popular press and are now affecting people’s receptiveness to fMRI information. This possibility is particularly relevant to results deriving from typical undergraduate psychology subject pools. Contemporary psychology undergraduates may be better trained to be critical thinkers, than the lay public and therefore less prone to becoming biased by the mere presence of neuroimagery. Other potentially important variables that should be investigated include the effects of socioeconomic status and culture. These dimensions, particularly where education level differences exist, may reveal important socio-demographic patterns that may be relevant to the public understanding and critical evaluation of science literature that uses neuroimaging.

References

: Gruber, D. and Dickerson, J. A. (2012). ‘Persuasive images in popular science: Testing judgments of scientific reasoning and credibility’. Public Understanding of Science 21 (8), pp. 938–948. https://doi.org/10.1177/0963662512454072 .
: Hook, C. J. and Farah, M. J. (2013). ‘Look again: effects of brain images and mind-brain dualism on lay evaluations of research’. Journal of Cognitive Neuroscience 25 (9), pp. 1397–1405. https://doi.org/10.1162/jocn_a_00407 . PMID: 23879877 .
: Ikeda, K., Kitagami, S., Takahashi, T., Hattori, Y. and Ito, Y. (2013). ‘Neuroscientific information bias in metacomprehension: the effect of brain images on metacomprehension judgment of neuroscience research’. Psychonomic Bulletin & Review 20 (6), pp. 1357–1363. https://doi.org/10.3758/s13423-013-0457-5 . PMID: 23728726 .
: Keehner, M., Mayberry, L. and Fischer, M. H. (2011). ‘Different clues from different views: The role of image format in public perceptions of neuroimaging results’. Psychonomic Bulletin & Review 18 (2), pp. 422–428. https://doi.org/10.3758/s13423-010-0048-7 .
: McCabe, D. P. and Castel, A. D. (2008). ‘Seeing is believing: The effect of brain images on judgments of scientific reasoning’. Cognition 107 (1), pp. 343–352. https://doi.org/10.1016/j.cognition.2007.07.017 .
: Michael, R. B., Newman, E. J., Vuorre, M., Cumming, G. and Garry, M. (2013). ‘On the (non)persuasive power of a brain image’. Psychonomic Bulletin & Review 20 (4), pp. 720–725. https://doi.org/10.3758/s13423-013-0391-6 .
: Racine, E., Bar-Ilan, O. and Illes, J. (2005). ‘fMRI in the public eye’. Nature reviews. Neuroscience 6 (2), pp. 159–164. https://doi.org/10.1038/nrn1609 . PMID: 15685221 .
: Schweitzer, N. J., Baker, D. A. and Risko, E. F. (2013). ‘Fooled by the brain: Re-examining the influence of neuroimages’. Cognition 129 (3), pp. 501–511. https://doi.org/10.1016/j.cognition.2013.08.009 .
: Struthers, W. M., Mulhauser, K., Smith, L. and Campos, D. (2013). ‘The persuasiveness of brain imaging and religious commitment’. Paper presented at the annual meeting of the Association for Psychological Sciences, Washington, DC, U.S.A.
: Weisberg, D. S., Keil, F. C., Goodstein, J., Rawson, E. and Gray, J. R. (2007). ‘The Seductive Allure of Neuroscience Explanations’. Journal of Cognitive Neuroscience 20 (3), pp. 470–477. https://doi.org/10.1162/jocn.2008.20040 .

Authors

Maria Popescu is a doctoral candidate in the neuroscience program at SUNY Upstate Medical University, Syracuse, NY. Her background is in Neurotoxicology and Teratology. E-mail: maria.popescu01@gmail.com .

R. Bruce Thompson, Ph.D. is Professor of Psychology in the Department of Psychology, University of Southern Maine. His primary focus of research is in language and cognitive development. E-mail: rbthompson@maine.edu .

William F. Gayton, Ph.D. is Professor of Psychology in the Department of Psychology, University of Southern Maine. His primary focus of research is in positive psychology. E-mail: bgayton@maine.edu .

Vincent Markowski, Ph.D. is Associate Professor of Psychology in the Department of Psychology, SUNY Geneseo. His primary focus of research is environmental toxicology and neuroscience. E-mail: markowski@geneseo.edu .