Plausibility of scientific findings: institutional factors in lay evaluations

1 Introduction

Since E. Husserl’s last major work, “The Crisis of European Sciences and Transcendental Phenomenology” [ 1989 ], the idea of reconnecting the basic notions of science back to their conceptual roots in the pre-scientific, everyday conceptual structure of “lifeworld” in order to prevent the general crisis in public confidence in science, its concepts, methods, and findings, has become a kind of intuitively accepted common ground for subsequent attempts to reconstitute the meaningfulness and trustworthiness of science for everyday life [Føllesdal, 1988 ; Heffernan, 2017 ]. Multiple attempts by philosophers, sociologists of science, and science communicators, to find the universal remedy for persistently self-reproducing disconnectedness between various sciences and relevant everyday “natural” belief-structures, have proved to be not fully successful.

This has ultimately led to the pronounced rates of distrust to some emerging scientific and technological developments (e.g., genetic engineering, automation, climate science), as well as to insufficient public confidence in the scientific method’s ability to produce plausible conclusions [House of Lords Select Committee on Science and Technology, 2000 ; Gauchat, 2012 ; Funk, 2020 ; WCIOM, 2020 ], with anti-science attitudes being prevalent, on average, worldwide, fluctuating across countries [Mede, 2022 ] and depending on country-level factors [O’Brien & Noy, 2018 ], social groups, and issues [Iyengar & Massey, 2019 ]. For these disturbing tendencies to be overcome, scientists and science communicators need not just to enhance public support but to deepen their understanding of the wider public’s interests and determinants of ordinary judgments regarding scientific research and the results of scientific knowledge production, as such understanding might be conducive to more effective communication with lay audiences.

Some authors argue that due to the inevitable ontological differences between scientific and everyday thinking most members of the general public are not able to decide based on their capacities which scientific claims to adopt as true. This, in turn, requires people to consider extra-scientific features, when judging scientific claims, thinking about science as a social system [Bromme & Gierth, 2021 ]. This means that laypeople will tend to resort to a more heuristic, instead of analytical, thinking relying in their judgment more on contextual cues [Taddicken & Krämer, 2021 ].

When reporting on recent developments, science communication usually provides additional information about institutional aspects of scientific research. It may include information about the scientific discipline associated with it (e.g., “… says Vilnius University biochemist” [McCarty, 2021 ]), organization (“Scientists at Nottingham Trent University have developed…” [Goodyer, 2021 ]), and funding of the research (e.g., “To investigate that hypothesis, the team has begun working with a $2.1 million, 5-year NIH grant…” [Picower Institute at MIT, 2021 ]). What is the possible role of such institutional factors in lay estimations of scientific studies?

Studies indicate disciplinary differences in public attitudes [National Science Board, 2014 ], while the research on public perceptions of science rarely considers their role. With a few recent exceptions [Scheitle & Guthrie, 2019 ], studies on public perceptions are mostly either focusing on science in general, are problem-specific [Critchley, Bruce, & Farrugia, 2013 ; Hmielowski, Feldman, Myers, Leiserowitz, & Maibach, 2014 ], or concentrating on particular disciplines [Wingen, Berkessel, & Englich, 2020 ; Hendriks & Jucks, 2020 ], not allowing to grasp possible effects of the discipline of the research. As for the institutional attribution and funding, the previous studies show that publicly funded university scientists are more trusted compared to those getting their funding from private companies [Critchley, 2008 ; Critchley et al., 2013 ], but they do not differentiate between universities and do not consider their prestige, nor do they investigate the purported specific effects of the funding amount.

The purpose of this study is to systematically evaluate possible effects of such institutional factors as the scientific field, size of research funding, and prestige of a university on the lay perception of the plausibility of scientific findings. It could broaden our understanding of the information cues determining the public authority of science. This may provide a further possibility of building more productive science communication with general audiences leading to their support and cooperation.

2 Context

2.1 Plausibility

The notion of plausibility is associated with the process of evaluation of concepts, ideas, and explanations, in particular, scientific ones. Plausibility is known to be determining their further acceptance among students [Blank, 2000 ; Grotzer & Mittlefehldt, 2012 ; Chen & Wang, 2016 ], and is produced and used explicitly or implicitly by both scientists and laypersons (such as students and the general public) [Lombardi, Nussbaum, & Sinatra, 2016 ; von der Mühlen, Richter, Schmid, Schmidt, & Berthold, 2016 ]. Plausibility judgement, when evaluating a knowledge claim, is based on perceiving the internal and external consistency of the claim with both its proposed premises and background assumptions, and prior knowledge and beliefs of a perceiver [Lombardi & Sinatra, 2012 ; von der Mühlen et al., 2016 ; Richter & Maier, 2017 ]. Plausibility might be based not only on existing empirical evidence, but also on hypothetical propositions inferred from other plausible assumptions that do not contradict the already known facts [Nussbaum, 2011 ]. It makes plausibility dependent both on the content of a statement and on the position of the one who evaluates it, which includes their experience, attitudes, and beliefs. Plausibility was found to be influenced by the credibility of the source [Lombardi, Seyranian, & Sinatra, 2014 ], comprehensibility and simplicity of the claim [Lombardi et al., 2016 ; Richter & Maier, 2017 ], its internal and external consistency [von der Mühlen et al., 2016 ], and subjects’ emotions about the topic [Lombardi & Sinatra, 2013 ].

In our definition of plausibility we are following a line of research developed in the realm of cognitive science mentioned above. We define plausibility as a judgment of potential truthfulness when evaluating knowledge claims, made by relating the incoming information to individual’s prior knowledge and assessing its “goodness of fit” [Lombardi et al., 2014 ] with all knowledge possessed by an individual, including both knowledge about the structure of an argument as well as with background conceptual and content knowledge [von der Mühlen et al., 2016 ]. Plausibility is distinct from but related to other criteria used for evaluating information cues such as credibility, comprehensibility, and coherence, being connected but not limited to argumentation, source evaluation, and reading comprehension. In particular, in the case of complex and controversial topics, credibility judgments can act as a prerequisite for assessing the plausibility of a claim, which though, may encompass a broader set of factors [Lombardi & Sinatra, 2012 ]. It also is connected to conceptual change, with perceived implausibility being able to function as one of the barriers for accepting knowledge and ideas, including scientific ones [Lombardi et al., 2016 ]. Therefore, if we want to explore how non-professionals perceive scientific information in terms of accepting it as valid while using all of the knowledge at hand and judging on a wide range of characteristics, we should investigate possible factors affecting the perceived plausibility of incoming scientific claims among them.

2.2 Scientific field

We assume that lay plausibility perceptions of a scientific claim are determined by representations people have of its academic domain, the scientific field it belongs to [Muis, Bendixen, & Haerle, 2006 ; Hofer, 2006 ]. For example, public opinion data show that Americans consider sociology, economics, and history as much less scientific than physics, biology, medicine, and engineering [National Science Board, 2014 ]. Studies also show that knowledge in social sciences and humanities (e.g., psychology and history) compared to natural sciences (e.g., biology), is usually perceived as less certain and unchanging [Hofer, 2000 ; Estes, Chandler, Horvath, & Backus, 2003 ; Barzilai & Weinstock, 2015 ]. Meanwhile, perceptions of certainty in science communication were found to be a significant predictor of plausibility judgments in the field of climate science [Lombardi et al., 2014 ].

Some discipline-specific elements of knowledge claims are also proved to have “seductive effects” on lay judgments. Recent studies have shown that adding neuroscientific [Weisberg, Keil, Goodstein, Rawson, & Gray, 2008 ; Weisberg, Taylor, & Hopkins, 2015 ] and mathematical [Eriksson, 2012 ] jargon and expressions (whatever their relevance or meaningfulness) leads to higher assessments of the quality of scientific information. Though, this effect was not apparent among people with relevant expertise.

Based on the aforementioned previous findings, we hypothesize that:

• Research results in such fields as sociology and criminology, both being social sciences, may be perceived as less plausible compared to those in neuroscience as well as genetics and physiology as branches of biology.

2.3 Funding

Our second assumption is that information about research funding might influences the way laypeople perceive scientific research and its results. Here, we follow the idea that funding may be used as an indirect indicator that provides information about the quality of supported research [Connolly, 1997 ; Blume-Kohout, Kumar, & Sood, 2009 ].

There is a view that the amount and type of research funding is an ultimate measure of scientific knowledge and contribution of a nation aiming at supporting “best science”, having “most potential to advance knowledge” [Murray et al., 2016 ], which necessarily includes expert evaluation and assessment of scientific units and research determining their funding [Geuna & Martin, 2003 ]. This may drive some further assumptions among readers about the quality of research. Scholars report that those scientific articles that mention grant-funding are more recognized by peers, being cited more frequently compared to those without such acknowledgment [Zhao, 2010 ; Rigby, 2013 ; Yan, Wu, & Song, 2018 ].

Studies also show a relationship between funding size and research output with a growth of productivity and impact associated with an increase in funding [Payne & Siow, 2003 ; Fortin & Currie, 2013 ; Rosenbloom, Ginther, Juhl, & Heppert, 2015 ]. Some authors even state that the grant volume is “generally seen as a good indicator of performance” [Ma, Mondragón, & Latora, 2015 ].

We suggest that lay assessments of plausibility may also be subject to the following effects: in addition to the fact that mentioning research funding should increase the perceived plausibility of research results, grant value may also make a difference. Therefore, we posit the following exploratory hypothesis:

• The greater the funding volume, the greater the lay assessment of perceived plausibility would be.

2.4 Institutional prestige

We suppose that institutional prestige, as a “relative esteem” in which an institution is held in an “ordered total system of differentiated evaluation” [Parsons, 1951 , p. 132], may play a role in how scientific findings are perceived. Our assumption is based on the idea that the institutional source of a claim contributes to its acceptance, with a highly credible or prestigious source being more persuasive, as opposed to a low-prestige one [Hovland & Weiss, 1951 ; Pornpitakpan, 2004 ].

Moreover, we base our assumption on the idea of the existing relationship between academic recognition and institutional affiliation, formulated within the framework of the sociology of science. Apart from scientists gaining more recognition when affiliated with major universities compared to minor ones [Crane, 1965 ], scientific work itself affiliated with prestigious institutions is more likely to be recognized, accepted, or selected when judged by scientists during the peer-review process. This was demonstrated for such forms of presenting scientific research as grant applications [Gillespie, Chubin, & Kurzon, 1985 ], manuscripts [Peters & Ceci, 1982 ; Bakanic, McPhail, & Simon, 1987 ], brief scientific reports [Garfunkel, Ulshen, Hamrick, & Lawson, 1994 ], and abstracts for conference papers [Ross et al., 2006 ].

We hypothesize that institutional prestige may not only promote higher professional recognition and justification of scientific findings within the academy, and thus professional scientists, but also enhance lay evaluations:

• Research results may be perceived as more plausible if they were obtained by scientists from a prestigious university.

3 Methods

3.1 Design and procedure

We conducted two separate factorial surveys using a mixed design with between-subjects and within-subject factors, with random assignment in both studies. We manipulated three factors in the factorial vignettes we created: (1) scientific field of the research, (2) information on funding — the size of the research grant, and (3) institutional prestige — researchers’ university affiliation selected depending on its ranking (based on the Webometric Ranking of World Universities).

In both studies, participants read a set of short textual vignettes describing scientific research. Texts were presented to them in random order. After reading each description, the participants assessed the plausibility of research results (see below for more details). In addition, they evaluated their competence level in different scientific fields and provided information about their sociodemographic and educational background. The average time to complete the questionnaire in both studies was roughly 8 minutes.

3.1.1 Study 1

We used a $2\times 2\times 3$ experimental design with scientific field (sociology, neuroscience) and institutional prestige (name of a specific university from the $1^{\text{st}}$ decile of the university ranking was provided ¹ or no affiliation mentioned) as within-subject factors, and funding (high — $500 000, low — $100 000, no funding mentioned) as a between-subject factor (see Table 1 for details). Each participant read and evaluated 4 textual vignettes in which we varied the field of study and information about institutional affiliation.

3.1.2 Study 2

In our second study, we added more levels for each experimental factor, also changing some of the ways of attributing them. We used a $5\times 3\times 3$ experimental design with disciplinary field (sociology, criminology, neuroscience, genetics, and physiology) as within-subject factors, and institutional prestige (names of specific universities from the $1^{\text{st}}$ and $6^{\text{th}}$ deciles of the university ranking were provided ¹ or no affiliation mentioned) and funding (high — $950 000, low — $50 000, no funding mentioned) as between-subject factors (see Table 2 for details). Each participant read and evaluated 5 vignettes, each attributed to one of the fields of study.

3.2 Participants

For sampling, we have chosen a specific group of the general public, i.e., university students. University students are pursuing academic training which possibly makes scientific information, as well as the strategies for searching and evaluating it, more relevant to them compared to other non-professionals. We made sure of their non-professional status by recruiting only those students who were not specializing in any of the scientific fields chosen.

The participants for both studies were recruited through the social network Vkontakte ² ( vk.com ) by placing ads in different student communities. Participation was voluntary and did not involve any extrinsic inducement. Those who volunteered to participate were directed to the 1KA platform, ³ where they gave informed consent to participate and complete the survey.

Only participants that fully completed the questionnaire were included in the analysis. As a result, out of those 1,055 (Study 1) and 785 (Study 2) volunteers who agreed to participate, we filtered out 481 and 307 incomplete cases accordingly. We also filtered our data based on the following criteria. We included data only from university students that were not majoring in the fields that were present in our experimental materials (Study 1: sociology or psychology; Study 2: sociology, psychology, biology or medicine), who evaluated themselves as moderately (or less) competent in those fields (see below for details), and were younger than 29 years old (based on Tukey’s method for detecting extreme outliers [Tukey, 1977 ]). Further details on the data acquisition can be accessed in the Supplementary Materials.

3.2.1 Study 1

The final sample consisted of 429 students, of which 54% were female, aged from 16 to 28 ( $M=20.6$ , $\text{SD}=2.18$ ) years. As for participants’ academic major, the breakdown was 25% engineering and technology, 23% humanities, 21% mathematics and computer science, 17% natural sciences, 11% medicine (including veterinary medicine), and 3% other (including sports, customs affairs, and military); 85% respondents were undergraduate students, 15% — graduate students.

3.2.2 Study 2

The final sample consisted of 344 students, of which 54% were female, age ranging from 17 to 28 years ( $M=20.7$ , $\text{SD}=1.99$ ). As for participants’ academic major, 18% of the participants were studying engineering and technology, 31% — humanities, 25% — mathematics and computer science, 24% — natural sciences, and 2% other (including veterinary medicine, tourism, customs affairs); 82% of the respondents were undergraduate students and 18% — graduate students.

3.3 Materials

In both studies, we created three-sentence long lay summaries, reading materials of approximately $50\pm 4$ words, each based on real scientific research studies published in Science and Nature , following existing recommendations [Kuehne & Olden, 2015 ]. Each text related to one disciplinary field and contained a brief research description, including information about its objectives, methodology, and main results. By adding information about funding and institutional affiliation in accordance with varying levels of two other factors, i.e., the size of research funding and institutional affiliation, we created a set of 12 (Study 1) and 45 (Study 2) vignettes.

Prior to the experimental phase, we tested all lay summaries for relative comprehensibility in a pre-test, with 32 and 61 university students respectively for Studies 1 and 2. In both cases, no significant differences between texts were found ( $p>0.05$ ). In addition, we pre-tested the correctness of scientific field attribution of the material for Study 2. All of the vignettes were correctly attributed to their disciplinary field or a broader domain. We also tested in a separate experiment ( $N=91$ ) whether students differentiate specific universities in terms of their prestige and recognize the different levels of the prestige factor. Higher ranked universities were perceived significantly more prestigious than the lower ranked ones. A more detailed description of the procedure, including lay summaries used, can be accessed in the Supplementary Materials section.

3.4 Measures

Perceived plausibility. After being exposed to each vignette, describing the research, participants evaluated the degree of plausibility of the research results on an 11-point scale (from 0 — “Absolutely implausible” to 10 — “Absolutely plausible”).

Individual characteristics. In addition, respondents answered questions related to their backgrounds. Participants assessed their level of competence in sociology and neuroscience (Study 1), or sociology, criminology, neuroscience, genetics, and physiology (Study 2) on a 5-point scale (from 1 — “Know nothing about it” to 5 — “Know almost everything about it”). They also specified their socio-demographic (age and gender) and educational backgrounds, including their academic major, education level, and year of study.

4 Results

Our main research question is focused on the possible effects that scientific field, funding, and institutional prestige can have on the perceived plausibility of research results. We used repeated-measures ANOVA to evaluate the main effects and pairwise interaction effects of all three experimental factors that were used in both studies.

4.1 Study 1

In our first study, we used sociology and neuroscience as disciplinary fields, two funding figures ($100 000 and $500 000), and a highly ranked university vs. no mention of the specific university, to test possible effects of the scientific field, funding, and institutional prestige on plausibility perceptions. Since the only between-subject factor in this study was funding, there were three separate groups of respondents. Table 3 provides the descriptive statistics for all combinations of conditions we used.

There was a significant main effect of research field ( $F_{1,426}=16.875$ , $p<.001$ , ${\eta }_p^2=0.038$ as a measure of effect size) with sociological research results perceived significantly less plausible than those in neuroscience ( $\text{EMM diff}=0.575$ , $p<.001$ ). The main effect of funding was not significant. There was no main effect on the plausibility scores of research results depending on whether there was any information provided regarding funding given or depending on the size of research funding mentioned ( $F_{2,426}=0.518$ , $p=.596$ ). We were also unable to find any significant main effect of institutional prestige. There was no overall difference in perceived plausibility of research results whether the researchers were affiliated with the highly ranked university or not ( $F_{1,426}=0.333$ , $p=.564$ ). The analysis also did not reveal statistically significant effects of pairwise interaction between factors.

4.1.1 Discussion

The purpose of this study was to identify how the field of research, its funding, and the presence of informational cues about high institutional prestige, can influence lay plausibility perceptions of the research results. Our results show that the scientific field may act as a factor influencing lay perceptions of the plausibility of research results: sociological findings are generally considered as less plausible compared to the results obtained in neuroscience. As for other factors, we did not find any statistically significant effect (or any statistically significant interactions) of funding or institutional prestige on the perceived plausibility of research results. Consequently, we found no evidence allowing us to sustain our hypotheses $H_2$ and $H_3$ regarding the corresponding informational cues role in how students not specializing in corresponding disciplinary fields perceive research results.

We conducted a second study to have the opportunity to rule out some of the possible alternative explanations for our negative results. It is possible that mentioning only high-ranking universities does not properly capture the prestige effect. We were also concerned about the sensitivity of our grant size scale as our initial levels might not be contrasting enough. Also, reading and evaluating one research description twice, with the need to present and evaluate each instance as a separate study based on only additional institutional information, might attract too much attention to that information while the task was to evaluate research results at large.

4.2 Study 2

In our second study, we expanded the range of scientific fields by including criminology, genetics, and physiology, made funding amounts more contrasting ($50 000 vs. $950 000), and added a relatively low ranked university to test the possible effects of the scientific field, funding, and institutional prestige on plausibility perceptions in more detail. We also made institutional prestige a between-subject factor (along with funding, similarly to study 1) creating 9 separate groups of respondents. Table 4 provides the descriptive statistics for all combinations of conditions we used.

A highly pronounced significant effect of research field was also found in our second study ( $F_{4,1340}=120.735$ , $p<.001$ , ${\eta }_p^2=0.265$ ), with sociology having lower plausibility scores compared to neuroscience ( $\text{EMM diff}=1.485$ , $p<.001$ ) as well as to physiology ( $\text{EMM diff}=1.609$ , $p<.001$ ), similar to criminology also being associated with lower plausibility scores compared to neuroscience ( $\text{EMM diff}=1.567$ , $p<.001$ ) and physiology ( $\text{EMM diff}=1.690$ , $p<.001$ ), and genetics research having significantly lower plausibility scores opposed to sociology ( $\text{EMM diff}=1.711$ , $p<.001$ ), criminology ( $\text{EMM diff}=1.630$ , $p<.001$ ), neuroscience ( $\text{EMM diff}=3.196$ , $p<.001$ ), and physiology ( $\text{EMM diff}=3.320$ , $p<.001$ ).

Similar to our first study, the main effect of funding was not significant. There was no overall difference in the plausibility scores of research results depending on whether there was any information provided regarding funding given or depending on the amount of funding mentioned when there was a wider gap between the amounts ( $F_{2,335}=0.260$ , $p=.771$ ). The main effect of institutional prestige was also not significant, with no overall difference in perceived plausibility of research results when comparing research results with no affiliation mentioned, and top-ranked and low-ranked university affiliation ( $F_{2,335}=1.338$ , $p=.264$ ). The analysis also did not reveal statistically significant effects of interaction between our experimental factors.

4.2.1 Discussion

The purpose of our second study was to further explore the relationship between perceived plausibility of research results and informational cues related to the field of research, its funding, and institutional prestige. Not only did we expand the range of scientific fields, but also created more contrast for other conditions, widened the gap between funding amounts, and assessed the influence of informational cues i.e., specific university names corresponding to either high-ranked or lower-ranked university affiliation.

Based on our analysis, the disciplinary field plays the only significant role in the perceived plausibility of research results among those three factors. We were able to discover that sociological and criminological research results are perceived as less plausible than neuroscientific and physiological research, but as more plausible compared to research results from the disciplinary field of genetics. We can say that results obtained in social research (sociological or criminological) are perceived by our participants as less plausible compared to research in some of the natural sciences (neuroscience and physiology), with the exception of genetics. Nevertheless, natural sciences are not that homogeneous since some of the branches, for instance genetics, are associated with even less plausibility than social sciences, though the latter finding might have been partly caused by our research design. Since in our study disciplinary fields were necessary conflated with a specific research, this effect may be hypothetically caused by our specific example of genetics research as such and not so much by the disciplinary field of genetics itself.

As for other experimental factors, we found no evidence allowing us to claim that they play some role in how students as a rather educated part of a general readership perceive scientific results in terms of their plausibility. There was no difference in how students perceive results of scientific research depending on whether its funding is mentioned or not or whether funding is large or small. They also did not respond to informational cues on institutional affiliation and its corresponding prestige when assessing research results unlike science professionals and experts in previous studies [e.g. Garfunkel et al., 1994 ]. This applies to both social and natural sciences.

5 General discussion

Our research aimed at examining the possible effects of the scientific field, research funding, and institutional prestige on students’ perceptions of plausibility regarding the results of scientific research. Here, we follow the concept of plausibility as a determinant of acceptance and justification, dependent in turn, on social context and public recognition, as well as on background assumptions of the perceiver providing some level of internal and external consistency of a claim being evaluated [Richter & Schmid, 2010 ; Lombardi & Sinatra, 2012 ; Richter & Maier, 2017 ].

Overall, our findings show that information on research funding does not determine the way students, non-specializing in disciplines used for vignette construction, evaluate scientific results. No matter how large (or small) the funding is or whether it is mentioned at all, research results are not perceived any different in terms of their plausibility. Analogously, results obtained by scientists from prestigious universities do not seem to differ from others in terms of their perceived plausibility in the eyes of students.

The way students are indifferent to the monetary as well as to the prestige factors may mirror the difference in more idealistic representations of scientific ethos translated to students during their studies and early stage of their professional or academic careers as compared to more instrumental and efficiency-oriented practices of professional scholars. Studies show that when compared to the mid-career scientists, early-career postdoctoral trainees are more likely to subscribe to the norm of organized skepticism and less likely to the “counter norms” of particularism or self-interestedness [Anderson, Ronning, De Vries, & Martinson, 2010 ]. Alternatively, the fact that funding information had no effect could be due to the problem that for some laypersons such information cue might point to the possibility that the researchers are biased when their research is privately funded as opposed to having public funding [Critchley, 2008 ]. Since we did not specify that in our research, it hypothetically could be perceived by our participants both ways, resulting in a mutual canceling out the effect of this variable. However, we didn’t find any significant differences between the groups of respondents depending on whether there was any information provided regarding funding at all. It is also possible that such institutional factors affect plausibility judgments only when conflicting or competing claims are present, creating a need to choose the best of the alternatives [Sinatra & Lombardi, 2020 ]. These purported explanations of our findings are rather hypothetical and require further research.

While factors of research funding and institutional prestige do not seem to be relevant for students when evaluating scientific research, there is an effect of the scientific field on the way students perceive research claims in terms of their plausibility. Based on our findings, ordinary people may have an unarticulated “ranking” of scientific fields in terms of plausibility. The social sciences’ research results appear as less plausible compared to those from natural sciences. Also, there are possible “marginal” disciplines that are not recognized, as in the case of genetics, due to its relative underrepresentation in public discourse [Sidler & Jones, 2008 ]. On the other hand, there are disciplinary “superstars” like neuroscience, which generally receives a great deal of public attention, having great public support and what some authors call a “seductive allure” [Weisberg et al., 2008 ]. As for genetics, such low levels of perceived plausibility of research results may also be caused by limited awareness of this field among people in Russia. According to a survey conducted in 2018 by the National Research Institute Higher School of Economics and the Levada Center, although there is a positive trend of increasing scientific literacy in Russia, Russians show the lowest results in their knowledge of human genetics. Nearly one-third of Russians believe that “the sex of the child is determined by the genes of the father” while almost as many are certain that this is not true [Levada-Center, 2018 ].

6 Conclusions, limitations, and further research directions

In our research, we were interested in how institutional factors may influence lay plausibility perceptions of scientific findings. We addressed this question by examining whether plausibility perceptions of research results vary depending on the scientific field, research funding, and institutional prestige. We conducted two factorial surveys on two separate student samples, asking participants to read brief descriptions of scientific research and evaluate their results in terms of their plausibility. Although it allowed us to assess some of the effects and the causal relationship between the characteristics of scientific research and the perceived plausibility of its results, our conclusions are limited in several aspects.

First, since we could not keep specific research content fixed while changing different disciplines for reasons of ecological validity, and did not use more than one study from a discipline for practical limitations, our disciplinary fields were necessarily conflated with a specific research specimen selected. Also, the number of scientific fields used in our study was itself limited since we only compared social and biological sciences. It means that our “scientific field” factor needs further research. Concerning its measurement we might consider implementing more than one research for each field and/or explicitly mentioning disciplinary affiliation of scholars. We also should consider widening its range, i.e., including some research from the physical sciences.

Our results also showed the indifference of laypeople to the monetary factor as well as to the prestige factor when judging scientific findings in terms of their plausibility. This might be partly explained by our sampling as participants, being students, possibly having more idealistic representations of scientific ethos [Anderson et al., 2010 ]. Thus, further research in wider segments of the popular science readership is needed. Alternatively, it might be a question of the context of a judgment itself. It is possible that such institutional factors in the form of informational cues affect plausibility judgments only when conflicting or competing claims are present [Sinatra & Lombardi, 2020 ]. Further studies on exploring plausibility factors might implement competing claims to be presented to participants creating a need to choose the best of the alternatives.

Taken together, our results contribute to a better understanding of the determinants of ordinary judgments of scientific findings, which may be used in building more effective science communication in the future. In sum, we found significant differences between plausibility perceptions of research results depending on the scientific field of the research. We find no evident effects of the funding or institutional prestige on how research results are perceived by lay observers in terms of their plausibility. We offer possible explanations for these results that require further research. We also show that ordinary perceptions of plausibility, and perhaps of research quality in general, may depend on other “information cues” than those perceived by professional audiences, which points to the importance of investigating such effects in the public perception of science as well as further consideration in science communication practices. Our results also shows that while university affiliation of the research and its funding volume are not that important, information cues as a part of science communication that report on the findings in some fields should attract extra attention. Taking this into consideration, our results might be more useful for setting further hypotheses and future research directions.

References

Anderson, M. S., Ronning, E. A., De Vries, R., & Martinson, B. C. (2010). Extending the Mertonian norms: scientists’ subscription to norms of research. The Journal of Higher Education 81 (3), 366–393. doi: 10.1353/jhe.0.0095

Bakanic, V., McPhail, C., & Simon, R. J. (1987). The manuscript review and decision-making process. American Sociological Review 52 (5), 631–642. doi: 10.2307/2095599

Barzilai, S., & Weinstock, M. (2015). Measuring epistemic thinking within and across topics: a scenario-based approach. Contemporary Educational Psychology 42 , 141–158. doi: 10.1016/j.cedpsych.2015.06.006

Blank, L. M. (2000). A metacognitive learning cycle: a better warranty for student understanding? Science Education 84 (4), 486–506. doi: 10.1002/1098-237X(200007)84:4<486::AID-SCE4>3.0.CO;2-U

Blume-Kohout, M. E., Kumar, K. B., & Sood, N. (2009). Federal life sciences funding and university R&D [Working paper 15146]. National Bureau of Economic Research. doi: 10.3386/w15146

Bromme, R., & Gierth, L. (2021). Rationality and the public understanding of science. In M. Knauff & W. Spohn (Eds.), The handbook of rationality (pp. 767–776). Cambridge, MA, U.S.A.: MIT Press.

Chen, Y.-T., & Wang, J.-H. (2016). Analyzing with Posner’s conceptual change model and Toulmin’s model of argumentative demonstration in senior high school students’ mathematic learning. International Journal of Information and Education Technology 6 (6), 457–464. doi: 10.7763/IJIET.2016.V6.732

Connolly, L. S. (1997). Does external funding of academic research crowd out institutional support? Journal of Public Economics 64 (3), 389–406. doi: 10.1016/s0047-2727(96)01626-x

Crane, D. (1965). Scientists at major and minor universities: a study of productivity and recognition. American Sociological Review 30 (5), 699–714. doi: 10.2307/2091138

Critchley, C. R. (2008). Public opinion and trust in scientists: the role of the research context, and the perceived motivation of stem cell researchers. Public Understanding of Science 17 (3), 309–327. doi: 10.1177/0963662506070162

Critchley, C. R., Bruce, G., & Farrugia, M. (2013). The impact of commercialisation on public perceptions of stem cell research: exploring differences across the use of induced pluripotent cells, human and animal embryos. Stem Cell Reviews and Reports 9 (5), 541–554. doi: 10.1007/s12015-013-9445-4

Eriksson, K. (2012). The nonsense math effect. Judgment and Decision Making 7 (6), 746–749. Retrieved from http://journal.sjdm.org/12/12810/jdm12810.pdf

Estes, D., Chandler, M., Horvath, K. J., & Backus, D. W. (2003). American and British college students’ epistemological beliefs about research on psychological and biological development. Journal of Applied Developmental Psychology 23 (6), 625–642. doi: 10.1016/s0193-3973(03)00002-9

Føllesdal, D. (1988). Husserl on evidence and justification. In R. Sokolowski (Ed.), Edmund Husserl and the phenomenological tradition: essays in phenomenology (pp. 107–130). Washington, DC, U.S.A.: The Catholic University of America Press.

Fortin, J.-M., & Currie, D. J. (2013). Big science vs. little science: how scientific impact scales with funding. PLoS ONE 8 (6). doi: 10.1371/journal.pone.0065263

Funk, C. (2020). Key findings about Americans’ confidence in science and their views on scientists’ role in society . Pew Research Center. Washington, DC, U.S.A. Retrieved July 28, 2021, from https://www.pewresearch.org/fact-tank/2020/02/12/key-findings-about-americans-confidence-in-science-and-their-views-on-scientists-role-in-society/

Garfunkel, J. M., Ulshen, M. H., Hamrick, H. J., & Lawson, E. E. (1994). Effect of institutional prestige on reviewers’ recommendations and editorial decisions. JAMA 272 (2), 137–138. doi: 10.1001/jama.1994.03520020063017

Gauchat, G. (2012). Politicization of science in the public sphere: a study of public trust in the United States, 1974 to 2010. American Sociological Review 77 (2), 167–187. doi: 10.1177/0003122412438225

Geuna, A., & Martin, B. R. (2003). University research evaluation and funding: an international comparison. Minerva 41 (4), 277–304. doi: 10.1023/b:mine.0000005155.70870.bd

Gillespie, G. W., Chubin, D. E., & Kurzon, G. M. (1985). Experience with NIH peer review: researchers’ cynicism and desire for change. Science, Technology, & Human Values 10 (3), 44–54. doi: 10.1177/016224398501000306

Goodyer, J. (2021). 3D-printed liver models will help surgeons ‘rehearse’ cancer surgeries. Science Focus . Retrieved from https://www.sciencefocus.com/news/3d-printed-liver-models-will-help-surgeons-rehearse-cancer-surgeries

Grotzer, T., & Mittlefehldt, S. (2012). The role of metacognition in students’ understanding and transfer of explanatory structures in science. In A. Zohar & Y. J. Dori (Eds.), Metacognition in science education: trends in current research (Vol. 40, pp. 79–99). doi: 10.1007/978-94-007-2132-6_5

Heffernan, G. (2017). The concept of Krisis in Husserl’s The crisis of the European sciences and transcendental phenomenology . Husserl Studies 33 (3), 229–257. doi: 10.1007/s10743-017-9209-0

Hendriks, F., & Jucks, R. (2020). Does scientific uncertainty in news articles affect readers’ trust and decision-making? Media and Communication 8 (2), 401–412. doi: 10.17645/mac.v8i2.2824

Hmielowski, J. D., Feldman, L., Myers, T. A., Leiserowitz, A., & Maibach, E. (2014). An attack on science? Media use, trust in scientists, and perceptions of global warming. Public Understanding of Science 23 (7), 866–883. doi: 10.1177/0963662513480091

Hofer, B. K. (2000). Dimensionality and disciplinary differences in personal epistemology. Contemporary Educational Psychology 25 (4), 378–405. doi: 10.1006/ceps.1999.1026

Hofer, B. K. (2006). Domain specificity of personal epistemology: resolved questions, persistent issues, new models. International Journal of Educational Research 45 (1–2), 85–95. doi: 10.1016/j.ijer.2006.08.006

House of Lords Select Committee on Science and Technology (2000). Science and society: third report . Her Majesty’s Stationery Office. London, U.K.

Hovland, C. I., & Weiss, W. (1951). The influence of source credibility on communication effectiveness. Public Opinion Quarterly 15 (4), 635–650. doi: 10.1086/266350

Husserl, E. (1989). The crisis of European sciences and transcendental phenomenology: an introduction to phenomenological philosophy . Evanston, IL, U.S.A.: Northwestern University Press.

Iyengar, S., & Massey, D. S. (2019). Scientific communication in a post-truth society. Proceedings of the National Academy of Sciences 116 (16), 7656–7661. doi: 10.1073/pnas.1805868115

Kuehne, L. M., & Olden, J. D. (2015). Lay summaries needed to enhance science communication. Proceedings of the National Academy of Sciences 112 (12), 3585–3586. doi: 10.1073/pnas.1500882112

Levada-Center (2018, February 5). For the day of Russian science . Levada Analytical Center. In Russian. Moscow, Russia. Retrieved from https://www.levada.ru/2018/02/05/ko-dnyu-rossijskoj-nauki-2

Lombardi, D., Nussbaum, E. M., & Sinatra, G. M. (2016). Plausibility judgments in conceptual change and epistemic cognition. Educational Psychologist 51 (1), 35–56. doi: 10.1080/00461520.2015.1113134

Lombardi, D., Seyranian, V., & Sinatra, G. M. (2014). Source effects and plausibility judgments when reading about climate change. Discourse Processes 51 (1–2), 75–92. doi: 10.1080/0163853x.2013.855049

Lombardi, D., & Sinatra, G. M. (2012). College students’ perceptions about the plausibility of human-induced climate change. Research in Science Education 42 (2), 201–217. doi: 10.1007/s11165-010-9196-z

Lombardi, D., & Sinatra, G. M. (2013). Emotions about teaching about human-induced climate change. International Journal of Science Education 35 (1), 167–191. doi: 10.1080/09500693.2012.738372

Ma, A., Mondragón, R. J., & Latora, V. (2015). Anatomy of funded research in science. Proceedings of the National Academy of Sciences 112 (48), 14760–14765. doi: 10.1073/pnas.1513651112

McCarty, N. (2021). Newly studied proteins expand CRISPR’s editing range. Scientific American . Retrieved from https://www.scientificamerican.com/article/newly-studied-proteins-expand-crisprs-editing-range

Mede, N. G. (2022). Legacy media as inhibitors and drivers of public reservations against science: global survey evidence on the link between media use and anti-science attitudes. Humanities and Social Sciences Communications 9 . doi: 10.1057/s41599-022-01058-y

Muis, K. R., Bendixen, L. D., & Haerle, F. C. (2006). Domain-generality and domain-specificity in personal epistemology research: philosophical and empirical reflections in the development of a theoretical framework. Educational Psychology Review 18 (1), 3–54. doi: 10.1007/s10648-006-9003-6

Murray, D. L., Morris, D., Lavoie, C., Leavitt, P. R., MacIsaac, H., Masson, M. E. J., & Villard, M.-A. (2016). Bias in research grant evaluation has dire consequences for small universities. PLoS ONE 11 (6). doi: 10.1371/journal.pone.0155876

National Science Board (2014). Science and engineering indicators 2014 . National Science Foundation. Arlington, VA, U.S.A.

Nussbaum, E. M. (2011). Argumentation, dialogue theory, and probability modeling: alternative frameworks for argumentation research in education. Educational Psychologist 46 (2), 84–106. doi: 10.1080/00461520.2011.558816

O’Brien, T. L., & Noy, S. (2018). Cultural authority in comparative context: a multilevel analysis of trust in science and religion. Journal for the Scientific Study of Religion 57 (3), 495–513. doi: 10.1111/jssr.12537

Parsons, T. (1951). The social system . Glencoe, IL, U.S.A.: Free Press of Glencoe.

Payne, A. A., & Siow, A. (2003). Does federal research funding increase university research output? Advances in Economic Analysis & Policy 3 (1).

Peters, D. P., & Ceci, S. J. (1982). Peer-review research: objections and obligations. Behavioral and Brain Sciences 5 (2), 246–255. doi: 10.1017/s0140525x00011754

Picower Institute at MIT (2021). Neuroscientists posit that brain region is a key locus of learning. ScienceDaily . Retrieved from https://www.sciencedaily.com/releases/2021/07/210723130840.htm

Pornpitakpan, C. (2004). The persuasiveness of source credibility: a critical review of five decades’ evidence. Journal of Applied Social Psychology 34 (2), 243–281. doi: 10.1111/j.1559-1816.2004.tb02547.x

Public Opinion Foundation (2016, January 28). Online practices of Russians: social networks . Public Opinion Foundation. In Russian. Moscow, Russia. Retrieved from http://fom.ru/SMI-i-internet/12495

Richter, T., & Maier, J. (2017). Comprehension of multiple documents with conflicting information: a two-step model of validation. Educational Psychologist 52 (3), 148–166. doi: 10.1080/00461520.2017.1322968

Richter, T., & Schmid, S. (2010). Epistemological beliefs and epistemic strategies in self-regulated learning. Metacognition and Learning 5 (1), 47–65. doi: 10.1007/s11409-009-9038-4

Rigby, J. (2013). Looking for the impact of peer review: does count of funding acknowledgements really predict research impact? Scientometrics 94 (1), 57–73. doi: 10.1007/s11192-012-0779-5

Rosenbloom, J. L., Ginther, D. K., Juhl, T., & Heppert, J. A. (2015). The effects of research & development funding on scientific productivity: academic chemistry, 1990–2009. PLoS ONE 10 (9). doi: 10.1371/journal.pone.0138176

Ross, J. S., Gross, C. P., Desai, M. M., Hong, Y., Grant, A. O., Daniels, S. R., … Krumholz, H. M. (2006). Effect of blinded peer review on abstract acceptance. JAMA 295 (14), 1675–1680. doi: 10.1001/jama.295.14.1675

Scheitle, C. P., & Guthrie, S. K. (2019). Public responses to scientific research: does disciplinary attribution matter? Public Understanding of Science 28 (2), 234–245. doi: 10.1177/0963662518814365

Sidler, M., & Jones, N. (2008). Genetics interfaces: representing science and enacting public discourse in online spaces. Technical Communication Quarterly 18 (1), 28–48. doi: 10.1080/10572250802437317

Sinatra, G. M., & Lombardi, D. (2020). Evaluating sources of scientific evidence and claims in the post-truth era may require reappraising plausibility judgments. Educational Psychologist 55 (3), 120–131. doi: 10.1080/00461520.2020.1730181

Smirnov, I., Sivak, E., & Kozmina, Y. (2016). In search of lost profiles: the reliability of VKontakte data and its importance for educational research. Educational Studies 4 , 106–122. doi: 10.17323/1814-9545-2016-4-106-122

Taddicken, M., & Krämer, N. (2021). Public online engagement with science information: on the road to a theoretical framework and a future research agenda. JCOM 20 (03). doi: 10.22323/2.20030205

Tukey, J. W. (1977). Exploratory data analysis . Reading, MA, U.S.A.: Addison-Wesley.

von der Mühlen, S., Richter, T., Schmid, S., Schmidt, E. M., & Berthold, K. (2016). Judging the plausibility of arguments in scientific texts: a student-scientist comparison. Thinking & Reasoning 22 (2), 221–249. doi: 10.1080/13546783.2015.1127289

WCIOM (2020, July 20). Scientists amidst the pandemic: crisis of public trust? WCIOM. In Russian. Moscow, Russia. Retrieved from https://wciom.ru/analytical-reviews/analiticheskii-obzor/nauka-i-uchyonye-na-fone-pandemii-krizis-obshhestvennogo-doveriya

Weisberg, D. S., Keil, F. C., Goodstein, J., Rawson, E., & Gray, J. R. (2008). The seductive allure of neuroscience explanations. Journal of Cognitive Neuroscience 20 (3), 470–477. doi: 10.1162/jocn.2008.20040

Weisberg, D. S., Taylor, J. C. V., & Hopkins, E. J. (2015). Deconstructing the seductive allure of neuroscience explanations. Judgement and Decision Making 10 (5), 429–441. Retrieved from https://psycnet.apa.org/record/2015-45545-004

Wingen, T., Berkessel, J. B., & Englich, B. (2020). No replication, no trust? How low replicability influences trust in psychology. Social Psychological and Personality Science 11 (4), 454–463. doi: 10.1177/1948550619877412

Yan, E., Wu, C., & Song, M. (2018). The funding factor: a cross-disciplinary examination of the association between research funding and citation impact. Scientometrics 115 (1), 369–384. doi: 10.1007/s11192-017-2583-8

Zhao, D. (2010). Characteristics and impact of grant-funded research: a case study of the library and information science field. Scientometrics 84 (2), 293–306. doi: 10.1007/s11192-010-0191-y

Authors

Elizaveta P. Sheremet.
Ph.D. Student, Assistant, National Research University Higher School of Economics, Moscow, Russia.
E-mail: esheremet@hse.ru .
https://orcid.org/0000-0003-3543-5488 .

Inna F. Deviatko.
Dr. Sci. (Soc.), Full Professor, National Research University Higher School of Economics, Moscow, Russia.
Chief Researcher, Institute of Sociology of the Federal Center of Theoretical and Applied Sociology of the Russian Academy of Sciences, Moscow, Russia.
E-mail: deviatko@gmail.com .
https://orcid.org/0000-0002-1955-7592 .

Supplementary material

Available at https://doi.org/10.22323/2.21050201

Endnotes