Visitors come to experience science: towards a non-obtrusive evaluation method based on immersive virtual reality

1 Introduction

Evaluation is a complicated task, albeit necessary, to optimise practices and increase the impact of science communication exhibitions. In science centres and science and technology museums, evaluation may conflict with visitors’ agenda and it is often dependent on obtrusive measures.

Evaluation in science centres should be a transformational and smooth experience, rather than an obtrusive one. Non-obtrusive measures integrated into the exhibition will be compatible with visitors’ agenda and help them to make sense of the experience. Aiming at providing an experience where evaluation is integrated into the science communication process and not as an appendix, we developed a virtual reality experience prototype with a transformational play narrative to evaluate an exhibition on a cutting-edge socio-scientific issue: the deep sea. This paper aims to describe this evaluation method and provide the first indicators of its reliability and validity in the context of science centres and museums.

In the literature review, we demonstrate that science communication’s currently available evaluation practices are obtrusive or conflicting with the visitors’ agenda. We suggest that a virtual reality experience based on a transformational play narrative might be used to evaluate visitors’ experiences and the effectiveness of science communication programs in terms of public awareness, understanding, and engagement with science and technology.

Detailed information on the sample, measures, and procedures follow. After the results section, we discuss the study’s contribution to situate evaluation strategies in science centres and museums, and how the inclusion of the evaluation in science communication activities adds value for constructing a significant experience for visitors.

2 Literature review

2.1 Evaluation in museums and science centres: a research gap

Science communication initiatives that aim “to improve people’s understanding of the decision-relevant issues and promote behaviour change” [Bruine de Bruin and Bostrom, 2013 , p. 1] are very demanding in planning and making information available to the public. Evaluation becomes essential because “high-quality impact evaluation that is judiciously employed, skilfully conducted and effectively shared can provide a basis for practitioners to discover what aspects of science communication initiatives are working, in what ways, with which audiences and why” [Jensen, 2014 , p. 1]. Moreover, according to Shettel [ 2008 ], conducting visitor studies may represent a paradigm shift because the power changes hands from managers or exhibit developers to visitors, as the latter engage in exhibition analysis, sharing control with decision-makers in what should be improved. Making this participatory turn possible requires a new approach and an original method.

According to Leister et al. [ 2016 ], studies about the impact of science exhibitions can be categorised as visitor-centric or installation-centric concerning analyses’ main focus. Visitor-centric studies vary from demographic data collection to visitors’ knowledge gains assessment. Installation-centric studies indirectly evaluate visitors’ engagement by focusing on installations and exhibits’ features, such as attractiveness, usability, or education transmission. In turn, Kirchberg and Tröndle [ 2012 ] classify empirical studies on visitors’ experience into three moments of evaluation: pre-visit, comprising elements such as visitors’ expectations, social background or state of mind; during the visit, in which case evaluation parameters cover experiences, pleasure or aesthetic elements; and post-visit, focusing on learning, satisfaction or psychological well-being.

Many reasons can be raised for the prevalent lack of quality in science communication evaluation. For example, according to Jensen [ 2014 ], there is a “fiction that ‘evaluation’ and ‘research’ are completely different entities” while “there is every reason to expect both knowledge and practical guidance to emerge from the same well-designed impact evaluation” [p. 2]. One can speak of a gap between the research agenda and museums and science centres’ agenda and practice. Even in the best-resourced science communication institutions, poor quality evaluation methods are routinely applied, leading “to questionable data, specious conclusions and stunted growth in the quality and effectiveness of science communication practice” [Jensen, 2014 , p. 1].

Besides, the evaluation may clash head-on with the visitors’ expectations and objectives [Falk, Moussouri and Coulson, 1998 ]. Among the many reasons that motivate visits, visitors hardly desire to become an evaluation target and submit to psychosocial inquiries’ typical procedures. These procedures, while available, might be too obtrusive to use recursively. According to Leister et al. [ 2016 ], evaluation of science communication in museums should be as non-intrusive as possible, using non-intrusive technologies, combined, if necessary, with short, targeted questions that take place during the experience.

In a nutshell, the literature on science communication in museums and science centres evaluation indicates that currently available methods often conflict with visitors’ agenda and are obtrusive. Research to better understand exhibitions’ impact on visitors needs to incorporate multidisciplinary, multi-perspective, and multi-methods approaches [Kirchberg and Tröndle, 2012 ].

Considering that science communication, either explicit or implicitly, may raise awareness, build understanding, and promote engagement [Burns, O’Connor and Stocklmayer, 2003 ], such dimensions are significant for evaluating the impact of exhibitions on visitors, but the Gordian knot is how to operationize them. In our view, attitudes, as “a psychological tendency, that is expressed by evaluating a particular entity with some degree of favour or disfavour” [Eagly and Chaiken, 1998 , p. 269], are a good approach to assess the awareness. Conceptual and processual knowledge can also be evaluated by inquiring visitors, but often it is easier to just measure perceived knowledge. Also, there are three major problems with measuring public engagement: lack of focus on outcomes, lack of standardised instruments and tools, and the variety of approaches being adopted. Moreover, researchers acknowledge that measuring engagement’s broader impact is very difficult and would require significant investment in data collection [Hanover Research, 2014 ]. Considering that engagement measures are time-consuming [Carver et al., 2021 ] and hard to implement, perceived engagement of behavioural intention is used instead. Even when engagement (interaction with the exhibition materials) is measured, it is often challenging to obtain visitors’ data after leaving the museum or science centre.

Our idea was to develop a method integrating evaluation in the science communication process (and not afterward) so that it does not conflict with visitors’ agenda, and with a medium as non-obtrusive as possible, such as in virtual reality. Such a method should provide indicators on public awareness, understanding, and engagement with science and technology.

2.2 Virtual reality: a promise of transparent immediacy

Virtual reality has also been frequently indicated as a possibility for communicating/experiencing environmental science, as it enables users to be deeply immersed in natural landscapes otherwise unavailable [Taylor and Disinger, 1997 ; Tsivitanidou and Ioannou, 2020 ].

Virtual reality seems to be a continuously postponed promise, often presented as a new, ultimate technology, although its digital roots go back as far as the 1980’s [Lanier, 1988 ]. For authors like Bolter and Grusin [ 2000 ], virtual reality represents better than any other media the logic of transparent immediacy, providing users with a high level of presence. Due to the videogame industry, the interest in the technology has been refreshed, worth more affordable equipment and applications more available to ordinary users. Education and healthcare are, right after entertainment, the areas where more investment was expected by industry [Perkins Coie, 2018 ]. However, even in so-called “first world” countries, like Portugal, virtual reality experiences are still rarely found in science centres and, when available, are still offered as a technological experience rather than a means for conveying scientific messages [Simões, Morais and Moreira, 2019 ].

Our argument in this paper is that virtual reality might be used to evaluate science centre visitors’ experiences as good or better than other (more obtrusive) instruments. However, we needed a framework to make the virtual reality experience compatible with the visitors’ agenda. We resorted on the transformational play framework for this purpose.

2.3 Transformational play

The transformational play [Arici Barab and Barab, 2013 ; Barab, Gresalfi and Ingram-Goble, 2010 ; Barab et al., 2012 ] was first conceived as a framework to design educational videogames based on the idea of a transaction taking place between the person and their environment at a given moment [Dewey, 1963 ]. For a game to be transformational, it requires players who act intentionally with their knowledge to take actions that transform a problematic, fictional context. Besides the context, the content understanding and the player’s perception as someone who resorted to knowledge to tackle the problem is transformed by the experience [Barab, Gresalfi and Ingram-Goble, 2010 ]. Thus, the framework interconnects three core elements: person with intentionality, context with consequentiality, and content with legitimacy (Figure 1 ).

The story unfolds according to the player’s responsible choices. The content is necessary to solve the dilemmas, and the context reflects the player’s actions, giving meaningful feedback to them. These characteristics of the transformational play framework make it useful to think of the visitors experience in a museum or science centre. In this study, the virtual reality experience should allow visitors to impersonate a character acting intentionally in a context that changes according to their actions resorting on content perceived as legitimate.

Finally, to test our idea, we needed a cutting-edge socio-scientific issue. As we will see, the deep sea suits our purpose.

2.4 The deep sea: a cutting-edge socio-scientific issue

The deep sea is referred to as the last frontier because more than 80% of it remains uncharted, onobserved, and unexplored due primarily to technological limitations and exploration costs [Da Ros et al., 2019 ; Santos et al., 2018 ]. It is the largest ecosystem on Earth [Colaço et al., 2017 ; Webb, Berghe and O’Dor, 2010 ] and houses unique habitats and species [Colaço et al., 2017 ], many of them still understudied [Kennedy et al., 2019 ]. As decision-makers and the public remain mostly uninformed about the importance of deep-sea ecosystems, it is of uttermost significance to equate their environmental significance with their communication [Jobstvogt et al., 2014 ]. The deep sea is important for humans because it regulates temperature and absorbs greenhouse gases [Colaço et al., 2017 ; Jobstvogt et al., 2014 ]. Also, it has resources, besides fishing, that might be economically attractive: deep mining for minerals used in new technologies is in perspective [Santos et al., 2018 ]. For these reasons, the deep sea became an important socio-scientific issue, about which communication is challenging [Santoro et al., 2017 ; Jobstvogt et al., 2014 ]. However, through digital technologies, one can virtually reproduce its depths and experience complex interactions between environmental and contextual factors [Santoro et al., 2017 ].

2.5 Research objectives

The main goal of this paper is to describe a non-obtrusive science communication evaluation method based on an immersive virtual reality experience with a transformational play narrative and assess its concurrent validity. Concurrent validity is established by comparing a new measurement with a more established one in a similar time frame [Frey, 2018 ; McIntire and Miller, 2007 ] and for this purpose, the non-obtrusive measure will be compared with a standard questionnaire.

3 Methods

3.1 I SEA exhibition: a multimedia exhibition on the deep sea

The current investigation was part of an international project [Morais, 2020 ]. The main goal was to develop and test a non-obtrusive method to evaluate science communication of cutting-edge socio-scientific issues, such as the science of the deep sea, in the context of museums and science centres. For this purpose, we conceived and developed an multimedia exhibition to communicate the deep-sea environments and, specifically, hydrothermal vents ecosystems to the general public. Digital technologies have the potential to make science and innovation more open to citizens offering new forms of inquiry, communication, collaboration, and identity with positive cognitive, social, and emotional impacts [Tsivitanidou and Ioannou, 2020 ]. Infographics are currently popular for communicating science [Li et al., 2018 ], including marine science [Lazard and Atkinson, 2015 ], and their effectiveness at increasing the audience’s ocean knowledge and engagement with environmental issues has already been demonstrated [Huang, Li and Li, 2019 ; Lazard and Atkinson, 2015 ; Teixeira, Morais and Moreira, 2018 ] as well as in museums [Trapani, 2019 ].

The I SEA exhibition was composed of three infographics — two static ones, and one animated. The two static infographics had different main topics: one focused on the deep sea, summing up its main environmental characteristics such as temperature, pressure and light variation across different ocean layers [Ramirez-Llodra et al., 2010 ], as well as examples of Mid-Atlantic Ridge deep-sea ecosystems (e.g. coral gardens, sponge aggregations), associated species and the bioluminescence phenomenon [Colaço et al., 2017 ; Santos et al., 2006 ]; the other static infographic focused on the Azorean hydrothermal vents, explaining their main biogeochemical features, as well as their typical chemosynthesis process and associated chemical elements [Colaço et al., 2017 ; Santos et al., 2006 ].

The animated infographic lasted for about 5 minutes. It encompassed more detailed information regarding the formation and main biogeochemical characteristics of the hydrothermal vents’ ecosystem (e.g. chemosynthesis, food chain, typical species), focusing on those near the Azorean islands [Colaço et al., 2017 ], particularly the Lucky Strike hydrothermal vent field [Langmuir et al., 1997 ].

The infographics’ content definition and validation of the scientific information were based on an intensive literature review and deep-sea experts’ consultation. The content was also adjusted to fit into the virtual reality narrative to be experienced by the participants and provide them with the necessary knowledge to overcome the challenges presented. Finally, the design and implementation of the infographics were achieved through close interaction with visual design experts.

The static infographics (Figure 2 ) were exposed as roll-up banners, whereas the animated one was exposed as a video on a TV screen. The two static infographics were displayed as to be seen before the animated one, with the one about the deep sea coming in the first place.

In addition to the infographics already described, the exhibition also integrated a virtual reality experience that allowed visitors to explore the deep sea. As we describe under the next topic, the idea with the virtual reality experience was to combine communication, about the deep sea, as a cutting-edge socio-scientific issue, with evaluation of science communication. The important point to stress is that evaluation for visitors is not perceived as an extraneous activity to the exhibition but as a part integrated into the exhibition.

3.2 A non-obtrusive evaluation method

The virtual reality experience was developed in the game engine UNIT 3D [ Unity 2020 ]. The system runs in Windows 10, with head-mounted displays. An eye-tracking system was implemented [for details, see Vieira et al., 2019 ]. The system was housed in a physical capsule, whose design was inspired by submersible vehicles. Figure 3 depicts the final version of the structure hosted in one of the science centres (Expolab — Azores) and Figure 4 shows scenes of the virtual reality environment.

The virtual reality experience was presented to visitors as a journey to explore the deep sea inside a submersible system driven by the visitors. The narrative was based on the transformational play framework [Barab, Gresalfi and Ingram-Goble, 2010 ] (Figure 5 ). Visitors impersonated a fictional character (scientist, economist, environmental activist) as well as a mission of their choice to whom they assigned age and gender. During the journey, visitors interacted with a fictional traveling companion in another submersible. They were allowed to explore the virtual environment to get used to the main functions [see Vieira et al., 2019 ].

The journey immersed visitors in a threatened, fragile deep-sea ecosystem: Lucky Strike hydrothermal vent field, located in the Atlantic Ocean, near Azores archipelago. As they coordinated the submersible system, they needed to make sense of scientific data (while descending through the water column and exploring the hydrothermal vent). For example, as they descended the water column, the fictional companion suddenly reported a problem with their sensors, and asked the visitor to check if they were already in the deep sea. To answer this question visitors needed to use their knowledge to interpret the environment (absence of light and sensors indicating the depth) (Table 1 ). Scientific data was communicated in the multimedia materials described above (static and animated infographics).

---

Table 1 : Excerpt of the interaction between the fictional companion and visitors.

---

In the hydrothermal vent, they also needed to make socio-scientific decisions that affected that ecosystem’s sustainability. The dilemma was delivered to the visitor by a radio contact from the surface. A representative of the Azores Government asked for advice about whether mining activities should or should not advance in hydrothermal vents. These consisted of two rounds: first, participants were asked whether mining should occur in the hydrothermal vent fields and, secondly, whether this should be allowed in an inactive hydrothermal vent.

As visitors ascended the water column, they were also asked to talk about their own experience — as if in a dialogue with the fictional traveling companion — via an automatic interview, inside the capsule, consisting of three questions (see appendix A ). In telling their own story about the adventure, they were expected to be more likely to do it later in their daily contexts. At the end of the virtual reality experience, visitors received an automatically generated QR-Code linking to a digital “deep-sea-gram”, summarizing the visitor’s pathway through the deep-sea environment (Figure 6 ). Data on participants’ actions and visual behaviour were tracked and registered.

3.3 Hypotheses and experimental design

The hypotheses to be tested are the following:

• Participants going through the exhibition materials (static and animated infographics) before the virtual reality journey will show better understanding (knowledge) about the deep sea than participants not going through the exhibition materials. We should note that our focus on understanding is based on the design of the exhibition. Because of the exploratory nature of the topic and the discussion we wanted to promote, neither exhibition materials nor the virtual reality experience conveyed a structure message that supported an alternative directional hypothesis about the eventual changes in visitors’ awareness or engagement with the science topic.
• The non-obtrusive method will have concurrent validity compared with a standard questionnaire: participants’ results in the non-obtrusive method (collected inside the virtual reality experience) will be positively associated with their results in the questionnaire in three dimensions of science communication objectives: awareness, understanding, and engagement.

To test these two hypotheses, we employed a mixed (within and between-subjects) experimental design (see notation in Figure 7 ). Participants were randomly assigned to one of two conditions. In condition A, participants were asked to visit the exhibition on the deep sea (X) and then to make the virtual reality journey (O ${}_1$ ); in condition B, participants were sent directly to the virtual reality journey (O ${}_1$ ). In the end, all answered the questionnaire (O ${}_2$ ). Figure 8 depicts the sequence of actions in a more detailed way.

3.4 Participants

Participants were recruited through a snowball, non probabilistic sampling method. Invitations were sent to acquaintances not involved in the research (e.g., students, colleagues, friends), older than 18 years-old, asking them to resend the invitation to other potential participants. The experiment was advertised on the social networks. We tried to overcome the limitations of not using a random sample by following Rosenthal and Rosnow [ 2009 ] recommendations. Namely, (i) make the participation interesting, using enthusiastic language and taking advantage of the theme’s novelty; (ii) emphasise participants’ contribution for research; (iii) make the participation non-threatening, reassuring anonymity and confidentially and not requiring any expertise or background prerequisites. In total, 96 participants took part in the experiment. However, data from 24 participants were discarded either because of technical difficulties during the virtual reality experience or because participants did not answer the final questionnaire.

Thus, the final sample consisted of 72 participants (Male: $n=35$ ; Female: $n=37$ ), with real ages as follows: 18–29 years old $=22$ ; 30–39 $=12$ ; 40–49 $=23$ ; over 50 $=15$ . The majority ( $n=60$ ) were Portuguese. As for the educational level, 6 had up to secondary education; 36 held a degree; 19 a Master degree and 11 a Ph.D.

Participants were randomly assigned to the conditions (with exhibition exposure: $n=34$ ; without exhibition exposure: $n=38$ ).

Participants reported being already engaged with science communication activities frequently ( $M=3.80$ , $\text{SD}=0.67$ ). As for virtual reality, they reported not having experience ( $M=2.09$ , $\text{SD}=0.81$ ), but scored high on interest and perceived usefulness ( $M=4.33$ , $\text{SD}=0.81$ ).

Differences in the number of cases and degrees of freedom in subsequent analyses are due to a small number of non-responses, failure in registering the answers via the virtual reality data-gathering system, or removal of outliers from participants on a specific variable.

3.5 Measures

To test hypotheses, data were collected via (i) standard method and (ii) non-obtrusive method (see Table 2 ).

---

Table 2 : Research dimensions and measures.

---

Sociodemographic data. The standard questionnaire collected sociodemographic data (age, gender, education, proximity to the sea). In the virtual reality method, visitors’ choices about the character were also expected to mirror their data because of a psychological identification process (age and sex) [Smith, 2014 ], thus providing a proxy for sociodemographic information about participants. The participants were also asked to select the characters’ gender (male, female, other) and age (18–29, 30–39, 40–49 or >50 years).

Knowledge about the deep sea. The standard measure consisted of fifteen items about scientific contents of the deep sea, varying in the level of difficulty (easy, medium and high). Items were assessed and revised by international deep-sea experts. Participants were asked to say if they thought the statements were correct or incorrect and the score was obtained by the sum of correct answers (ranging from $0=$ none to $15=$ all answers correct). Five questions about the deep sea were integrated into the virtual reality narrative in a non-obtrusive manner (see Table 1 ), being asked by the fictional traveling companion in another submersible: — Do you think we are in the deep sea?, Is the water temperature adequate to this ocean depth?, Is the water pressure adequate to this ocean depth?, Do you think we are at the aphotic zone?, Which set of chemical elements were found in our rock sample?

Attitudes towards the deep sea. The attitudes towards the deep sea were measured via a 19 items scale, which was previously validated with a Portuguese sample [Morais et al., 2021 ], with the following answer options: $1=$ strongly disagree, $2=$ disagree, $3=$ neither agree nor disagree, $4=$ agree, $5=$ strongly agree. The scale has three components: science exploration (five items), economic exploration (seven items), and preservation (seven items). In the previous study, internal consistency analysis (Cronbach’s $\alpha$ ) revealed good values for the economic exploration (.89), preservation components (.79), (.77), and for scientific exploration (see appendix B ).

In the non-obtrusive method, the character’s choice was expected to be associated with the dimensions of the attitudes towards the deep sea (science exploration, economic exploration, and preservation). The visitor’s choice of character provided a measure of visitor awareness of the scientific endeavour.

Engagement. The questionnaire did not include measures on engagement; such information was collected via interview after the virtual reality experience (see appendix C ). In the virtual reality experience, several measures refer to engagement during the experience. For example, the decision regarding the dilemma between supporting mining or not was measured by registration of the decision and the time participants took to make the decision. We also registered the number of accesses to the deep-sea-gram (a measure of behavioural engagement after the experience and outside the science centre).

Additional measures. To characterise the sample, the questionnaire included measures about exposure to science communication and virtual reality, as we would expect that people already exposed to virtual reality would more easily navigate the system and require less habituation time. The scale on exposure to science communication consisted of three items (i.e., visiting science museums, watching science communication programs, and reading text or news about science [Paiva, Morais and Moreira, 2019 ; Paiva et al., 2020 ; Morais et al., 2021 ], with five answer options: 1 — never; 2 — seldom; 3 — sometimes; 4 — frequently; 5 — very often.

The virtual reality scale consisted of six items with five answer options (1 — strongly disagree to 5 — strongly agree). It assessed participants’ perceived experience with virtual reality (four items, e.g., I have experience with virtual reality or I have knowledge about virtual reality), as well as how interesting or useful they considered it (two items, i.e., I am interested in virtual reality, I consider virtual reality to be useful). ¹

Qualitative data. As referred before, the interview script (both within the virtual reality experience and in-person) included questions about what was the main message that they retained from their journey and what were the reasons for their choices of character and narrative decisions. These data complemented the quantitative data of the experiment.

Procedures. The questionnaire was created and administered via the university version of LimeSurvey [ 2020 ], together with a digital consent form explaining and authorizing the data collection. Quantitative data were exported and analysed using IBM SPSS Statistics version 26 [IBM, 2020 ], using t-student, chi-square, and correlation statistics. Statistical significance was specified at $p<.05$ value. As for the qualitative data, a content analysis was conducted with the support of NVivo, version 12 for Mac [QSR International, 2020 ].

4 Results

4.1 Exhibition efficacy on understanding

The score on the questionnaire’s knowledge scale of the participants that have been exposed to the exhibition ( $M=12.91$ ; $\text{SD}=1.59$ ) was higher than that of the participants that have not ( $M=11.13$ ; $\text{SD}=2.13$ ). We performed an independent samples t-test that showed the differences to be significant, $t\left(69\right)=3.93$ , $p<.001$ .

As for the scores on knowledge in the virtual reality method, the participants’ values that have been exposed to the exhibition ( $M=2.32$ ; $\text{SD}=0.79$ ) are slightly lower than that of the participants that have not ( $M=2.46$ ; $\text{SD}=0.87$ ), but the differences were not significant, $t\left(66\right)=-0.67$ , $p=.50$ .

Results from the questionnaire partially supported the claim that the exhibition has effectively communicated the research goals. However, results from the virtual reality method did not. In the next section, we will investigate the method’s concurrent validity and seek to understand the difference between the results.

4.2 Reliability of the virtual reality method

Knowledge dimension — questionnaire and virtual reality knowledge scales. In line with the results presented before, there was no correlation between the scores of the knowledge scales (questionnaire and virtual reality), $\text{rs}\left(70\right)=.07$ , $p=.58$ . Most participants did not look at the screen, displaying relevant information to answer them (ranging from 50.9% of non-lookers in analysing the chemical elements’ animation to 89.5% in the depth screen).

Moreover, most participants (87.5%) provided no real justification for their answers to the virtual reality knowledge questions, either due to the inexistence of the interview or by reported inadequacy/lack of knowledge during their choice of answers. Amongst the few respondents to questions regarding their virtual reality knowledge questions ( $n=9$ , 12.5%), most affirmed to have answered randomly ( $n=5$ , 6.9%) while others mentioned the exhibition ( $n=3$ , 4.2%) or logic ( $n=1$ , 1.4%) as bases for their answers.

Awareness dimension — choice of the character and scale of attitudes. First, we observed an association between participants’ gender and chosen characters’ gender ( ${\chi }^2\left(2\right)=39$ , $p<.001$ ) and with participants’ age and chosen characters’ ( ${\chi }^2\left(9\right)=113.12$ , $p<.001$ ).

The virtual reality data-gathering system showed that 34 participants chose the scientist, 28 the environmental activist, and seven the economist (Missing $=3$ ).

Education level seemed to be associated with the character choice, with participants having higher education levels (Master or Ph.D.) choosing scientist and those with lower education levels (up to graduation) choosing environmental activist ( ${\chi }^2\left(6\right)=16.39$ , $p=.012$ ). With age, however, such an association did not exist ( ${\chi }^2\left(6\right)=5.66$ , $p=.462$ ).

As for attitudes, participants were strongly favourable to preservation ( $M=4.34$ ; $\text{SD}=0.52$ ) and scientific exploration ( $M=4.05$ ; $\text{SD}=0.70$ ), and against economic exploration ( $M=2.07$ ; $\text{SD}=0.78$ ). Given that only seven participants chose the economist character, we further assessed if the choice of the other two characters (scientist and environmental activist) was somehow related to the attitudes.

Participants who chose the environmental activist showed more extreme attitudes in preservation ( $M=4.54$ ; $\text{SD}=0.35$ ) than participants that chose the scientist (preservation: $M=4.24$ ; $\text{SD}=0.57$ ). They also showed slightly higher scores, although still unfavourable, towards economic exploration (environmental activist: $M=2.03$ ; $\text{SD}=0.69$ ; scientist: $M=1.84$ ; $\text{SD}=0.58$ ). Moreover, they showed positive attitudes towards scientific exploration (environmental activist: $M=4.23$ ; $\text{SD}=0.42$ ; scientist: $M=4.08$ ; $\text{SD}=0.76$ ). These differences were significant only for the preservation dimension, $t\left(60\right)=2.37$ , $p=.021$ .

Concerning the reasons mentioned for participants’ choice of character, while the majority (55.6%) mentioned an identification with their chosen character (in terms of participants’ occupation or educational field), some pointed the character’s characteristics (scientist’s natural curiosity or the activist drive for preservation) as drivers of their choice (19.4%). Others affirmed to choose merely out of curiosity (9.7%) (to see how the chosen character would interact in that environment) or that they did not know/did not provide any explanation for their choice (15.3%).

Engagement dimension — decision taken, deep-sea gram and interviews. As for the decision taken, most participants first decided against mining in hydrothermal vents (77.8%) (Figure 8 ), but in the second round the majority decided in favour (52.8%).

There was no association between the chosen character and the decision taken (first decision: ${\chi }^2\left(4\right)=3.15$ , $p=.533$ ; second decision: ${\chi }^2\left(4\right)=2.50$ , $p=.644$ ). There were, however, significant correlations between some dimensions of the attitudes scale and the time spent to make decisions: the score on economic exploration was positively correlated with the time spent to take the first decision ( $\text{rs}\left(70\right)=.36$ , $p=.006$ ). A negative correlation between the score on preservation and the time spent to take the second decision ( $\text{rs}\left(70\right)=-.25$ , $p=.04$ ) was observed. Other correlations were not statistically significant.

Participants were asked to respond to a series of open questions regarding their I SEA experience. Most participants answered to this interview (95.8%) and 69.4% gave their answers while inside the virtual reality system (this moment was previously designated as an automatic interview). Six participants preferred to have their answers recorded outside the virtual reality system (8.3%), 13 preferred not to record their voices and to have the researchers writing down their answers (18.1%), and the remaining three participants did not want to answer to the interview in any form (4.2%).

Data from interviews showed that, in the first case, reasons for deciding against mining were mostly motivated by participants’ will to preserve the ecosystem (42.4%), negative representations of economic exploration (23.7%), and fear of damaging the ecosystem (13.5%). In fewer cases, motivation for deciding against mining was related to consequences of mining to the deep sea (5.1%), the unfeasibility of mining (3.4%), weighing mining pros and cons (3.4%), participants’ curiosity (3.4%), maintaining nature untouched (1.7%), needing more information to decide (1.7%), and, finally, deciding against mining because that is the politically correct decision (1.7%).

In the second moment of decision, most respondents who initially decided against mining were pro-mining, changing their initial position. Most participants considered that mining in inactive hydrothermal vent fields would be less harmful than mining in active ones (53%), seven participants (20.6%) wanted to observe what would happen to the ecosystems, four respondents (11.8%) considered the benefits of mining and two (5.9%) wanted to analyse the possibility of an alternative solution. To a lower extent, one respondent (3%) wanted to weigh mining’s pros and cons, another participant (3%) considered that mining could be an option depending on the desired outcome and finally, one respondent (3%) reported deciding in favour of mining due to technical difficulties.

Most participants did not access their deep-sea gram after leaving the experience ( $n=57$ , 79.2%). Among those who accessed it ( $n=15$ , 20.8%), most accessed it only once ( $n=13$ , 18.1%), but in two cases, the deep-sea gram was accessed more than once ( $n=2$ , 2.7%).

5 Discussion

In this paper, we focused on first steps towards developing and assessing a non-obtrusive and transformative method to evaluate science exhibitions in science centres and museums. The method was tested with an exhibition about the deep sea, a cutting-edge socio-scientific issue. Three dimensions of science communication were evaluated (awareness, understanding, and engagement) with a within and between-subjects experimental design. We hypothesised that (1) the exhibition would promote a better understanding of the deep sea and (2) participants’ results in the non-obtrusive method (collected inside the virtual reality experience) would correlate positively with their results in standard method (questionnaire and interview).

As for the first hypothesis, results from the standard questionnaire partially support the claim that the exhibition has effectively communicated science content, as participants showed increased knowledge regarding deep-sea ecosystems, but results from the non-obtrusive method did not. As for the second hypothesis, results support the reliability of the non-obtrusive method but also revealed weak spots that must be discussed.

It appears that, during the virtual reality experience, participants were paying attention to other aspects of the surrounding environment rather than the scientific content or were too bewildered to be paying attention to all the available information, thus answering mindlessly to the questions presented. This could be due to the fact that most participants had never experienced virtual reality before, and were feeling overwhelmed by the experience but were also experiencing difficulties with the joysticks. These results resemble the ones obtained with $360^{\circ }$ presentations either in the computer or virtual reality headsets that detected lower levels of focused attention and memory [Barreda-Ángeles, Aleix-Guillaume and Pereda-Baños, 2021 ]. Participants might also have experienced cognitive overload [Mayer and Moreno, 2003 ], which requires further redesign of the experience. It is not clear from this study alone whether the perception of being overwhelmed stemmed from the amount of novelty to process, dazzlement about the technology versus the content being addressed, or both. Regardless of the cognitive overload source, from a design perspective, it becomes clear that the experience needs to give participants time to become familiar with the environment but also adjust the introduction of the content into the flow of the experience to promote focused attention and prevent cognitive overload. The concepts of space and time, as used by Stiegler [ 2010 ], might contribute for this reflection. Some media, like video, are temporal in that they control the flow of the experience. The viewers do not control the way information is delivered. Other media, like photography, are spatial in that they let participants control the flow of the experience, exploring information in their own pace. The way virtual reality is designed can make the experience more or less spatial. To the challenges defined by Barab, Gresalfi and Ingram-Goble [ 2010 ] regarding the quality of the content (which might be more or less explicit), we need to add the pace of narrative unfolding, allowing participants to explore the scenario and make sense of the information. However, the drawback is that the experience becomes more time-consuming, which is not always compatible with the visitors’ agenda and the resource management of science centres. Moreover, given the scarcity of offer of virtual reality offer in Portuguese science centres [Simões, Morais and Moreira, 2019 ] and that equipment is still expensive, long individual experiences are, perhaps, less attractive for science centres.

As for the awareness dimension, the virtual reality measures showed good concurrent validity. The character’s choice reflected participants’ attitudes towards the deep sea, besides their gender and age. Choosing characters that closely depict the self is common in virtual reality. For instance, in the context of merged identities, virtual reality can be used to project the self [Jerome and Jordan, 2007 ], and this presence of the self in the game is a phenomenon with a wide array of possibilities for self-betterment, such as prejudice reduction or health management [Slater and Sanchez-Vives, 2014 ].

However, we might also reason that the nature of the exhibition could have influenced choices. Being it about science, participants could have thought that we expected them to play scientists, functioning like availability/accessibility heuristics [Tversky and Kahneman, 1974 ] or something along the lines of social desirability.

As for the decision taken, most participants first decided against mining, but in the second round, the majority decided in favour. A possible explanation is that they act out of curiosity to understand the consequences of a different choice, one that does not necessarily reflect their personal beliefs. There were, however, significant correlations between some dimensions of the attitudes scale and the time spent to make decisions: the score on economic exploration was positively correlated with the time spent to take the first decision, as well as a negative correlation between the score on preservation and the time spent to take the second decision. People’s thoughts are, in general, ruled by two subsystems [Chaiken and Trope, 1999 ; Smith and DeCoster, 2000 ]: one more automatic, peripheral, heuristic, and another more controlled, central, systematic, [e.g., Chen and Chaiken, 1999 ; Greenwald and Banaji, 1995 ; Petty and Cacioppo, 1986 ]. When processing information, they are economy-minded, putting cognitive effort only until they achieve good-enough confidence in their judgments [Judd and Park, 1993 ; Leyens, Yzerbyt and Schadron, 1992 ]. This compromise between accuracy and efficiency [Heitz, 2014 ] can be inferred from the time taken in processing information: individuals take more time answering if the threshold between actual and desired confidence in a judgment is not reached. We can infer that participants needed more time making decisions they were not confident about.

Analysing participants’ reasons for their decisions, one gets the impression that the experience enabled them to develop a critical judgment of their decisions and the deep-sea mining topic — they pose questions, admit they need to ponder on pros and cons, position themselves into the debate as many deep-sea experts and stakeholders currently do [Santos et al., 2018 ]. As such, the I SEA experience seems to have successfully engaged participants, even if superficially with deep-sea ecosystems’ long-term sustainability.

Also noteworthy is the fact that around 20% of the participants accessed the deep-sea-gram after the experience. Although the figure might not seem impressive, it shows that the strategy might be used to trace participants’ engagement.

The automatic interview showed that participants are willing to express their views, even before challenging dilemmas, such as mining or not mining in hydrothermal vent fields, showing that the evaluation method was a transformative experience [Dewey, 1963 ]. Results showed that it is possible to automatically customize questions based on visitors’ choices and systematically collect relevant qualitative feedback from visitors. Considering time and human-resources constrains that affect many science centres and museums, this is a major advantage of the method. However, data analysis is still time consuming and requires specialized skills. The challenge is to integrate automatic and reliable content analysis that might provide timely information for practioners about visitors’ experience.

6 Conclusions

Science communication faces significant challenges. As information in contemporary society seems to travel faster and to be more accessible, knowledge seems to be at risk of becoming fragmented, interest dispersed, and commitment fragile. Systematic and reliable evaluation of science exhibitions in science centres and museums might improve communication practice, but it needs to become less intrusive, more transformative, and more systematic to become significant to researchers, practioners and visitors.

This paper contributed to the field of science communication in several ways:

1. It proposed a non-obtrusive and transformational method based on the idea that visitors come to science centres to experience science. By transforming evaluation into an worthy and meaningful experience, the method meets visitors’ agenda and helps them to make sense of their visit. The method provided a map to visitors’ experience. As all maps, this one is not the territory but a help to navigate the waters.
2. It highlighted the challenges of using virtual reality for communication and evaluation purposes (considering three dimensions of science communication: public awareness, understanding, and engagement). Virtual reality is still not familiar for many visitors, but offers a window to reproduce complex or not easily accessible systems. It is critical to find a balance between the novelty of the technology (for the public), the pace and quality of the narrative, the relevance of communication goals, and the reliability of indicators.
3. It also exemplified the usefulness of testing the concurrent validity of new measures resorting to experimental designs. Experimental approaches complement descriptive, qualitative, or ethnographic studies, bringing methodological diversity to the field and allowing to infer causal relationships [Carver et al., 2021 ]. In particular, experimental studies allow to develop a critical reflection about the quality of the evidence for supporting or challenging theoretical frameworks about visitors experience in science museums. They allow to experiment in a safe environment creative solutions that might then be progressively generalized and systematically introduced into museums and science centres practices of evaluation. Generalization implies to study different profiles of visitors and to consider their specific agendas, different contexts and scientific themes.

Despite the study’s contributions to the field, several constraints were present and are still necessary to overcome. Several participants reported having technical difficulties using the virtual reality system, were constrained about talking to an automatic system, had trouble paying attention to the contents and the narrative taking place inside the virtual environment, and did not access the deep-sea gram after leaving the experience.

Moreover, in this first test of the virtual reality evaluation method, we did assess the role of social, physical, and temporal factors, which have been shown to be important for an optimal learning experience at museums [Falk and Dierking, 2000 ]. Because, in this case, virtual reality is an individual, one at the time experience, it is important to capture contextual features as well. This would imply the time-consuming assessment of 12 indicators, but Falk and Storksdieck’s [ 2005 ] results suggest that measuring prior knowledge and experiences, quality of designs, engagement with explainers, and interaction with adults would be good candidates for a reduced version.

Nevertheless, at the end of the whole experience, the vast majority of participants expressed how much they enjoyed it, and several participants discussed with the investigators a couple of aspects of the experience, from the virtual reality technology itself to the choices they made; as well as broader thematic subjects such as environmental concerns or questions about the deep sea. Post-experience focus groups would bring valuable insight as many participants seemed to want to discuss the experience further. Our proposal represents a preliminary step towards a promising way of exhibiting and evaluating science communication events.

Acknowledgments

This work was supported by Fundação para a Ciência e a Tecnologia [grant number PTDC-CED-EDG-31480-2017, UIDB/00081/2020, UTAP-EXPL/CD/0106/2017, DL57/2016/CP1359/CT0027]. The authors express their gratitude to Expolab — Centro Ciência Viva and Planetário do Porto — Centro Ciência Viva.

Appendix A Automatic interview script

In this experiment, you had the chance to get to know the deep sea a little better. What is the most important piece of information you’ve taken away from this experiment?

In this exhibition, you could choose between three professions. Why did you choose to be the [chosen character automatically]?

At a certain point, you were asked to make a decision regarding mining in hydrothermal vents. What led you to decide to [first decision made] about mining?

Appendix B Attitudes towards the deep sea

Please read the following statements.

Indicate with an X the degree of agreement with each of them according to the following scale:

• Strongly disagree;
• Disagree;
• I do not agree or disagree;
• I agree;
• Strongly agree.

1. I agree with mining in deep sea ecosystems.
2. I would vote for economic exploitation of deep sea ecosystems.
3. I would vote in favour of deep sea mining.
4. I am in favour of the exploitation of deep sea ecosystems with economic objectives.
5. I disagree on the economic exploitation of deep sea ecosystems.
6. I do not believe that mining in the ecosystems of the deep has negative effects.
7. I think humanity has the right to take advantage of what the deep sea has to offer.
8. I agree that more institutions should be set up to protect deep sea ecosystems.
9. I am in favour of allocating economic funds for the protection of the deep sea.
10. The preservation of the deep sea is something that worries me.
11. I consider it urgent that marine areas be designated as protected against human action.
12. I think that if we don’t preserve the deep seas, we are putting the human species at risk.
13. I think that everyone can take measures to avoid deep sea pollution.
14. In my opinion it is preferable to preserve rather than to advance with deep-sea mining.
15. I consider that scientific exploration of deep sea ecosystems is beneficial.
16. I would vote in favour of the scientific exploration of deep sea ecosystems.
17. I am in favour of exploring deep sea ecosystems for the purposes of scientific research.
18. I do not agree with the scientific exploration of deep sea ecosystems.
19. I would vote against scientific exploration of deep sea ecosystems.
20. I believe that scientific exploration of deep sea ecosystems has harmful effects.
21. I do not consider it essential to create more measures for the regulation of marine areas.
22. I consider it urgent to take initiatives to reduce marine pollution.
23. I believe that combating pollution on the seas is not a priority at this moment.
24. I believe that humanity must limit its action in the deep sea.
25. In my opinion, mankind should not interfere at all with the deep sea.
26. I do not think we should advance with ore mining in the deep sea.

Appendix C Interview after the virtual reality experience (standard measure)

Can you explain your decisions about mining in hydrothermal vents?

Do you think the exhibition helped you to answer the questions? If so, how?

How did you decide regarding the questions you have been asked?

Did you have any knowledge about the deep sea?

References

Arici Barab, A. and Barab, S. (2013). ‘Transformational play; using 3D game-based narratives to immerse students in literacy learning’. In: 7th European Conference on Games Based Learning, ECGBL 2013 (Porto, Portugal, 3rd–4th October 2013), pp. 35–44.

Barab, S. A., Gresalfi, M. and Ingram-Goble, A. (2010). ‘Transformational play: using games to position person, content, and context’. Educational Researcher 39 (7), pp. 525–536. https://doi.org/10.3102/0013189x10386593 .

Barab, S. A., Pettyjohn, P., Gresalfi, M., Volk, C. and Solomou, M. (2012). ‘Game-based curriculum and transformational play: designing to meaningfully positioning person, content, and context’. Computers & Education 58 (1), pp. 518–533. https://doi.org/10.1016/j.compedu.2011.08.001 .

Barreda-Ángeles, M., Aleix-Guillaume, S. and Pereda-Baños, A. (2021). ‘Virtual reality storytelling as a double-edged sword: immersive presentation of nonfiction 360-video is associated with impaired cognitive information processing’. Communication Monographs 88 (2), pp. 154–173. https://doi.org/10.1080/03637751.2020.1803496 .

Bolter, J. D. and Grusin, R. (2000). Remediation: understanding new media. Cambridge, U.S.A.: MIT Press.

Bruine de Bruin, W. and Bostrom, A. (2013). ‘Assessing what to address in science communication’. Proceedings of the National Academy of Sciences 110 (Supplement 3), pp. 14062–14068. https://doi.org/10.1073/pnas.1212729110 .

Burns, T. W., O’Connor, D. J. and Stocklmayer, S. M. (2003). ‘Science communication: a contemporary definition’. Public Understanding of Science 12 (2), pp. 183–202. https://doi.org/10.1177/09636625030122004 .

Carver, A., Garner, J. K., Kaplan, A. and Pugh, K. J. (2021). ‘Visitors’ attendance motivation and meaning making at a public science event’. International Journal of Science Education, Part B 11 (1), pp. 75–89. https://doi.org/10.1080/21548455.2021.1874620 .

Chaiken, S. and Trope, Y., eds. (1999). Dual-process theories in social psychology. New York, NY, U.S.A.: Guilford Press.

Chen, S. and Chaiken, S. (1999). ‘The heuristic-systematic model in its broader context’. In: Dual-process theories in social psychology. Ed. by S. Chaiken and Y. Trope. New York, NY, U.S.A.: Guilford Press, pp. 73–96.

Colaço, A., Carreiro e Silva, M., Giacomello, E., Gordo, L., Vieira, A. R., Adão, H., Gomes-Pereira, J. N., Menezes, G. and Barros, I. (2017). Ecossistemas do Mar Profundo. Lisboa, Portugal: DGRM — Direção-Geral de Recursos Naturais, Segurança e Serviços Marítimos.

Da Ros, Z., Dell’Anno, A., Morato, T., Sweetman, A. K., Carreiro-Silva, M., Smith, C. J., Papadopoulou, N., Corinaldesi, C., Bianchelli, S., Gambi, C., Cimino, R., Snelgrove, P., Van Dover, C. L. and Danovaro, R. (2019). ‘The deep sea: the new frontier for ecological restoration’. Marine Policy 108, 103642. https://doi.org/10.1016/j.marpol.2019.103642 .

Dewey, J. (1963). Experience and education. New York, NY, U.S.A.: Collier Books.

Eagly, A. H. and Chaiken, S. (1998). ‘Attitude structure and function’. In: The handbook of social psychology. Ed. by D. T. Gilbert, S. T. Fiske and G. Lindzey. 4th ed. New York, NY, U.S.A.: McGraw-Hill, pp. 269–322.

Falk, J. H. and Dierking, L. D. (2000). Learning from museums: visitor experiences and the making of meaning. Walnut Creek, CA, U.S.A.: AltaMira Press.

Falk, J. H., Moussouri, T. and Coulson, D. (1998). ‘The effect of visitors’ agendas on museum learning’. Curator: The Museum Journal 41 (2), pp. 107–120. https://doi.org/10.1111/j.2151-6952.1998.tb00822.x .

Falk, J. H. and Storksdieck, M. (2005). ‘Using the contextual model of learning to understand visitor learning from a science center exhibition’. Science Education 89 (5), pp. 744–778. https://doi.org/10.1002/sce.20078 .

Field, A. (2017). Discovering statistics using IBM SPSS statistics: North American edition. London, U.K.: SAGE Publications.

Frey, B. B. (2018). The SAGE encyclopedia of educational research, measurement, and evaluation. Thousand Oaks, CA, U.S.A.: SAGE Publications. https://doi.org/10.4135/9781506326139 .

Greenwald, A. G. and Banaji, M. R. (1995). ‘Implicit social cognition: attitudes, self-esteem, and stereotypes’. Psychological Review 102 (1), pp. 4–27. https://doi.org/10.1037/0033-295x.102.1.4 .

Hanover Research (2014). Public engagement strategies and evaluation . URL: https://www.hanoverresearch.com/media/Public-Engagement-Strategies-and-Evaluation.pdf .

Heitz, R. P. (2014). ‘The speed-accuracy tradeoff: history, physiology, methodology, and behavior’. Frontiers in Neuroscience 8, 150. https://doi.org/10.3389/fnins.2014.00150 .

Huang, G., Li, K. and Li, H. (2019). ‘Show, not tell: the contingency role of infographics versus text in the differential effects of message strategies on optimistic bias’. Science Communication 41 (6), pp. 732–760. https://doi.org/10.1177/1075547019888659 .

IBM (2020). IBM SPSS Software . URL: https://www.ibm.com/analytics/spss-statistics-software .

Jensen, E. (2014). ‘The problems with science communication evaluation’. JCOM 13 (01), C04. https://doi.org/10.22323/2.13010304 .

Jerome, L. W. and Jordan, P. J. (2007). ‘Psychophysiological perspective on presence: the implications of mediated environments on relationships, behavioral health and social construction’. Psychological Services 4 (2), pp. 75–84. https://doi.org/10.1037/1541-1559.4.2.75 .

Jobstvogt, N., Hanley, N., Hynes, S., Kenter, J. and Witte, U. (2014). ‘Twenty thousand sterling under the sea: estimating the value of protecting deep-sea biodiversity’. Ecological Economics 97, pp. 10–19. https://doi.org/10.1016/j.ecolecon.2013.10.019 .

Judd, C. M. and Park, B. (1993). ‘Definition and assessment of accuracy in social stereotypes’. Psychological Review 100 (1), pp. 109–128. https://doi.org/10.1037/0033-295x.100.1.109 .

Kaiser, H. F. (1974). ‘An index of factorial simplicity’. Psychometrika 39 (1), pp. 31–36. https://doi.org/10.1007/bf02291575 .

Kennedy, B. R. C., Cantwell, K., Malik, M., Kelley, C., Potter, J., Elliott, K., Lobecker, E., Gray, L. M., Sowers, D., White, M. P., France, S. C., Auscavitch, S., Mah, C., Moriwake, V., Bingo, S. R. D., Putts, M. and Rotjan, R. D. (2019). ‘The unknown and the unexplored: insights into the Pacific deep-sea following NOAA CAPSTONE expeditions’. Frontiers in Marine Science 6, 480. https://doi.org/10.3389/fmars.2019.00480 .

Kirchberg, V. and Tröndle, M. (2012). ‘Experiencing exhibitions: a review of studies on visitor experiences in museums’. Curator: The Museum Journal 55 (4), pp. 435–452. https://doi.org/10.1111/j.2151-6952.2012.00167.x .

Langmuir, C., Humphris, S., Fornari, D., Van Dover, C., Von Damm, K., Tivey, M. K., Colodner, D., Charlou, J.-L., Desonie, D., Wilson, C., Fouquet, Y., Klinkhammer, G. and Bougault, H. (1997). ‘Hydrothermal vents near a mantle hot spot: the Lucky Strike vent field at 37N on the Mid-Atlantic Ridge’. Earth and Planetary Science Letters 148 (1–2), pp. 69–91. https://doi.org/10.1016/s0012-821x(97)00027-7 .

Lanier, J. (1988). ‘A vintage virtual reality interview’. Whole Earth Review . URL: http://www.jaronlanier.com/vrint.html .

Lazard, A. and Atkinson, L. (2015). ‘Putting environmental infographics center stage: the role of visuals at the elaboration likelihood model’s critical point of persuasion’. Science Communication 37 (1), pp. 6–33. https://doi.org/10.1177/1075547014555997 .

Leister, W., Tjøstheim, I., Schulz, T., Joryd, G., Larssen, A. and de Brisis, M. (2016). ‘Assessing visitor engagement in science centres and museums’. International Journal on Advances in Life Sciences 8 (1&2), pp. 50–64.

Leyens, J.-P., Yzerbyt, V. Y. and Schadron, G. (1992). ‘The social judgeability approach to stereotypes’. European Review of Social Psychology 3 (1), pp. 91–120. https://doi.org/10.1080/14792779243000032 .

Li, N., Brossard, D., Scheufele, D. A., Wilson, P. H. and Rose, K. M. (2018). ‘Communicating data: interactive infographics, scientific data and credibility’. JCOM 17 (02), A06. https://doi.org/10.22323/2.17020206 .

LimeSurvey (2020). URL: http://www.limesurvey.org/ .

Mayer, R. E. and Moreno, R. (2003). ‘Nine ways to reduce cognitive load in multimedia learning’. Educational Psychologist 38 (1), pp. 43–52. https://doi.org/10.1207/s15326985ep3801_6 .

McIntire, S. A. and Miller, L. A. (2007). Foundations of psychological testing: a practical approach. 2nd ed. Thousand Oaks, CA, U.S.A.: SAGE Publications.

Morais, C. (2020). I SEA Project — Immersive virtual reality environments to evaluate audience attitudes about science communication projects . Grant no. UTAP-EXPL/CD/0106/2017. Fundação para a Ciência e a Tecnologia. URL: https://www.fc.up.pt/isea/ .

Morais, C., Moreira, L., Teixeira, A. S. and Aguiar, T. (2021). ‘No waves from surface knowledge: diving into the social representations of the deep sea’. International Journal of Science Education, Part B . https://doi.org/10.1080/21548455.2021.2017507 .

Paiva, J. C., Morais, C. and Moreira, L. (2019). ‘If neither from evolution nor from the Bible, where does tension between science and religion come from? Insights from a survey with high school students in a Roman Catholic society’. In: Science and religion in education. Ed. by B. Billingsley, K. Chappell and M. J. Reiss. Cham, Switzerland: Springer International Publishing, pp. 277–290. https://doi.org/10.1007/978-3-030-17234-3_21 .

Paiva, J. C., Rosa, M., Moreira, J. R., Morais, C. and Moreira, L. (2020). ‘Science-religion dialogue in education: religion teachers’ perceptions in a Roman-Catholic context’. Research in Science Education . https://doi.org/10.1007/s11165-020-09941-x .

Perkins Coie (2018). 2018 Augmented and virtual reality survey report. Industry insights into the future of AR/VR . URL: https://www.perkinscoie.com/images/content/1/8/v2/187785/2018-VR-AR-Survey-Digital.pdf .

Petty, R. E. and Cacioppo, J. T. (1986). ‘The elaboration likelihood model of persuasion’. Advances in Experimental Social Psychology 19, pp. 123–205. https://doi.org/10.1016/S0065-2601(08)60214-2 .

QSR International (2020). NVivo . URL: https://www.qsrinternational.com/nvivo-qualitative-data-analysis-software/home/ (visited on 29th December 2020).

Ramirez-Llodra, E., Brandt, A., Danovaro, R., De Mol, B., Escobar, E., German, C. R., Levin, L. A., Martinez Arbizu, P., Menot, L., Buhl-Mortensen, P., Narayanaswamy, B. E., Smith, C. R., Tittensor, D. P., Tyler, P. A., Vanreusel, A. and Vecchione, M. (2010). ‘Deep, diverse and definitely different: unique attributes of the world’s largest ecosystem’. Biogeosciences 7 (9), pp. 2851–2899. https://doi.org/10.5194/bg-7-2851-2010 .

Rosenthal, R. and Rosnow, R. L. (2009). ‘The volunteer subject’. In: Artifacts in behavioral research. Ed. by R. Rosenthal and R. L. Rosnow. Classic books. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195385540.003.0003 .

Santoro, F., Santin, S., Scowcroft, G., Fauville, G. and Tuddenham, P. (2017). Ocean literacy for all: a toolkit. Vol. 80. IOC Manuals and Guides. UNESCO Publishing.

Santos, M. M., Jorge, P. A. S., Coimbra, J., Vale, C., Caetano, M., Bastos, L., Iglesias, I., Guimarães, L., Reis-Henriques, M. A., Teles, L. O., Vieira, M. N., Raimundo, J., Pinheiro, M., Nogueira, V., Pereira, R., Neuparth, T., Ribeiro, M. C., Silva, E. and Castro, L. F. C. (2018). ‘The last frontier : coupling technological developments with scientific challenges to improve hazard assessment of deep-sea mining’. Science of The Total Environment 627, pp. 1505–1514. https://doi.org/10.1016/j.scitotenv.2018.01.221 .

Santos, R. S., Gomes, C. d. L. C., Porteiro, F. M. and Gallagher, L. (2006). Centro de Interpretação Marinha Virtual (CIMV) — Uma viagem emocionante aos ecossistemas marinhos dos Açores . DVD interactivo educacional. ImagDOP/Universidade dos Açores.

Shettel, H. (2008). ‘No visitor left behind’. Curator: The Museum Journal 51 (4), pp. 367–375. https://doi.org/10.1111/j.2151-6952.2008.tb00323.x .

Simões, R., Morais, C. and Moreira, L. (2019). ‘Multimedia and virtual reality into communication practices of science centers: a social representations perspective’. In: ICERI2019 Proceedings . 12th annual International Conference of Education, Research and Innovation (Seville, Spain, 11th–13th November 2019). IATED, pp. 4746–4752. https://doi.org/10.21125/iceri.2019.1165 .

Slater, M. and Sanchez-Vives, M. V. (2014). ‘Transcending the self in immersive virtual reality’. Computer 47 (7), pp. 24–30. https://doi.org/10.1109/mc.2014.198 .

Smith, E. R. and DeCoster, J. (2000). ‘Dual-process models in social and cognitive psychology: conceptual integration and links to underlying memory systems’. Personality and Social Psychology Review 4 (2), pp. 108–131. https://doi.org/10.1207/s15327957pspr0402_01 .

Smith, K. S. (2014). ‘Projecting the self into a virtual world’. EKU Libraries Research Award for Undergraduates , 1. URL: http://encompass.eku.edu/ugra/2014/2014/1 .

Stiegler, B. (2010). Taking care of youth and the generations. Stanford, CA, U.S.A.: Stanford University Press.

Taylor, G. L. and Disinger, J. F. (1997). ‘The potential role of virtual reality in environmental education’. The Journal of Environmental Education 28 (3), pp. 38–43. https://doi.org/10.1080/00958964.1997.9942828 .

Teixeira, A., Morais, C. and Moreira, L. (2018). ‘Digital infographics on marine litter: social representations and science communication’. In: EDULEARN18 Proceedings . 10th International Conference on Education and New Learning Technologies (Palma, Spain, 2nd–4th July 2018). IATED, pp. 6772–6781. https://doi.org/10.21125/edulearn.2018.1607 .

Trapani, V. M. V. (2019). ‘Communication design for the dissemination of scientific knowledge. Languages, tools, technologies, collaborative processes for museum education’. In: Proceedings of the 1st International and Interdisciplinary Conference on Digital Environments for Education, Arts and Heritage. EARTH 2018 . Ed. by A. Luigini. Cham, Switzerland: Springer, pp. 716–725. https://doi.org/10.1007/978-3-030-12240-9_73 .

Tsivitanidou, O. E. and Ioannou, A. (2020). ‘Citizen Science, K-12 science education and use of technology: a synthesis of empirical research’. JCOM 19 (04), V01. https://doi.org/10.22323/2.19040901 .

Tversky, A. and Kahneman, D. (1974). ‘Judgment under uncertainty: heuristics and biases’. Science 185 (4157), pp. 1124–1131. https://doi.org/10.1126/science.185.4157.1124 .

Unity (2020). URL: https://unity.com/ (visited on 29th December 2020).

Vieira, J., Nóbrega, R., Pereira, V., Coelho, A., Jacinto, A. and Morais, C. (2019). ‘Knowledge analysis automatic evaluation in virtual reality immersive experiences’. In: ICGI 2019 Proceedings . 2019 International Conference on Graphics and Interaction (ICGI) (Faro, Portugal, 21st–22nd November 2019). IEEE, pp. 122–129. https://doi.org/10.1109/icgi47575.2019.8955099 .

Webb, T. J., Berghe, E. V. and O’Dor, R. (2010). ‘Biodiversity’s big wet secret: the global distribution of marine biological records reveals chronic under-exploration of the deep pelagic ocean’. PLoS ONE 5 (8), e10223. https://doi.org/10.1371/journal.pone.0010223 .

Authors

Carla Morais has a degree in Chemistry, a Master in Multimedia Education and a Ph.D. in Science Education and Communication from the Faculty of Science of the University of Porto. She is an Assistant Professor and member of the Science Education Unit at the same Faculty. She is also a member of the Centre for Research in Chemistry at the University of Porto (CIQUP — RG5: Education, Science Communication and Society).
Affiliation: Chemistry Research Center (CIQUP), University of Porto, Porto, Portugal; Science Teaching Unit (UEC), University of Porto, Porto, Portugal; Department of Chemistry and Biochemistry of the Faculty of Sciences, University of Porto, Porto, Portugal. E-mail: cmorais@fc.up.pt .

Luciano Moreira received his Ph.D. in Digital Media from the University of Porto in 2021. He is an integrated researcher at CIQUP (RG5: Education, Science Communication and Society). He teaches at the Faculty of Engineering of the University of Porto in the Master in Multimedia. His areas of interest include representations and practices on digital media and science, technological and digital ecologies and scientific research methods.
Affiliation: Chemistry Research Center (CIQUP), University of Porto, Porto, Portugal; Department of Informatics Engineering of the Faculty of Engineering, University of Porto, Porto, Portugal. E-mail: lucianomoreira@fe.up.pt .

Ana Teixeira has a degree in Biology from the Faculty of Science of the University of Porto and a Master in Multimedia with a specialization in Education from the Faculty of Engineering of the same university. She has been doing research focused on science communication via infographics and virtual reality about marine environmental topics and investigating how such affects audiences’ attitudes and social representations.
Affiliation: Chemistry Research Center (CIQUP), University of Porto, Porto, Portugal. E-mail: ana.teixeira@fc.up.pt .

Teresa Aguiar has a degree in Developmental Psychology and Child Education and a master in Psychology Themes from the Faculty of Psychology and Education Sciences of the University of Porto (FPCEUP). She has worked in research projects in the areas of Early Childhood Education Quality in FPCEUP and Science Communication in the Faculty of Sciences of University of Porto.
Affiliation: Chemistry Research Center (CIQUP), University of Porto, Porto, Portugal. E-mail: teresa.aguiar@fc.up.pt .

António Fernando Coelho is an Associate Professor with Habilitation at the Department of Informatics Engineering of the Faculty of Engineering (FEUP), University of Porto (UP), where he coordinates the Doctoral Program in Digital Media at the University of Porto and is one of the academic Leaders of the EUGLOH: European University Alliance. He is also a Senior Researcher at INESC TEC where he coordinates of the Computer Graphics and Virtual Environments area.
Affiliation: Department of Informatics Engineering of the Faculty of Engineering, University of Porto, Porto, Portugal. INESC Tecnologia e Ciência (INESC TEC). E-mail: acoelho@fe.up.pt .

Vasco Magalhães Pereira obtained his Master in Informatics Engineering from the Faculdade de Engenharia da Universidade do Porto in 2019. He is currently a game developer engineer.
Affiliation: Faculty of Engineering, University of Porto, Porto, Portugal. E-mail: vasco_mag_pereira@hotmail.com .

Alexandre Jacinto is a Bachelor in Design by Escola Superior de Artes e Design (ESAD) and the Multimedia Course Coordinator at ESAD.
Affiliation: Esad-idea, Research in Design and Art, Matosinhos, Portugal. E-mail: alexandrejacinto@esad.pt .

Marta Varzim has a Ph.D. in Arts from the Universidade de Aveiro. She is Associate Professor at the Escola Superior de Artes e Design (ESAD) and director of the Degree in Arts at ESAD.
Affiliation: Esad-idea, Research in Design and Art, Matosinhos, Portugal. E-mail: martavarzim@esad.pt .

João Carlos de Matos Paiva is Associate Professor in the Chemistry Department (Education) of the Faculty of Sciences of the University of Porto — Portugal. He is the coordinator of the research group Education, Science Communication and Society in the Centre for Research in Chemistry of the University of Porto (CIQUP).
Affiliation: Chemistry Research Center (CIQUP), University of Porto, Porto, Portugal; Science Teaching Unit (UEC), University of Porto, Porto, Portugal; Department of Chemistry and Biochemistry of the Faculty of Sciences, University of Porto, Porto, Portugal. E-mail: jcpaiva@fc.up.pt .

Miriam Rosa is a researcher and professor at Iscte-University Institute of Lisbon (CIS). She is a social and organizational psychologist, conducting research and training on relations between social groups of asymmetric status, and processes of social influence.
Affiliation: Instituto Universitário de Lisboa (ISCTE-IUL), CIS-IUL, Lisboa, Portugal. E-mail: miriam.rosa@iscte-iul.pt .

Endnotes