A standard for public consultation on science communication: the CONCISE project experience

1 Context

Over the last two decades, novel science and innovation spaces have gone beyond commitment to stakeholders to include members of the general public. Discussions about the relationship between science and society have attached importance to the “democratisation of science”, a concept through which science and technology issues are perceived as a responsibility shared among a broad range of stakeholders, including citizens [Stilgoe, Owen and Macnaghten, 2013 ]. Society has begun to be perceived as one of the four major actors in the innovation system, as stated by the quadruple helix model of innovation [Schütz, Heidingsfelder and Schraudner, 2019 ].

During this period, citizen participation activities have begun to be included in science and technology policy decision-making, for example, in the case of genetically modified food [Berg and Lidskog, 2018 ]. The democratisation of science means paying at least as much attention to the societal outcomes of research as to the scientific and technological findings. However, the radical democratisation of science, which includes a broader set of proposals for public responses to existing scientific framings of environmental issues, could lead to identification of unforeseen environmental risks or even modify understanding of what constitutes such a risk [Huttunen and Hildén, 2014 ; Irwin and Michael, 2003 ].

Moreover, public involvement in the policy decision-making process may foster consensus and public acceptability, as Ward et al. [ 2019 ] reported in relation to France’s citizen consultation on vaccination, whereby 10,435 public comments posted to the consultation’s official website were analysed by researchers. Increasing citizen participation in decision-making about vaccinations may increase citizens’ sense of democratic participation.

Recent years have also seen the emergence of informal links between society and science policies, such as the #FridaysForFuture phenomenon initiated by Greta Thunberg, which has engaged unprecedented numbers of youth to join the climate movement around the world [Fisher, 2019 ]. The power of this growing movement stems, not only from its potential impact on climate policy, but also from its creation of a cohort of citizens who will become active participants in democracy.

Also growing in popularity are “dialogue events” such as citizen participation activities in science and technology not intended to inform public policy [Davies et al., 2009 ]. Active citizen participation using different forms of knowledge to find the solution to a given problem fulfils the reported will of society to acquire more responsibility [Rogers, 2006 ]. Citizens are also given a vision of a shared future [Jacobi et al., 2017 ] and a feeling of belonging [Calder, 2002 ]. Bernauer and Gampfer [ 2013 ] report a clear example of this phenomenon, finding that the popular legitimacy of global climate governance increases or decreases according to whether civil society is involved or excluded.

A number of different models of citizen involvement in science and technology-related issues exist. Rowe and Frewer [ 2005 ] identified three aspects of public engagement: public communication, public consultation and public participation. According to the Organisation for Economic Co-operation and Development [Rodrigo and Amo, 2008 ], “Public consultation is one of the key regulatory tools employed to improve transparency, efficiency and effectiveness of regulation”. The OECD further specifies that the consultation process involves “actively seeking the opinions of interested and affected groups. It is a “two-way flow of information” [Rodrigo and Amo, 2008 ].

In public consultation, information elicited from the public is thought to be representative of currently held opinions on the topic in question. Relevant information from the organiser can also be maximised through efficient information processing by participants [Rowe and Frewer, 2005 ]. This makes public consultation quite a useful tool for gathering information from a considerable number of people on a specific topic.

Other novel models of consultation stress the “dialogue approach”, in which interactions between science and citizens are mutually informing or symmetrical [Davies et al., 2009 ]. These consultation strategies shape discussions, highlight personal aspects of the consulting issue, and may affect how professionals view the public [Armstrong et al., 2020 ]. As a result, some of these consultation processes have been designed to engage the audience with specific science-related issues without necessarily being part of a policy process [Davies et al., 2009 ]. The World Wide Views consultation [Blue and Medlock, 2014 ], for example, was the first initiative to organise large-scale international public debate.

This article addresses a specific type of deliberative citizen consultation, designed within the framework of the European CONCISE (Communication role on perception and beliefs of European citizens about science) project. The main aim of the project was to provide qualitative knowledge about the means and channels (media and social media networks, life experience, relatives, religion, political ideology and educational system) through which European citizens acquire their science-related knowledge, and how this knowledge influences their beliefs, opinions and perceptions with respect to four socially relevant topics: vaccines, complementary and alternative medicine, genetically modified organisms and climate change.

In 2019, the partners involved in the CONCISE project carried out five citizen consultations in five European countries (Italy, Poland, Portugal, Slovakia and Spain) to explore understanding of science communication with a total of 497 individuals. In designing these consultations, the consortium partners were inspired by the World Wild Views consultation approach [Blue and Medlock, 2014 ; Rask and Worthington, 2017 ; Riedy and Herriman, 2011 ]. The five consultations were organised in their respective countries according to a standard format: one hundred citizens, selected to represent the diversity of their society, were gathered into small groups together with a moderator/facilitator to discuss an identical set of questions on four selected topics [Delicado et al., 2022 ; Delicado, Rowland and Estevens, 2021 ; Moreno-Castro, Mendoza-Poudereux and Vengut-Climent, 2020 ; Krzewińska et al., 2021 ].

The design of the consultation methodology sought to answer the following research questions:

• How are citizens informed about the four selected topics: vaccines, climate change, GMO and alternative medicine?
• Which information sources do citizens consider “reliable” for each topic?
• How do citizens propose improving scientific communication regarding the four topics?

The main objective of this paper is to present and reflect on the methodology used in the CONCISE project to carry out citizen consultations. Thus, the main benefit considered by the authors is the practical knowledge acquired after this experience that is shared in this article. Based on the international experiences of the CONCISE project partner consortium, a standard for conducting public consultation on science communication was developed and tested in a European context. We believe that thanks to this paper’s reflections, the replication, and even the improvement, of citizen consultations of this type can be promoted.

2 The CONCISE project methodology: insights gained from practical experience

The consultation in each country was held on a single day, with researchers acting as discussion moderators in small citizen groups, using a methodology similar to a focus group approach. Focus groups are a consultation method designed to collect data through group interaction and illuminate citizen perceptions that may be missed using other survey techniques [Rothwell, Anderson and Botkin, 2016 ; Shipley and Utz, 2012 ]. One of the most critical issues for the CONCISE partners was to promote a feeling of participation in an initiative with a global reach, as well as offer a space in which participants could express their proposals, demands and expectations of communication on the four topics.

The CONCISE consultations were designed to answer the research questions presented above as well as provide a participatory experience for all the attendees. In addition to seeking information, we also wished to offer citizens a unique opportunity to be empowered through actively participating and providing ideas for possible solutions. Joining in an experience with 100 people for a whole day generates a greater degree of involvement than investing an hour or two in a group discussion. At each CONCISE consultation, all the citizen participants interacted with the dozen people in their group in addition to sharing the common spaces with the rest of the 100 participants at mealtimes and during the welcome and closing receptions, for example. Thus, during the event, citizens were able to contribute to the development of science-related issues, and they knew that their contribution would help to prompt specific practical actions to improve communication on socially relevant topics. The citizens’ felt empowered by being asked for their opinion, which would then be transmitted to science communication stakeholders. During the consultations the citizens not only shared information, they also influenced each other’s opinions and changed their statements accordingly, adding to their feelings of efficacy. Moreover, the very opportunity to take part in such an event for the first time offered the citizens a space to increase their disposition to participation. Personal benefits may also derive from participating in this type of consultation. By taking part in a singular event, the citizen participants had the opportunity to exercise their democratic rights, expand their perspective and knowledge, express their opinions freely, make suggestions and be listened to.

With these objectives, the focus group methodology was adapted by gathering a considerable number of people in a single place, where they were divided into informal small-group discussions focused on a particular topic or set of issues [Wilkinson, 2004 ]. This approach enabled data to be collected from multiple individuals simultaneously in a non-threatening environment that invited participants to discuss their perceptions, ideas, opinions and thoughts [Hennink, 2014 ].

The benefits of using the discussion group method for public consultation on science communication are many. Group discussion strategies are an economical, fast and efficient method of obtaining data from multiple participants [Krueger and Casey, 2000 ], thereby potentially increasing the overall number of participants in a given qualitative study [Onwuegbuzie et al., 2009 ]. The group members take part in a participatory, decision-making process; they are considered knowledgeable about the study subject and participate in the research work [Krueger and Casey, 2000 ]. The sense of belonging to a group can also increase participants’ sense of cohesiveness [Peters, 1993 ] and help them to feel safe sharing information [Hennink, 2014 ]. Furthermore, interactions that occur among participants can yield essential data [Kitzinger, 1994 ], foster more spontaneous responses [Wilkinson, 2004 ] and provide a setting in which participants can discuss personal problems and offer possible solutions [Onwuegbuzie et al., 2009 ].

The innovative nature of the CONCISE public consultations derives from the fact that participants were asked, not only how they obtained scientific information and how they perceived it, but also how they believed the scientific content should be communicated. At the end of each round of the discussions, participants were asked, among other things, how they would like to receive information about a topic that they had discussed, the communication channels they preferred, the kind of information they would be looking for, and the type of format they found most useful.

2.1 Large-scale citizen consultation characteristics

In each of the five countries, all four topics — vaccines, climate change, genetically modified organisms and alternative medicine — were discussed in a single consultation. A total of 100 attendees participated in each consultation, where they were organised into subgroups and the different discussions were carried out in parallel.

The sessions were divided into two blocks, morning and afternoon, which alternated between environmental and health discussions to make the debate more dynamic. In four out of the five countries, the same group of participants remained in their assigned places throughout all four rounds of the discussion. However, in the fifth country, the consultation groups were reorganised during the lunch break, and a new set of participant groups was established, the idea being to avoid monotony and provide participants and facilitators with an additional energy boost. The organisers who opted not to change the group composition did so in the belief that staying with the same group provided participants with a feeling of security and comfort.

At the beginning of the consultation, an ice-breaking activity was carried out to enable participants, moderators and observers to introduce themselves. This was followed by a short coffee break before commencing the discussion on the first topic. Coffee and lunch were provided by the consultation organisers.

2.1.1 Preparations for public consultation

To address the organisational needs of the public consultations, the CONCISE project consortium produced Guidelines , which included a detailed explanation of the methodology to be used, the script for the four topics under discussion, information on key issues for consideration during the consultation, instructions on the selection and invitation of participants, and risk mitigation strategies [Moreno-Castro, Mendoza-Poudereux and Vengut-Climent, 2020 ]. These guidelines then became the Implementation Toolkit document, containing specific tools and solutions to secure a standard for operations: (1) a project management template and a Gantt chart, (2) a communication set to streamline promotion of the event and the participant and sponsor recruitment process, (3) participant structures for $n=100$ (expected sample size) and $n=150$ (including the back-up list), (4) sets for moderators and observers, (5) forms, (6) checklists, (7) an evaluation set, and (8) a summary report template. A collection of over 30 documents served the project partners as common resources, most of which were translated into the respective languages of the countries involved.

After the consultations had taken place, all the organising partners emphasised the importance of the Guidelines and the usefulness of the Implementation Toolkit . They also highlighted the value of their teams’ project management competencies, especially in relation to time and scope management.

2.1.2 Agenda structure

The sessions were divided into two blocks, morning and afternoon, and all the events followed the same agenda structure.

In most countries, the event commenced at 8:00 a.m. with registration or transport to the venue, and the official opening of the consultation was scheduled for 9:00 a.m. The discussions came to a close at around 5:00 p.m., thus taking approximately eight hours of the participants’ time to run the four rounds of talks.

Organisers were free to decide whether to alter the composition of the groups during the day, or rotate the facilitators and observers to modify group dynamics. The first discussion round was preceded by an ice-breaking activity to allow participants to get to know each other. If the group composition was altered after the lunch break, a second ice-breaking activity was held prior to the start of the third round of discussions. Each discussion round lasted between 60 and 90 minutes, as suggested by Onwuegbuzie et al. [ 2009 ].

2.1.3 Venue and timing

It is important that consultation organisers create a non-judgemental environment in which individuals feel free to express their values and their emotions. Venues must be comfortable, quiet and accessible to favour group interaction.

The venue must also be free of symbolic meanings and separated from contexts that could condition the discourse. The identification system used to track the participants’ interventions must be agreed on beforehand; we decided to use ID numbers for the CONCISE consultation. We recommend delegating this task to the observer.

In a context in which several discussion groups are held simultaneously, it is crucial to take background noise into account to ensure that the session proceeds correctly. Each discussion group should have sufficient space for its members to relax, walk, talk and conduct their discussion in a noise-free environment.

In all cases, the partners treated the choice of venue as a strategic decision that could offer participants an additional incentive to attend the event. Trnava University and the University of Lodz decided to hold their consultations on university premises, while the Italian, Spanish and Portuguese partners held the event at premises leased for the purpose. The Italian consultations took place at the historic Villa Valmarana, a 16th century Renaissance villa designed by Andrea Palladio. The Botanical Garden in Madrid was the site for the Spanish consultation, while the Belém Cultural Centre in Lisbon hosted the Portuguese discussions. While the university venues added academic prestige to the events, the three leased venues provided historical, ecological and cultural contexts that enriched the activities with a consultation-specific climate.

The noise that accompanied the consultations was, to some extent, distracting for both participants and moderators, as well as rendering some of the recordings unintelligible in a few cases. On the other hand, being grouped in large spaces gave participants the impression of taking part in a sizeable public event where their opinions mattered [Blue and Medlock, 2014 ]. Based on these experiences, we recommend maintaining the open-space formula whilst ensuring adequate space between tables.

Choosing the date was also a strategic decision. Organising the consultation on a weekday might have resulted in poor attendance by people who work full-time, while holding it at the weekend requires additional incentives for people to participate. All the CONCISE project consultations were conducted on Saturdays.

2.1.4 Scripts

A detailed script was developed for all the topics, and all the consultations followed the same script translated into the respective languages of the five countries. Moderators and observers received the scripts beforehand. The complete CONCISE consultation scripts is provided in appendix A .

The purpose of this consultation was to obtain original, genuine, unbiased responses from citizens to understand and gather data about their behaviours, beliefs, perceptions and experiences in relation to the seeking of scientific knowledge on the topics discussed. Evidently, the difference between what people say and what they really think is a limitation of this type of study.

Each discussion group used the same scenario for discussing a chosen topic, and for each topic participants were asked how they received information about scientific issues, the sources and channels they used, and how the communication of scientific messages could be improved. These elements of the discussion were repeated in the same order during each round of discussion. The friendliness and predictability of the research situation had a positive effect on the atmosphere, and this was expressed by participants on the post-consultation evaluation forms.

2.1.5 Composition and number of groups

To ensure a diversity of opinions were represented [Krueger and Casey, 2000 ], the CONCISE consultations involved 10 to 14 different consultation groups in several countries. Each group was relatively homogeneous in terms of level of education and age to promote interaction and gather different voices on the issues discussed. This approach was intended to help establish a pleasant atmosphere for discussion to ensure that none of the participants were excluded and all had the opportunity to speak and share their opinions and experiences.

A minimum of two groups were formed for each segmentation variable to obtain the saturation of each voice. The individuals that form a discussion group must be unknown to each other, as most people tend to be more comfortable talking about certain issues in front of people they do not know [Krueger and Casey, 2000 ]. Each discussion group, therefore, was composed of 8 to 10 participants: enough to yield diversity in the information provided, but not so many as to create an environment in which participants might be uncomfortable sharing their thoughts, opinions, beliefs and experiences [Onwuegbuzie et al., 2009 ].

In this type of qualitative public consultation, the participant population is not representative of the country. The participant selection criteria during recruitment for the CONCISE consultations in each of the five countries was as inclusive as possible considering the diversity of their societies. The factors taken into account were gender, age, level of education (no education, primary education, secondary or higher), place of residence (urban and rural areas), special needs, national minorities (determined for each country separately according to the minorities found there) and immigrant population.

The number of people sitting at the tables was also determined by the venue capacity, type of table available and possible table layout. In other words, the spaces in which the consultations were carried out impacted on the number and layout of the tables, which directly translated into the number of participants at individual tables.

2.1.6 Selection of participants

The literature on citizen consultations alludes to a number of aspects of organising such meetings and selection of consultation methods, but only to a small extent does it relate to issues connected to the range of participants. The reasons for this are largely rooted in the fact that public consultations, as they have traditionally been understood, impose virtually no restrictions on the possibility of participation.

Stewart and Shamdasani [ 1990 , p. 53] use the term “convenience sampling” to refer to the practice whereby qualitative researchers do not transfer the results obtained in the study from the sample to the population. The sample therefore contains all the values of the variables of interest to the researchers, which also occur in society. Because of this, it is possible to make cautious generalisations and talk about certain tendencies in the distribution of opinions, shared views and frequency of behaviours in the population from which the sample was taken. Adopting an identical list of features (variables) together with their values also allows for comparisons between countries.

Evidently, such a selection of participants also has some limitations, derived primarily from the fact that a sample of 100 people from any one country is too small to be an inclusive sample or one that reflects all possible voices. A further limitation is that, during the recruitment of participants, no account was taken of features that did not belong to the socio-demographic group, but which were related to the topic of the consultation. For example, aspects related to the use of electronic media were not audited because no reliable information was available for the distribution of this feature in the respective societies. Also worthy of note is the fact that the introduction of too much diversity in groups may adversely affect the discussions, due to excessive fragmentation of the features under consideration. The CONCISE partners took into account the possibility of non-attendance among those invited, providing each organiser with a standby list of citizens willing to participate.

2.1.7 Recruitment strategies

The project partners developed a country-based tool set for use during the recruitment process of the CONCISE project, which was based on the following arguments:

1. development of science: participants could contribute to the development of a given discipline;
2. development of practical principles: participant input could be translated into specific concrete actions;
3. personal benefits: broadening their perspective and knowledge, making suggestions and being heard;
4. taking part in an exciting event: a different and unusual way to spend leisure time;
5. additional incentives: e.g., an opportunity to visit the city, free museum entries, small gifts from sponsors, etc.

To increase the chances of successful recruitment, a combination of arguments was used for each potential participant. The success of the recruitment activities was also achieved by multiplying participant search strategies to reach a number of different audiences. The CONCISE project team provided information about the consultations using traditional media channels (press, radio and television), contemporary media platforms (websites, Facebook, Instagram and Twitter), distributed leaflets and posters in crowded public places, and sent letters and emails of invitation to social institutions and organisations. Had financial compensation been provided to participants, a greater degree of diversity among the participants could have been achieved. However, the literature remains divided as to whether such a solution is actually beneficial to research studies [Head, 2009 ; Surmiak, 2020 ; Krzewińska, 2017 ].

2.1.8 Registration procedures

The consultation participants registered to participate in the event via an online registration form (translated into all languages), which made it possible to gather, store and manage the participants’ data securely. The participants’ different levels of computer literacy meant that the organisers were required to assist with online registration throughout the recruitment process.

Once recruited, participants were informed about the type of data collected, how the data would be anonymised and stored, and how they could withdraw their consent at any time. The CONCISE project followed the personal data management principles indicated under European and national legislation. Online registration was followed by onsite registration at the venue, where each participant was required to sign participation and video recording consent forms before taking part in the project.

Part of each of the CONCISE consultations was filmed to make dissemination videos of the event. Only general shots were used to show the type of event that took place; the intervention of the participants was not filmed in any detail. The participants were aware of the purpose of the filming and were given the option not to appear.

For those participants who chose not to be filmed, the partners used different methods of labelling, e.g., different coloured badges (Poland), stickers (Italy, Spain and Portugal), or seating in a discrete area (Slovakia). The most effective of these options was the Slovak approach. The other countries’ methods left room for human error, e.g., the loss of a badge or sticker, or the label being concealed by clothing, making it hard to identify the participant’s status. To overcome this limitation, video drafts of each consultation were reviewed by CONCISE partners.

2.2 Moderator and observer roles

A discussion group will ideally have a moderator and an observer. The moderator is responsible for facilitating the discussion, prompting members to speak, requesting that overly talkative members allow others to talk, and encouraging all members to participate. In contrast, the observer’s responsibilities include recording the session, taking notes, creating an environment conducive to group discussion, providing verification of data, and helping the moderator to analyse and interpret the focus group data [Krueger and Casey, 2000 ]. In each CONCISE consultation, 10 to 14 moderators and 10 to 14 observers were recruited and trained.

2.2.1 Recruitment and training of moderators and observers

A list of general competencies and operationalised behaviours was compiled to standardise the competencies required by the moderators (Table 1 ).

The CONCISE guidelines recommend recruiting moderators with previous experience in focus group discussions, such as sociologists, journalists or communicators. The role of the observer, however, can be performed by less experienced people (CONCISE Deliberable 1.2, pp. 35–36).

Detailed decisions regarding the recruitment and pre-consultation training of moderators were made by each partner individually (Table 2 ).

The train-the-trainer model was used to train moderators. First, several trainers from each participating country were trained online and equipped with materials in English. They then trained moderators and observers in their own languages. The training focused on the general principles of working with a group, pointing out potential difficulties as well as giving specific guidelines with respect to consultation scenarios, activities and evaluation. The reason for preparing the facilitators in such detail was to standardise their work across the partnership as well as reduce their stress levels and provide them with security and comfort.

2.3 Recording and transcription

The sessions were audio recorded from beginning to end. During each session, the observer took notes of the participants’ attitude (non-verbal language), participation level and interactions, and discussed these with the moderator at the end of the session. These impressions on the part of the observer were useful for the data analysis. The sessions were transcribed, identifying each participant, and the citizens’ opinions were analysed with corpus linguistic software.

The overall methodology was tested in a pilot consultation with 20 participants in Barcelona (Spain) in March 2019 [Revuelta and Llorente, 2020 ], after which both scripts and specific characteristics were discussed and improved before scaling up the consultation with a more significant number of participants.

2.3.1 Data gathering

According to Krueger [ 1994 ], the recording of one focus group will commonly result in 50 to 70 pages of text. In the CONCISE project, audio recordings and subsequent transcriptions were chosen as the data collection method (Table 3 ). In four countries, two recorders were allocated to each table to mitigate any risks of losing the data due to either device malfunction or human error, such as accidental switching off of the device or insufficient battery life in the equipment.

The instructions for the data gathering included a recommendation that the sessions be transcribed as soon as possible, ideally by the facilitator or the observer of the meeting. Various approaches were used by the consortium partners to produce the transcriptions (see Table 4 ).

The CONCISE project recordings were stored in a secure digital space by each of the partners, using popular software — T-Lab, N-Vivo 12 and SPSS — that enabled the data to be analysed in all five languages.

3 Reflections and conclusions

Citizen consultations have been carried out in democratic countries since the 1950s as public participation mechanisms. It is fascinating to observe how those first consultations were organised and how they have evolved into spaces that promote citizen participation in public policy-making [Parker, 1954 ]. Currently, citizen consultations are widespread in local and international policy-making processes. The most commonly used element of these is the online model, for its ability to obtain the views of greater numbers of citizens [Boucher, 2009 ]. For instance, the public consultation model is one of the global initiatives to address the United Nations Sustainable Development Goals.

According to Fox and Stoett [ 2016 ], citizen participation should be perceived as complementary to governmental representation, not as a substitute for it. In most cases, policies will achieve better overall results when citizen perspectives are taken into account as part of a broader process of democratic deliberation that involves a full spectrum of the actors concerned [Fox and Stoett, 2016 ].

The CONCISE project’s public consultations are unlike other general citizen participatory strategies that seek consensus formulae leading to specific public policies, or elicit proposals from citizens with the idea of implementing specific measures to resolve the problems of local, national and international government administrations. Instead, CONCISE sought citizens’ viewpoints and analysed the responses obtained during the consultation to determine exactly how they viewed the issues under debate and use the findings to produce good practice guidelines for science communicators, educators and policymakers. In this way, the CONCISE public consultations have become an opportunity to test and synthesise methodological assumptions with respect to the collection of citizen perspectives on science communication. The reflections included in previous sections helps to promote inights of the CONCISE methodology to replicate, or even improve, it in different context or in oder periods of time.

Recruitment was subject to a careful procedure designed to ensure the socio-demographic diversity of the participants. However, a number of limitations remained. Persuading citizens to participate in public consultations is a key stage in the process and one that requires extensive advertising, promotion and incentives. Although citizens may declare a positive interest, it is never certain whether they will eventually attend the event. The results will also be affected by the bias that tends to characterise outreach activities, in that the same interested public are always involved. Certain audiences are difficult to reach and those who come forward to participate in a citizen consultation tend to have a very proactive attitude. Hence the challenge is how to reach those citizens who do not participate in activities that promote science culture, and whose opinions are therefore systematically unknown [Holliman and Jensen, 2009 ].

Great care was taken to recruit citizens across a broad spectrum of society. Beyond the characteristics of people that wish to participate in citizen consultations, the act of participating itself seems to have a direct effect on democratic participation in science and technology. Above all, people feel that they are being listened to and that their opinions are being taken into account [Carvalho, Pinto-Coelho and Seixas, 2019 ].

As discussed in previous sectons, the standard for CONCISE consultations requires a professional approach to their management. Thus, the different reflections dhared in this paper will be usefull in this regard. Preparing for such an event should be operationalised through a number of standard operating procedures to ensure the reliability and validity of the data collected. Using well-prepared tools offers the opportunity to standardise and harmonise the approach across many countries. The choice of time and place of the event should be treated as strategic decisions with which to encourage citizens to participate. The events should therefore be well-located, with plenty of space for participants to feel comfortable while taking part in the discussions. The spaces used need to provide adequate privacy and acoustics conducive to discussion, the potential to share space during breaks and meals, and allow the participants to take advantage of tourist and cultural attractions beyond the venue. Moderators, who play a particularly important role, must receive training, support and scenarios to facilitate the consultations.

As a summary of the results obtained, a policy brief ¹ was produced in English. Policy briefs were also produced in the partner languages (Polish, Slovak, Portuguese, Italian and Spanish), and these were focused on different needs according to the particular socio-economic political context. The procedure for the CONCISE consultations was universal and, at the same time, open to any modifications that might need to be introduced by each country with respect to its own unique organisational and cultural conditions. Of course, this methodology can be used in other countries or in other contexts to generate more local or regionl policy briefs according to specific realities.

The organisers faced a number of dilemmas and had to seek compromises and maintain a balance between the following: (1) the flexibility and rigidness of the procedures across the countries; (2) the comparability and uniqueness of the cultural perspectives and experiences presented in the partner countries; (3) a qualitative approach to collecting the opinions and an attempt to standardise certain elements of the procedures; (4) an inclusive approach to participant recruitment and ensuring appropriate representation of various participant groups; (5) trust in the moderators’ professionalism and standardising their work through the provision of discussion scenarios.

The authors are convinced that the experiences described in this article will offer insights into the standard developed for public consultation on science communication without discouraging the reader from taking up the challenge. Experiencing such a complex, important undertaking expands perspective on cooperation with public consultation stakeholders to an inclusive, pro-democratic level.

Although the CONCISE public consultations were held on-site, the new circumstances brought about by the COVID-19 pandemic give cause to consider online versions of such events. Various opinion-sharing platforms are being created to facilitate the exchange of opinions. Cobo [ 2012 ] studied how public websites facilitated the creation of networks for citizen consultation, analysing European public-sector sites to determine the level to which they adopted digital mechanisms and strategies to enable citizen participation and collaboration. The selected websites were investigated and classified according to a taxonomy created by Dutton [ 2008 ], who defines three levels of digital citizen engagement: (1) sharing, (2) contributing, and (3) co-creating knowledge. The vast majority of EU public websites primarily undertook strategies to foster the first level of digital engagement [Cobo, 2012 ].

We believe that the standard for public consultation on science communication developed and presented in this article can be used to prepare and implement online as well as on-site consultations.

Acknowledgments

We are particularly grateful to the CONCISE project consortium, with whom we have worked for over two years and from whom we have learnt so much. We especially wish to thank Ana Delicado, Giuseppe Pellegrini and Ľubomír Šottník, who led three of the five consultations. Finally, we would like to thank Isabel Mendoza and Edurne Gastón for their role in supporting all citizen consultations.

Funding

The work reported in this paper was performed in the context of the “Communication role on perception and belief for European citizens about science” (CONCISE) project, which has received funding from the European Union’s Horizon 2020 research and innovation programme under Grant Agreement No. 824537.

Appendix A. CONCISE public consultation scripts

Here, we present the different scripts for each of the topics to be discussed during the consultations.

The moderator must have the script translated into the language in which the consultation will be developed beforehand.

Appendix A.1 Climate change consultation script

This document must be translated into the vehicular language of the consultation.
Total time : 1 h

Note to the moderator. Questions may vary depending on how the discussion goes. Please keep in mind that these questions are just suggestions to ensure that we treat all topics of interest. If any of the topics are mentioned spontaneously during the discussion, there is no need to ask again.

Always consider the gender dimension. For example, if the participants mention that they have received or have shared information with someone, ask if they remember the gender. Or if they mention a specific news, ask if they remember who appeared in the image, who spoke etc.

Appendix A.2 Vaccines consultation script

This document must be translated into the vehicular language of the consultation.
Total time : 1 h

Note to the moderator. Questions may vary depending on how the discussion goes. Please keep in mind that these questions are just a suggestion to ensure that we treat all topics of interest. If any of the topics are mentioned spontaneously during the discussion, there is no need to ask again.

Always consider the gender dimension. For example, if the participants mention that they have received or have shared information with someone, ask if they remember the gender. Or if they mention a specific news, ask if they remember who appeared in the image, who spoke etc.

Appendix A.3 Genetically Modified Organisms consultation script

This document must be translated into the vehicular language of the consultation.
Total time : 1 h

Note to the moderator. Questions may vary depending on how the discussion goes. Please keep in mind that these questions are just a suggestion to ensure that we treat all topics of interest. If any of the topics are mentioned spontaneously during the discussion, there is no need to ask again.

Always consider the gender dimension. For example, if the participants mention that they have received or have shared information with someone, ask if they remember the gender. Or if they mention a specific news, ask if they remember who appeared in the image, who spoke etc.

Appendix A.4 Alternative medicine consultation script

This document must be translated into the vehicular language of the consultation.
Total time : 1 h

Note to the moderator. Questions may vary depending on how the discussion goes. Please keep in mind that these questions are just a suggestion to ensure that we treat all topics of interest. If any of the topics are mentioned spontaneously during the discussion, there is no need to ask again.

Always consider the gender dimension. For example, if the participants mention that they have received or have shared information with someone, ask if they remember the gender. Or if they mention a specific news, ask if they remember who appeared in the image, who spoke etc.

References

Armstrong, M. J., Gronseth, G. S., Gagliardi, A. R. and Mullins, C. D. (2020). ‘Participation and consultation engagement strategies have complementary roles: a case study of patient and public involvement in clinical practice guideline development’. Health Expectations 23 (2), pp. 423–432. https://doi.org/10.1111/hex.13018 .

Berg, M. and Lidskog, R. (2018). ‘Deliberative democracy meets democratised science: a deliberative systems approach to global environmental governance’. Environmental Politics 27 (1), pp. 1–20. https://doi.org/10.1080/09644016.2017.1371919 .

Bernauer, T. and Gampfer, R. (2013). ‘Effects of civil society involvement on popular legitimacy of global environmental governance’. Global Environmental Change 23 (2), pp. 439–449. https://doi.org/10.1016/j.gloenvcha.2013.01.001 .

Blue, G. and Medlock, J. (2014). ‘Public engagement with climate change as scientific citizenship: a case study of World Wide Views on Global Warming’. Science as Culture 23 (4), pp. 560–579. https://doi.org/10.1080/09505431.2014.917620 .

Boucher, S. (2009). If citizens have a voice, who’s listening? Lessons from recent citizen consultation experiments for the European Union . EPIN working paper No. 24. Brussels, Belgium: Centre for European Policy Studies.

Calder, I. R. (2002). ‘Forests and hydrological services: reconciling public and science perceptions’. Land Use and Water Resources Research 2, pp. 2.1–2.12. https://doi.org/10.22004/ag.econ.47860 .

Carvalho, A., Pinto-Coelho, Z. and Seixas, E. (2019). ‘Listening to the public — enacting power: citizen access, standing and influence in public participation discourses’. Journal of Environmental Policy & Planning 21 (5), pp. 563–576. https://doi.org/10.1080/1523908x.2016.1149772 .

Cobo, C. (2012). ‘Networks for citizen consultation and citizen sourcing of expertise’. Contemporary Social Science 7 (3), pp. 283–304. https://doi.org/10.1080/21582041.2012.683445 .

Davies, S., McCallie, E., Simonsson, E., Lehr, J. L. and Duensing, S. (2009). ‘Discussing dialogue: perspectives on the value of science dialogue events that do not inform policy’. Public Understanding of Science 18 (3), pp. 338–353. https://doi.org/10.1177/0963662507079760 .

Delicado, A., Rowland, J. and Estevens, J. (2021). ‘Bringing back the debate on mediated and unmediated science communication: the public’s perspective’. JCOM 20 (03), A10. https://doi.org/10.22323/2.20030210 .

Delicado, A., Rowland, J., Vengut Climent, E., Mendoza-Poudereux, I. and Gaston, E. (2022). ‘Citizen consultations on science communication: a citizen science approach’. Mètode Science Studies Journal 12, pp. 47–53. https://doi.org/10.7203/metode.12.17510 .

Dutton, W. H. (2008). ‘The wisdom of collaborative network organizations: capturing the value of networked individuals’. Prometheus 26 (3), pp. 211–230. https://doi.org/10.1080/08109020802270182 .

Fisher, D. R. (2019). ‘The broader importance of #FridaysForFuture’. Nature Climate Change 9 (6), pp. 430–431. https://doi.org/10.1038/s41558-019-0484-y .

Fox, O. and Stoett, P. (2016). ‘Citizen participation in the UN sustainable development goals consultation process: toward global democratic governance?’ Global Governance: a Review of Multilateralism and International Organizations 22 (4), pp. 555–574. https://doi.org/10.1163/19426720-02204007 .

Head, E. (2009). ‘The ethics and implications of paying participants in qualitative research’. International Journal of Social Research Methodology 12 (4), pp. 335–344. https://doi.org/10.1080/13645570802246724 .

Hennink, M. M. (2014). Understanding focus group discussions. New York, NY, U.S.A.: Oxford University Press. https://doi.org/10.1093/acprof:osobl/9780199856169.001.0001 .

Holliman, R. and Jensen, E. (2009). ‘(In)authentic science and (im)partial publics: (re)constructing the science outreach and public engagement agenda’. In: Investigating science communication in the information age: implications for public engagement and popular media. Ed. by R. Holliman, E. Whitelegg, E. Scanlon, S. Smidt and J. Thomas. Oxford, U.K.: Oxford University Press, pp. 35–52.

Huttunen, S. and Hildén, M. (2014). ‘Framing the controversial: geoengineering in academic literature’. Science Communication 36 (1), pp. 3–29. https://doi.org/10.1177/1075547013492435 .

Irwin, A. and Michael, M. (2003). Science, social theory and public knowledge. Maidenhead, U.K.: Open University Press.

Jacobi, J., Mathez-Stiefel, S.-L., Gambon, H., Rist, S. and Altieri, M. (2017). ‘Whose knowledge, whose development? Use and role of local and external knowledge in agroforestry projects in Bolivia’. Environmental Management 59 (3), pp. 464–476. https://doi.org/10.1007/s00267-016-0805-0 .

Kitzinger, J. (1994). ‘The methodology of focus groups: the importance of interaction between research participants’. Sociology of Health and Illness 16 (1), pp. 103–121. https://doi.org/10.1111/1467-9566.ep11347023 .

Krueger, R. A. (1994). Focus groups: a practical guide for applied research. 2nd ed. Thousand Oaks, CA, U.S.A.: SAGE Publications.

Krueger, R. A. and Casey, M. A. (2000). Focus groups: a practical guide for applied research. 3rd ed. Thousand Oaks, CA, U.S.A.: SAGE Publications.

Krzewińska, A. (2017). Deliberacja. Idea — metodologia — praktyka. Łódź, Poland: Wydawnictwo Uniwersytetu Łódzkiego.

Krzewińska, A., Dzimińska, M., Warwas, I. and Wiktorowicz, J. (2021). Komunikacja naukowa w Polsce. Szczepionki, medycyna alternatywna, zmiany klimatyczne, GMO pod lupą. Łódź, Poland: Wydawnictwo Uniwersytetu Łódzkiego. https://doi.org/10.18778/8220-553-4 .

Moreno-Castro, C., Mendoza-Poudereux, I. and Vengut-Climent, E., eds. (2020). CONCISE’s public consultations. Valencia, Spain: ScienceFlows, Science Culture and Innovation Unit of the University of Valencia. URL: https://hdl.handle.net/10550/76516 .

Onwuegbuzie, A. J., Dickinson, W. B., Leech, N. L. and Zoran, A. G. (2009). ‘A qualitative framework for collecting and analyzing data in focus group research’. International Journal of Qualitative Methods 8 (3), pp. 1–21. https://doi.org/10.1177/160940690900800301 .

Parker, W. R. (1954). The national interest and foreign languages: a discussion guide and work paper for citizen consultations . Washington, D.C., U.S.A.: U.S. Government Printing Office.

Peters, D. A. (1993). ‘Improving quality requires consumer input: using focus groups’. Journal of Nursing Care Quality 7 (2), pp. 34–41. https://doi.org/10.1097/00001786-199301000-00006 .

Rask, M. and Worthington, R., eds. (2017). Governing biodiversity through democratic deliberation. London, U.K. and New York, NY, U.S.A.: Routledge.

Revuelta, G. and Llorente, C. (2020). ‘Barcelona pilot public consultation (Spain)’. In: CONCISE’s public consultations. Ed. by C. Moreno-Castro, I. Mendoza-Poudereux and E. Vengut-Climent. Valencia, Spain: ScienceFlows, Science Culture and Innovation Unit of the University of Valencia, pp. 12–15. URL: https://hdl.handle.net/10550/76516 .

Riedy, C. and Herriman, J. (2011). ‘Deliberative mini-publics and the global deliberative system: insights from an evaluation of World Wide Views on Global Warming in Australia’. PORTAL Journal of Multidisciplinary International Studies 8 (3), pp. 1–29. URL: http://hdl.handle.net/10453/18959 .

Rodrigo, D. and Amo, P. A. (2008). Background document on public consultation . Organisation for Economic Co-operation and Development (OECD).

Rogers, K. H. (2006). ‘The real river management challenge: integrating scientists, stakeholders and service agencies’. River Research and Applications 22 (2), pp. 269–280. https://doi.org/10.1002/rra.910 .

Rothwell, E., Anderson, R. and Botkin, J. R. (2016). ‘Deliberative discussion focus groups’. Qualitative Health Research 26 (6), pp. 734–740. https://doi.org/10.1177/1049732315591150 .

Rowe, G. and Frewer, L. J. (2005). ‘A typology of public engagement mechanisms’. Science, Technology, & Human Values 30 (2), pp. 251–290. https://doi.org/10.1177/0162243904271724 .

Schütz, F., Heidingsfelder, M. L. and Schraudner, M. (2019). ‘Co-shaping the future in Quadruple Helix innovation systems: uncovering public preferences toward participatory research and innovation’. She Ji: the Journal of Design, Economics, and Innovation 5 (2), pp. 128–146. https://doi.org/10.1016/j.sheji.2019.04.002 .

Shipley, R. and Utz, S. (2012). ‘Making it count: a review of the value and techniques for public consultation’. Journal of Planning Literature 27 (1), pp. 22–42. https://doi.org/10.1177/0885412211413133 .

Stewart, D. W. and Shamdasani, P. N. (1990). Focus groups: theory and practice. Vol. 20. Applied Social Research Methods Series. Newbury Park, CA, U.S.A.: SAGE Publications.

Stilgoe, J., Owen, R. and Macnaghten, P. (2013). ‘Developing a framework for responsible innovation’. Research Policy 42 (9), pp. 1568–1580. https://doi.org/10.1016/j.respol.2013.05.008 .

Surmiak, A. (2020). ‘Ethical concerns of paying cash to vulnerable participants: the qualitative researchers’ views’. The Qualitative Report 25 (12), pp. 4461–4481. https://doi.org/10.46743/2160-3715/2020.4441 .

Ward, J. K., Cafiero, F., Fretigny, R., Colgrove, J. and Seror, V. (2019). ‘France’s citizen consultation on vaccination and the challenges of participatory democracy in health’. Social Science & Medicine 220, pp. 73–80. https://doi.org/10.1016/j.socscimed.2018.10.032 .

Wilkinson, S. (2004). ‘Focus group research’. In: Qualitative research: theory, method and practice. Ed. by D. Silverman. Thousand Oaks, CA, U.S.A.: SAGE Publications, pp. 177–199.

Authors

Carolina Llorente, Ph.D. Associate Professor at the Department of Experimental and Health Sciences at Pompeu Fabra University (UPF). She is also the assistant director of the Science, Communication and Society Studies Centre of the UPF and the deputy director of the Master’s Degree in Scientific, Medical and Environmental Communication at the UPF — Barcelona School of Management. Her research focuses on citizen participation in science, the perception of researchers on the public and Responsible Research and Innovation (RRI). E-mail: carolina.llorente@upf.edu .

Gema Revuelta, Ph.D. Associate Professor at the Department of Experimental and Health Sciences at Pompeu Fabra University (UPF). She is also the director of the Science, Communication and Society Studies Centre (CCS-UPF) and the director of the Master’s Degree in Scientific, Medical and Environmental Communication at UPF — Barcelona School of Management. Her lines of research include: Social perception of science, Science and health communication and journalism and Responsible Research and Innovation (RRI). E-mail: gema.revuelta@upf.edu .

Małgorzata Dzimińska, Ph.D. Assistant Professor, holds a Ph.D. in social sciences with the specialty in pedagogy from the University of Social Sciences in Warsaw, Master degree in English Philology from University of Łódź. Her main research interests revolve around higher education where she focuses on the topics of quality management, knowledge management, science communication, community-based participatory research, data use for educational improvement and policy making. E-mail: mdziminska@spoleczna.pl .

Izabela Warwas, Ph.D. Professor at the Faculty of Economics and Sociology at the University of Lodz. Chair of the Department of Labour and Social Policy. Her scientific interests focus on the issues related to human resources management, age and diversity management. Author and co-author publications 120 (scientific and popular science publications). From 2002 she participated, as a technical coordinator, specialist on evaluation and expert, in over 35 European projects. Moreover, she participated in several national projects ordered by the Ministry of Science and Higher Education, Ministry of Regional Development, marshal offices. In the years 2002–2019, auditor of several dozen companies within the nationwide competition on Leader of Human Resources Management organised by the Institute of Labour and Social Affairs in Warsaw. E-mail: izabela.warwas@uni.lodz.pl .

Aneta Krzewińska, Ph.D. Associate Professor, Ph.D. in the humanities in the field of sociology, head of the Department of Methods and Techniques of Social Research (Faculty of Economics and Sociology at the University of Lodz, Poland). She is an author of numerous methodological publications. She implemented 30+ qualitative and quantitative research studies. Her scientific interests include: deliberative techniques, social consultations, social research methodology. E-mail: aneta.krzewinska@uni.lodz.pl .

Carolina Moreno, Ph.D. Full Professor of Science Communication and a member of the Research Institute on Social Welfare Policy (POLIBIENESTAR), at the University of Valencia. During the last decade, she was leading a research team, Scienceflows, where she was coordinating several research projects about risk communication and science communication. She has published over 100 works about the representativeness, social treatment, and perception of health, science, technology and environment on media and social networks. E-mail: carolina.moreno@uv.es .

Endnotes