Participant motivation to engage in a citizen science campaign: the case of the TESS network

1 Introduction and context

Volunteers participate in citizen science initiatives for a multiplicity of reasons [Land-Zandstra et al., 2016 ; Aoki et al., 2017 ]: personal enjoyment, desire for improvement or achievement, establishment of personal relationships, care for the environment, etc. Studying motivation and investigating the factors influencing people’s participation in citizen science projects is therefore an essential aspect in the analysis of citizen science communities. Understanding the reasons that foster people to engage can support the successful design and implementation of effective participant involvement tasks, as well as pave the way for long-term engagement [Richter et al., 2018 ].

ACTION ¹ (Participatory science toolkit against pollution) is an ongoing H2020 project aiming to combat and prevent major forms of pollution in different European countries and internationally by supporting several existing case study citizen science projects and also running an accelerator program for additional initiatives. The project is currently developing a global community of pollution-related citizen science practitioners; the aim is to support citizen science research by providing a socio-technical toolkit, constituted by digital tools, methodologies and best practices.

With reference to participant motivation analysis, in the context of the ACTION project, we carried out a dedicated study to investigate the main factors affecting and influencing participants’ motivation. In particular, we defined a specific methodology, constituted by a configurable questionnaire and its approach to analyse the collected responses, and we created an engaging digital tool to administer the survey.

2 Objective

Participation and motivation are broadly studied in the context of citizen science. One of the main methodology to investigate motivation is the well-known Schwartz Theory of Basic Values [Schwartz, 2012 ]. Schwartz’s theory identifies ten motivationally distinct values, grouped in four main groups (Openness to change, Self-enhancement, Conservation and Self-transcendence), and describes the dynamic relations between them. Several models have been developed to measure the basic values, usually in the form of a questionnaire.

In the context of citizen science, Schwartz’s theory has been extended and adapted, with the goal to include some additional factors characterizing the specific field. In particular, in our work to investigate motivation to participate, we adopted a questionnaire derived from the best practices from citizen science research [Levontin, Gilad and Chako, 2018 ], which is indeed inspired from Schwartz’s theory.

As a first case study, we employed such methodology to analyse the motivation of a specific citizen science community focused on fighting light pollution: the TESS network. ² Volunteers of this network accept to host and install sensors to monitor sky brightness in order to collect data for measuring the level of light pollution in many areas of the Earth. The sensor, called TESS photometer (Telescope Encoder and Sky Sensor [Zamorano et al., 2016 ]) , is a compact device to monitor sky brightness every night developed by the Universidad Complutense de Madrid within the STARS4ALL project, ³ an H2020 project that created and spread awareness about light pollution. Figure 1 shows the visual appearance and the size of a TESS photometer.

Initially, most of the photometers were provided to the volunteers free of charge as beta testers; they were selected among the (professional and amateur) astronomer communities and the people interested in fighting Light Pollution. As the project became more popular, additional collaborators asked (and paid) for hosting a photometer. As of today, about 180 photometers are installed in Asia, Africa, North and South America, Europe and Australia (cf. map in Figure 2 ). All the sensor measurements are periodically published in the STARS4ALL Zenodo Community ⁴ following the principles of open science.

The first objective of the work presented in this paper, therefore, is to analyse the specific motivational factors influencing the TESS network participants. The adopted methodology and tool, however, are not strictly connected to the TESS network. Indeed, we devised them to be partially configurable and completely reusable to inquiry other citizen science campaigns, in line with the goals of the ACTION project. Moreover, from the comparison of the TESS network survey results with the information collected from other citizen science projects and participatory initiatives, we can obtain a holistic view on the reasons why different people engage with different communities.

3 Methods

As explained in the previous section, we set up a questionnaire to measure the motivational factors to participate in citizen science.

First we focused on the variables to investigate and we identified 10 main factors: self-direction, stimulation, hedonism, achievement, power, conformity, benevolence and universalism (from Schwartz [ 2012 ]), plus routine and belongingness (from the mentioned citizen science specialization of Schwartz’s theory [Levontin, Gilad and Chako, 2018 ] ). We also added a global motivation factor and an additional investigation variable, related to the specificity of the TESS network, which is data usage, i.e. the participants’ interest in the information collected through the photometers. ⁵

For each variable, we selected or defined two question items to include in the survey. We also added a question item to measure the global motivation and an open-ended question to collect more qualitative feedback from the citizen scientists. All closed-question items included 5 answer options, numerically coded with a 1–5 Likert scale. Finally, we completed the questionnaire with some demographic questions. The variables and questions are summarised in Table 1 . We performed a pre-test of the survey with a limited set of knowledgeable respondents to assess the method reliability (internal consistency).

---

Table 1 : Question items in the motivation survey, formulated for the TESS network.

---

Since the TESS community is geographically distributed throughout the entire world, we needed the survey to be administered in a digital form. We implemented our motivation questionnaire in a digital web-based system named Coney, ⁶ a conversational survey toolkit with an interactive chat interface. ⁷ The choice of this tool was driven by the desire to improve the survey respondents’ engagement; it was demonstrated that this approach is perceived as more enjoyable and intriguing by end users [Celino and Re Calegari, 2020 ] who, consequently, are more engaged and pay more attention while filling the survey with respect to traditional web-based questionnaires. Coney also provides the possibility to export the collected information in an open science fashion and to analyse results [Scandolari et al., 2019 ]. Therefore, we inserted the question items illustrated in Table 1 within a coherent and colloquial “conversation”, to administer the survey and engage the involved citizen scientists in a more personal and effective way, soliciting their self-reflection. Figure 3 shows how the TESS motivation appears in the Coney interface.

The TESS network coordinators sent out the link to the survey by email to 120 people hosting photometers. One reminder email after a week from the initial sending solicited the response. All the answers were collected anonymously and the data do not include any detail that could lead to de-anonymization. Within 3 weeks in November 2019, we collected the citizen scientists’ responses and we proceeded with their processing. We performed different statistical analysis (mean values for each variable, correlation between factors and global motivation) to collect quantitative insights and we also analysed the free-text answers to the open-ended question to derive a qualitative understanding.

4 Results

We collected answers from 83 volunteers, corresponding to the 69% of our target users. This response rate is very high compared to both the average Web survey response rate (33%) and the email survey response rate (30%) [Lindemann, 2019 ]. Our respondents were mostly male (85%) and older than 45 years old (70%).

To better understand the target group, we asked them to select one or more predefined options in response to the question “Which of the following categories identifies you the most?”. As shown in Figure 4 , professional and amateur astronomers, together with light pollution fighters were the most selected categories.

From this categorization, it is already clear that most of the respondents care about sky darkness, therefore it is not surprising that the answer distribution for the global motivation is highly right-skewed (cf. Figure 5 ), with a mean value of 4.39. None of the participants selected the lowest valued-answer and a striking 87.2% expressed a high or very high motivation.

We then computed the mean value for each of the investigated variables, i.e. each of the motivating factors included in our questionnaire, and we also measured the correlation between those variables and the global motivation, in an attempt to discover the relation between the factors and the declared level of motivation. The results are shown in Table 2 , in decreasing order of the mean value; as previously explained, all individual values are in a 1–5 Likert scale.

---

Table 2 : Mean values on different factors and correlation with global motivation; the asterisks represent the confidence value of the correlation (p-value: *** < 0.001, ** < 0.01, * < 0.05).

---

With specific reference to the usage of the data collected through the photometers, Figure 6 shows the distribution of answers to the respective questions, which are specific to the TESS network citizen science project. Almost 3/4 of the respondents expressed their interest in using the collected data.

We also analysed the effect of the free/paid photometer on the results. A test for the difference in distribution of the global motivation value between the two groups (free and paid photometer hosts) did not yield any statistically significant difference. In other words, we can say that there is no evidence that the purchase of the device had any influence on the participants’ motivation. Indeed, 78% of participants received TESS for free as beta testers and 70% of them declared to be willing to pay for it. Those who bought the photometer judged the price as right (88%). This is another evidence of the high level of participation of this community.

Finally, we analysed the free-text answers to the global motivation question “In your own word, which is the main motivation why you decided to host a photometer?”. We derived the word cloud displayed in Figure 7 , which shows that the main self-reported drivers are indeed related to light pollution fighting and sky darkness preservation.

In order to foster reproducibility, by following the FAIR principles of Open Science [Wilkinson et al., 2016 ], we made the survey structure, the collected answers and the analysis of correlation in CSV and RDF formats [Scandolari et al., 2020 ] available on Zenodo under an open license (CC-BY).

5 Discussion

The results illustrated in the previous section give a clear idea about the participation of the citizen scientists engaged in the TESS network. They are adult people, either with a professional or personal interest in astronomy and sky darkness, who were highly motivated to join the network and are active participants (cf. Figure 5 ): even if their active role is mainly limited to hosting a photometer, they also show a clear interest in the data collected through the sensor network (cf. Figure 6 ).

Analysing the motivating factors (cf. Table 2 ), it is clear that the main drivers of the citizen scientists to participate are: the interest in the topic and the willingness to learn ( Self-direction mean value of 4.43); the goodwill to contribute to scientific research ( Benevolence mean value of 4.42); the possibility to raise public awareness by making data more accessible ( Universalism mean value of 4.33). Other motivations with high mean values are Hedonism , Stimulation and Achievement (4.17, 4.14 and 4.13 respectively), which highlight the respondents’ interest in pursuing personal passions, challenging themselves and achieving meaningful results.

On the other hand, the TESS network participants answered the questions related to Power and Conformity by selecting low-valued options (mean values of 2.83 and 2.35 respectively). In other words, the respondents showed very limited interest in gaining recognition or getting something in return, and they did not join the network because they felt somehow obliged by social pressure.

Those results highlight the fact that the TESS network is a healthy community of citizen scientists who are genuinely interested in the project value and outcomes. The main motivational factors are directed to the benefit of the community. This is also apparent from the analysis of the free-text answers on the participants’ motivation (cf. the word cloud in Figure 7 ): light pollution, sky brightness/darkness and data were the most frequently used words. A manual analysis of the answers also highlighted the strong interest of participants to support and contribute to local measurements in their geographical area and the high recognition and the great respect for the scientific goal of the TESS network initiators and coordinators.

The correlations between the investigated variables and the global motivation also seem to suggest the same explanation (cf. again Table 2 ): Benevolence , Universalism and Hedonism show the highest correlation values, meaning that this citizen science project profits by a good combination of participants’ personal interest and awareness with a scientific goal.

Indeed, most of the respondents declared a clear ongoing or intended use of the data collected through the photometers (cf. Figure 6 ): only 25.6% answered that they are not using the data. On the other hand, the actual reuse of the sensor data by the researchers is less known, showing a potential for improvement with respect to communication and dissemination. Learning about the project discoveries has in itself been framed as a reward and so increasing the sharing of the results can be an added value for the community.

The results and findings discussed above are of course limited to the TESS network citizen science project. However, we can reuse and adapt the followed methodology and questionnaire to carry out the same investigation on different citizen science and participatory initiatives. The goal is twofold: on the one hand, the adoption of the same approach allows for a proper comparison of results between different projects and, on the other hand, the analysis of the different factors can lead to interest insights on the different nature of the citizen scientist communities and the potential different incentive schemes that can be adopted to foster participation in the long term.

As a preliminary proof for this approach, we repeated the same investigation by involving a quite contrasting community, i.e. the workers of a crowdsourcing platform named Prolific. ⁸ Crowd workers usually participate in exchange for a (small) monetary reward; in this respect the Prolific platform is quite peculiar because it is mostly oriented to social science research and its participants have a high education level.

Therefore, we collected from 100 Prolific workers their answers to the same questionnaire (same investigated variables and global motivations, we only removed the questions related to data usage) and we compared the results with those of the TESS network.

The global motivation was much lower (mean value: 3.58 vs. 4.39), probably because participating in crowdsourcing campaigns is less stimulating. The highest difference was recorded with respect to Universalism , Hedonism and Self-direction : this may indicate a more limited interest in raising awareness, a reduced appeal to personal passions and a smaller curiosity in the activity. Also, the main factors that correlate with the global motivation are the (lack of) Self-Direction and Stimulation (correlation values of 0.623 and 0.591); it is also interesting that, in the case of the crowd workers, a statistically significant correlation with the global motivation also exist for the Power variable (correlation value of 0.326): this highlight the fact that Prolific participants indeed expect something in return.

6 Conclusions

In this paper, we illustrated our investigation on citizen scientists’ motivation to participate. We explained our conceptual approach, the followed methodology and the digital implementation of the motivation questionnaire. We also provided the results and the insights coming from the concrete investigation carried out with reference to a specific citizen science project, the TESS network.

It is worth underlining that we designed the proposed methodology independently of the specific application to an individual citizen science initiative. On the contrary, we devised it to easily replicate this investigation with different participatory communities, also to facilitate the comparison between different project and the exploitation of the findings to better design incentive schemes to support participants’ engagement.

This is indeed in line with the overall goal of the ACTION toolkit, which aims to collect a set of guidelines, best practices and digital tools to support citizen science. Figure 8 graphically represents the global framework to citizen science implementation [Passani, 2020 ].

In particular, in this paper we offered our approach to study citizen scientists’ motivation. We schematise the different activities needed to apply our methodology, with respect to the steps in Figure 8 , in the following Table 3 .

---

Table 3 : Steps of our proposed methodology to investigate citizen scientists’ motivation.

---

Of course, the formulation of the questions can be slightly adapted to reflect the specificity of each citizen science project. However, the questionnaire that we implemented for the TESS network study can be reused to save time in the Survey Design step. Moreover, the kind of analysis we carried out on the responses from the TESS community can be easily replicated to enable a proper comparison between different initiatives and save time in the last two steps of our methodology. That is the reason why we released both the questionnaire and the anonymously collected data in line with Open Science principles.

Finally, we would like to note that we offering not only the methodology but also our engaging digital tool for conversational surveys to interested citizen science project, as part of the ACTION acceleration program. In addition, initiatives that are not formally affiliated with the ACTION project are very welcome to contact us. This will not only represent a concrete support for citizen science researchers, but it will also constitute a means to extend the results presented in this paper to better understand the factors affecting citizen scientists’ motivation and the diversity of our participatory communities.

Acknowledgments

We would like to thank all the citizen scientists participating in the TESS network, as well as the Prolific crowdsourcing workers, for their time and effort to compile our motivation survey. This research was partially supported by the H2020 ACTION project (Grant Agreement 824603), co-funded by the European Commission.

References

Aoki, P., Woodruff, A., Yellapragada, B. and Willett, W. (2017). ‘Environmental Protection and Agency: motivations, capacity and goals in participatory sensing’. In: Proceedings of the 2017 CHI conference on human factors in computing systems . New York, NY, U.S.A.: ACM, pp. 3138–3150. https://doi.org/10.1145/3025453.3025667 .

Celino, I. and Re Calegari, G. (2020). ‘Submitting surveys via a conversational interface: an evaluation of user acceptance and approach effectiveness’. International Journal of Human-Computer Studies 139, p. 102410. https://doi.org/10.1016/j.ijhcs.2020.102410 .

Land-Zandstra, A. M., Devilee, J. L. A., Snik, F., Buurmeijer, F. and van den Broek, J. M. (2016). ‘Citizen science on a smartphone: Participants’ motivations and learning’. Public Understanding of Science 25 (1), pp. 45–60. https://doi.org/10.1177/0963662515602406 .

Levontin, L., Gilad, Z. and Chako, S. (2018). Questionnaire for the motivation for citizen science scale . Technical report, Citizen Science COST Action. URL: https://www.cs-eu.net/news/questionare-motivation-citizen-science-scale .

Lindemann, N. (2019). What’s the average survey response rate? [2019 Benchmark]. Technical report, Survey Anyplace. URL: https://surveyanyplace.com/average-survey-response-rate/ .

Passani, A. (2020). ‘Participatory research lifecycle’. Zenodo . https://doi.org/10.5281/ZENODO.3687582 .

Richter, A., Hauck, J., Feldmann, R., Kühn, E., Harpke, A., Hirneisen, N., Mahla, A., Settele, J. and Bonn, A. (2018). ‘The social fabric of citizen science — drivers for long-term engagement in the German butterfly monitoring scheme’. Journal of Insect Conservation 22 (5-6), pp. 731–743. https://doi.org/10.1007/s10841-018-0097-1 .

Scandolari, D., Scrocca, M., Re Calegari, G. and Celino, I. (2019). ‘Coney: a conversational approach to enhance engagement in surveys’. In: Proceedings of the Open Science Fair 2019 (Demo) (Porto, Portugal, September 2019).

Scandolari, D., Re Calegari, G., Scrocca, M. and Celino, I. (2020). ‘TESS network motivation survey dataset’. Zenodo . https://doi.org/10.5281/ZENODO.3739058 .

Schwartz, S. H. (2012). ‘An overview of the Schwartz theory of basic values’. Online Readings in Psychology and Culture 2 (1). https://doi.org/10.9707/2307-0919.1116 .

Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., Silva Santos, L. B. da, Bourne, P. E., Bouwman, J., Brookes, A. J., Clark, T., Crosas, M., Dillo, I., Dumon, O., Edmunds, S., Evelo, C. T., Finkers, R., Gonzalez-Beltran, A., Gray, A. J. G., Groth, P., Goble, C., Grethe, J. S., Heringa, J., ’t Hoen, P. A. C., Hooft, R., Kuhn, T., Kok, R., Kok, J., Lusher, S. J., Martone, M. E., Mons, A., Packer, A. L., Persson, B., Rocca-Serra, P., Roos, M., van Schaik, R., Sansone, S.-A., Schultes, E., Sengstag, T., Slater, T., Strawn, G., Swertz, M. A., Thompson, M., van der Lei, J., van Mulligen, E., Velterop, J., Waagmeester, A., Wittenburg, P., Wolstencroft, K., Zhao, J. and Mons, B. (2016). ‘The FAIR guiding principles for scientific data management and stewardship’. Scientific Data 3 (1), pp. 1–9. https://doi.org/10.1038/sdata.2016.18 .

Zamorano, J., García, C., Tapia, C., Sánchez de Miguel, A., Pascual, S. and Gallego, J. (2016). ‘STARS4ALL night sky brightness photometer’. International Journal of Sustainable Lighting 35, pp. 49–54. https://doi.org/10.26607/ijsl.v18i0.21 .

Authors

Irene Celino is the Head of the Knowledge Technologies group at Cefriel. She is an expert in Web and Data technologies and her research activities span across Human Computation, Citizen Science, Human-in-the-loop Machine Learning, Data Analytics, Semantic Web and Semantic Interoperability, with a focus on Smart Cities and transport/mobility. She has 15+ years of experience in 30+ cooperative research projects, at regional, national and European level, and she is author of 100+ scientific publications. E-mail: irene.celino@cefriel.com .

Gloria Re Calegari is a Researcher who has been working since 2014 in Cefriel. She graduated in Computer Engineering at Politecnico di Milano and she obtained a Master on “Analytics & Business Intelligence” held by Cefriel and MIP Politecnico di Milano in 2017. Her research activities focus on Data Analytics, Data Mining and Human Computation technologies to the development of gamified application, human in the loop solutions and tools for supporting data collection, especially in the citizen science field. She participated to research projects at national/regional level (PROACTIVE, SPICA) and at European level within EIT Digital (PIAZZA, Ninja Riders, Bankable, AIDE) and H2020 (STARS4ALL, ACTION). E-mail: gloria.re@cefriel.com .

Mario Scrocca is a researcher at Cefriel in the Knowledge Technologies group. His research is focused on Semantic Interoperability and Human Computation. He holds a MSc in computer science engineering from Politecnico di Milano where he specialised on interactive applications and artificial intelligence. E-mail: mario.scrocca@cefriel.com .

Jaime Zamorano is a professor of Physics at Universidad Complutense de Madrid. He received his PhD in astrophysics in 1985 and has been the advisor of 12 PhD thesis in astronomy and of multiple graduate research works related with light pollution. He is a teacher of the master in astrophysics at UCM. He leads the Advanced Scientific Instrumentation Laboratory. Coordinator of the Spanish Light Pollution Research Network, he is the principal investigator of the TESS photometer network. E-mail: jzamorano@fis.ucm.es .

Esteban González is a Software Engineer by ETSIINF (Universidad Politecnica de Madrid) and Master in Artificial Intelligence at the same university. His research area focuses on the study of reputation systems to improve the results of citizen science projects. He has worked on the STARS4ALL project when he was responsible for the management of the open research data and the portal to visualize them. Currently he is working in ACTION project managing the data portal. E-mail: egonzalez@fi.upm.es .

Endnotes