Peer-reviewed publishing of results from Citizen Science projects

Citizen Science (CS) is gaining momentum on a national and international level in both quantity and scale of CS projects. Reasons for this increase are manifold. Enabling technology, such as the availability of built-in sensors in smartphones, supported a wide use by scientists via the Internet to collect sound data on large scales. Also, specific funding programs for engaging the general public in scientific research projects emerged. No single definition of CS exists, [Eitzel et al., 2017 ] but in general, CS is understood as the active engagement of volunteers without scientific training in the scientific process [Shirk et al., 2012 ]. The popularity of CS lead to the development of the “Ten Principles of Citizen Science” by the European Citizen Science Association (ECSA) and the “Quality criteria for citizen science projects on Österreich forscht” by the Citizen Science Network Austria (CSNA) to ensure quality of projects, ethical standards and work towards a definition of CS. In this principles/criteria it is stated that CS projects must have a genuine science outcome, and project data, meta-data and results are made publicly available whenever possible. Additionally, citizen scientists must be acknowledged in project results and publications [ECSA, 2015 ; Heigl et al., 2018 ]. Peer-reviewed, open-access publications would fulfill these requirements perfectly. However, when working with CS it seems that publishing in peer-reviewed scientific journals is not always priority and results are rather shared via project websites, blogs, or in popular scientific journals to reach the general public. What could be the reasons for not publishing in open-access peer-reviewed journals when conducting a CS project? In this letter, we present collected experiences in publishing results from CS projects in peer-reviewed journals as presented by the authors at the Austrian Citizen Science Conference in February 2018.

The traditional scientific method focuses on data quality and advancing the scientific theory, therefore the main selection criteria for publishing data obtained by citizen scientists were identical: data quality and quantity, and scientific novelty. Datasets needed to be complete and comprehensible, with metrics for evaluating data quality post hoc in place. Authors’ strategies to accomplish this included pilot studies, standardized training of participants, streamlined and simplified collection protocols, controls, intuitive online databases and robust analysis tools. Finding appropriate analysis methods can be very time-consuming, and authors recommend doing so very early in the process, preferably even before or during data collection. Whenever possible, CS data were combined with expert data [Heigl et al., 2016 ] to identify potential bias and outliers, especially if data sets were large. In terms of scientific novelty, CS as method may be suitable enough to add an attractive twist to a publication that may be otherwise of low interest for a journal. Carefully designed small-scale citizen science projects have shown great potential to produce scientifically exciting results [Marizzi et al., 2018 ; Molina et al., 2018 ].

The authors’ main criteria for selecting a journal to publish CS projects were journal scope, impact factor, open-access and cost. Authors found that although publishing in indexed, high-impact factor journals has been strongly recommended, or even required, by the institutions (for example due to Ph.D. thesis compliances), funds to cover publication fees were limited. Therefore, institutional agreements for reduced publication fees with specific publishers were a deciding factor. Generally, small-scale projects with limited datasets but interesting story were submitted to non-indexed, open access journals. Projects around curated campaigns using a robust sampling strategy that met scientific novelty expectations were submitted to open-access, indexed journals with impact factor. Authors also preferred journals that had previously published papers that were co-created with citizen scientists or based on CS data.

Contributions of citizen scientists were mostly mentioned in the acknowledgement section. This solution is most convenient and accepted by journals, as all co-authors need to fulfill all four criteria of International Committee of Medical Journal Editors (ICMJE) authorship recommendations [ICMJE, 2018 ]. If appropriate, citizen scientists were included in the byline as co-authors, either as part of a multi-author group or listed individually. When a large multi-author group has conducted the work, the group decided who would be listed as author. In select cases, citizen scientists were first authors. Getting citizen scientists’ permission to be listed as co-authors can be challenging, especially when working with youth. In one case, youth citizen scientists were recruited by another agency that did not release contact information once the lead author tried to obtain youth’s written permission to be included in the manuscript. Citizen scientists’ focus with their contribution often on the process and not the product [Freitag and Pfeffer, 2013 ] and therefore some even declined to be co-author in a peer-reviewed publication. We therefore strongly recommend discussing potential peer-reviewed publications directly with participants as early as possible.

Once submitted, interdisciplinary studies tended to be harder to publish and attracted more criticism as thematically focused manuscripts, as they were seemingly a “less good fit” for journals. For example, youth focused CS studies had to meet dual goals of rigorous science and science education, and editors suggested to publish the science and the education part separately in the respective journals. CS as method may draw the focus of the reviewers’ comments on data quality. As documented by Burgess et al. [ 2017 ] in the field of conservation research, some bias amongst scientists against data collected by citizen scientists exists, especially if they are young, like high school students, or new to the field. If not already mentioned in the paper itself, authors should be prepared to write detailed rebuttal letters explaining data collection and rigorous internal quality controls to dilute these concerns. In one case, a reviewer criticized writing style of a youth citizen scientist as “not scientific enough”, and requested to rework the manuscript to make it even more technical to meet his standards. This comment indicates, that regardless of the story, if citizen scientists wish to publish in peer-reviewed journals, they need to adapt to the scientific jargon. It also demonstrates the need for a dedicated journal that allows citizen scientists to publish for citizen scientists in their own voice.

We conclude that publishing results from CS projects can be very demanding, but as part of a global effort to democratize science well worth it. One of the authors was even invited to write a blog entry regarding their scientific article that lead to being “highlight of 2017” [Heigl, 2017 ].

References

Burgess, H. K., DeBey, L. B., Froehlich, H. E., Schmidt, N., Theobald, E. J., Ettinger, A. K., HilleRisLambers, J., Tewksbury, J. and Parrish, J. K. (2017). ‘The science of citizen science: exploring barriers to use as a primary research tool’. Biological Conservation 208, pp. 113–120. https://doi.org/10.1016/j.biocon.2016.05.014 .

ECSA (2015). ‘Ten principles of citizen science’. European Citizen Science Association . URL: https://ecsa.citizen-science.net/sites/default/files/ecsa_ten_principles_of_citizen_science.pdf (visited on 15th April 2018).

Eitzel, M. V., Cappadonna, J. L., Santos-Lang, C., Duerr, R. E., Virapongse, A., West, S. E., Kyba, C. C. M., Bowser, A., Cooper, C. B., Sforzi, A., Metcalfe, A. N., Harris, E. S., Thiel, M., Haklay, M., Ponciano, L., Roche, J., Ceccaroni, L., Shilling, F. M., Dörler, D., Heigl, F., Kiessling, T., Davis, B. Y. and Jiang, Q. (2017). ‘Citizen Science Terminology Matters: Exploring Key Terms’. Citizen Science: Theory and Practice 2 (1), pp. 1–20. https://doi.org/10.5334/cstp.96 .

Freitag, A. and Pfeffer, M. J. (2013). ‘Process, Not Product: Investigating Recommendations for Improving Citizen Science “Success”’. PLOS ONE 8 (5). Ed. by C. Nardini, pp. 1–5. https://doi.org/10.1371/journal.pone.0064079 .

Heigl, F. (2017). Using citizen science to monitor roadkills . URL: https://blogs.biomedcentral.com/bmcseriesblog/2018/02/21/bmc-ecology-highlights/ (visited on 8th July 2018).

Heigl, F., Stretz, C., Steiner, W., Suppan, F., Bauer, T., Laaha, G. and Zaller, J. (2016). ‘Comparing road-kill datasets from hunters and citizen scientists in a landscape context’. Remote Sensing 8 (10), p. 832. https://doi.org/10.3390/rs8100832 .

Heigl, F., Dörler, D., Bartar, P., Brodschneider, R., Cieslinski, M., Ernst, M., Fritz, S., Greilhuber, I., Hatlauf, J., Hecker, S., Hübner, T., Kieslinger, B., Kraker, P., Krennert, T., Oberraufner, G., Paul, K. T., Tiefenthaler, B., Vignoli, M., Walter, T., Würflinger, R., Zacharias, M. and Ziegler, D. (2018). ‘Quality criteria for citizen science projects on Österreich forscht — version 1.1’. Open Science Framework . https://doi.org/10.31219/osf.io/48j27 .

ICMJE (2018). Defining the role of authors and contributors . URL: http://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html (visited on 8th July 2018).

Marizzi, C., Florio, A., Lee, M., Khalfan, M., Ghiban, C., Nash, B., Dorey, J., McKenzie, S., Mazza, C., Cellini, F., Baria, C., Bepat, R., Cosentino, L., Dvorak, A., Gacevic, A., Guzman-Moumtzis, C., Heller, F., Holt, N. A., Horenstein, J., Joralemon, V., Kaur, M., Kaur, T., Khan, A., Kuppan, J., Laverty, S., Lock, C., Pena, M., Petrychyn, I., Puthenkalam, I., Ram, D., Ramos, A., Scoca, N., Sin, R., Gonzalez, I., Thakur, A., Usmanov, H., Han, K., Wu, A., Zhu, T. and Micklos, D. A. (2018). ‘DNA barcoding Brooklyn (New York): a first assessment of biodiversity in Marine Park by citizen scientists’. PLOS ONE 13 (7), e0199015. https://doi.org/10.1371/journal.pone.0199015 .

Molina, J., Sherpa, C., Ng, J., Sonam, T. and Stuhr, N. (2018). ‘DNA barcoding of online herbal supplements: crowd-sourcing pharmacovigilance in high school’. Open Life Sciences 13 (1), pp. 48–55. https://doi.org/10.1515/biol-2018-0007 .

Shirk, J. L., Ballard, H. L., Wilderman, C. C., Phillips, T., Wiggins, A., Jordan, R., McCallie, E., Minarchek, M., Lewenstein, B. V., Krasny, M. E. and Bonney, R. (2012). ‘Public Participation in Scientific Research: a Framework for Deliberate Design’. Ecology and Society 17 (2), p. 29. https://doi.org/10.5751/ES-04705-170229 .

Authors

Gabriele Gadermaier is working as senior scientist in the field of molecular biology and immunology at the University of Salzburg, Austria. Her research focuses on identification and characterization of allergen molecules from pollen and food. One key question is how and why innocuous sources trigger altered immune reactions leading to allergic sensitizations. Within the project “ALRAUNE — Allergy research in rural, alpine and urban networks”, more than 500 young citizen scientists contributed to the success of the epidemiological allergy research project. E-mail: Gabriele.Gadermaier@sbg.ac.at .

Daniel Dörler is a researcher at the University of Natural Resources and Life Sciences in Vienna, Austria. He is founder and coordinator of the Citizen Science Network Austria and its associated platform Österreich forscht. Daniel is a pioneer in Open Innovation in Science in Austria, member in various advisory boards and organizer of the Austrian Citizen Science Conference. E-mail: daniel.doerler@boku.ac.at .

Florian Heigl is a postdoc at the University of Natural Resources and Life Sciences in Vienna, Austria. He is also founder of the project Roadkill and initiator and coordinator of the Citizen Science Network Austria and its associated platform Österreich forscht. He is member of various advisory boards and an active networker and organizer of the annual Austrian Citizen Science Conference. E-mail: florian.heigl@boku.ac.at .

Stefan Mayr is head of the research group “Ecophysiology” and holds a full professorship at the Department of Botany, University of Innsbruck. He has been working on plant physiology and ecology for several years with focus on plant hydraulics, winter-stress and mountain vegetation. In applied projects, he dealt with restoration techniques, apple trees, rust-infection of conifers and the effects of forest fires, and he has long-term experience in science communication activities. E-mail: stefan.mayr@uibk.ac.at .

Johannes Rüdisser has vast expertise in landscape ecology with focus on biodiversity, geo-spatial analysis and ecological modeling including habitat models and spatial indicator development. In addition to his research on biodiversity, Johannes Rüdisser works as environmental educator. He has developed and executed various environmental education programs, projects and didactical concepts. The award winning citizen science project “Viel-Falter” (viel-falter.at) aims to monitor butterfly diversity combining the advantages of citizen science approaches with professional assessments executed by experts. E-mail: Johannes.Ruedisser@uibk.ac.at .

Robert Brodschneider is a biologist at the University of Graz. Since his master and PhD thesis he is studying honey bees. In his research he focuses on the nutrition of honey bees on adult, larval and colony level. In 2008 he established the investigation of colony losses in Austria. Robert has a great interest in involvement of beekeepers, and their bee colonies as biomonitors in scientific research, by closely interacting with them on various research topics. E-mail: robert.brodschneider@uni-graz.at .

Christine Marizzi is an award winning scientist and educator and currently leads the Urban Barcode Project at Cold Spring Harbor Laboratory’s DNA Learning Center. With 12+ years in urban and international science communication, she dedicates her time, energy and intellect to help students realize their greatest potential in STEM fields and provides traditionally under-represented minorities with multiple entry points to academic and professional careers in STEM disciplines. Find her on Twitter @cmarizzi. E-mail: cmarizzi@cshl.edu .