User experience of digital technologies in citizen science

17/01/2019

Abstract: 

The growing interest in citizen science has resulted in a new range of digital tools that facilitate the interaction and communications between citizens and scientists. Considering the ever increasing number of applications that currently exist, it is surprising how little we know about how volunteers interact with these technologies, what they expect from them, and why these technologies succeed or fail. Aiming to address this gap, JCOM organized this special issue on the role of User Experience (UX) of digital technologies in citizen science which is the first to focus on the qualities and impacts of interface and user design within citizen science. Seven papers are included that highlight three key aspects of user-focused research and methodological approaches. In the first category, "design standards", the authors explore the applicability of existing standards, build and evaluate a set of guidelines to improve interactions with citizen science applications. In the second, "design methods", methodological approaches for getting user feedback, analysing user behaviour and exploring different interface designs modes are explored. Finally, "user experience in the physical and digital world" explores crossovers with other fields to improve our understanding of user experiences and demonstrate how design choices not only influence digital interactions but also shape interactions with the wider world.

Keywords: 


Citizen science involves the collaboration or partnership between professional scientists and amateurs, volunteers, and even scientists outside their prescribed role, who jointly take part in scientific endeavours. While citizen science as an activity has existed for centuries, it has recently gained momentum partly due to the growth and penetration of publicly accessible information and communication technologies (ICT). This has resulted in a whole range of new tools that facilitate the interaction and communications between citizens and scientists. Like social media, citizen science platforms are creating new configurations of people and issues that radically transform the way the environment is monitored and challenge the status quo of scientific knowledge production and public deliberation. Yet, we know surprisingly little about how volunteers interact with these technologies, what they expect from them, and why these technologies succeed or fail. The aim of this special issue is, therefore, to advance our understanding and capture the state-of-the-art in research and practice regarding the user experience aspects of digital citizen science technologies.

In her 2016 paper ‘Citizen Science: New Research Challenges for Human-Computer Interaction’, Jenny Preece states that “HCI [Human-Computer Interaction] researchers can empower citizen scientists to dramatically increase what they do and how they do it” [Preece, 2016, p. 585] and argues that HCI is a critical part of citizen science. Yet in citizen science, digital technologies are often developed without HCI principles and methodologies in mind. Thus, it is not surprising that many citizen science applications fail or cause problems for researcher and users. These problems can impact adoption, continuous participation, data quality, and other aspects.

This special issue consists of seven papers and represents one of the first coordinated attempts to examine the qualities and impacts of interface and user design within citizen science. The needs and experiences of users, participants and volunteers require much attention when designing the infrastructures that underpin citizen science projects. These texts illustrate that there are important lessons to be learnt from user interactions with citizen science technologies that need to be more communicated across the HCI design and citizen science communities, so that we can better understand user needs and create successful projects. This special issue highlights three key components that should guide the design of citizen science: design standards, design methods and participant experiences. These three categories are explored in the next sections.

1 Design standards

Standards have traditionally been used in product development to impose amongst others compatibility, interoperability and for ensuring public health and safety. An increasing number of standards have an HCI focus and attempt to promote good practice principles across user interface design, usability assurance, usability and software quality and the human-centred design process [Bevan, 2009]. While these have been fundamental building blocks in the design and development of many interfaces, in citizen science the main reference with respect to standards has been about data recording, collection and sharing protocols [see Hecker et al., 2018].

In their paper, Robert Houghton and co-authors [Houghton et al., 2019] make reference to BS ISO 27500:2016 “The human-centred organisation” standard, which highlights the importance of socio-technical infrastructures and context to support the design of a citizen science project. This paper is therefore a step towards demonstrating the importance of human-centred design practices towards a technical audience. The standard includes a set of principles that highlight the importance of usability and accessibility; concepts that despite their popularity in the broader digital space are less prevalent in citizen science especially as components of the broader socio-technical system, rather than as individual pieces. In this way, the standard overlaps with the ethos of citizen science in terms of promoting openness, trustworthiness, social responsibility and might encourage a focus on these elements. The authors show how design and interaction can be improved at different levels; e.g. from considering human perception and cognition aspects in the way tasks are designed, to improve usability through user-centred design that involves users at all stages of project development.

The second paper in this category, by Artemis Skarlatidou and co-authors [Skarlatidou et al., 2019] takes a pragmatic approach to develop and evaluate a set of guidelines for supporting interface design of citizen science applications. It emphasises “a lack of detailed analysis of volunteers’ needs and requirements, common usability mistakes and the kinds of user experiences that citizen science applications generate”. Given the increasing development of applications for mobile devices and platforms that involve citizen scientists, this analysis is timely.

In the HCI community, ‘user experience’ attempts to capture end-user needs and their requirements and incorporate them into the development of new or existing technologies [Garrett, 2011]. Through a systematic literature review Skarlatidou and co-authors explore published research that discusses user and design issues for environmental citizen science applications. They synthesise this knowledge to build and evaluate a set of design guidelines. The significance of design guidelines in HCI is well-established and can help citizen science scientists and practitioners incorporate fundamental principles [e.g. see Jennett and Cox, 2014] into their designs and evaluate the user experiences generated by their applications.

2 Design methods

Robert Houghton and his co-authors make the argument that usability and accessibility should be strategic objectives in a user-centric design and development process for citizen science. They mention methodological approaches such as eye-tracking, qualitative interviews, usability testing with think aloud and quantitative data such as performance metrics. Yet, Ulrike Sturm and Martin Tscholl’s paper [Sturm and Tscholl, 2019] highlights important limitation of these methods. Although they are effective at uncovering usability issues, their applicability in specific citizen science scenarios may be problematic, especially if specific functionalities (e.g. data collection audio recordings) cannot be fully evaluated even in lab-based user-testing sessions.

User-centred design (UCD) is one of the most popular approaches in HCI and despite being limited there are some examples which demonstrate how it is used in citizen science [e.g. Kim et al., 2011; Newman et al., 2010] for the development of user-friendly applications. Nevertheless, there are fewer examples to demonstrate how this can be done in an agile context, which has been subject of extensive discussion in HCI [McInerney and Maurer, 2005]. Ulrike Sturm and Martin Tscholl explore UCD in an agile context and focus on the role of user feedback, which is a feature that within the context of citizen science has not received enough attention. This study is a step in this direction; it provides insight into the relevance of different feedback types at different stages of product development. The authors observe three types of feedback; a general type of feedback — mostly in terms of answering general satisfaction questions; contributory user feedback — which provides a deeper insight into usability problems and error reports; and co-creational user feedback — which is more reflective in terms of usability and other interaction issues and which can provide suggestions for overcoming barriers and better design of features. Robert Houghton and his co-authors make the assumption that “giving too early access to a project might be seen as wasting an opportunity if elements are not right the first time”. Nevertheless, Strum and Tscholl provide the first evidence that early access co-creational user feedback can lead to a better understanding of what users think of market competitors, their needs and additional user audiences. Exploiting this knowledge early in the project design is undoubtedly a valuable source of information.

Helen Spiers and her co-authors [Spiers et al., 2019] take a different approach to explore user issues and volunteer behaviour in virtual citizen science projects. Zooniverse is perhaps the most widely-studied platform in the citizen science community, and the authors suggest that with 63 projects it is “the most comprehensive collection of online citizen science project data gathered to date”. While a virtual citizen science platform provides the tools to set up an online project and reach a pool of volunteers, this does not necessarily ensure a consistent user experience across its projects. In their analysis, they observe heterogeneity across projects, along with differences when considering specific domains. The authors suggest that controlling the release of data over time creates a ‘gamified’ aspect which may support volunteer retainment. Nevertheless, this approach may have other design implications that can limit the project to a smaller group of volunteers, sacrificing inclusivity and volunteer diversity for an increase in classification rate. This raises questions about balancing scientific aims with social and ethical responsibilities towards the volunteer community and questions the position of citizen science in this regard.

Gamification is another area which is widely used in citizen science to reach new audiences, for providing a fun user experience and for sustaining engagement. In their paper, Artemis Skarlatidou and co-authors touch on gamification and provide a relevant design guideline for this feature. In addition, Thomas Muender and his co-authors [Muender et al., 2019] describe a case which explores the usability of gamification features, user preference and enjoyment in a well-structured, within-subjects experiment. They take into account; e.g. actual usability, in terms of completion times; perceived attention and spatial presence and; subjective measures for enjoyment and frustration. Although the results do not significantly favour one input mode over the other, the authors conclude that multi-touch interfaces may promote — for tasks which involve guided rigid body manipulations — accuracy and enable the completion of the task in fewer moves.

3 User experiences in the digital and physical world

An interesting angle for studying user experience is taken by Liz Dowthwaite and James Sprinks [Dowthwaite and Sprinks, 2019] who examine the ‘professional-amateur’ divide in citizen science and compare it with the creative domain of webcomics. The authors argue that the mechanisms of online citizen science have narrowed the professional-amateur divide and citizen science is not the only domain that has had this influence in its online practices. Via the comparison with webcomics, they highlight three areas: mutual acknowledgement — the perceived value of professional and amateurs in the community; infrastructural support — services made available online that were previously only available to professionals; and platform specialisation — the use of specialised tools and websites. While attribution has already been tested in citizen science, the comparison with webcomics highlights the suggestion of recognising individual contribution such as the provision of tailored individual feedback mechanisms on performance and impact to make citizen scientists feel more involved with the process. While this might happen more frequently ‘offline’ when professionals and volunteers have a direct relationship it is rare to observe online. It is simple design solutions like this one that as the authors argue, have the potential to blur barriers to professional practice and improve the user experience.

The design of the technologies of citizen science can take a powerful role in shaping the affective and embodied experience of participants and their relationship with the environment. Nirwan Sharma and his co-authors [Sharma et al., 2019] offer the important perspective that citizen science is not just a means for extracting data points from participants but can be used to stimulate affective encounters with nature for “cultivating non-binary ontologies of nature”. The paper shows that citizen science projects create complicated ‘nature experiences’ that are facilitated by the research tools and involve both human and non-human species. In the paper, the physical act of using a camera to photograph bees becomes a way of learning to relate to bees as fellow garden users and develop a relationship with them. For a HCI perspective, this paper highlights that the design of the scientific research apparatus is critical for defining and enabling the participant’s experience. Good design in citizen science not only enables better scientific data but also allows participants to build novel relationships with the natural world as a living entity. In this way Sharma and his co-authors illustrate that citizen science can be respectful and sensitive to different kinds of knowledge and bridge disciplines and epistemologies. Alongside other emerging research [Nold, 2017], the paper points towards new forms of citizen science that can work with fields such as Science and Technology studies to build platforms for the co-production of knowledge between scientists and broader publics.

The special issue presents a collection of papers that aim to improve our knowledge of how users interact with citizen science technologies and to subsequently initiate a more in-depth discussion around UX design and user issues in citizen science, which will eventually bridge knowledge and expertise from various fields. We hope that the papers will inspire researchers, developers and citizen science practitioners to reflect more on their experiences and share their anecdotal evidence of the kind of user experiences their applications create, and the key lessons learned; presenting their unique contextual characteristics and disciplinary insights.

Acknowledgments

We would like to thank everyone who contributed in this special issue, especially the authors who submitted their manuscripts. All papers presented have been peer reviewed by at least two reviewers and each paper has undergone at least one revision after receiving extensive comments. We would therefore like to thank all the reviewers for their efforts in providing useful feedback to the authors, but we would also like to thank all other reviewers who provided feedback to the more than 40 submissions that were made in this special issue. We would also like to thank for its support the European Union's COST Action CA15212 on Citizen Science to promote creativity, scientific literacy, and innovation throughout Europe. This project has received funding from the European Research Council (ERC) under the Horizon 2020 research and innovation programme (Grant agreement No. 694767).

References

Bevan, N. (2009). ‘International standards for usability should be more widely used’. Journal of Usability Studies 4 (3), pp. 107–113.

Dowthwaite, L. and Sprinks, J. (2019). ‘Citizen science and the professional-amateur divide: lessons from differing online practices’. JCOM 18 (01), A06. https://doi.org/10.22323/2.18010206.

Garrett, J. (2011). The elements of user experience: user-centred design for the web and beyond. 2nd ed. Berkeley, CA, U.S.A.: New Riders publishing.

Hecker, S., Haklay, M., Bowser, A., Makuch, Z., Vogel, J. and Bonn, A. (2018). Citizen science: innovation in open science, society and policy. London, U.K.: UCL Press.

Houghton, R., Sprinks, J., Wardlaw, J., Bamford, S. and Marsh, S. (2019). ‘A sociotechnical system approach to virtual citizen science: an application of BS ISO 27500:2016’. JCOM 18 (01), A01. https://doi.org/10.22323/2.18010201.

Jennett, C. and Cox, A. L. (2014). ‘Eight guidelines for designing virtual citizen science projects’. In: HCOMP ’14 (Pittsburgh, PA, U.S.A. 2nd–4th November 2014), pp. 16–17. URL: https://www.aaai.org/ocs/index.php/HCOMP/HCOMP14/paper/view/9261.

Kim, S., Robson, C., Zimmerman, T., Pierce, J. and Haber, E. M. (2011). ‘Creek watch: pairing usefulness and usability for successful citizen science’. In: Proceedings of the 2011 annual conference on Human factors in computing systems — CHI ’11 (Vancouver, BC, Canada, 7th–12th May 2011), pp. 2125–2134. https://doi.org/10.1145/1978942.1979251.

McInerney, P. and Maurer, F. (2005). ‘UCD in agile projects: dream team or odd couple?’ Interactions 12 (6), pp. 19–23. https://doi.org/10.1145/1096554.1096556.

Muender, T., Gulani, S., Westendorf, L., Verish, C., Malaka, R., Shaer, O. and Cooper, S. (2019). ‘Comparison of mouse and multi-touch for protein structure manipulation in a citizen science game interface’. JCOM 18 (01), A05. https://doi.org/10.22323/2.18010205.

Newman, G., Zimmerman, D., Crall, A., Laituri, M., Graham, J. and Stapel, L. (2010). ‘User-friendly web mapping: lessons from a citizen science website’. International Journal of Geographical Information Science 24 (12), pp. 1851–1869. https://doi.org/10.1080/13658816.2010.490532.

Nold, C. (2017). ‘Device studies of participatory sensing: ontological politics and design interventions’. Doctoral thesis. London, U.K.: University College London.

Preece, J. (2016). ‘Citizen science: new research challenges for human-computer interaction’. International Journal of Human-Computer Interaction 32 (8), pp. 585–612. https://doi.org/10.1080/10447318.2016.1194153.

Sharma, N., Greaves, S., Siddharthan, A., Anderson, H., Robinson, A., Colucci-Gray, L., Wibowo, A., Bostock, H., Salisbury, A., Roberts, S., Slawson, D. and van der Wal, R. (2019). ‘From citizen science to citizen action: analysing the potential for a digital platform to cultivate different ontologies of nature’. JCOM 18 (01), A07. https://doi.org/10.22323/2.18010207.

Skarlatidou, A., Hamilton, A., Vitos, M. and Haklay, M. (2019). ‘What do volunteers want from citizen science technologies? A systematic literature review and best practice guidelines’. JCOM 18 (01), A02. https://doi.org/10.22323/2.18010202.

Spiers, H., Swanson, A., Fortson, L., Simmons, B., Trouille, L., Blickhan, S. and Lintott, C. (2019). ‘Everyone counts? Design considerations in online citizen science’. JCOM 18 (01), A04. https://doi.org/10.22323/2.18010204.

Sturm, U. and Tscholl, M. (2019). ‘The role of digital feedback in a user centred development process in citizen science’. JCOM 18 (01), A03. https://doi.org/10.22323/2.18010203.

Authors

Artemis Skarlatidou is a senior research associate in the Extreme Citizen Science group at University College London. She is chairing WG4 ‘Enhance the role of citizen science for civil society’ of the COST Action to ‘promote creativity, scientific literacy, and innovation throughout Europe’ and she is a co-chair of the International Cartographic Association Commission on Use, User and Usability Issues. Her research interests include usability, user experience and interface design of geographic interfaces and she also is interested in risk communication and trust design for technologies which are used to support public engagement and participation in various contexts. E-mail: a.skarlatidou@ucl.ac.uk.

Marisa Ponti is Assistant Professor at the Department of Applied Information Technology at the University of Gothenburg in Sweden. She was Vice-Chair of the COST Action to promote creativity, scientific literacy, and innovation throughout Europe before taking a position as a Scientific Project Officer at the EU DG Joint Research Centre. Her main research interests include the study of Games with a Purpose to solve scientific problems and contribute to social causes. However, her focus is not restricted to games and she is interested in the use of digital technologies to support processes of co-production and co-creation. E-mail: marisa.ponti@ait.gu.se.

James Sprinks is a Research Associate in the Medical Design Research Group at Nottingham Trent University, based within the faculty of Science and Technology. His research considers how citizen science platforms can be designed and implemented to both ensure that the data generated is in a usable format and scientifically robust, while still maintaining a user experience that results in volunteers enjoying and learning from the process. This process encompasses elements of science communication, remote sensing, Human-Computer Interaction, GIS, ergonomics and human factors research. E-mail: james.sprinks@ntu.ac.uk.

Christian Nold is an artist, designer and researcher who builds participatory technologies for collective representation. He is a Research Associate in the Department of Geography at UCL, coordinating the large-scale European ‘DITOs’ citizen science project. In the last decade he created public art projects such as the widely acclaimed ‘Bio Mapping’, ‘Emotion Mapping’ and ‘Bijlmer Euro’ projects, which were staged with thousands of participants across the world. He wrote and edited Emotional Cartography: Technologies of the Self, (2009) The Internet of People for a Post-Oil World (2011) and Autopsy of an Island Currency (2014). His PhD explored the ontological politics of participatory sensing and the potential of design interventions. E-mail: christian@softhook.com.

Muki Haklay is a Professor of Geographical Information Science in the Department of Geography at UCL, where he is the co-director of the Extreme Citizen Science Group (ExCiteS). His research focus on public access to environmental information, participatory mapping and Geographical Information Systems (PGIS), and citizen science. His interest in Human-Computer Interaction and usability stems from the need to make digital technologies useful within participatory processes. E-mail: m.haklay@ucl.ac.uk.

Eiman Kanjo is an Associate Professor in Mobile Sensing & Data Science, in the Computing Department at Nottingham Trent University. She is a technologist and an active researcher in the areas of mobile sensing, smart cities, spatial analysis, and data analytics, who worked previously at the University of Cambridge, Mixed Reality Laboratory and the University of Nottingham as well as the International Centre for Computer Games and Virtual Entertainment, Dundee. She has authored some of the earliest papers in the research area of Urban Sensing and currently carries out work in the area of Smart Campus, Pervasive Retail, Proximity Detection, and the Internet of Things for Behaviour Change in collaboration with many industrial partners and end-user organizations. E-mail: eiman.kanjo@ntu.ac.uk.

How to cite

Skarlatidou, A., Ponti, M., Sprinks, J., Nold, C., Haklay, M. and Kanjo, E. (2019). ‘User experience of digital technologies in citizen science’. JCOM 18 (01), E. https://doi.org/10.22323/2.18010501.