The landscape of online visual communication of science

Photography would not exist but for scientific investigation, and science would hardly have the form it has today without photography [Wilder, 2009 ]. Images have long been used by scientists to illustrate scientific theories and principles. Whether in the form of natural history illustrations, hand-drawn anatomical tables, symbolic reproductions, creative imagery used to visually represent non-physical data, graphics or macro and microscope photographs, pictures have been playing a prominent role both in the evolution of sciences (medicine in particular) and in the perception of science by the general public [Stafford, 1998 ].

However, in the past few years, the vast majority of the imagery produced by scientists has been functional to scientific papers rather than to science popularisation [Frankel, and Depace, 2012 ]. At the same time, science popularisers have relegated images to a merely decorative role, integrating them less and less with the text and deemphasising their explanatory and informative function [Rodríguez Estrada and Davis, 2015 ]. With the advent of the new web-based information channels in the media landscape, such function appears to have faded up to almost disappear.

1 On the web, scientific images acquire new functionalities: attractiveness and integrability

The evolution of digital technology has been accompanied by the constant reshaping of the web platforms that emerged to compete in the information market. This evolution brought about new media formats in which texts are no longer written, but designed for the contemporary use on different media [Kress and Van Leeuwen, 1998 ]. Web interfaces are no longer structured to provide only information; they also offer entertainment, emotions, and spur user interaction with the contents published. They should also look attractive, as “attractive things work better because interacting with them is more pleasant” [Norman, 2007 ].

Also scientific imagery should adapt to such stringent aesthetic models by acquiring a new important functionality: the ability to attract. This is implicit in the vast majority of astronomical, naturalistic and microscopy images, but it is less predictable in other disciplines. For example, think of the several websites of clinics and patients’ associations: the relation between digital technology and visual communication has reshaped the colours and pictures that appear on them to adapt to the new standards. Websites once showing anatomical drawings having the purpose of illustrating the clinical course of a desease or surgery procedures, have now almost entirely replaced explanatory pictures with stock photos of smiling people looking extremely healthy [Thompson, 2012 ].

The web has changed the definition of a good or bad scientific picture. In a lab, a good picture may highlight a scientific result more effectively than words, whereas a bad quality picture may jeopardise a result or a discussion [Watson and Lom, 2008 ]. On the web, a good picture is one that best integrates with the platform hosting it. It will function as a filter for the user that — drawn in by the picture — will visualise it for a quick reference on the content of the text, to work out at a glance the information of interest within a virtually infinite landscape of contents scattered over thousands of pages. In choosing the picture to publish online along with an educational article, the most important parameter is no longer its explanatory function, but its seamless integration with the layout. The size and the orientation of a picture become the primary requirements in making the choice, together with colours and the type of visuals. Accuracy, understandability and mapping of the picture appear to be concepts relegated to the illustration tables of scientific books from the past. Attractiveness, accessibility and integrability are the mandatory requirements in the current landscape.

The distinction between pictures having the primary function of offering a visual experience to draw the attention of the users and those having the function of providing an informative solution therefore becomes more and more blurred, so much so that the explanatory function is often subordinate to technical functionality. Scientific pictures turn into a promotional product, having the purpose of drawing the attention of web users, having them linger on an article and click. Sometimes, the visual function may prevail so much that there is not any relation whatsoever between the content of the article and the picture. In cases as such, articles illustrating the concept of statistical significance, describing climate change or explaining quantum physics are published along with pictures of kittens.

However, this does not mean that the visual communication of science is inevitably compromised by the need to emotionally and visually involve the reader. On the contrary, this need has been a driver to the enrichment of visual communication of science, which becomes much more than mere knowledge dissemination. It turns into an engagement instrument, arouses interest in scientific topics, creates trust in the scientific community, and reaches that large share of people that are not interested in science, but may be attracted by a video, a spectacular picture or eye-catching graphics.

2 Transmedia and recontextualization of scientific issues

With millions of pictures loaded online every day “experts and audiences alike live constantly immersed in an environment in which the visual part plays a central role, especially in the presentation of technical-scientific issues” [Bucchi and Canadelli, 2015 ]. The web provides a unique environment to combine — in a highly structured and interactive way — visual contents of different types (pictures, videos, maps, graphs, animations) to explain a complex concept within a single screen and with a minimum quantity of text.

The relation between not always linear pieces of information, such as those relating to health, can be displayed through visual supports such as interactive maps, a genogram and infographics. Complex concepts can be understood by interacting with transmedia objects that overcome and expand the information boundaries that drawings and photographic pictures imply. Whereas in the past visual communication of science served the purpose of documenting the world to spread knowledge, in an attempt to fill the cultural gaps of society, today the wide range of communication instruments made available by technological development makes it possible to design visual products that are ideally effective in having users understand complex scientific concepts at a very high level of detail.

The storytelling technology — designed to meet the need for uniformity in e-publishing and to provide access to data to external apps though adaptive layouts — has further encouraged the search for new formats of visual communication of science, more oriented towards the emotional involvement of users through interaction and co-creation of contents. Social media platforms, along with their apps and other easy-to-use software tools, make available the creation and sharing of multimodal artefacts to an unprecedented number of people. This provides a creative space for multimodal text production including the emergence of new genres, that is, new configurations of typified communicative forms that fulfil specific and diversified social functions [Adami and Jewitt, 2016 ].

One of the most famous examples is the communication campaign for the Rosetta mission. The European Space Agency (ESA) leveraged visual communication through social media platforms to engage the public with each step of the mission. This aroused interest in all the unanswered questions of science that the public perceives as distant from their lives. By telling the space adventure in real time through compelling visual storytelling that gives room to involvement and interaction, ESA succeeded in fascinating the public and the media all over the world. By drawing the public’s attention to the most critical moments of the mission, it increased people’s awareness on the risk and therefore its acceptance [Bauer, McCaughrean and Landeau-Constantin, 2016 ].

Cartoons, digital storytelling, videos and twitter newsfeeds were the key elements of this campaign that took the public’s breath away at each risky step of the mission. The space manoeuvre technology of the Philae lander was metaphorically displayed as a difficult task that the lander had to complete. “The anthropomorphic approach chosen for the visual representation of the robot made it possible to present the unexpected chain of events, in terms of common feelings: fear, surprise, commitment, and even humour” [Mignone et al., 2016 ].

Therefore, in the digital era, visual communication of science should no longer be regarded as an instrument used to simplify complex concepts. Instead, it should be conceived as a way to recontextualise scientific issues within other domains, depicting scientific phenomena in different ways to reach different aims [Luzón, 2013 ].

3 The public takes part in the production of the visual culture

However, the production of scientific images is not the sole preserve of scientists or popularisers. On the web, people use pictures to share information and news, express opinions, elicit emotions, persuade [Chung and Yoon, 2013 ]. Everyone can produce pictures, share them, modify pictures shared by others, reproduce them. Everyone becomes a prosumer (producer and consumer) of contents whose “depiction parameter meets a specific cultural standard that determines practices and methods” [Branzaglia, 2011 ], blurring the boundaries of what were once considered to be firmly established, clear-cut categories and distinctions between media products and production processes, distribution and consumption, amateur and professional, public and private, official and unofficial, authentic and inauthentic [French, 2014 ].

The phenomenal rise in user generated content — especially the uploading and sharing of image content across a range of different social media platforms — is possibly the most significant development of all in terms of how non-media professionals can now participate in the production and reproduction of visual culture in ways previously unimaginable [French, 2014 ]. Such exponential growth of image content on the web is highly encouraged by mobile technology ¹ and the social networks.

The more attractive is an image, the more it elicits emotions, the more people wants to share it through social networks. As image content prosumers, web users take part in communication flows, and change them.

Through ‘copy-and-paste’, visual artefacts are easily assembled, bricolaged, edited, manipulated and reused, from one media to another [Adami and Jewitt, 2016 ]. Filters are used with pictures in order to iron out dissonances, represent the user’s emotional experience rather than reality, emphasise the dramatic character, compensate for an absence [Benzaquen, 2013 ]; pictures are cut to adapt them to the format of the platform, square or rectangular, and to the orientation, vertical or horizontal. Finally, captions and reference to sources are removed, thus completely decontextualising the image. When a picture is taken out of its context and shared again across other virtual platforms, it may become a visual product conveying meanings that overturn the original one.

Such reworkings give rise to new visual science communication content created by users: monographic collections reminiscent of the Cabinets of Wonder or Wunderkammer, depicting several aspects of a single subject in many different ways, are hosted by the Tumblr platform; animated GIFs created by teachers for educational purposes; internet science memes, that is collections of products linked to the internet subcultures — connoted by a humorous intent that one can hardly grasp without adequate scientific literacy; video parodies made by doctoral students to explain scientific concepts or to show the backstage of a researcher’s life. All the online visual scientific material may ideally contribute to scientific information — or even misinformation.

4 The connection between the decontextualisation of images and science fakes

Whilst the recontextualisation of scientific concepts through multimedia visual content helps understanding information, decontextualisation of images may result in the opposite and lead to the creation of pseudo-scientific contents.

When a user takes a picture from a website, shares it through different social media platforms removing its caption or source references, comments it attaching new meanings to it, i.e. decontexualises it to then share it in other virtual places stripped of its initial information, fakes emerge almost in all cases.

This is how pictures of the moon surface dated 1885 and polar bears stranded on the Scottish coasts emerge on the web. Stock photos of an earthquake are used as visual evidence of the violence of a tsunami somewhere in the other hemisphere of the world. Pictures of a tornado in New York magically become the imagery of Hurricane Sandy. April Fools’ Day spoofs spread on YouTube become “science” when shared by the majority of the 1.9 million followers of the @scienceporn Twitter account (on Twitter, the suffix “porn” denotes trash content) who relabelled them with the hashtag #science. The picture of actress Susan Marie Frontczak taken during a theatre performance on the life of Marie Curie is mistaken for a portrait of the scientist and used instead of the actual picture on the commemorative stamps of Zambia, Togo, Mali and Guinea.

Falling into error is not unusual. Distinguishing what is true and what is false requires a high level of visual literacy. One need only think that the vast majority of the scientific imagery we know results from very high-level photo editing. In disciplinary sectors such as astronomy and biomedicine where pictures are nothing else than a large quantity of raw data often incomplete and colourless, image processing is fundamental to help understanding what pictures depict. Colourless molecules and astronomical bodies are coloured to be recognised by the public and changed to make them suitable for telling a story, triggering emotions, striking the collective imagination.

Whereas words can be deceptive, videos and pictures have always been held as evidence of reality. We tend to think of scientific images as real and objective because we consider their source as being authoritative [Sturken and Cartwright, 2009 ], but the objectivity of these images is also the result of craft on the part of the makers, rather than an essential quality of these images [de Rijcke and Beaulieu, 2007 ]. Although a picture can be accurate in reflecting a scientific principle, and as much as its source is credible, it is nothing else but a point fixed in time. Without a context, it is not possible to know what is around the picture, what was omitted in the capture, and therefore our understanding of it is under the influence of our cultural experience and our feelings. What we see in a picture is subjective, and therefore it is further susceptible to any information provided with it, either true or false. Every single picture contains a lot of information connected to the culture that produced it, but is decoded and interpreted according to the cultural background of the viewer. A picture may be worth a thousand words, but it does not speak for itself. The more complex it is, the more difficult its interpretation, and the more we need its context to grasp what we see: we need a guide to look beyond appearances. Insofar as what we see is consistent with what we know, we can interpret pictures. However, when facing scientific images that are particularly complex, our ability to decipher the information declines. For example, while a DNA double helix can be quite easily recognised, when looking at the picture of a DNA microarray, one would quite probably be puzzled and think it is a set of coloured dots or a pixellated image. The more an image is removed from its context, the more it may carry different interpretations and meanings with it.

Things are even more complicated when images are edited, i.e. artificially manipulated. Although we may think that image manipulation is something relatively new — a product of the digital era — there are countless cases of fake photographs in history [Farid, 2008 ]. Despite all this, people still have difficulty at identifying forgery in digital images, even in a context where they have been explicitly told to look for it. Quite the opposite, they often question the authenticity of original pictures [Schetinger et al., 2015 ]. One of the main problems connected with the spreading of forged pictures is that they not only make us fall into error, tricking us into believing fake news, but they also create false memories. When people are exposed to new and misleading information on a past event through manipulated pictures, their memories can often be distorted [Sacchi, Agnoli and Loftus, 2007 ] and they will tend to remember doctored information instead of real one, thus contributing further to the spreading on the web of such phoney or manipulated information.

5 The confluence of social research and digital culture as a tool for analysis

Pictures have become such an important part of our reality that we no longer conceive them as mere representations describing the surrounding world. We have made them real and capable of generating new meanings [Johnson, 1999 ]. It is so easy to create, modify, break down and rearrange pictures in different ways, or even put them into a new context, that this has generated a huge quantity of new images originating from a single matrix, matched by as many meanings. In addition, the fact they are shared across different platforms and contexts is crucial in shaping the way they are understood.

Within this landscape, it is harder and harder to tell whether scientific visual content also serves the function of spreading knowledge and having people understand complex concepts. All the statistical tools that the web provides make it possible to count how many people are reached by the contents published, how many like them and how many interact by sharing such contents with others, commenting and even modifying them. We can analyse the components of an image to establish its meanings, and how an astronomical picture coloured with different filters is perceived by the public, or define what role a caption plays in the understanding of a scientific image by different audiences [Smith et al., 2011 ].

The current social research methods having images as their subject of study are now consolidated, both in the fields of qualitative and quantitative analysis of space and time dynamics. Today there is an array of digital tools able to identify patterns within the so-called big data.

What we cannot survey yet is the depth of understanding, nor how the audience’s knowledge improves after looking at the visual content. We know the number of engagements, of impressions, but not the quality of what is learned by our followers. We know that the original message moves from node to node, but not how it is transformed or decontextualised. Big data, a real treasure trove of information for quantitative research, is also the limit of qualitative research.

Therefore, before approaching the “cultural” analysis of digital scientific imagery, it is fundamental to develop a good knowledge of online platforms, understand how they work, see into the algorithms that govern them, the changeability of the serendipity models, the dynamics that dominate the sharing of scientific images connected to controversial issues, the intersections between the technology we use to produce images and the one we use to attach a meaning to them, the confluences of sharing platforms and instruments promoting (or curbing) intertexuality, which is no longer made of captions and texts alone, but now includes tags, emoticons and comments from the users.

With common people uploading millions of pictures daily through a variety of constantly evolving online instruments, the knowledge of the digital environment is now an unavoidable requirement to any study wishing to approach online scientific images, together with the relations existing between the products co-created and enjoyed by the users and the ways all this unfolds.

6 Conclusion

The landscape of online visual communication of science is very complex because images and visualisation instruments are intertwined with the technology that generates them, and their requirements and functionality change to conform to technological development. In this landscape, it is harder and harder to tell whether scientific visual content also serves the function of spreading knowledge and having people understand complex concepts. We know the mechanics of the web that allow us to identify the space and time dynamics of pictures or the characteristics that make them an information product, we can extrapolate any image layer to see whether it was artificially manipulated, measure the quantity and quality of the pictures shared online by people, but in order to identify the interpretation context of a picture, social research has to add new capabilities and interdisciplinary collaborations.

Also science communicators have to face new challenges as the use of pictures to illustrate, to explain, to facilitate understanding is no longer enough. Present-day audiences must be emotionally and proactively involved. The web constantly generates forms of participatory communication with contents being co-created, shaped, modified and regenerated. New genres emerge thanks to the public’s activity and large part of the information shared is managed by the users instead of communicators. The risk implied here is that, when it happens that the public lacks the visual and scientific literacy to understand visual content, misinformation deriving from image manipulation or decontextualisation starts to spread.

References

Adami, E. and Jewitt, C. (2016). ‘Special Issue: Social media and the visual’. Visual Communication 15 (3), pp. 263–270. https://doi.org/10.1177/1470357216644153 .

Bauer, M., McCaughrean, M. and Landeau-Constantin, J. (2016). ‘The Strategy and Implementation of the Rosetta Communication Campaign’. Communicating Astronomy with the Public Journal 19, pp. 5–11. URL: http://adsabs.harvard.edu/abs/2016CAPJ...19....5B .

Benzaquen, S. (2013). #Holocaust: social networks, production of images and the cultural memory of the holocaust. Presented at the New Scholars . New Research on the Holocaust, University of Toronto, Canada. URL: https://www.researchgate.net/publication/266201134_holocaust_social_networks_production_of_images_and_the_cultural_memory_of_the_holocaust .

Branzaglia, C. (2011). Comunicare con le immagini. Milano, Italy: Bruno Mondadori.

Bucchi, M. and Canadelli, E. (2015). Nature immaginate. Immagini che hanno cambiato il nostro modo di vedere la natura. Sansepolcro (AR), Italy: Aboca Edizioni.

Chung, E. and Yoon, J. (2013). ‘An Analysis of Image Use in Twitter Message’. Journal of the Korean BIBLIA Society for library and Information Science 24 (4), pp. 75–90. https://doi.org/10.14699/kbiblia.2013.24.4.075 .

de Rijcke, S. and Beaulieu, A. (2007). ‘Essay Review: Taking a Good Look at Why Scientific Images Don’t Speak for Themselves’. Theory & Psychology 17 (5), pp. 733–742. https://doi.org/10.1177/0959354307081626 .

Farid, H. (2008). ‘Photography changes what we are willing to believe’. Australia Policy Online . URL: http://apo.org.au/node/16963 .

Frankel, F. C., and Depace, A. H. (2012). Visual Strategies: A Practical Guide to Graphics for Scientists & Engineers. Design by Sagemeister Inc. London, U.K.: Yale University Press.

French, L. (2014). ‘Researching Social Media and Visual Culture’. In: Social Media in Social Research: Blogs on Blurring the Boundaries. Ed. by K. Woodfield. London, U.K.: National Centre for Social Research / Sage.

Johnson, S. A. (1999). Interface Culture: How New Technology Transforms the Way We Create & Communicate. New York, NY, U.S.A.: Basic Books.

Kress, G. and Van Leeuwen, T. (1998). ‘Front Pages: (The Critical) Analysis of Newspaper Layout’. In: Approaches to Media Discourse. Ed. by P. Garret and A. Bell. Hoboken, NJ, U.S.A.: Wiley-Blackwell, p. 304.

Luzón, M. J. (2013). ‘Public Communication of Science in Blogs: Recontextualizing Scientific Discourse for a Diversified Audience’. Written Communication 30 (4), pp. 428–457. https://doi.org/10.1177/0741088313493610 .

Mignone, C., Baldwin, E., O’Flaherty, K. S., Homfeld, A. M., Bauer, M., McCaughrean, M., Marcu, S. and Palazzari, C. (2016). ‘How a Cartoon Series Helped the Public Care about Rosetta and Philae’. Communicating Astronomy with the Public Journal 19, pp. 12–18. URL: http://adsabs.harvard.edu/abs/2016CAPJ...19...12M .

Norman, D. (2007). Emotional Design: Why We Love (or Hate) Everyday Things. New York, NY, U.S.A.: Basic Books.

Rodríguez Estrada, F. C. and Davis, L. S. (2015). ‘Improving Visual Communication of Science Through the Incorporation of Graphic Design Theories and Practices Into Science Communication’. Science Communication 37 (1), pp. 140–148. https://doi.org/10.1177/1075547014562914 .

Sacchi, D. L. M., Agnoli, F. and Loftus, E. F. (2007). ‘Changing history: doctored photographs affect memory for past public events’. Applied Cognitive Psychology 21 (8), pp. 1005–1022. https://doi.org/10.1002/acp.1394 .

Schetinger, V., Oliveira, M. M., da Silva, R. and Carvalho, T. J. (2015). ‘Humans Are Easily Fooled by Digital Images’. arXiv: 1509.05301 .

Smith, L. F., Smith, J. K., Arcand, K. K., Smith, R. K., Bookbinder, J. and Keach, K. (2011). ‘Aesthetics and Astronomy: Studying the Public’s Perception and Understanding of Imagery From Space’. Science Communication 33 (2), pp. 201–238. https://doi.org/10.1177/1075547010379579 .

Stafford, B. M. (1998). Good Looking: Essays on the Virtue of Images. Cambridge, MA, U.S.A.: The MIT Press.

Sturken, M. and Cartwright, L. (2009). Practices of Looking: An Introduction to Visual Culture. Oxford, New York: Oxford University Press.

Thompson, R. (2012). ‘Looking healthy: visualizing mental health and illness online’. Visual Communication 11 (4), pp. 395–420. https://doi.org/10.1177/1470357212453978 .

Watson, F. L. and Lom, B. (2008). ‘More than a Picture: Helping Undergraduates Learn to Communicate through Scientific Images’. Cell Biology Education 7 (1), pp. 27–35. https://doi.org/10.1187/cbe.07-07-0045 .

Wilder, K. (2009). Photography and Science. London, U.K.: Reaktion Books.

Author

Cristina Rigutto is social media editor at Public Understanding of Science Journal and a communications consultant. She trains researchers to build the communications skills, networks, and relationships they need to engage effectively in the public discourse about the research issues. She is a co-author of “Manuale di redazione medico-scientifica” and author of “Twitter per ricercatori” and she writes a monthly column — “Connecting minds” — for TechEconomy Magazine on the use of digital technology to improve science outreach. Previously she was managing Director for Italy of a multinational media company. Earlier she spent twenty years in industry leading corporate marketing and communications for major consulting and contracting companies. She has completed studies in Economics, Social Media Marketing and Science Communication. E-mail: mail@cristinarigutto.com .

Endnotes