The audience experience of science storytelling: impact profiles from a Q methodology study

Sickler, Jessica; Lentzner, Michelle

doi:10.22323/2.21010203

1 Background

In the last decade, there has been growing theoretical and practical interest in the role and potential impact of storytelling and narrative in science communication, well-illustrated by the special issue of the Journal of Science Communication [Joubert, Davis and Metcalfe, 2019 ]. Recent literature applies the principles of storytelling and narrative in varying ways and they are, perhaps more importantly, aligned with a variety of science communication goals. Literature has discussed the power of narrative for outcomes including persuasion [Dahlstrom, 2014 ], information gain [Riedlinger et al., 2019 ], behavior change [Finkler, 2018 ], and perceptions of science [Brounéus, Lindholm and Bohlin, 2019 ]. In much of the literature, these outcomes are associated with unidirectional goals, that is, where storytelling is a vehicle for conveying facts or changing minds in a specific direction.

Our research was specifically interested in first-person, personal storytelling, such as the shows produced by The Story Collider ( www.storycollider.org ), which differs in substantial ways from other forms of using narrative in science communication. Its goals are not to inform or persuade toward a conceptual understanding, takeaway, or even opinion or position on a topic. The Story Collider’s style of science engagement shares first-person stories about science with public audiences, told by scientists and non-scientists. Their stories are meant to highlight the many ways science intersects with our lives and spotlight the human dimensions of science and scientists. The communication goals are largely open-ended, including shifting stereotypes of scientists, generating feelings of identity and belonging, and emphasizing the accessibility of science to all [Neeley et al., 2020 ; Schinske et al., 2016 ].

This application of storytelling aligns well with the idea of a “cultural approach to science communication”, which positions science communication as a meaning-making process that is intertwined with society’s cultural understanding and meanings [Davies et al., 2019 ]. This framework proposes that experience, identity, narrative, and emotion are all routes by which meaning-making happens. This theoretical framework aligns with the principle of narrative transportation [Green and Brock, 2000 ; Mazzocco et al., 2010 ], which is the guiding principle that underlies The Story Collider’s theory of change.

Through first-person stories, told by scientists and non-scientists, The Story Collider’s live shows use the principles of narrative transportation to activate emotion, empathy, and identity work in relation to the disciplinary umbrella of science. This theoretical underpinning means that personal science storytelling does not strive for a focused learning outcome, recognizing that there is no “single, objectively true meaning that is understood by all audiences at all times” [Neeley et al., 2020 , p. 3]. Rather, the meaning-making process, including activation of identity, emotion, and cognitive engagement is subject to the story-consumer’s incoming mindset, experience, emotion, and interpretation of the story they hear.

1.1 Parallels with the Performing Arts

The Story Collider’s live shows have more in common with the performing arts than to traditional, directional science outreach (e.g., talks, science cafés, film, or exhibitions). Researchers studying the impact and value of the arts have long grappled with the complexities of how to frame research about the benefits and value of aesthetic experiences [e.g., Belfiore and Bennett, 2007 ; Gray, 2007 ; McCarthy et al., 2004 ; National Endowment for the Arts, 2012 ]. In particular, the arts research field has discussed the importance of examining and explaining the intrinsic impacts that result from arts experiences, including frameworks that have moved the field away from an emphasis on instrumental impacts (e.g., knowledge gain, test scores, skills, etc.), which are not the true purpose of an artistic work.

Researchers in the arts have wrestled with the challenges of measuring impacts of aesthetic experiences because they are intrinsic, subjective, and highly personal, and therefore often difficult for an audience member to articulate. A promising body of research, which looked across audience experience and reaction to a variety of performing arts formats, has led to a concrete framework of six constructs of intrinsic impact [Brown and Novak-Leonard, 2013 ]. The six constructs of intrinsic impact were: captivation, intellectual stimulation, emotional resonance, aesthetic growth, social bonding/bridging, and spiritual value. We found this framework to have substantial parallels to anecdotal reports from The Story Collider’s staff and producers, and it became an organizing framework for exploring our study of audience experience and impacts from live science storytelling shows.

Methodologically, the discourse in the arts research field also guided our approach to this study. In the effort to move past measuring impact in instrumental or attendance-based terms, Belfiore and Bennett [ 2010 ] suggest looking to methodologies of humanities and social science research to understand the complex phenomena of aesthetic experience, rather than using instrumental measures for comparing audiences to pre-defined outcomes. While some have emphasized the importance of deep, qualitative research [Radbourne, Glow and Johanson, 2010 ], others found that asking audiences to articulate highly personal, potentially subconscious, impacts is difficult, and noted the value in carefully constructed statements that aid a person in reflecting on what has and has not resonated with their experience [Brown and Novak-Leonard, 2013 ]. In our effort to study the impacts of science storytelling experiences, we wanted a methodology that would build upon both of these points — allowing an exploration where audiences could express their unique, personal experience, while also recognizing the difficulty of articulating the complexity of those experiences off-the-cuff.

2 Study context & objectives

From this grounding in the arts, our study focused on the experience of audiences to The Story Collider’s live shows. These shows are carefully produced with the mission of sharing stories that highlight the many ways science intersects with our lives and that show the human side of scientific endeavor and scientists. Since 2010, The Story Collider has produced more than 300 storytelling shows, each featuring five storytellers, who come from all walks of life — e.g., a senior researcher, a graduate student, or someone whose last science class was in high school. The unifying feature is that each person’s personal story tells of some way that science touched their life experience. Before the onset of the COVID-19 pandemic, The Story Collider hosted live shows regularly at Home Stages in over a dozen cities in the United States, Canada, and New Zealand, in venues such as bars and theaters. In 2020, The Story Collider began producing a weekly live virtual show, which quickly cultivated an audience from around the world to share these live (but no longer in-person) storytelling shows.

Over the years, Story Collider staff and producers have noted examples and anecdotes of a variety of ways people report impact from shows and storytelling. But it has been extremely difficult to systematically study and document these impacts, due to many of the challenges described in research on the impact of performing arts. Our research sought to explore and articulate the complex ways that experiences with live shows create personal meaning for audiences. Specifically, the study was guided by two questions: 1) What are common profiles of impact, that is, common ways that audience members internalize the impact of live shows? 2) Are there any patterns that suggest a relationship between impact profiles and audience characteristics?

The study shared in this paper was conducted between August 2020 and January 2021, in the midst of the shift to virtual live shows during the ongoing pandemic. For the purpose of the study, we combined audiences’ reflections of impact from pre-pandemic in-person shows and the current virtual shows.

3 Method

Our study was grounded in the idea that the impacts of live, personal storytelling shows, much like performing arts experiences, are inherently subjective. Each audience member constructs their own meaning, as the stories and storytelling interact with their life experience, prior beliefs, worldviews, and emotions. However, based upon anecdotal evidence from The Story Collider’s staff, we also felt it was possible to identify several typical, coherent perspectives about what made live shows personally meaningful — social perspectives. A social perspective is a generalizable pattern of views, or a distinct way of thinking, about a given topic that is held across individuals [Webler, Danielson and Tuler, 2009 ]. While a social perspective does not precisely represent every nuance of an individual’s thinking, it identifies the underlying patterns that shape a view on a given topic.

For this goal, we used Q methodology, which is designed to capture subjectivity and multi-faceted perspectives in a way that allows researchers to construct portraits of common social perspectives [Brown, 1980 ; Brown, 1993 ; McKeown and Thomas, 1988 ; Stephenson, 1952 ]. Q methodology has primarily been used in the social and political sciences including public health [Barrense-Dias et al., 2020 ; Cross, 2004 ], environmental policy [Addams and Proops, 2000 ; Brown, 2019 ; Danielson, Webler and Tuler, 2009 ], and perceptions of animals and animal rights [Kalof, 2000 ; Sickler et al., 2006 ]. In the education sector, the method has been applied to understand how visitors conceive what is fun about zoo-going [Sickler and Fraser, 2009 ], and teachers’ views of instructional practices and professional development [Baron et al., 2018 ; Rodl, Cruz and Knollman, 2020 ].

In Figure 1 , we present an overview of the steps of Q methodology and how we applied those steps in this study. In brief, Q methodology looks for coherent, multi-faceted, commonly-held viewpoints that exist about a given topic. A viewpoint is a distinctive construction of all the ideas that exist on the topic. Through this methodology, a small set of participants who hold distinct points-of-view on the topic are asked to sort a set of statements, which are a representative sample of the full population of opinions on the focal topic. By sorting the sample of statements, each participant creates a unique picture that represents their experience or viewpoint. Q analysis applies an inverted factor analysis to find patterns in the individual sorts, highlighting similarities and differences between the complete, multi-dimensional perspectives of individuals. These patterns reveal common social perspectives, or groups of sorts that were similar in key ways. By looking at those underlying patterns, Q methodology reveals a complex pattern within a set of viewpoints on the topic.

We selected Q methodology because of the distinctive way it generated insights into audience perceptions, as compared to traditional quantitative (survey-based) or qualitative (focus groups or interviews) methods. In contrast with a traditional survey, Q doesn’t try to assess an audience against the researchers’ viewpoints, but uncovers the ideas and relationships that are deeply embedded in the audience’s viewpoints [Brown, 1993 ], and constructs meaning from those patterns. A benefit Q methodology provides over qualitative methods is that it forces each study participant to reflect and prioritize what was more or less valuable or relevant within their experience. This process of contextualized prioritization combats desirability bias (strongly agreeing with many things) and provides quantifiable data about the relative strength of each type of impact within a given participant’s lived experience. Focus group discussions alone revealed the range of categories impact, but not how they might fit together for each individual’s story nor what might be common patterns within those perspectives.

Figure 1 : Depiction of the generalized steps of a Q methodology study and an overview of how each step was applied in this study.

3.1 Our application of Q methodology

Step 1: defining the concourse. To ensure we had the most complete starting point of the full range of ideas and opinions that exist among audiences about the value or meaning derived from Story Collider shows, we began with a process to collect “stories of impact” from a breadth of current and past audience members. We collected short reflections with an online form, distributed as an open call to Story Collider audiences. The call simply asked: “Tell us a way that Story Collider has influenced you”. A small incentive ($5 gift card) was offered as a thank-you for participation. We also held three virtual focus group discussions with 14 past audience members for deeper discussions to elicit descriptions and examples of all of the possible ways that people experienced personal meaning from attending live shows. The group discussion allowed participants to build on and contrast with one another’s statements, which helped unearth and clarify a variety of perspectives. From these qualitative data, we identified each specific statement of impact made, looking for coherent excerpts that expressed an opinion of value or meaning derived from Story Collider show(s). These statements, which were direct excerpted quotations from audience members, were the concourse.

Step 2: identifying a representative sample of statements. We engaged in an initial round of inductive coding, identifying and describing emergent themes represented by these excerpts. Our initial draft of inductive codes included themes of being absorbed by the storytellers, of feeling connection with storytellers and audience members, of experiencing strong emotions, of self-reflection, of having a new perspective on the practice of science, and of having curiosity piqued. With this first draft of themes, we revisited the literature on intrinsic impact in the performing arts [Brown and Novak, 2007 ; Brown and Novak-Leonard, 2013 ] and found substantial similarities between constructs of intrinsic impacts in the arts and the themes of impact voiced by Story Collider audiences. This became the starting point for creating the conceptual framework for the Q methodology study. We made some changes to the arts framework, modifying the definitions to suit what we heard from audiences about The Story Collider experience (e.g., the science emphasis), expanding or adding constructs to reflect distinctive views we heard about The Story Collider (e.g., perspective on science), and eliminating constructs from the arts research that had not emerged as relevant (e.g., spiritual value). The final conceptual framework for our study is shown in Table 1 , which is what we used to code and organize the ideas and statements that came from the Step 1 data — the concourse for the Q methodology study.

Table 1 : Conceptual framework of the range of ways The Story Collider live shows impact audiences [adapted from Brown and Novak-Leonard, 2013 ].

This conceptual framework was the grounding of our process to ensure that we selected a representative sample of statements from the full concourse (the population of ideas, in the framing of a Q methodology study). All excerpted statements were re-coded using this framework, so that each individual statement of impact was associated with a construct in Table 1 . We reviewed hundreds of the statements, looking within each construct (many of which were individuals saying the same idea with very similar words and phrases), and engaged in an iterative process to identify a sample of statements that represented the full range of ideas captured within each construct. The goal was to select statements that covered all essential ideas, without too many statements that were redundant as restatements of one another. Staff from The Story Collider aided in this review and vetting process. We ultimately selected 28 statements (S1–S28) within the seven conceptual categories. Some constructs contained a wider range of specific sentiments (e.g., social bridging and bonding); for those categories, we selected more statements to ensure all of the ideas were represented. The sample of statements (S1–S28) is presented in Table 2 , organized by construct. This sample represents the breadth of ideas expressed by audience members, and statements use their phrasing verbatim, as much as possible, with any editing done primarily for clarity and brevity.

Table 2 : Final sample of statements (S1–S28) used as prompts for participant sorting, organized by conceptual framework category.

Step 3: sorting by participants with distinct points-of-view. In the final step, we recruited 29 past Story Collider audience members to be study participants (P1–P29; their characteristics are described below). They were asked to complete a sorting activity (Q sort), using an online interface that allowed them to sort the 28 statements into a distribution from “most like my experience” to “least like my experience” into the pyramid distribution shown in Figure 2 . Participants were given the following sorting instructions:

“We are interested in the many different ways Story Collider audience members have experienced personal meaning or influence from attending live shows. On the next screens, you will review a series of statements. Each one is something someone has said about how Story Collider has been meaningful to them. They are all opinions. You will sort the cards based on your personal feelings and experiences about how The Story Collider has impacted you most significantly. There are no right or wrong answers. We want you to focus on the experience of attending live shows as an audience member — whether in-person or virtual. (But not the podcast or workshops.)”

After the sorting exercise, participants were asked two follow-up questions to reflect on their thinking behind the extreme ends of their sorts. They were asked to “Look at the statements you put in the far right of your sort — most like you. Tell us about why you feel those most/least describe your experience”. Sorters had the ability to re-view their sort while answering. The responses to these questions would be used to support and verify our interpretation of the results of the quantitative analysis of the sorts; quotations from these responses are included alongside the quantitative results in this paper. Participants also provided demographic data, which we used to look for evidence or absence of patterns in who tended to contribute to each social perspective.

Figure 2 : Image of the distribution shape into which participants sorted the 28 statements; at the end of sorting, each box contained one statement, ranked from most to least like the participant’s view of their experience with Story Collider shows.

3.2 Study participants

Q methodology does not require a large number of sorts; rather, it relies on sorts from participants with clear points-of-view on the topic [Webler, Danielson and Tuler, 2009 ]. We collected sorts over a month-long period, monitoring responses and engaging in targeted recruitment to ensure we obtained a set of sorts from participants with a variety of characteristics. Specifically, we sought a group of participants who represented a variety of experience with live shows (e.g., virtual, in-person, and both), professions (STEM, non-STEM), ages, gender identities, and racial/ethnic identities. As the study asked people to reflect generally on the impact of The Story Collider’s live shows (rather than a momentary reaction to a single show), we limited participants to audience members who had experience with at least two live shows in the past. A $25 gift card was offered as an incentive for the time to complete this online activity (typically 15–20 minutes).

The final set of 29 participants (P1-P29) whose sorts are included in this study represented a diversity of the characteristics (demographic, career, and Story Collider-related) we thought could contribute to distinct views of impact. About two-thirds identified as women and one-third as men; ages ranged from 18 to 74; and while most described their race/ethnicity as white, one-quarter of participants described themselves with another racial or ethnic identity (which included Black, Latina, Arab, Indian, and Asian). Participant sorters represented a range of professions; just under half were science, technology, engineering, and/or mathematics (STEM) professionals, while 21% worked entirely outside of the STEM fields and more than one-third worked in science communication. And while we expected participants would have some science interest (given their participation in a leisure activity with a science theme), the group of participating sorters showed a range of interest levels from 6 to 10 (on a 10-point scale of interest). Finally, under one-third of participating sorters had only attended in-person shows (pre-pandemic), under one-third had only attended virtual shows (post-pandemic), and 45% had attended shows in both conditions. Participants’ estimates of the total number of live shows they had ever attended ranged from two to 20.

3.3 Q analysis & interpretation

Analysis was conducted using the Q analysis software KADE [Banasick, 2019 ], which computes a correlation matrix among all of the Q sorts from the 29 participants, followed by factor analysis on the correlation matrix; we selected Principal Components Analysis (PCA) for this study. This analysis looks for underlying factors that account for differences derived from a collection of individual people whose sorts were highly correlated with one another. For our final analysis, we selected four factors to carry forward for rotation. This was done after several rounds of exploratory analysis, examining the results of solutions with three, four, five, or six factors [as recommended in Webler, Danielson and Tuler, 2009 ]. We found that four factors provided the strongest solution in terms of its clarity (minimizing non-loaders or confounders), stability (maintaining clusters of consistently similar sorts), and distinctness of the factors (that they are not too strongly correlated with one another),

Following a process of factor rotation, which prioritized a solution in which each individual sort is strongly associated with just one factor, the analysis calculates how strongly each person’s sort is correlated with each factor — called a “loading score”. Researchers must identify which participants’ sorts had loading scores high enough to indicate they should be used to define a given factor. In this study, we set a threshold of loading scores of 0.45 or higher in order to associate a participant’s sort as helping to define a factor. While this was slightly more generous than the threshold indicated by an alpha <0.01 level (0.48), we found the lower threshold better included several sorts that had loading scores between the 0.45 and 0.48 level, and felt it was appropriate that those sorts be included to help define the emerging profiles.

The four factors that resulted from this analysis were the foundation for our interpretation of the commonly-held profiles of impact. Most participants loaded (associated strongly) on just one of the four factors, which meant their sorts were used to help construct the composite factor (see Table 3 ). One participant (P22) did not load significantly on any of the profiles, indicating they held a very different opinion from all others. Two participants (P11 and P28) loaded significantly, but negatively, on a factor; this means they sorted in a pattern essentially opposite of the other participants who defined that factor. Their inverse perceptions are factored into creating the composite profile. We will discuss the meaning of these later in the results.

After identifying which participants’ sorts define each factor (via the loading scores), the analysis software uses those individual sorts to generate a weighted composite sort of all statements, which represents the commonalities and patterns of the sorts that defined that factor. The composite sort represents the patterns of the underlying, commonly-held perspective among the stakeholder audience. We refer to that composite as a “profile” for the remainder of this paper. Researchers then review the composite sort data about what statements were higher and lower ranked, how each profile is distinguished from the others, and writes a narrative interpretation of that viewpoint.

Table 3 : Factor Loading Matrix: Results from Q analysis after factor rotation. Highlighted cells indicate a participant’s sort loaded significantly on that factor and ultimately was used to create the definition of the composite sort that defined that profile.

4 Results

Four profiles emerged from the analysis. Each profile represents a distinct point-of-view, which is a composite based directly on the rankings given by the (very similar) sorts that defined it (see Table 3 ). These profiles paint a picture of four distinctive ways that Story Collider audiences experience, think about, and internalize the value of their live show experiences over time. Analytically, the data provide an idealized composite sort for each profile (included with each profile description below). In interpreting these data, we have given each profile a short descriptive title, based on our interpretation of the data:

Profile 1: humanizing science & piquing interest
Profile 2: feeling empathetic & empowered
Profile 3: shared community of science insiders
Profile 4: absorbed in the live show

4.1 Profile 1: humanizing science & piquing interest

Profile 1 derives the strongest impact from live shows in a combination of ways that are at the core of The Story Collider’s stated mission to highlight the many ways science intersects with our lives and reveal the human experience behind scientific work. (The profile’s composite sort is in Figure 3 .) This profile attributes impact to the emotional resonance of the shows and stories, specifically through experiencing the intimacy created between storyteller and audience (S6) and the wide range of emotions it prompts (S5); moreover, it creates a valued, shared emotional experience among the audience as a whole (S10). At the same time, Profile 1 also expresses that shows build their connection with science as a human endeavor (S25), which is exhibited by seeing scientists as complex and emotional people (S22) whose passion for their fields inspires curiosity (S8). This profile exhibits how live shows, despite their intentional lack of didactic information, can be valued for connecting audiences with science; Profile 1 strongly identifies the value of Story Collider shows as being informative “in a stealthy way” (S3).

Introspection and creative stimulation are decidedly not experienced by Profile 1; six of the seven statements ranked lowest in this perspective are in the aesthetic growth (S12, S4, S23) and introspective value (S2, S20, S11) constructs. In other words, someone who holds Profile 1 tends not to walk away from a show feeling inspired to get involved in creative activities or storytelling, nor do the shows prompt them to think deeply about aspects of their own life or careers. For Profile 1, Story Collider shows are impactful as a contained show experience; they are not the prompt for larger individual change.

Figure 3 : Composite Sort for Profile 1: Humanizing Science & Piquing Interest. Dark green statements are those that significantly distinguished this profile from others at the p<.01 level; light green statements were significantly distinguishing at the p<.05 level.

4.2 Profile 2: feeling empathetic & empowered

Profile 2 stood out from all of the other perspectives by strongly connecting with impacts in the categories of emotional resonance, aesthetic growth, and social connection. (The profile’s composite sort is in Figure 4 .) For Profile 2, shows create a sense of shared experience, emotion, and vulnerability, which seems to drive desire to tell one’s own story. The language of the statements that most strongly resonate for this profile clearly reference the emotional qualities of true, personal storytelling. They feel impacts from the intimacy (S6), vulnerability (S9), profound empathy (S21), and absorption (S24) created through storytelling shows. It seems that The Story Collider’s approach of narrative transportation really helps Profile 2 feel connected and seen; this viewpoint was also defined by statements about feeling social connection, that Story Collider shows help them feel validated (S19) and connected with others (S18, S27) through the shared stories (S13). It also, quite uniquely among all of the perspectives, prompted a desire to engage in greater creative expression, including storytelling (S12, S4). In a way, Profile 2 expresses that shows help them feel seen and want to be heard.

In contrast, Profile 2 does not feel that live shows spark any insights on the content (S14, S16, S8, S3) or process (S28, S17) of science. Based on data collected from sorters about their reasoning for placement, it’s not that those in Profile 2 is unaware of those themes in the story, but they are already very familiar with the facts, process, and human endeavor of science. For Profile 2, science storytelling shows are not about making them feel curious (S8) or making science understandable (S14); they already know that science is non-linear (S28) and that things go wrong (S25).

Figure 4 : Composite Sort for Profile 2: Feeling Empathetic & Empowered. Dark green statements are those that significantly distinguished this profile from others at the p<.01 level; light green statements were significantly distinguishing at the p<.05 level.

4.3 Profile 3: shared community of science insiders

Profile 3 stands out from the others by centering its strongest impacts around the various ways Story Collider live shows create a shared community of science enthusiasts. (The profile’s composite sort is in Figure 5 .) Looking closely at the specific statements that rise to the top for Profile 3, there is a clear emphasis on the construct of humanizing science, particularly how shows delve into science as a complex, human profession. For example, a defining statement is that Story Collider shows make Profile 3 “feel more connected to other scientists and science enthusiasts” (S1). Other strong impacts come from the way stories showcase that scientists are complex humans (S22), doing complicated, messy work (S28, S25), and that emotion is part of the work (S13). This sense of shows creating a feeling of shared identity around science careers also emerges in the sense that shows prompt “reflect[ion] on my career or life path” (S2).

While Profile 3 experiences great impact from exploring the complex, messy process of science with others, they are more neutral about the value of the emotional nature of stories (S5, S21). Profile 3 also does not connect with a sense of captivation or absorption in the moment (S24, S7, S15, S10). Much like Profile 2, this viewpoint does not see the value of shows for science learning (S14, S2); as a community of science enthusiasts, it seems likely they already feel science is understandable (S14) prior to a show and aren’t interested in “stealthy” learning (S3). Similarly, they did not strongly connect with the idea that shows can make science feel “more accessible and less authoritarian” (S17). With an identity as science insiders, this statement may have felt less applicable.

Figure 5 : Composite Sort for Profile 3: Shared Community of Science Insiders. Dark green statements are those that significantly distinguished this profile from others at the p<.01 level; light green statements were significantly distinguishing at the p<.05 level.

4.4 Profile 4: absorbed in the live show

The final profile largely experiences impact through the transportive power of live storytelling — becoming absorbed in the stories and emotional experience. (The profile’s composite sort is in Figure 6 .) The construct of captivation (S15, S7, S24) was a clear and defining characteristic of Profile 4; these statements were ranked higher than by the other profiles. For this viewpoint, the value of live shows is being pulled deeply into the moment, experiencing stories that really stick with you, and a generally having an enjoyable experience. Similar to Profile 1’s views, Profile 4 also connects with the intimate environment (S6) and the full range of emotions that they experience as an audience member (S5).

In general, statements within the social connections construct were not ranked highly for this profile. One exception was that Profile 4 does feel impacted by “seeing the world” through others’ point-of-view (S26), but in the context of their sort, this statement seems to have more ties with their value of personal captivation. But Profile 4 clearly does not attribute impact to the category of the humanity of science as a field or process (S17, S28, S25, S22). This difference distinguishes Profile 4 from Profile 1. Finally, although those within Profile 4 are impacted by the emotional experience of being an audience member, they did not derive their impact from the chance to really get to the “core” of the storyteller (S9) or of feeling a shared experience (S19, S21). Like Profile 1, they also do not experience a creative spark from the shows; two of these statements were ranked very low (S23, S4).

Figure 6 : Composite Sort for Profile 4: Absorbed in the Live Show. Dark green statements are those that significantly distinguished this profile from others at the p<.01 level; light green statements were significantly distinguishing at the p<.05 level.

4.5 Relationships between profiles & audience characteristics

With the four profiles established, we qualitatively explored the characteristics of the study participants who had contributed to defining each viewpoint, looking for emergent patterns in who helped define each profile. It is important at this stage to reiterate that Q methodology is used to reveal and understand coherent perspectives that exist among stakeholders, but does not measure their prevalence. Similarly, demographic trends help us understand and explain the perspectives, but are not conclusive associations.

The sorters who defined Profile 1 ( Humanizing Science & Piquing Interest ) were the most diverse and covered the widest range of audience characteristics, including gender, age, science interest, and past show attendance. The eight sorters who comprised Profile 1 ranged in age from 25 to 74; and this was the only profile that included the oldest sorters in our study (those over age 65). Sorters who identified as white and people of color comprised this group at about the same proportion as the full group of sorters. Of particular interest, sorters who defined Profile 1 had the widest range of science interest levels of any of the profiles, ranging from 6 (the lowest level reported) to 10. This group also had the strongest representation of audience-members who do not work in STEM-related jobs; only three of the eight sorters indicated they worked in a STEM profession. Given that Profile 1 was the viewpoint that most strongly connected with statements about discovery and learning about science, it is notable that these outcomes are resonant for audience members who are demographically diverse and those with less direct connection to STEM fields. Some of the comments left by sorters highlight how the show experience and science topics were very meaningful to them, such as a sorter who noted:

“For me it’s all about curiosity and connection! I love learning about new things.” (Sorter P16)

The sorters who comprised Profile 2 ( Feeling Empathetic & Empowered ) were predominantly women who work in STEM or science communication professions. It stood out that this profile, which placed a higher value on the impact of emotional resonance and empathetic connections, was defined largely, although not exclusively, by women.

“Story Collider shows kindle my empathy with others as humans (with all the vulnerabilities and biases than come with being human) in a context (science) whose PR seems to strip away its humanity. …The humanity is essential!” (Sorter P13)

It was also notable that all but one of the defining sorters worked in a STEM profession (5 people) or in science communication (3 people).
These data, along with comments left during sorting, confirm that this professional experience drove their low ranking of statements about learning science content or process; they see its value, but it is not something they need from the shows. In contrast, the statements about “feeling seen” in science stories resonated far more strongly for these women working in STEM fields. Another defining quality of Profile 2 was sparking the desire to engage their creative side and tell their own stories; the data indicate that some (but not all) of these sorters had opportunities to realize these interests; sorters defining this profile tended to have engaged with multiple strands of Story Collider programming (e.g., podcasts, workshops, telling stories).

Profile 3 ( Shared Community of Science Insiders ) was primarily defined by sorters who were young STEM professionals with very high science interest. Everyone who comprised Profile 3 was under age 44, including all of the youngest sorters (in the 18–24 age group). This group was split between genders, and all but one sorter identified as white. The other major characteristic of this group was that nearly everyone had a professional connection to STEM, including being a STEM professional (5 of 7), one science communicator, and one college student (psychology). All rated themselves as very highly interested in science. The personal values evident in Profile 3 clearly aligns with these sorters’ identity as science professionals and enthusiasts. They are impacted by the shared stories — that are authentic, diverse, and speak to the authentic scientific world they know. As one defining sorter put it:

“Listening to live shows has always made me feel a part of a wide community with non-linear paths.” (Sorter P4)

Much like the STEM professionals in Profile 2, their knowledge of science means they do not tend to connect with impacts around science information. In contrast to Profile 2, these sorters did not connect with the emotional resonance of stories or the more personal, empathetic outcomes of the ways stories are told. One sorter summed it up this way: ““I’m not as emotional as the statements” (Sorter P24).

Profile 4 ( Absorbed by the Live Show ) was a less common perspective, defined by just two people who held this distinct view. While extrapolating about demographic details is more difficult with so few people defining this profile, the two had some key similarities. Both defining sorters were women who have attended in-person shows, and neither listened to the podcast. As this profile was largely characterized by the strong sense of captivation, reiterated in their post-sort comments, this is possibly connected to their basis of experience coming primarily from in-person shows, where The Story Collider has more control over the overall atmosphere and conditions of the show experience. One Profile 4 sorter described it this way:

“I was thinking in particular about a show I went to years ago that I *still* think about and talk about. Stories are the way to get people to remember and empathize, and that’s what Story Collider does so well, in part because of the intimacy of the experience.” (Sorter P5)

Also helping define this perspective was one sorter (P28) who had a distinctly opposite view; his sort was negatively correlated with Profile 4. This means that he, for example, ranked a sense of captivation quite low as his experience. In the post-sort comments of this contrary (but defining) view, it was clear that he also felt motivated by the potential of a storytelling show, but felt that one of his show experiences did not fully deliver: “I have found the quality very uneven thus far. Not everyone is a gifted storyteller” (Sorter P28).

5 Discussion

The results of this study emphasize the importance of thinking about the impacts of some science communication experiences as subjective and individualistic. For science engagement programs that are arts-based and non-didactic, like The Story Collider’s live shows, it is critical that organizers and funders do not expect linear, narrow audience impacts; that the engagement experience not be considered instrumental for specific learning or attitudinal outcomes. Just as these issues are being discussed to understand the value of arts participation [Belfiore and Bennett, 2007 ; Gray, 2007 ; McCarthy et al., 2004 ; National Endowment for the Arts, 2012 ], science communication may need to find ways of conceiving of impact differently, more open-endedly, and more subjectively. The data presented here helped confirm that, even as producers and organizers support and build the quality of the craft of storytelling within all live shows, there are many ways audiences are impacted by live storytelling of personal science stories. While the nature of the shows’ impact differs for each individual, clear patterns did emerge. These patterns and profiles had an internal logic that resonated with the anecdotal observations of producers and staff, validated that the core elements of the program’s mission are happening, and provided language to describe the range and complexity of impacts.

One implication of this research is how substantially impact is affected by the “inputs” that each audience member brings to the show experience. For example, two of the profiles (Profile 2 and 3) very similarly de-emphasized the role of shows on enlightening them about science process, information, or topics. Both profiles were largely comprised of people working in STEM or STEM-adjacent fields, and this prior knowledge influenced what wasn’t meaningful for them. However, this analysis of the subjectivity in responses allowed us to move beyond this similarity to understand the complexity and substantial differences between these two perspectives — which were extremely different in what did generate personal meaning. While Profile 2 strongly connected with the emotional resonance of stories and the mutual empathy between performers and audiences of very different backgrounds, views, and experiences, Profile 3 largely had the reverse experience. The sorters who comprised Profile 3 focused not on social bridging, but on a very specific type of bonding — being part of a shared community of science insiders, talking about the real experience of the process of science. Sorters in Profile 3 were not moved by the impacts of emotional resonance or aesthetic stimulation (wanting to tell their story); they simply enjoy being part of this community of people with shared values and experiences.

We feel these distinctions highlight that the impact of science storytelling is very much the result of the live show interacting with each audience member as a whole person, past experience, identity, and present experiences. In post-sorting comments, some sorters even reflected on this view of how the perception of value in Story Collider shows might change over time, in step with their journey. One sorter commented that they wondered if their perception of show impact would have been different for “a past me”, while another could imagine that her sort would change when she was further in her career and having differing needs and identities.

The existence of profiles that emerged in this study highlights an important strength of arts-based science communication as a way to resonate with science enthusiasts and casually science-interested audience members at the same time and with the same communication experience. Because the emphasis is on authentic, personal stories with science as the unifying thread, there is not a content-driven or persuasive communication objective. That means, in any given show, some science enthusiasts are gaining value from feeling part of a like-minded community, where they share in the authenticity of life as a scientist. But in the exact same theater (or virtual theater), audiences who would comprise Profiles 1, 2, and 4 are gaining value from intriguing science ideas, feeling a shared humanity, or being transported to another worldview. The nature of science storytelling, and we imagine other arts-based public engagement modes, creates a much more open-ended way for each audience member to find their personal meaning.

5.1 Limitations

We acknowledge that there is an assumption built into this study design that there is some type of personal meaning or impact experienced by audience members. Our study purpose was to uncover the potential impact areas and dig into the qualities of those impacts. However, if there are segments of the audience who experience little or negative value, they were less likely to have participated in the study or be captured in these data. Additionally, the purpose of this study, and the selected methodology, was to reveal the existence of socially-held perspectives of impact, and to more comprehensively define those profiles to aid The Story Collider team in describing the value of their work. A benefit of Q methodology is that it requires a relatively small number of sorters to achieve this goal. However, it does not assess prevalence of viewpoints in a population of people. In order to say anything definitive about how common each of these profiles is among audiences as a whole, how they vary between audiences at particular venues, or relationships with demographic characteristics, an additional quantitative study would be necessary.

While our team feels confident that the characteristics of the sorters in this study represent typical variation in characteristics of repeat show-goers, this is largely based on anecdotal experience of staff, as there are not systematically collected demographic data about show attendees. Our data were also limited to repeat attendees, so the experiences of one-time or first-time attendees may not align as well with these perspectives. We also note that three sorters did not associate strongly positively with perspectives in our analysis, indicating they hold views that either oppose or are quite different from the viewpoints presented here. This is typical for a Q method study, which allows for very distinct or idiosyncratic views, however it also reiterates that consideration of these results must allow that the perspectives cannot perfectly represent all views within The Story Collider’s audiences, even as they represent common, consistent views for many.

6 Conclusion

The results of this research point to a substantial research opportunity to understand what happens when programs work at the nexus of science communication and the arts. This study began from an interest to capture the voice, perspectives, and subjectivity of how audiences experience Story Collider’s science communication strategy of storytelling. As we examined the audiences’ self-described impacts, it was remarkable how tightly their experiences mirrored frameworks of intrinsic impact in the performing arts [Brown and Novak-Leonard, 2013 ; Brown and Novak, 2007 ]. From captivation to social connection to aesthetic stimulation, Story Collider audiences were expressing nearly identical themes as audiences of theater, music, and dance performances. For The Story Collider, these findings lead to thinking about opportunities to leverage their distinctive, arts-informed value for public science engagement and thinking about the intersections of these values within a larger ecosystem of science communication and education.

In many contexts, there is certainly great value to using an instrumental approach to science communication, one that uses focused communication objectives to inform, persuade, or create a specific feeling [Besley, Dudo and Yuan, 2018 ; Yuan, Besley and Dudo, 2018 ]. For a talk, an op-ed, an activity, or a museum exhibition, there is immense value to designing communication to achieve specific outcomes or objectives. But these may not capture the value of less message-oriented approaches to science communication [Davies et al., 2019 ], particularly those that draw upon the distinctive strengths and affordances provided by artistic media. By expanding our understanding of the resonance between the goals, needs, and approaches of the arts and science communication, we could further the societal aims of both fields.

Acknowledgments

This work was supported by a grant from the Simons Foundation (Award #621445 to The Story Collider, PI: Erin Barker). Our thanks to the staff and leadership of The Story Collider for providing support and input throughout this study.

References

: Addams, H. and Proops, J. L. R. (2000). Social discourse and environmental policy: An application of Q methodology. Cheltenham, U.K.: Edward Elgar.
: Banasick, S. (2019). ‘KADE: A desktop application for Q methodology’. Journal of Open Source Software 4 36, p. 1360. https://doi.org/10.21105/joss.01360 .
: Baron, C., Sklarwitz, S., Bang, H. and Shatara, H. (2018). ‘Understanding what teachers gain from professional development at historic sites’. Theory & Research in Social Education 47 1, pp. 76–107. https://doi.org/10.1080/00933104.2018.1489927 .
: Barrense-Dias, Y., Akre, C., Suris, J.-C. and Berchtold, A. (2020). ‘Opinions of Adolescents on Prevention Related to Sexting: a Q-Methodology Study’. Sexuality Research and Social Policy 17 4, pp. 753–764. https://doi.org/10.1007/s13178-020-00431-3 .
: Belfiore, E. and Bennett, O. (2007). ‘Determinants of Impact: Towards a Better Understanding of Encounters with the Arts’. Cultural Trends 16 3, pp. 225–275. https://doi.org/10.1080/09548960701479417 .
: — (2010). ‘Beyond the “Toolkit Approach”: Arts Impact Evaluation Research and the Realities of Cultural Policy-Making’. Journal for Cultural Research 14 2, pp. 121–142. https://doi.org/10.1080/14797580903481280 .
: Besley, J., Dudo, A. and Yuan, S. (2018). ‘Scientists’ views about communication objectives’. Public Understanding of Science 27 (6), pp. 708–730. https://doi.org/10.1177/0963662517728478 .
: Brounéus, F., Lindholm, M. and Bohlin, G. (2019). ‘Telling it straight — a focus group study on narratives affecting public confidence in science’. JCOM 18 (05), A03. https://doi.org/10.22323/2.18050203 .
: Brown, A. S. and Novak, J. L. (2007). Assessing the Intrinsic Impacts of a Live Performance .
: Brown, A. S. and Novak-Leonard, J. L. (2013). ‘Measuring the intrinsic impacts of arts attendance’. Cultural Trends 22 3-4, pp. 223–233. https://doi.org/10.1080/09548963.2013.817654 .
: Brown, S. R. (1980). Political subjectivity: Applications of Q methodology in political science. New Haven, CT, U.S.A.: Yale University Press.
: — (1993). ‘A Primer on Q Methodology’. Operant Subjectivity 16 (3/4), pp. 91–138. https://doi.org/10.15133/J.OS.1993.002 .
: — (2019). ‘Q Methodology in Research on Political Decision Making’. Oxford Research Encyclopedia of Politics. 29. https://doi.org/10.1093/acrefore/9780190228637.013.980 .
: Cross, R. M. (2004). ‘Exploring attitudes: the case for Q methodology’. Health Education Research 20 2, pp. 206–213. https://doi.org/10.1093/her/cyg121 .
: Dahlstrom, M. F. (2014). ‘Using narratives and storytelling to communicate science with nonexpert audiences’. Proceedings of the National Academy of Sciences 111 (Supplement 4), pp. 13614–13620. https://doi.org/10.1073/pnas.1320645111 .
: Danielson, S., Webler, T. and Tuler, S. P. (2009). ‘Using Q Method for the Formative Evaluation of Public Participation Processes’. Society & Natural Resources 23 1, pp. 92–96. https://doi.org/10.1080/08941920802438626 .
: Davies, S. R., Halpern, M., Horst, M., Kirby, D. A. and Lewenstein, B. (2019). ‘Science stories as culture: experience, identity, narrative and emotion in public communication of science’. JCOM 18 (05), A01. https://doi.org/10.22323/2.18050201 .
: Finkler, W. (2018). ‘The science communication of whale watching: a people-focused science communication approach’. Doctor of Philosophy Thesis. Dunedin, New Zealand: University of Otago. URL: http://hdl.handle.net/10523/8012 .
: Gray, C. (2007). ‘Commodification and instrumentality in cultural policy’. International Journal of Cultural Policy 13 2, pp. 203–215. https://doi.org/10.1080/10286630701342899 .
: Green, M. C. and Brock, T. C. (2000). ‘The role of transportation in the persuasiveness of public narratives’. Journal of Personality and Social Psychology 79 (5), pp. 701–721. https://doi.org/10.1037/0022-3514.79.5.701 .
: Joubert, M., Davis, L. and Metcalfe, J. (2019). ‘Storytelling: the soul of science communication’. JCOM 18 (05), E. https://doi.org/10.22323/2.18050501 .
: Kalof, L. (2000). ‘The multi-layered discourses of animal concern’. In: Social discourse and environmental policy. Ed. by H. Addams and J. Proops. Cheltenham, U.K.: Edward Elgar, pp. 174–195.
: Mazzocco, P. J., Green, M. C., Sasota, J. A. and Jones, N. W. (2010). ‘This Story Is Not for Everyone: Transportability and Narrative Persuasion’. Social Psychological and Personality Science 1 (4), pp. 361–368. https://doi.org/10.1177/1948550610376600 .
: McCarthy, K. F., Ondaatje, E. H., Zakaras, L. and Brooks, A. (2004). Gifts of the Muse: Reframing the Debate About the Benefits of the Arts". Santa Monica, CA, U.S.A.: RAND Corporation. https://doi.org/10.7249/MG218 .
: McKeown, B. and Thomas, D. B. (1988). Q methodology. 1st ed. Vol. 66. Quantitative applications in the social sciences. Sage Publications, Inc.
: National Endowment for the Arts (September 2012). How art works: The national endowment for the arts’ five-year research agenda, with a system map and measurement model, appendix a and appendix b . Measuring the Arts: Research Agendas and a System Map of How Art Works.
: Neeley, L., Barker, E., Bayer, S. R., Maktoufi, R., Wu, K. J. and Zaringhalam, M. (2020). ‘Linking Scholarship and Practice: Narrative and Identity in Science’. Frontiers in Communication 5. https://doi.org/10.3389/fcomm.2020.00035 .
: Radbourne, J., Glow, H. and Johanson, K. (2010). ‘Measuring the intrinsic benefits of arts attendance’. Cultural Trends 19 4, pp. 307–324. https://doi.org/10.1080/09548963.2010.515005 .
: Riedlinger, M., Massarani, L., Joubert, M., Baram-Tsabari, A., Entradas, M. and Metcalfe, J. (2019). ‘Telling stories in science communication: case studies of scholar-practitioner collaboration’. JCOM 18 (05), N01. https://doi.org/10.22323/2.18050801 .
: Rodl, J. E., Cruz, R. A. and Knollman, G. A. (2020). ‘Applying Q methodology to teacher evaluation research’. Studies in Educational Evaluation 65, p. 100844. https://doi.org/10.1016/j.stueduc.2020.100844 .
: Schinske, J. N., Perkins, H., Snyder, A. and Wyer, M. (2016). ‘Scientist Spotlight Homework Assignments Shift Students’ Stereotypes of Scientists and Enhance Science Identity in a Diverse Introductory Science Class’. Cell Biol. Educ.—Life Sciences Education 15 3. Ed. by P. Marsteller, ar47. https://doi.org/10.1187/cbe.16-01-0002 .
: Sickler, J. and Fraser, J. (2009). ‘Enjoyment in zoos’. Leisure Studies 28 3, pp. 313–331. https://doi.org/10.1080/02614360903046649 .
: Sickler, J., Lemcke, K., Gruber, S., Fraser, J., Webler, T., Reiss, D., Lyn, H. and Boyle, P. (2006). ‘Social narratives surrounding dolphins: A Q method study’. Society and Animals 14 (4), pp. 351–382.
: Stephenson, W. (1952). ‘Q-methodology and the projective techniques’. Journal of Clinical Psychology 8 (3), pp. 219–229. https://doi.org/10.1002/1097-4679(195207)8:3%3C219::AID-JCLP2270080302%3E3.0.CO;2-J .
: Webler, T., Danielson, S. and Tuler, S. (2009). Using Q method to reveal social perspectives in environmental research. Greenfield, MA, U.S.A.: Social and Environmental Research Institute.
: Yuan, S., Besley, J. C. and Dudo, A. (2018). ‘A comparison between scientists’ and communication scholars’ views about scientists’ public engagement activities’. Public Understanding of Science 28 1, pp. 101–118. https://doi.org/10.1177/0963662518797002 .

Authors

Jessica Sickler (M.S.Ed., Bank Street College of Education) is Principal and Owner of J. Sickler Consulting, an independent research and evaluation firm. Her research focuses on understanding what works and how people learn through a variety of informal learning experiences, particularly public engagement with science. E-mail: jessica@jsickler.net .

Michelle Lentzner (M.A., Indiana University-Purdue University Indianapolis) is an Evaluation & Research Associate at J. Sickler Consulting. She studies how people learn in museums and other informal learning settings, with strong background in museums of art and culture. E-mail: michelle@jsickler.net .