One size does not fit all: gender implications for the design of outcomes, evaluation and assessment of science communication programs

1 Context

Science communication training has become a growing element in the education and professional development of scientists around the world [Baram-Tsabari and Lewenstein, 2017 ; Bray, France and Gilbert, 2012 ]. There is limited research on the effectiveness of science communication workshops and courses on communication practice [Miller, Fahy and the ESConet Team, 2009 ; Rodgers et al., 2018 ], with many studies on the efficacy of such programs based on “anecdotes and basic self-report evaluations” [Baram-Tsabari and Lewenstein, 2017 , p. 296] and not on necessarily on communication theory or validated protocols. The lack of comparative data between science communication training programs prevents the development of an evidence-base from which to identify the most effective programs for different scientists or contexts [Baram-Tsabari and Lewenstein, 2017 ]. In the absence of such data “programs focus primarily on developing specific skills and are only loosely based on social science research about what makes science communication effective” [Besley, Dudo and Storksdieck, 2015 ]. The first two authors of this paper often deliver science communication training to scientists, researchers and policy makers around the world. Comments from workshop participants, and the personal experiences of all three authors and their colleagues, suggest that these workshops influence far more than specific communication skills. Anecdotal evidence suggests a variety of benefits may arise from participating in science communication training, including the development of confidence, leadership skills, interpersonal empathy, anxiety reduction, and many others. Gender bias in science has been shown to start as early as kindergarten and propagate throughout a women’s career [Bian, Leslie and Cimpian, 2017 ; Cimpian et al., 2016 ]. Therefore the inherent experience of being a woman ‘doing’ science communication is different to that of men, so perhaps the experience of science communication training may also differ by gender? This paper describes a preliminary test of ways to assess whether science communication training influenced more than just communication ability for participants. We conducted this research in the context of improvisation (theatre game) based workshops, but with a goal of addressing the broader question of effects of in person, live science communication training and whether these effects were gendered.

1.1 Science communication and gender

Sexism and gender bias is well documented in science, including glaring statistics on harassment and barriers to career development [Bian, Leslie and Cimpian, 2017 ; Cimpian et al., 2016 ; Moss-Racusin et al., 2012 ; National Academies of Sciences, Engineering and Medicine, 2018 ]. This cultured sexism does not escape communication — from writing (articles and CVs), speaking (conference talks and teaching), and social media. For example, studies have shown that women are evaluated less positively than male colleagues in the hiring process from written job materials alone [Milkman, Akinola and Chugh, 2015 ; Moss-Racusin et al., 2012 ; Steinpreis, Anders and Ritzke, 1999 ]. In certain fields, women receive more negative comments or rejections of papers in the peer review process [Budden et al., 2008 ].

Women face more disincentives and threats when communicating science, compared to men, from negative teaching evaluations to blatant harassment. For example, female professors are evaluated more on personality and appearance and receive lower teaching evaluations than their male colleagues [Mitchell and Martin, 2018 ]. When women in STEM choose to communicate in the public sphere, they can be subject to harassment and abuse on social media [Veletsianos, 2012 ]. Female scientists often report disparaging and abusive comments on online platforms such as Twitter and YouTube. A 2018 study showed that female STEM communicators on YouTube received significantly more hostile and negative commentary, including on their appearance and of a sexual nature [Amarasekara and Grant, 2019 ]. All of these compounding factors can have a direct effect on how someone chooses to communicate, including how often.

Additionally, women are not only subject to more harassment and gender bias when they communicate, but are given fewer opportunities to communicate. Studies have shown that female scientists are less represented in news stories about science, and in professional communication forums such as colloquium speakers at prestigious universities and scientific meetings in certain disciplines [Isbell, Young and Harcourt, 2012 ; McCullagh et al., 2019 ; Nittrouer et al., 2018 ].

1.2 Science communication training

While not much is known about the effectiveness of scientific communication training programs, expert and participant perceptions of what makes for effective scientific communication is better understood. Science communication is improved when clear goals are articulated in advance [Ogawa, 2013 ]. This tenet can also be applied to the learning of science communication and developing a high-level set of learning goals or outcomes that program providers can use as a benchmark [Baram-Tsabari and Lewenstein, 2017 ]. However, Besley, Dudo, Yuan et al. [ 2016 ] believe that the current goals of scientists and providers of science communication are not broad or long-term enough. Communication is a two-way street and is often more about listening than talking. Baram-Tsabari and Lewenstein [ 2017 ] assert that open listening and responding honestly to your audience is the basis of true communication, creating more positive interactions between scientists and the many publics they interact with. This echoes the hope expressed by Besley and Tanner [ 2011 ] that regular formal and informal interactions between publics and scientists increases empathy for both parties, leading to mutual positive changes on how each view the other and the process of science communication. Bray, France and Gilbert [ 2012 ] concluded that one of the most important tenets of science communication training is to know your audience, which requires the development of empathy and a shift of focus to your audience’s needs. Therefore communication that sets clear goals, promotes listening and empathy, and meets the audience where they are (in terms of skill, knowledge and/or needs for example) is most effective. By extension, training programs that focus on developing these skills should also be the most effective.

1.3 Why improvisation?

The pioneer of improvisation games for learning, Viola Spolin, states “theatre games are applicable to any field, discipline, or subject matter which creates a place where full participation, communication and transformation can take place” [Hyams, 1974 ]. Improvisation — or improv — has “an emotional and intellectual impact on the participants…and deepens our understanding of human motivation and behaviour” [Toivanen, Komulainen and Ruismäki, 2011 , p. 63]. It also helps participants fully engage in the moment and develop an authentic and personal connection with their audience [Bernstein, 2014 ].

Fundamentally, “the techniques of theatre are the techniques of communicating” [Spolin, 1999 , p. 14]. Improvisation facilitates participants to be in the moment, and to be mentally, emotionally, and physically engaged in listening and responding to their audience. The core tenets of improvisation, developed from Spolin’s work in the 1920s, involve the principals of ‘Yes, and’ and ‘Making your partner look good’: saying yes to the situation presented to you, listening, paying deep attention to and supporting your partner [Bernstein, 2014 ; Rossing and Hoffmann-Longtin, 2016 ]. These involve the building of empathetic relationships and a shift in your focus from yourself to audience [Kaplan-Liss et al., 2018 ; Bernstein, 2014 ]. Given the identified importance of listening [Baram-Tsabari and Lewenstein, 2017 ] and audience focus [Evia et al., 2017 ; Besley and Tanner, 2011 ; Bray, France and Gilbert, 2012 ] as key component skills of effective science communication, this study aims to explore the impact of workshops which attempt to develop those skills via theatre improvisation games.

1.4 Improvisation and science communication

There is a small but growing body of literature on the influence of improv on science communication skill development. Rutgers University introduced a ‘Communicating Science’ class for doctoral students in science, finding students had improved confidence in speaking and tailoring content to meet diverse audiences [Ponzio et al., 2018 ]. In 2011, pharmacy students evaluated the use of improvisation games used to develop listening skills, yielding mixed reactions, but results were such that improv activities were going to be included in future relevant lectures [Guiguis, 2011 ]. Other published materials about improvisation tools to develop communication skills either describe particular exercises [Aurbach et al., 2018 ], explore ways to include science communication training in university programs [Kuehne et al., 2014 ; Rossing and Hoffmann-Longtin, 2016 ] or are pieces in news and popular writing describing improvisation and why scientists are finding it useful [Bernstein, 2014 ; Chang, 2015 ; Nason, 2018 ]. The impact and effectiveness of science communication on scientists, and what that impact might be beyond the acquisition of communication skills, is largely underexplored.

The Alan Alda Center for Communicating Science, established in 2009 within the Stony Brook School of Journalism at Stony Brook University in New York, is a regular provider of improv-based science communication workshops for scientists. The Center has produced some research outlining the relevance and effectiveness of their program content [Bass, 2016 ; Kaplan-Liss et al., 2018 ] but largely within a content and program development lens as the Center itself is comparatively young — opening in 2009. The lead author is employed by Stony Brook University and helped develop the program that is used there. Actor Alan Alda, who founded the Center, credits improvisation with boosting his own communication skills and, after years interviewing scientists on television show Scientific American Frontiers, wondered if scientists would find improv similarly useful [Bernstein, 2014 ]. The end result is a series of improvisational techniques designed to facilitate an emotional connection between people and ‘clear and vivid communication’ [Alda Communication Training, 2019 ].

The Alda Center courses and workshops are open to scientists, engineers and health professionals, aiming to develop effective communication skills to suit a wide range of publics, including students, community members, potential donors and employers, policymakers, the press, and potential collaborators in other disciplines. The Alda Center has held an annual Science Communication Bootcamp at Stony Brook University since 2014. Participants come to the Alda Center Bootcamp for an intensive experiential learning workshop in science communication, with the goal of changing the approaches used by scientists to better engage with various audiences. These Bootcamps form the basis of this study.

2 Objective

Earlier program assessments were based on instructor grading in courses and participant self-assessment after workshops [Kaplan-Liss et al., 2018 ], typical of the kind of evaluations identified in recent studies [Baram-Tsabari and Lewenstein, 2017 ]. In 2016, 2017 and 2018 we surveyed participants before and after participation in an intensive improv-based training program to measure changes in participants across the program, looking specifically for any influence on perceptions of, attitudes towards and confidence in science communication.

Anecdotal reports from earlier participants’ experiences within an Alda Center science communication workshop suggested a variety of benefits from the workshop experience including the development of confidence, leadership skills, interpersonal empathy and anxiety reduction to name a few. This study is the first attempt to quantify if these skills are a consistent, previously unexpected, outcome of this type of science communication training.

We aimed to explore the extent to which improv might influence empathy and humility. These qualities both include a focus on others rather than the self — empathy is an emotional response to the perceived needs of others [Batson et al., 2002 ], whereas humility is a characteristic way of thinking and feeling that focuses on others over the self and represents comfort with one’s strengths and weaknesses [Davis, Worthington and Hook, 2010 ]. We hypothesised that the emphasis of these workshops to focus on the audience (others) rather than the self, as well as the training to be comfortable communicating one’s expertise may have influenced these qualities in participants. Further, we hypothesized the training may have broader impacts on qualities like self-esteem and optimism. Finally, the workshops aim to increase participants’ confidence in their ability to communicate science. Part of the workshops focus on the role of scientists to inform the publics they serve. This is based on the growing practice of adopting training which builds on social-scientific approaches to help scientists to engage with publics in meaningful discussion to ensure scientists’ views are viewed as legitimate additions to public debates [Webler, 2013 ] and to ensure that science is applied to social problems [National Science Board, 2014 ]. We hypothesize that this may create a sense of personal responsibility in the scientist. Based on the previously described anecdotal feedback that the training increased leadership skills, we wished to examine whether this ‘responsibility’ manifested as an increase in participants’ perceptions of their ability to be a leader.

3 Methods

The participants in this study were recruited as part a five-day residential science communication training program, called a Bootcamp, at the Alda Center. These residential programs have been run at the Alda Center since 2014. Participants applied and paid tuition to attend the Bootcamp; the application procedure was to limit the number of participants per session to roughly 32 and ensure attendees were scientists or researchers rather than professional science communicators or consultants. All attendees stayed for the duration of the Bootcamp. Respondents were from three different workshop cohorts held one year apart each June. Participants varied between groups however the workshops had consistent content and delivery method. Participants are a self-selecting group who range in age, stage of career, gender, field, and experience in communication — from graduate students to senior researchers/faculty, and from private industry, government, non-profit and academia. Fields of study included medical professionals, environmental scientists, physicists, biologists, and other applied and basic researchers.

3.1 Workshop content

As described earlier, the workshops use an interdisciplinary and experiential learning process drawing on the fields of improvisational theatre, journalism, science, and communications. Participants spend time working in large and small groups, presenting their work and getting feedback, being interviewed in the Stony Brook University television studio, writing a letter to an editor and participating in improv activities. Emphasis is placed on using clear, vivid, conversational language, on explaining meaning before technical details, and on using storytelling techniques, based on best-practices in science communication.

The included improv exercises focus on building empathy for the audience and shifting focus to connect with other people, paying dynamic attention to verbal and nonverbal cues and responding based on those cues. Each improv exercise is explicitly linked to science communication. One warm up exercise asks participants to work in pairs and alternate saying the numbers to repeatedly count to three, as follows:

Person one: “one”

Person two: “two”

Person one: “three”,

Person two: “one” and so on.

Each round, a number is replaced by an action. Instead of saying “two” for example, it is replaced by a clap of the hands, then three is replaced with a stamp of the foot. When the participants make a mistake, they must both loudly yell ‘Ta dah!’ and celebrate the error. As the rounds get more complicated, mistakes happen more frequently. This exercise aims to develop confidence in participants who may shy away from communication activities as they fear making a mistake, or they cannot make a public presentation that is not meticulously scripted and prepared. This exercise encourages participants to acknowledge that everyone makes mistakes and it is not ‘the end of the world’ nor a justifiable reason for not communicating. It also introduces them to the core principles of improvisation; responding to the situation in front of you, connecting with other people to create context and shared understanding. It also invariably generates a lot of laughter and helps create an environment conducive to introducing more challenging improvisation activities and learning to acknowledge (and even celebrate) failure and move forward.

About half of each day was spent in groups of up to 20 people, learning improvisation exercises. The remaining time was allocated to small (about eight people) group interactive activities where participants practiced talking about their work in front of different audiences, telling stories about turning points in their life or science, being ‘interviewed’ in the Stony Brook University campus Journalism School television studio, or role-playing meetings with policy makers. Participants received feedback and support from the trainers, as well as the other participants, over the course of the week.

3.2 Recruitment

Alda Center workshop participants were contacted via e-mail in the week prior to their training and asked to complete a series of pre-test measures online via Qualtrics Survey Software (Qualtrics, Provo, UT), described below. Following basic approaches to measuring change [Bock, 1976 ] in the week following their training, participants were contacted via e-mail and asked to complete a follow-up survey which included the pre-test measures as well as opportunities for participants to reflect in open-ended questions about their experiences. There are limitations to our pre-post test design including response drop-off with paired measurements, response-shift bias (e.g., do participants have the same frame of reference and baseline information pre/post trainings?), and possible ceiling effects (e.g., participants starting the Bootcamps with already extremely high empathy scores) [Allen and Nimon, 2007 ; Howard, 1980 ; Pratt, McGuigan and Katzev, 2000 ] which means the results presented should be interpreted with this in mind.

3.3 Measures

Multiple measures were explored via closed (Likert-scale) and open-ended questions. We focus our analyses on the constructs identified anecdotally by previous Alda Center boot camp participants. These constructs represent the key traits that participants identified as being challenged or strengthened by their experiences. We used well-established self-report measures to investigate developments in those constructs. We also wished to determine if these were constructs that were potentially more broadly applicable or if they were simply a product of those particular individuals from the earlier bootcamps.

3.4 Empathy

Trait Empathy was measured using on Davis’ Interpersonal Reactivity Index, specifically the Empathic Concern subscale [Davis, 1983 ]. Participants read a statement about their personality and determined whether that statement described them well or not on a scale from 1 (Does not describe me well) to 7 (Describes me well). For example, “I often have tender, concerned feelings for people less fortunate than me.”

3.5 Self-esteem

Self-Esteem was measured with the social subscale of Heatherton and Polivy’s [ 1991 ] Self-Esteem Questionnaire. Participants indicated on a 1 (strongly disagree) to 7 (strongly agree) scale the extent to which they agreed with items like, “I am worried about looking foolish” and “I am worried about whether I am regarded as a success or a failure.”

3.6 Humility

Participants responded to a humility semantic differential scale [Rowatt et al., 2006 ], which asks people to indicate on a 1 to 7 scale how close they were to two poles (e.g., arrogant-humble; intolerant-tolerant) across 7 different items.

3.7 Optimism

General optimism was measured using the Life Orientation Test — Revised [Scheier, Carver and Bridges, 1994 ] which asks participants to indicate on a 1 (strongly disagree) to 7 (strongly agree) scale with items like, “Overall, I expect more good things to happen to me than bad.”

3.8 Leadership self-efficacy

Self-Efficacy of Leadership was measured via Hoyt and Blascovich’s [ 2010 ] measure which asks participants to indicate on a 1 (strongly disagree) to 7 (strongly agree) scale the extent to which they agree with items like, “I know what it takes to make a group accomplish its task.”

3.9 Desire for communication

Desire for communication was measured through newly-constructed individual items regarding the extent to which participants felt comfortable and their desire to communicate about their research. A total of ten questions asked about three key areas; the value participants placed on science communication (“I feel strongly about the importance of science communication”); their perceived skills (“I feel confident in my ability to communicate science to different audiences”); and their desire for more opportunity to communicate (“I want more opportunities to talk about scientific research with the public”). These were all closed response questions requiring participants to respond on a 1 (strongly disagree) to 7 (strongly agree) scale. These questions were asked before and after the workshops. As it is newly developed specifically for this study, the desire for communication scale used has not been internally or externally validated, and the reliability of the questions is unknown.

Before the workshop participants were also asked what specific skills or outcomes they hoped to get from participating in the workshop. Immediately after the workshop participants were asked to identify the most valuable aspect of the workshop for them. These responses were transcribed and coded thematically by one of the researchers into four different groups which described the focus of the response, similar to the method used by Besley, Dudo, Yuan et al. [ 2016 ].

4 Results

Of a possible 101 scientists, 74 participated in this study, however only 49 participants submitted both complete pre- and post- workshop surveys. Only the complete pre/post data sets are included in this paper. Of the original 74 scientists, 62% identified as female, 38% as male, with 85% White/Caucasian, 4% African American, 4% Hispanic, and 4% Asian/Pacific Islander. Twenty percent were graduate students or post-doctoral associates, 47% were faculty, and 32% were administrators/researchers/other. Of the 49 participants included in this analysis; 35 identify as women and 14 as men. Although this sample size does not allow for the derivation of accurate estimates of effect size, and quantitative analysis should be considered exploratory, it has allowed us observations of preliminary trends. As such, no significant differences were observed, but statistical tests are reported for thoroughness. Because the samples violated assumptions of parametric tests, we conducted non-parametric comparisons using Wilcoxon’s Z. Analysis of the results combining all participants showed some differences, but when the responses were separated by gender, larger differences and trends begin to emerge. Both are presented here.

4.1 Empathy

In looking at empathy, we expected to see an increase pre/post workshop, which occurred (insignificantly) with an average of 5.57 pre-workshop ( SD = .83) and 5.70 post ( SD = .84; Wilcoxon Z = .90; p = .371). When exploring the results based on gender, the greatest difference was observed in men. Women show a mean increase of 0.05 on the empathy measure (pre- M = 5.65, SD = .96; post- M = 5.70, SD = .91 Wilcoxon Z = .26; p = .793), whereas men’s empathy score increased by 0.26 (pre- M = 5.44, SD = .56; post- M = 5.70, SD = .74 Wilcoxon Z = 1.29; p = .196; Figure 1 ). Of all participants, 71% of the men increased in empathy in comparison to 47% of women. On average, men had a slightly lower empathy score pre-workshop ( M = 5.44, SD = .56) than women ( M = 5.65, SD = .96, U = 218.5, p = .071).

---

---

4.2 Humility

The Humility SD scale showed very little difference with a pre-workshop average score of 5.52 ( SD = .58) and 5.57 post-workshop ( SD = .71; Wilcoxon Z = .74; p = .462). When examined by gender, men reported an decrease in their average score post workshop by -0.16 (pre- M = 5.41, SD = .70; post- M = 5.25, SD = .83 Wilcoxon Z = -.69; p = .489), whereas women reported an increase by 0.17 (pre- M = 5.59, SD = .50; post- M = 5.76, SD = .57 Wilcoxon Z = 1.66; p = .097; Figure 1 ). Women reported higher humility on average than men at both time points.

4.3 Optimism

We observed virtually no change overall in optimism across the workshop (pre- M = 5.33, SD = 1.31; post- M = 5.32, SD = 1.36 Wilcoxon Z = -.713; p = .476). Comparing by gender, we observed an increase in optimism for men (pre- M = 4.71, SD = 1.61; post- M = 4.91, SD = .82 Wilcoxon Z = -1.14; p = .256). and a decrease for women (pre- M = 5.70 SD = .95; post- M = 5.57, SD = 1.57 Wilcoxon Z = 0; p = 1.00; Figure 1 ). Despite the small decrease, women typically were more optimistic than men at both time points.

4.4 Leadership

Average scores on leadership self-efficacy increased significantly across the workshop (pre- M = 5.39, SD = 1.13; post- M = 5.60, SD = 1.04 Wilcoxon Z = 2.16; p = .031. This effect seems to be driven by changes for women (pre- M = 5.38, SD = 1.07; post- M = 5.71, SD = 1.01 Wilcoxon Z = 2.73; p = .006) rather than men (pre- M = 5.39, SD = 1.27; post- M = 5.41, SD = 1.10 Wilcoxon Z = -.55; p = .581; Figure 1 ). The majority of women (about 74%) increased in their individual leadership scores in comparison to 40% of men. This was the only scale measured where men had higher average scores than women across both time points.

4.5 Self-esteem

Participants reported an insignificant increase in their mean self-esteem levels from 4.66 pre-workshop ( SD = 1.38) to 4.86 post ( SD = 1.34; Wilcoxon Z = 1.13; p = .259). Both men (pre- M = 4.41, SD = .98; post- M = 4.52, SD = 1.11 Wilcoxon Z = .11; p = .916) and women (pre- M = 4.82, SD = 1.58; post- M = 5.06, SD = 1.45 Wilcoxon Z = 1.18; p = .237; Figure 1 ) reported similar small increases.

4.6 Communication measures

In all 10 measures across the categories of value, skill and opportunity, men tended to have higher pre-workshop scores than women. There was an increase in all post scores except for one: ‘I feel like I have the skills to explain how scientific research is relevant’. When separated for gender, the men showed a small decrease in their sense of their ability to do this, whereas women showed a small increase. For the majority of measures, women showed the largest increase in scores, especially for measures relating to personal beliefs in skills — which were the main expected learning outcomes of the workshop — and opportunity to speak to the public and policy makers. Women were unchanged in their desire to speak to other scientists; however, men showed an increase in this measure and wanted more opportunities to communicate with other scientists about their work.

4.7 Qualitative responses

The pre-workshop questions asked participants what they most wanted to gain from the experience, which provided insight into what participants’ felt were the most valuable or useful elements for their communication. The post-workshop survey asked participants what they found to be the most useful parts of the workshop. The responses presented here are matched (pre and post) for each individual. All responses were collated and coded into four categories which denoted the focus of the participant: self, audience, goal/outcome of communicating, and specific skill (Table 2 ). Participants often gave answers which encompassed more than one category.

Pre-workshop responses showed a strong focus on the self, with the most common responses (from 32 of 49 participants) outlining a desire for confidence and an ability to represent their science appropriately (Figure 2 ). Only nine participants responded with some desire to develop skills with an audience, with five respondents noting a specific communication goal and 11 a specific skill focus. The post-surveys showed a marked difference in focus in what participants found most valuable or useful. More participants (21) noted that specific skills, such as being able to distil a message, tell stories or effectively use metaphors or analogies, were what they felt most valuable after the workshops, especially women.

---

---

The greatest difference was seen in the shift in focus from self to the audience. Only eight participants had responses with a focus on self, whereas 33 responses included an audience focus. This occurred for both male and female participants as shown in the following examples:

PRE: I would like to improve my ability to communicate my research (and the importance of my research) to non-academic persons.

POST: Focusing on a singular message, rather than trying to convey the entirety of my research. Knowing the audience and breaking the artificial barrier between me as a speaker and the audience as listeners through deliberate audience engagement.

(Female participant)

PRE: Executive presence, an ability to act in any situation

POST: I learned to develop a “people-first” approach to presenting. That evoking the personal first will draw the audience closer to you, and get a better connection.

(Male participant)

5 Discussion

Preliminary results show interesting areas for further exploration for science communication training programs, the impacts of which appear to extend beyond simple skill acquisition, with gender possibly playing an important role.

Participants reported positive changes in key objectives of the workshops (increase in science communication skills, value, and desire for more opportunities to practice). All participants showed similar levels of perceived skill development and expressed a desire for further communication opportunities. Results provide a clear indication that improvisation-based science communication training has a positive influence when it comes to desire for communication and having more of a focus on audience and goal. Further research is needed to better understand the development of these specific skills and qualities.

What participants said they wanted (and by inference what they perceived would be most useful or valuable to them) showed a marked difference before and after the workshop. The focus was very much inward-looking pre-workshop, with the majority of respondents showing a far greater focus on their audience after the workshop. This could be associated with the greater levels of empathy or the perceived value of science communication activities. Or it may be as Besley and colleagues [ 2015 ] postulated; that scientists “need to develop a stronger appreciation for just how valuable it is to ‘connect’ with an audience” (p. 215). They argued that this should be tied to communication style, and that the role of a scientist’s communication style should be examined by future research. We contend that this could also be applied to examination of the style of communication training, should scholars and practitioners in the discipline wish for a more complete understanding of impact.

Initial results suggest gender may be an important component to consider in program design, objectives and evaluation. Because of the nature of improvisation, an influence in empathy was expected, however this was far more pronounced looking at the individual scores of male participants. With the majority of male participants, the increase in empathy came with a decrease in humility but an increase in optimism, leadership self-efficacy and self-esteem. The lower pre-empathy score for men could be framed in terms of common societal norms telling men that showing emotion, especially sadness, is weak and wrong [Nelson, 2015 ]. Improv training likely increases male reports of empathy because they learn in this context it is okay to have and express those feelings to achieve a communication goal.

In comparison, fewer women had an increase in empathy, but their leadership self-efficacy and self-esteem markedly increased. In STEM culture, women are often perceived and stereotyped as less effective leaders and seen as lesser than their male counterparts with equal levels of achievement [Knobloch-Westerwick, Glynn and Huge, 2013 ; Moss-Racusin et al., 2012 ]. This kind of systematic bias might influence someone’s sense of self, including their accomplishments and abilities, and can increase feelings of imposter syndrome in STEM. Perhaps improvisation-based science communication training specifically helps respond to culturally-imposed gender stereotypes? Perhaps this allows women to realize their less-constrained potentials when it comes to science communication? Further studies are required to determine if this is indeed the case.

6 Conclusions

Previous studies [Besley, Dudo and Storksdieck, 2015 ; Dudo and Besley, 2016 ] have proposed that there is a disconnect between the objectives of the providers of science communication workshops and/or courses and those of their participants. We argue that perhaps both parties are not fully aware of the potential impacts of these workshops and the objectives on both sides could aim far higher. There is little to no evidence of the effectiveness and impact of the different kinds of science communication workshops and courses. This self-report change analysis approach is a starting point for systematically investigating the effects of such programs. The results presented here do have limitations and as such should be interpreted cautiously. This is a preliminary study with a self-selecting audience conducting an initial exploration of additional constructs which science communication training may benefit. Further studies should supplement these investigations, for example by objectively measuring scientific communication skills of participants, validating the desire for communication scale and further exploring the applicability of influences on empathy, leadership and self-esteem as outcomes of effective training programs. Despite these limitations, these preliminary findings provide a starting point for future research particularly exploring how outcomes of improvisation-based science communication manifest in participants with respect to gender. The results presented here are intended as an indicator of fruitful areas for future research to better understand how best to support the development of effective science communicators.

References

Alda Communication Training (2019). Alda method: improvisational exercises that spark clear and vivid communication . URL: http://www.aldacommunicationtraining.com/alda-method/ (visited on 19th March 2019).

Allen, J. M. and Nimon, K. (2007). ‘Retrospective pretest: a practical technique for professional development evaluation’. Journal of Industrial Teacher Education 44 (3), pp. 27–42.

Amarasekara, I. and Grant, W. J. (2019). ‘Exploring the YouTube science communication gender gap: a sentiment analysis’. Public Understanding of Science 28 (1), pp. 68–84. https://doi.org/10.1177/0963662518786654 .

Aurbach, E. L., Prater, K. E., Patterson, B. and Zikmund-Fisher, B. J. (2018). ‘Half-life your message: a quick, flexible tool for message discovery’. Science Communication 40 (5), pp. 669–677. https://doi.org/10.1177/1075547018781917 .

Baram-Tsabari, A. and Lewenstein, B. V. (2017). ‘Science communication training: what are we trying to teach?’ International Journal of Science Education, Part B 7 (3), pp. 285–300. https://doi.org/10.1080/21548455.2017.1303756 .

Bass, E. (2016). ‘The Importance of Bringing Science and Medicine to Lay Audiences’. Circulation 133 (23), pp. 2334–2337. https://doi.org/10.1161/circulationaha.116.023297 .

Batson, C. D., Ahmad, N., Lishner, D. A. and Tsang, J.-A. (2002). ‘Empathy and altruism’. In: Handbook of positive psychology. Ed. by C. R. Snyder and S. J. Lopez. New York, NY, U.S.A.: Oxford University Press, pp. 485–498.

Bernstein, R. (2014). ‘Communication: Spontaneous scientists’. Nature 505 (7481), pp. 121–123. https://doi.org/10.1038/nj7481-121a .

Besley, J. C., Dudo, A. D., Yuan, S. and Ghannam, N. A. (2016). ‘Qualitative Interviews With Science Communication Trainers About Communication Objectives and Goals’. Science Communication 38 (3), pp. 356–381. https://doi.org/10.1177/1075547016645640 .

Besley, J. C., Dudo, A. and Storksdieck, M. (2015). ‘Scientists’ views about communication training’. Journal of Research in Science Teaching 52 (2), pp. 199–220. https://doi.org/10.1002/tea.21186 .

Besley, J. C. and Tanner, A. H. (2011). ‘What Science Communication Scholars Think About Training Scientists to Communicate’. Science Communication 33 (2), pp. 239–263. https://doi.org/10.1177/1075547010386972 .

Bian, L., Leslie, S.-J. and Cimpian, A. (2017). ‘Gender stereotypes about intellectual ability emerge early and influence children’s interests’. Science 355 (6323), pp. 389–391. https://doi.org/10.1126/science.aah6524 .

Bock, R. D. (1976). ‘Basic issues in the measurement of change’. In: Advances in psychological and educational measurement. Ed. by D. N. M. DeGruijter and L. J. T. Van der Kamp. New York, NY, U.S.A.: John Wiley & Sons, pp. 75–96.

Bray, B., France, B. and Gilbert, J. K. (2012). ‘Identifying the Essential Elements of Effective Science Communication: What do the experts say?’ International Journal of Science Education, Part B 2 (1), pp. 23–41. https://doi.org/10.1080/21548455.2011.611627 .

Budden, A., Tregenza, T., Aarssen, L., Koricheva, J., Leimu, R. and Lortie, C. (2008). ‘Double-blind review favours increased representation of female authors’. Trends in Ecology & Evolution 23 (1), pp. 4–6. https://doi.org/10.1016/j.tree.2007.07.008 .

Chang, K. (2nd March 2015). ‘Attention, all scientists: do improv, with Alan Alda’s help’. The New York Times . URL: https://www.nytimes.com/2015/03/03/science/attention-all-scientists-do-improv-with-alan-aldas-help.html .

Cimpian, J. R., Lubienski, S. T., Timmer, J. D., Makowski, M. B. and Miller, E. K. (2016). ‘Have gender gaps in math closed? Achievement, teacher perceptions and learning behaviors across two ECLS-K cohorts’. AERA Open 2 (4), p. 233285841667361. https://doi.org/10.1177/2332858416673617 .

Davis, D. E., Worthington, E. L. and Hook, J. N. (2010). ‘Humility: review of measurement strategies and conceptualization as personality judgment’. The Journal of Positive Psychology 5 (4), pp. 243–252. https://doi.org/10.1080/17439761003791672 .

Davis, M. H. (1983). ‘Measuring individual differences in empathy: evidence for a multidimensional approach’. Journal of Personality and Social Psychology 44 (1), pp. 113–126. https://doi.org/10.1037/0022-3514.44.1.113 .

Dudo, A. and Besley, J. C. (2016). ‘Scientists’ Prioritization of Communication Objectives for Public Engagement’. PLoS ONE 11 (2), e0148867, pp. 1–18. https://doi.org/10.1371/journal.pone.0148867 .

Evia, J. R., Peterman, K., Cloyd, E. and Besley, J. (2017). ‘Validating a scale that measures scientists’ self-efficacy for public engagement with science’. International Journal of Science Education, Part B 8 (1), pp. 40–52. https://doi.org/10.1080/21548455.2017.1377852 .

Guiguis, L. (2011). ‘Improvisation games in a pharmacy communications course: “it was kind of interesting to get to step out of my science-oriented mind and get to be creative!”’ Pharmacy Education 11, pp. 201–204.

Heatherton, T. F. and Polivy, J. (1991). ‘Development and validation of a scale for measuring state self-esteem’. Journal of Personality and Social Psychology 60 (6), pp. 895–910. https://doi.org/10.1037/0022-3514.60.6.895 .

Howard, G. S. (1980). ‘Response-shift bias: a problem in evaluating interventions with pre/post self-reports’. Evaluation Review 4 (1), pp. 93–106. https://doi.org/10.1177/0193841x8000400105 .

Hoyt, C. L. and Blascovich, J. (2010). ‘The role of leadership self-efficacy and stereotype activation on cardiovascular, behavioral and self-report responses in the leadership domain’. The Leadership Quarterly 21 (1), pp. 89–103. https://doi.org/10.1016/j.leaqua.2009.10.007 .

Hyams, B. (26th May 1974). ‘Spolin game plan for improvisational theater’. Los Angeles Times .

Isbell, L. A., Young, T. P. and Harcourt, A. H. (2012). ‘Stag parties linger: continued gender bias in a female-rich scientific discipline’. PLoS ONE 7 (11), e49682. https://doi.org/10.1371/journal.pone.0049682 .

Kaplan-Liss, E., Lantz-Gefroh, V., Bass, E., Killebrew, D., Ponzio, N. M., Savi, C. and O’Connell, C. (2018). ‘Teaching medical students to communicate with empathy and clarity using improvisation’. Academic Medicine 93 (3), pp. 440–443. https://doi.org/10.1097/acm.0000000000002031 .

Knobloch-Westerwick, S., Glynn, C. J. and Huge, M. (2013). ‘The Matilda effect in science communication: an experiment on gender bias in publication quality perceptions and collaboration interest’. Science Communication 35 (5), pp. 603–625. https://doi.org/10.1177/1075547012472684 .

Kuehne, L. M., Twardochleb, L. A., Fritschie, K. J., Mims, M. C., Lawrence, D. J., Gibson, P. P., Stewart-Koster, B. and Olden, J. D. (2014). ‘Practical Science Communication Strategies for Graduate Students’. Conservation Biology 28 (5), pp. 1225–1235. https://doi.org/10.1111/cobi.12305 .

McCullagh, E. A., Nowak, K., Pogoriler, A., Metcalf, J. L., Zaringhalam, M. and Zelikova, T. J. (2019). ‘Request a woman scientist: a database for diversifying the public face of science’. PLOS Biology 17 (4), e3000212. https://doi.org/10.1371/journal.pbio.3000212 .

Milkman, K. L., Akinola, M. and Chugh, D. (2015). ‘What happens before? A field experiment exploring how pay and representation differentially shape bias on the pathway into organizations’. Journal of Applied Psychology 100 (6), pp. 1678–1712. https://doi.org/10.1037/apl0000022 .

Miller, S., Fahy, D. and the ESConet Team (2009). ‘Can Science Communication Workshops Train Scientists for Reflexive Public Engagement?’ Science Communication 31 (1), pp. 116–126. https://doi.org/10.1177/1075547009339048 .

Mitchell, K. M. W. and Martin, J. (2018). ‘Gender bias in student evaluations’. PS: Political Science & Politics 51 (03), pp. 648–652. https://doi.org/10.1017/s104909651800001x .

Moss-Racusin, C. A., Dovidio, J. F., Brescoll, V. L., Graham, M. J. and Handelsman, J. (2012). ‘Science faculty’s subtle gender biases favor male students’. Proceedings of the National Academy of Sciences 109 (41), pp. 16474–16479. https://doi.org/10.1073/pnas.1211286109 .

Nason, S. (26th September 2018). ‘How improvisation and acting can boost your science (really!)’ Rapid Ecology . Blog entry. URL: https://rapidecology.com/2018/09/26/how-improvisation-and-acting-can-boost-your-science-really/ .

National Academies of Sciences, Engineering and Medicine (2018). Sexual harassment of women: climate, culture and consequences in academic sciences, engineering and Medicine. Washington, DC, U.S.A.: The National Academies Press. URL: http://nap.edu/24994 .

National Science Board (2014). Chapter 7. Science and technology: public attitudes and understanding . Science and engineering indicators. URL: https://www.nsf.gov/statistics/seind14/index.cfm/chapter-7/c7h.htm .

Nelson, A. (24th January 2015). ‘Why don’t many men show their emotions?’ Psychology Today . URL: https://www.psychologytoday.com/au/blog/he-speaks-she-speaks/201501/why-don-t-many-men-show-their-emotions .

Nittrouer, C. L., Hebl, M. R., Ashburn-Nardo, L., Trump-Steele, R. C. E., Lane, D. M. and Valian, V. (2018). ‘Gender disparities in colloquium speakers at top universities’. Proceedings of the National Academy of Sciences 115 (1), pp. 104–108. https://doi.org/10.1073/pnas.1708414115 .

Ogawa, M. (2013). ‘Towards a “design approach” to science communication’. In: Communication and engagement with science and technology — issues and dilemmas. Ed. by J. K. Gilbert and S. Stocklmayer. New York, NY, U.S.A.: Routledge, pp. 3–18.

Ponzio, N. M., Alder, J., Nucci, M., Dannenfelser, D., Hilton, H., Linardopoulos, N. and Lutz, C. (2018). ‘Learning science communication skills using improvisation, video recordings, and practice, practice, practice’. Journal of Microbiology & Biology Education 19 (1). https://doi.org/10.1128/jmbe.v19i1.1433 .

Pratt, C. C., McGuigan, W. M. and Katzev, A. R. (2000). ‘Measuring program outcomes: using retrospective pretest methodology’. American Journal of Evaluation 21 (3), pp. 341–349. https://doi.org/10.1177/109821400002100305 .

Rodgers, S., Wang, Z., Maras, M. A., Burgoyne, S., Balakrishnan, B., Stemmle, J. and Schultz, J. C. (2018). ‘Decoding Science: Development and Evaluation of a Science Communication Training Program Using a Triangulated Framework’. Science Communication 40 (1), pp. 3–32. https://doi.org/10.1177/1075547017747285 .

Rossing, J. P. and Hoffmann-Longtin, K. (2016). ‘Improv(ing) the academy: applied improvisation as a strategy for educational development’. To Improve the Academy 35 (2), pp. 303–325. https://doi.org/10.1002/tia2.20044 .

Rowatt, W. C., Powers, C., Targhetta, V., Comer, J., Kennedy, S. and Labouff, J. (2006). ‘Development and initial validation of an implicit measure of humility relative to arrogance’. The Journal of Positive Psychology 1 (4), pp. 198–211. https://doi.org/10.1080/17439760600885671 .

Scheier, M. F., Carver, C. S. and Bridges, M. W. (1994). ‘Distinguishing optimism from neuroticism (and trait anxiety, self-mastery and self-esteem): a reevaluation of the Life Orientation Test’. Journal of Personality and Social Psychology 67 (6), pp. 1063–1078. https://doi.org/10.1037/0022-3514.67.6.1063 .

Spolin, V. (1999). Improvisation for the theater: a handbook of teaching and directing techniques. 3rd ed. Evanston, IL, U.S.A.: Northwestern University Press.

Steinpreis, R. E., Anders, K. A. and Ritzke, D. (1999). ‘The impact of gender on the review of the curricula vitae of job applicants and tenure candidates: a national empirical study’. Sex Roles 41 (7/8), pp. 509–528. https://doi.org/10.1023/a:1018839203698 .

Toivanen, T., Komulainen, K. and Ruismäki, H. (2011). ‘Drama education and improvisation as a resource of teacher student’s creativity’. Procedia — Social and Behavioral Sciences 12, pp. 60–69. https://doi.org/10.1016/j.sbspro.2011.02.010 .

Veletsianos, G. (2012). ‘Higher education scholars’ participation and practices on Twitter’. Journal of Computer Assisted Learning 28 (4), pp. 336–349. https://doi.org/10.1111/j.1365-2729.2011.00449.x .

Webler, T. (2013). ‘Why risk communicators should care about the fairness and competence of their public engagement processes’. In: Effective risk communication. Ed. by J. L. Arvai and L. Rivers. London, U.K.: Routledge, pp. 124–141. https://doi.org/10.4324/9780203109861 .

Authors

Christine O’Connell is an Assistant Professor at the School of Journalism and the Alan Alda Center for Communicating Science at Stony Brook University. She was the founding Associate Director at the Alda Center where she helped build the Center and many of its flagship programs. She currently teaches and builds research and curriculum initiatives in science communication, with a focus on best practices, science policy, novel collaborations, and women in STEM. E-mail: christine.oconnell@stonybrook.edu .

Merryn McKinnon is a lecturer at the Centre for the Public Awareness of Science at The Australian National University in Canberra. Her research interests include the relationship between science, publics and the media; and the impacts and influence of gender in science communication practice and development. E-mail: merryn.mckinnon@anu.edu.au .

Jordan Labouff is an Associate Professor of Psychology and Honors at the University of Maine. His research investigates how beliefs interact with personal virtues and social situations to influence attitudes (e.g., prejudice, impression formation) and behaviors (e.g., helping those in need, discrimination, and public behaviors). He pursues this question across three primary research areas: (1) virtue and prosocial behavior, (2) belief and intergroup bias, and (3) applications of implicit social cognition to public policy and social issues. E-mail: Jordan.LaBouff@maine.edu .