Reflecting on four Living Labs in the Netherlands and Indonesia: a perspective on performance, public engagement and participation

1 Introduction

With the increased incidence of phenomena such as heavy rainfall, thunderstorms, flooding’s, heatwaves and wildfires, a wider reconnaissance of the Anthropocene is discernible. The sustainable development goals (SDGs),¹ indicate global recognition of contemporary challenges such as hunger, climate change, biodiversity losses, protracted conflicts and poverty, and thereby constitute a call for partnerships between diverse actors to act accordingly. It is further recognised that action is not only required for the well-being of current generations but even more concerning future generations which is referred to as acknowledging intergenerational equity (see for example decision Cp. 27 of the COP 2022).² The rights of future generations’ voices are also heard from the youth claiming climate change adaptation and ecological justice. The 19-year-old Swedish climate activist Greta Thunberg described the lack of action at the recent COP conference³ stating, “The COPs are mainly used as an opportunity for leaders and people in power to get attention, using many different kinds of greenwashing”. She further expressed the need for radical action, stating that conferences such as COP “are not really meant to change the whole system, but instead encourage gradual progress”.

The urgent and systemic call for environmental action synchronizes with a realization that conventional approaches have failed or insufficiently achieved sustainability-oriented engagement and results. Policies and practices are required that sharply articulate and activate responsibility of current generations or, as framed by Foth [2018, p. 13]: In search of answers for how to transition humanity towards sustainable futures, it suffices to say that progressive, disruptive, and radical change is required. Acting on such contemporary challenges paved the way for the Living Lab approach as a response “to do different things and do things differently”, referring to a common saying about change.

In line with many authors who argue that there is not a single definition of Living Labs, it is generally agreed that Living Labs are active in real-life environments and refer to open innovation or co-creation processes with quadruple helix participation: governments, academia, civil society, and industry. Working within a Living Lab configuration is relatively recent and experimental and assumed as promising [Beaudoin et al., 2022]. McPhee et al. [2021, p. 1] describe this as follows: “Living Labs appear to have the potential to accelerate co-creation and adoption throughout the value chain, because of their user-centric approach used to develop and co-create innovative solutions in partnership with stakeholders and tested in the users’ real-life context”. In a Living Lab, innovation development is not only accelerated, but also considered cost-effective because the different parties have access to various networks and funds that can support research and implementation [Smits, Straatsma & Fliervoet, 2019]. Living Labs (LLs) are defined by the European Network of Living Labs (ENoLL) as ‘user-centred, open innovation ecosystems based on a systematic user co-creation approach integrating research and innovation processes in real-life communities and settings’.⁴ Westerlund and Leminen [2011] define Living Labs as ‘physical regions or virtual realities where stakeholders form public-private-people partnerships (4Ps) of firms, public agencies, universities, institutes, and users all collaborating for creation, prototyping, validating, and testing of new technologies, services, products and systems in real-life contexts.’ In elaborating on such a multi-actor focus, it is widely agreed that Living Labs require collaboration and learning between all societal actors [Veeckman, Schuurman, Leminen & Westerlund, 2013; Westerlund & Leminen, 2011; Ståhlbröst & Holst, 2017].

Unfortunately, it has to be admitted that achieving public engagement and participation with a Living Lab approach is sometimes experienced as a kind of fairy tale awaiting happy endings. Such expectations can be acknowledged as naïve but have persistent contributing outlooks resulting from science practices dominated by a disbalance of academic versus local knowledge, or ‘tokenism’ as Reed and Rudman [2022] mention or a defined time frame resulting from funding conditions such as a PhD fellowship or a project grant. In line with Monno and Khakee [2012] many other authors have written about public participation and the disillusionment of participants because of the discrepancy between their input in the planning discourse and the outcomes.

The authors of this paper have participated in Living Labs as a promising concept of a sustainability-oriented approach in regional transition processes in attachment to higher education institutes with functions that combine academic work and the practice of environmental projects. Over the years they witnessed a reconceptualization of learning strategies in Higher Education Institutions (HEIs) moving from lecture centred learning to an orientation on (simulated) learning options in the real world. An example of such developments are curriculum adaptations mentioned by the Academic Senate of the Institute of Technology Bandung (ITB) referring to 21st-century challenges and competences such as lifelong learning, system thinking, non-routine problem solving and working in cross-disciplinary teams.⁵ Beyond curriculum adaptation and innovative learning strategies, HEIs increasingly focus on their societal role as noticeable in “The Times Higher Education,⁶ which calibrates indicators for diverse university dimensions, such as teaching, research, knowledge transfer, outreach, and stewardship against the Sustainable Development Goals” [L. Witteveen et al., 2022, p. 7].

In our HEIs, pressing contemporary challenges are responded with a search to address sustainability transition processes with a dedicated multi-actor system orientation.

After a first decade of actively using the Living Lab concept, evaluations and critical reviews are becoming available. A recent literature review by Lie, van Paassen and Witteveen [2023] describes the current state of insight as “Although we are starting to get a grip on the functioning of Living Labs and Innovation Platforms, there is still a lot to learn about how to make them operate responsibly and successfully”. Fuglsang, Vorre Hansen, Mergel and Taivalsaari Røhnebæk [2021] indicated, based on the integrative review of Living Labs, that knowledge gaps exist on “how are Living Labs created and recreated as environments for public sector innovation over time” [2021, p. 16]. Bronson, Devkota and Nguyen [2021] specify the knowledge gap by stating that Living Labs are relatively underused in the environmental and agricultural sectors, which limits their understanding of the life sciences domain.

We align with Fuglsang et al. [2021], stating that “Living Labs remain a somewhat elusive concept and phenomenon, and there is a lack of understanding of its versatile nature”. This study seeks to reflect on four Living Labs — two in the Netherlands and two in Indonesia — to contribute to further conceptualising Living Lab. It aims to better understand Living Labs’ performance in a long-term perspective with a focus on engagement and participation.

The next section of this paper presents the analytical framework for analyzing Living Labs by elaborating on three Living Lab dimensions and a life cycle perspective. Subsequently, the reflection of the four Living Labs in the Netherlands and in Indonesia is presented. The paper winds up with a discussion and conclusion on the importance of considering public engagement, participation, and a life cycle perspective and formulates topics for future research on Living Labs.

2 Analytical framework

It is increasingly recognized that responding to the urgency of contemporary environmental challenges requires integrated and collaborative solutions by facilitating an open process that enhances creativity, broader participation, and public engagement [Beaudoin et al., 2022]. Arguments for a Living Lab approach come with an array of assumptions on multi-actor systems and re-interpretations of conventional participation approaches. Perceived benefits of a Living Lab approach link to expectations that ‘working together’ will lead to a better consideration of diverse interests and local conditions provide an environment for learning, an exchange of knowledge and ideas. Collaboration on concrete solutions and experiments is often mentioned as a benefit which also can be characterized as democratization of public sector innovation [Fuglsang et al., 2021]. A focus on better use of available resources in an area is foreseen to lead to shaping stewardship [Gomez & Derr, 2021]. Getting started at the local and regional level and creating a space for non or informal learning — see a.o. Hughes, Foth, Dezuanni, Mallan and Allan [2018] — are also frequently mentioned as undisputable benefits of Living Labs.

To withstand throwing out the baby away with the bathwater, the ambition for contemporary approaches with a predictable happy ending urged for searching a fresh understanding of the Living Labs’ hindrances to continue acting on basic premises of collaborative action. With this research, we do not aim to give a complete study on Living Labs. Nevertheless, we seek insights into the dynamics of Living Labs, focusing on sustainability transition in open network configurations where diverse actors collaborate and learn under the explicit condition of public engagement and participation.

The framework used for analysing Living Labs builds on the recent Scopus based literature review by Lie et al. [2023] which identified three themes to categorize the body of knowledge on Living Labs and Innovation platforms: (1) setting and system, (2) stakeholders and collaboration and (3) approach, aim and focus. In line with this literature review and further inspired by Veeckman et al. [2013] and Fuglsang et al. [2021], the following dimensions constitute the framework as visualised in Figure 1:

1. environment and focus,

2. methods and collaborative action,

3. outcomes.

As literature frequently indicates promising benefits and reasons to activate a Living Lab, the pragmatic, complex, dynamic, and unstructured nature of Living Labs demands the analytical framework to allow addressing such dynamics of ambitious prospects at the start of a Living Lab and all hurdles it faces in its lifetime.

Therefor the framework combines dimensions of Living Labs with a life cycle perspective adapted from the life cycle model of network governance processes developed by Imperial, Johnston, Pruett-Jones, Leong and Thomsen [2016]. Hereafter the three dimensions and the life cycle perspective are elaborated with the use of literature.

2.1 Living Lab dimensions

Dimension: Living Lab environment and focus. Living Labs are physical or virtual spaces with probably hybrid elements. Living Lab environments are constantly changing, as social relationships between actors and organizations are dynamic and programs are created, reshaped, or stopped from existing [Genskow & Born, 2006]. In the Living Labs that we reflect on in this paper, the environment is also physical and regional, with dynamics attributed to its very nature with exposure to the climate, the seasons and resource use. Portraying a Living Lab environment as a landscape is part of the ‘fairy tale’ we try to untangle. To avoid romanticism, we follow Hossain, Leminen and Westerlund, quoting Dvarioniene et al. [2015], who describes Living Labs as “experiential environments where participants engage in a creative space to design and experience products and services”. Referring to our domains, we merge such a description with writings from authors like Fuglsang et al. [2021] who characterize Living Labs as networks or platforms and thereby focus on “the ecosystem context of innovation beyond the user perspective, i.e., a Living Lab is not an isolated space disconnected from its environment”. This results in Living Lab environments which have physical or geographical components and are to be considered in a wider institutional and governance system.

Living Labs for integrated river management and sustainable regional development are often provider- or enabler-driven due to the very nature of the issue at stake. Enablers include various public-sector actors, non-governmental organizations, and financiers, such as towns, municipalities, or area-development organizations. For example, enabler-driven Living Labs aim to develop a specific region or city area to reduce local unemployment or solve diverse social and structural problems. Often, such Living Lab processes of public engagement and participation are initiated and coordinated by governmental institutions supported by knowledge institutes as these Living Lab actors have access to social-ecological system knowledge considered essential for innovation processes in the Living Lab.

A systemic view of the functioning of a Living Lab requires distinguishing between the enabling space and Living Lab activities, where participating actors meet in physical or virtual encounters. This distinction may suggest a hierarchical ranking contradicting the main methodological approaches of Living Labs. Articulating the enabling space is not to overrule a dominant role to aspects such as equality and horizontal communication but indicates a ‘reality check’ considering the scarcity of resources required to put a Living Lab in action. It is realistic to distinguish between the management of the Living Lab and the Living Lab as an open innovation network [Nyström, Leminen, Westerlund & Kortelainen, 2014].

Westerlund and Leminen [2011] use the term enablers to indicate organizations which enable the Living Lab activities with logistic, administrative, and financial support, amongst others. A distinction between the enabling space and the space of dialogue and deliberative action contributes to the unravelling of the Living Labs, also to disclose persisting ambitious expectations or fairy tale happy endings. Similarly, Kamols, Foth and Guaralda [2021, p. 7] add a critical note on stating, “While the issues of engagement and innovation theatre are acknowledged, there is often a lack of interest or ability in addressing the root causes as this would require substantial institutional change that goes beyond the capability of any one individual planner. Therefore, efforts to advance or reform participatory planning practices generally focus on incremental improvements to methods at hand rather than tackling systemic institutional change”.

Dimension: Living Lab methods and collaborative action. Living Labs originate from perspectives on participation, which move away from hierarchical models of participation to a focus on co-creation. Whereas conventional models can frame citizens as subjects who are entitled to participate by (dominant) institutional actors, co-creation builds on a realization of interdependencies and acts on synergy resulting from facilitated dialogues [Veeckman et al., 2013]. In this context, Wals [2015, p. 8] states “the fundamental task in the coming decades is to redesign our socio-political-economic system in ways that reintegrate the dependencies between people and their underpinning ecological systems”. Discourses of urgency, disruption and radical change are mentioned in a context of a Living Lab configuration, for example, Fuglsang et al. [2021] describes the development of Living Labs as “they disrupt traditional top-down, internally driven and efficiency-oriented innovation practices, and seek to replace them with inclusive, experimental and iterative approaches to innovation”.

Living Labs give articulated attention to methods for envisioning a future different from contemporary times. Diverse methods are used, from more conventional research to open innovation, from dialogue-based strategies to design thinking and mapping exercises with a prevailing transdisciplinary focus which Fuglsang et al. [2021] describes as “it is essentially the eclecticism that forms a defining trait of Living Lab methods”.

Operationalizing or facilitating a Living Lab is approached in different ways. From a research perspective, Living Labs may be defined, conceptualized, and theorized as empirical studies of research or experiments to produce new knowledge [Bergvall-Kareborn & Stahlbrost, 2009]. This paper is written against the background of our involvement in actual Living Labs in Indonesia and the Netherlands, representing our respective Higher Education Institutes (HEIs) which each have a sustainability transition focus. As HEIs’ staffs, we have taken up diverse roles such as providers, researchers, facilitators, and workshop organizers, and we have witnessed how a low impact of Living Lab activities comes at the cost of involved citizens and communities. Reed and Rudman [2022] describe this as “Current narratives around impact are implicitly positive in their definition and fail to recognize that there are often both winners and losers”. Disappointments resulting from frictions between ambitions and achievements may directly affect these actor groups’ resilience and social capacity and are our concern in terms of accountability.

Dimension: Living Lab outcomes. Living Labs focusing on the governance of transformative processes are often envisioned to create societal impact or public value. Increased recognition of a diversity of values beyond the most tangible and conventional is also observable in the academic textbook ‘Policy Analysis of Multi-Actor Systems’ [Enserink et al., 2022, 2^nd edition], referring to problem dimensions as technical, social, institutional, and normative whereas the first edition [Enserink et al., 2010] only detailed technical and social dimensions. Acknowledging a diversity of outcomes carries implications for monitoring and evaluation, as not all outcomes can be measured with quantified methods and qualitative methods may fall short.

Living Labs as spaces for innovation and experiments may lead to tangible or product innovations. The Living Labs studied here do not envision a utilizer-driven approach with ex-ante-defined technologies or services to design and develop but instead refer to a regional situation with potential conflicts between actors, resource use and intra-generational interests. Such a contentious open-ended space implies that expected outcomes may differ significantly for diverse Living Lab partners, and direct and indirect actors may have different affective attachments to possible outcomes. In situations where the Living Lab configuration is likely to be defined by most active and facilitating actors, sometimes acting individually rather than representative, transparency on the impact of participation and expected outcomes require due attention. Outcomes resulting from envisioning methods will not always be concrete and tangible but rather represent values for an envisaged future. As Leminen, DeFillippi and Westerlund [2015] discussed, Living Labs also create possible results not initially targeted. Nguyen and Marques [2021, p. 2] link what we call the happy endings expectations to the role of researchers and policymakers and mention the “discrepancy between expectations about the collaboration and the actual performance”.

2.2 Life cycle perspective

Analysing dynamics of Living Labs indicates that it is too easy to assume that facilitating actors’ collaboration is automatic process, smooth and without risks. Living Labs are commenced with an array of expectations addressing ambitions for the environment, methods, and outcomes. As dynamics cannot build on solid, undisputable foundations and models there is not one single approach that is spot-on for every Living Lab. To confront the paradox of longing for project outcomes versus due respect to an open process, which allows embarking on sustainable futures, we opt for a life cycle perspective. Westerlund and Leminen [2011] developed a typology distinguishing between actor activities in different phases of activation and innovation. Imperial et al. [2016] distinguish between four stages of development: activation, collectivity, institutionalization and a stage of stability, decline, re-orientation, and re-creation. The activation stage can be described as initial Living Lab network’s activities, involving the quadruple helix. The collectivity stage is characterized by a high commitment to the network and jointly formulated and reliable network processes. The institutionalization stage indicates a stable network which can deal with changing conditions. The final stage referred to as stability, decline, re-orientation, or recreation, recognizes the dynamics and various paths the development of a Living Lab network may encounter. Imperial et al. [2016] elaborate that the development of a network does not have to be a linear process because “network participants may confront challenges from different stages simultaneously” [2016, p. 136].

The life cycle model to understand the challenges in designing governance and participation processes for transformative change governance processes as developed by Imperial et al. [2016] means unravelling a ‘healthy and useful’ network life. Comparably, a life cycle perspective will be applied in this reflection to gain insight into how and why events unfolded as witnessed and to allow a review with a sense of sympathy for recent of inactive or dormant periods without blaming people and bringing resistance into the process. The recent lockdowns during the Corona pandemic endorse such empathy considering the demand for approaches of unprecedented nature.

3 Reflecting on four Living Labs

The following section reflects on the four Living Labs using the analytical framework with three dimensions and the life cycle perspective as elaborated for four Living Labs in the Netherlands and Indonesia. These Living Labs present differences in their year of activation and environment and focus. They are similar for the recent establishment and focus on sustainability transition processes. See Table 1 for an overview.

The consecutive Corona lockdowns influenced the continuation of these Living Labs despite dedicated efforts by societal partners and HEIs to proceed as ‘normal’ as possible under the given circumstances. Despite significant technological advancements in virtual communication, the performance of the Living Labs has been severely affected, and activities slowed down.

In March–October 2022, the dynamics in the Living Labs have been reviewed in meetings and workshops (Table 2). To make the life cycle approach workable for all Living Lab participants we distinguished between start (activation), continuation (Imperial et al.’s collectivity and institutionalization) and consolidation, referred to as a mid and long-term action plan (Imperial et al.’s [2016] stability, decline, re-orientation, or re-creation).

3.1 Living Lab Upper Citarum

Living Lab environment & focus. Most big rivers in Java Island are severely polluted [Resosudarmo, 2003; Roosmini et al., 2018] due to anthropogenic activities (domestic, industrial, agricultural). River restoration efforts receive attention and for many years, local and central government spent a lot of funds, but a significant result that change community health and welfare are far from the target.

Citarum River in West Java Province and its riverbank areas constitute the environment of the Living Lab Upper Citarum. Van Ginkel, Ozerol and Lufiandi [2015] indicate that water management for the situation in the Upper Citarum River watershed is strongly affected by the governmental decentralization policy. The large autonomy of local governments is at odds with the upstream-downstream relations in a river basin because basin issues are crossing administrative borders. Consequently, close collaboration between local authorities is essential in a decentralized government. Recognizing the need for strong coordinating bodies and to get a clear understanding of the positions of all actors while involving local communities, activated the Living Lab Upper Citarum. Since 2016, by partners from government, knowledge institutes, industry, and environmental community groups, the Living Lab focuses on sustainable river management in a context of the circular economy and community-based interventions.

The Living Lab has a focus on dealing with water quality, waste management, and reforestation and has a strong role for the university however the Living Lab focus on community participation is not aligned with common student research which repeatedly research the low water quality without community engagement, the latter not being part of their curriculum. In addition to the Living Lab Upper Citarum the Citarum Harum program is active in the watershed area. The Citarum Harum program was initiated in 2018 by the special Citarum Watershed Team base on President Regulation no. 15/2018: The Acceleration of Pollution and Watershed Degradation Control in Citarum River in association with the regional Military. The strict focus of the Citarum Harum program on direct and tangible results, such as solid waste removal and minimizing domestic and industrial wastewater, leads to a limited focus on community engagement.

Living Lab methods and collaborative action. A recent Living Lab meeting (Figure 2) illustrates community engagement between institutional participants. During this meeting a disbalance of the network was discussed by NGO’s who are supporting communities with waste management and other initiatives whereas other Living Lab partners are currently more awaiting. Local leaders proposed improving the cohesion of the Living Lab and requested “to be trained in more communication and negotiation skills, especially to communicate with and to involve local governmental bodies”. This demands the network leaders and enablers to facilitate such an activity. Network leadership was not only mentioned for governmental partners but also for communities, “community leaders are important to show the right way of doing”.

A workshop at ITB campus indicated a high institutional responsibility by the governmental organization (West Java Environmental Protection Agency) and knowledge institutes (ITB and Telkom) based on high personal commitments to the Living Lab. The network struggles with the low community stewardship after a period of inactivity. “I have been thinking is force become the only way for us to save the entire upstream quality of Citarum river and also the downstream quality? Because if the Citarum Harum program changes or the transition to new program happens and the community near river does not feel like they own the river it will just going to cause another environment problems in the future” (member of Telkom University).

Living Lab outcomes. The Living Lab Upper Citarum envisions stewardship and the river to be restored as a valuable resource. The complexity of technological challenges in the catchment, the multiple stakeholders and diverse time frames of immediate and long-term sustainability present options for aspiring interventions. Topics such as sediment management, agro-forestry in the critical lands and oxbow river restoration have been discussed. Quite some attention was given to the area of Majalaya due to occurring floods and pollution by (textile) industries and domestic waste. Diverse projects have been taking place but not all in the context of the Living Lab or resulting from community-based approaches and stewardship affecting the sustainability of certain achievements.

A distinctive activity is the poetry route ‘Citarum voices’ following a community art approach in two villages upstream of Bandung [Roosmini, Witteveen, Dwi Mayangsari, Nastiti & Botden, 2023]. Twelve banners compiling poems and paintings made by community members reflect a dialogue between the river and riverbank communities. The tangible outcome of the Poetry route constitutes a non-tangible outcome of voicing the value of the river by community members and by the river itself as a non-human voice. The poetry route, travelled to other communities establishing a conversation space. The community art activities also served as a 21^st century skills training for Living Lab institutional partners and ITB students.

Institutional partners of the Living Lab stated, in the evaluation, their surprise about implementing an activity which was so unfamiliar for them and so clearly responded by the community. This discrepancy shed light on the relevance of articulating experience. Eggens and Chavez-Tafur [2019, p. 8] describe such activities as experience capitalization, which they define as “an approach that allows people to systematically analyse and document real-life experiences, in a participatory way. This creates knowledge, which can be shared and used to generate change, both internally and in other initiatives”. A focus on experience capitalization might be an additional role for knowledge institutes or other providing partners in a Living Lab.

Desirable and achieved outcomes on water quality are difficult to establish as monitoring systems to evaluate the progress are not well-informed and need laboratory analysis data “unlike solid waste that we can see has already been removed from the river”. Also, it is acknowledged that the Living lab is not the only configuration or network active for river restoration or sustainable river management.

Life cycle. The Living Lab Upper-Citarum is currently reframing the continuation stage after a period of inactivity caused by the Corona pandemic. Communication between ITB university and the communities in the field slowed during the pandemic, and the community in Ciwalengke (Elingan NGO) became more expressive by showing their Citarum River activities on Instagram media. As Instagram is becoming prevalent in society, most local and central governments use it to show their activities and achievement. The West Java Provincial harness this social media power in their concept of Penta helix for their development programs, involving the parties of government, communities, academia, entrepreneur, and media (social media).

As the Living Lab network exist, partners express the dynamics of the configuration of the diverse partners and aim to develop a more cohesive network. A recent community meeting in the Living Lab has been organized by the university to re-activate the Living Lab sessions. The poetry route ‘Citarum Voices’ was exhibited and functioned as conversation starter which brought the community to express their satisfaction for an apporach “not blaming us for doing things wrong.” In this context the local NGO in the community formed a facilitator team in the level of subdistrict. The local initiative was perceived as allowing more interaction with the community and finding some idea or obstacle in river water management.

Another point of attention is the Citarum Harum program which will be stopped in 2024, thereby challenging the Living Lab to reconfigure. The end of the Citarum Harum program may change the role and contribution of the university and thereby become a multidisciplinary hub for enhancing community engagement.

3.2 Living Lab Delta East

Living Lab environment & focus. Dutch river management has a mainly sectoral perspective as it “seeks to optimize single societal functions or only a few functions” [Havinga, 2020], for example combining flood protection with developing nature in floodplains [Fliervoet, Van den Born, Smits & Knippenberg, 2013] and integrated river management has not yet been achieved. According to den Haan, Fliervoet, van der Voort, Cortes Arevalo and Hulscher [2018] other challenges are creating flexibility in a controlled river system, adopting the integrated approach to the maintenance phase and developing climate proof river management policies. Consequently, institutional partners of Living Lab Delta East envision an integrated and sustainable future for the riverine landscape and focus on challenges in a river ecosystem (interactions on a river scale), in the Eastern part of the Dutch Delta. Universities also initiated the Living Lab in 2017 searching for real-life learning opportunities for students in ongoing projects. All partners are directly involved in integrated river management: the Dutch Water Authority (flood safety), provincial government (landscape, biodiversity, and sustainable development), Water Board (flood safety, sustainable water system, sewage treatment), nature conservation organisations (biodiversity), sand- and clay-mining industry (business), municipalities (liveability, safety, and sustainability) and knowledge institutes.

The enabling space was organized by the National Water Authority, provincial government, and knowledge institutes. The facilitator of the Living Lab sessions was a staff member of the national water authority. Independently of the current activities of the Living Lab, it is undeniable that the Living Lab configuration exists according to the partners and regional actors involved.

Living Lab methods and collaborative action. During the activation stage the partners focused on creating a shared vision for the riverine landscape. In 2019 and 2020, several Living Lab sessions were organized by the enablers to elaborate on a shared vision, analyse the baseline situation, and develop possible sustainable solutions for three case study areas (Upper Waal, Middle-Waal, and Ijssel) as documented on the website:⁷ The website does not indicate to what extent activities are discontinued or if the website maintenance just finished after the projects and project budgets ended.

Since its start in 2017, Living Lab Delta East applied the concept of mediated participation [L. M. Witteveen, Enserink & Lie, 2009] in creating a learning environment about sustainable river management. The Visual Problem Appraisal (VPA) Rhine River Branches creates a learning environment with a longitudinal series of filmed actor interviews (Figure 3). A continuation of the production to include more actor interviews is an ambition on the agenda.

Living Lab outcomes. Since the start, multi-actor sessions have been organized and the participants formulated a shared future vision. Institutional the participants show a high personal commitment to the network and before the COVID lockdowns joint-formulated projects were executed, such as Circular Floodplain Management and ‘SteenGoed’.

The VPA ‘Rhine River Branches’ has been used in formal and non-formal learning environments, also during Lockdowns and successfully contributes to mediated participation.

Life cycle. Currently, the Living Lab Delta East is inactive and not engaging with the public. As personal commitments and institutional agendas are discernible, the Living Lab might be at the brink of a stage of re-orientation and re-creation. One explanation is the COVID-19 pandemic as no meetings were held between since 29th of January 2020. Based on interviews with Living Lab partners other aspects that hinder the progress of Living Lab were mentioned, such as the complexity of working in a multi-actor setting, missing urgency and the perceived absence of actions and outcomes. Collaborating in a quadruple helix configuration is challenging for many partners, because it is not their conventional approach, which leads to question institutional mandates network leadership and the framing of ownership.

In recent meetings between the enablers (National water authority, the provincial government and knowledge institutes) opportunities or strategies were discussed to re-activate Living Lab Delta East. All enablers indicated an interest to continue although one participant questioned the term ‘Living Lab’ and it was stated that ‘regional cooperation’ would also do to re-activate the focus of the Living Lab to water, soil, and public participation in sustainable river management.

Involving citizens seems undervalued in Living Lab Delta East, especially in developing policies for sustainable river management. The partners addressed the knowledge infrastructure of the Living Lab and referred websites of finished projects which are no longer accessible. This is partially attributed to a project approach in Living Lab activities as compared to a long-term research, innovation, and knowledge agenda.

3.3 Living Lab Enrekang

Living Lab environment & focus. Considering the challenges on sustainable rural economy in the mountainous area of Enrekang, the District Agriculture Office (DAO), District Livestock and Fisheries Office (DLFO), District Environment Office (DEO), District Health Office (DHO) and the University of Muhammadiyah Enrekang (UNIMEN) joined hands and decided to opt for a Living Lab approach envisioning further alignment in policies and interventions. The COVID-19 pandemic brought the District Health Office on board and more recently the disaster management office joined the Living Lab team.

Acting on the felt need to develop sound communication strategies the Living Lab partners explored innovative approaches to engage with farmers in a local innovation process to overcome the disconnection between rural development policies and governance versus the implementation of the agricultural and environmental knowledge systems. Soon after the start of the Living Lab the partners agreed to start the design and develop of a Digital Farmer Field School (DFFS) as a communication interface with rural farmer groups. For DFFS design and development the Living Lab Enrekang cooperates with the Dutch partner Van Hall Larenstein University (VHL).

Living Lab methods and collaborative action. Since the start in 2018, the Living Lab Enrekang organised diverse activities. Quarterly coordination meeting in conjunction with regular agriculture coordination meeting led by DAO succeeded in sharing of expertise among the Living Lab partners.

A major activity of the Living Lab has been a design training for DFFS implemented during COVID lockdowns.

The achievement of producing a DFFS Prototype and the consecutive testing with farmer groups created a shared enthusiasm (Figure 4). The development of DFFS creates an essential collaborative action for the institutional partners in the Living Lab configuration. Related assignments such as the Social Ecological System Analysis supported synchronisation towards sustainable agriculture by different departmental services.

The partnership approach in the Living Lab is highly appreciated by the institutional partners and perceived as a governance achievement. Considering farmers as direct partner is not a new approach for the institutional partners, however it is the realisation that the institutional participants have a sound ability to act in line with the preconceived plan for considering farmers as a partner in the Living Lab which adds affective outcomes to the Living Lab. A recent prototype testing activity gave a nice impression that farmers similarly appreciate to be considered as equal participants⁸ leading to a further appreciation of farmer groups as partners in the Living Lab.

Living Lab outcomes. Through the Living Lab, participating actors gained more insight in the complexity of issues they are facing in relation to sustainable rural economy. The transdisciplinary perspective resulting from contributions by diverse (institutional) actors reframed many problems as multi-facetted and interdependent. Such a perception of issues at stake, improved decision-making process as all information and expertise from Living Lab members was incorporated.

A DFFS workshop in 2019 established a LL’s ethical standard “We do not want to disappoint Farmers”. This statement expresses the presence of normative values among the participants which links to a high personal commitment to the network.

The active roles of the farmer groups in the DFFS design process of the innovation DFFS contributed to provide input and perspectives on their priority information needs and how rural issues cross boundaries among government institutions. Public participation in the context of Living Lab Enrekang also contributed to changing the way of the local government is working as re-considering farmers from beneficiaries to end users is quite a radical development.

Life cycle. The design and development of DFFS offers a conducive space to explore intersectoral collaboration and activates the participation of farmers as the main users and as co-creators of content in all stage of the development process. The current situation of the Living Lab Enrekang is highly active in relation to DFFS activities, the Living Lab is dynamic, and more partners are participating.

During workshops in Enrekang the Living Lab approach was in the spotlight. A community prototype testing activity created a sound context for Living lab partners to experience the relevant contribution of local communities and farmer groups to regional developments. In further workshop activities the Living Lab were reviewed according to the stages of network development and participants were clear about concrete steps to take in the continuation stage.

The Enrekang institutional partners and the local DFFS team are committed to continue the Living Lab approach. It became however apparent that to consolidation stage requires mobilizing local funding as a possible dependency on external funding is notable. Whereas activities in the continuation stage could be described in concrete terms with achievable outcomes; the consolidation stage also referred to conflict resolution capacity and communication challenges among participants.

The Living Lab partners search to create a conducive context for continuing collaboration by establishing a legal foundation for the Living Lab which embedded within government structure to prevent that DFFS would be the only focus. The Living Lab approach has drawn attention of the district’s planning, research, and development agency as its in line with the approach that the district government is promoting to achieve its development goal.

3.4 Living Lab GEUS

Living Lab environment & focus. The Living Lab actors relate to the villages Garderen, Elspeet, Uddel and Speuld located in a rural area with heathlands, forests, fens, lakes and streams, currently known for its large number of veal-calf farms [Wennemers, Spek & van Eijk, 2021]. This highly intensive farming results in a high nitrogen deposition in the area which has a negative impact on the surrounding nature conservation area.

Current environmental policies challenge the Living Lab to deal with high nitrogen depositions, impact of intensive agriculture, climate change, low water quality and nature management. The interrelated challenges are spread over various policies, programs, and agendas. Enforced by all these challenges, the main guiding principle of water and soil are steering in spatial planning policies. The aim of the Living Lab to facilitate collaboration between transition pathways and parties to sustainably develop the area. The following goals are formulated by the enabler (Water board) for the Living Lab:

• Develop concrete proposals for an integrated area approach to decrease the nitrogen deposition in which landscape quality, nature goals, cultural identity, social and economic needs are aligned;

• Create a cooperation model between governments, knowledge institutes, organizations, residents and entrepreneurs for the long term that should lead to more efficiency, transparency and understanding of actions to be taken;

• Exploring key instruments and new ways of collaboration with public and private partners in the context of the new Environmental Planning Act, like the Blue Spatial Vision 2050 (www.BOVI2050.nl);

• Accelerate transitions by courageously trying things out and experiment during a collective (action) learning process.

Living Lab methods and collaborative action. The Living Lab GEUS started in 2022 bringing together governments, residents, entrepreneurs, knowledge institutes and social parties to achieve an integrated approach to the area. The first Living Lab meetings aimed to discuss the context, explore expectations and advantages of a Living Lab method, and jointly create a future vision using World Café and envisioning methods (Figure 5).

From the start the main point on the agenda were nitrogen policies by the government. The nitrogen deposition discussions indicate that discussion leaders should be able to deal with sharp conversations exemplified by civil participants statements’ that “nothing is allowed anyway”, and “they’re coming to take something away from us”. The contentious policy context revealed the need for independent facilitation to enable dialogue with great controversies occur between citizens and institutional participants. The assumption is that dialogue is a boundary condition to achieve a shared vision and shared understanding for each other interests towards viable societal, spatial, and environmental development.

Living Lab outcomes. The waterboard acted as (financial) enabler. Professional facilitators were hired for initial Living Lab sessions. The waterboard also organised the production of a landscape biography to feed into the discussion on the historical identities of the area and multi-actor sessions have been organized. The participants established core values and a future vision (Figure 6) and thematic working groups are established,

The Living sessions achieved increasing representativity with attention for youth participation, the distribution of the villages and other social-cultural aspects. The realization of higher involvement required articulation of interest and increased the commitment of provincial and municipal partners

Life cycle. The Living Lab has, by the time of writing, a short active life and is yet to prove its relevance. Since the activation, there has been an increase in participants contributing to the Living Lab participants represented all four villages although with a dominant representation of the agricultural sector. The Living Lab meetings led to the development of shared ambitions in envisioned outcomes, agreements on methods and the creation of four thematic working groups. These groups aim to elaborate concrete and feasible follow-up proposal for experiments.

The start of the Living Lab showed challenges regarding commitment, effectiveness of the process and the management of the Living Lab. The Living Lab meetings demonstrated a precarious cooperation between different parties, as civil actors expressed doubts in relation to past experiences with collaborative processes organized by the (regional) government some thirty years ago which were not followed up.

It is remarkable that Waterboard is not formally in charge concerning spatial development issues, takes the lead in the Living Lab and thereby shows and independent and trustful role and their specific knowledge of the area. The waterboard, provinces, municipalities and regional business and civil actors are all confronted with national policies regarding the agricultural transitions. The pressures experienced by the national policies the mandate and scope of influence of the Living Lab partners. Major enabling partners realise the relevance of continuity.

4 Discussion and conclusion

Enhancing the performance of a Living Lab is legitimate in contemporary times of social and natural sustainability transitions where urgent challenges require strategic responses. While recognizing the pressing need for an approach with a quality to bring diverse participants, perceptions, and engagements together, we acknowledge a risk when simple claims on outcomes become an implicit guiding principle.

The analytical framework with three dimensions and a life cycle perspective allows reflecting on Living Labs. The dimensions have not been perceived as completely distinctive but supported unravelling diverse aspects of performance of Living Labs. The life cycle perspective is supportive to articulate ordinary changes as they are to happen while creating space for more unintended changes, without a sense of victim blaming.

It remains ambiguous for the first dimension how focus and environment define each other; it is illustrative how the Living Lab Enrekang has an articulated focus on the Digital Farmer Field School (DFFS) design and development while DFFS enhances Living Lab ambitions. Situating the enabling space in the dimension Environment and focus is not yet optimal to elaborate on diverse roles and functions, some aspects referring to funding and other resources gives a glimpse of appreciating diverse power stands.

The dimension of Methods and collaborative action indicates a plead for physical multi-actor meetings as a major characteristic of a Living Lab. Considering participation, overlooked aspects of access, inclusion and conditions for participation shaping this dimension as can be learned from praxis are:

• Civil stakeholders are participating for free, while all other stakeholders are in (salaried) functions,

• Non-civil stakeholders are also citizens and should deal accordingly with potential conflictive value sets discrepancies,

• Materializing participation and voicing links to communicative and social-cultural aspects such as eloquence and educational background.

Based on experiences with envisioning methods for stewardship (LLab Upper Citarum, LLab Geus, LLab Enrekang) and inspired by Latour [2020] it might be considered to recognise the non-human as an actor in sustainability-oriented Living Labs which motivates Foth [2018, p. 13] to suggest expanding the quadruple helix to a quintuple or penta helix model.

Living Lab methods may require paying attention to rewarding aspects. Envisioning and dreaming is easier than dealing with disappointments, especially if these are constituted by interpersonal conflicts, natural disasters, or detested policies. In this context the Rathenau Institute [Van den Broek, van Elzakker, Maas & Deuten, 2020, p. 54] rightly calls on all stakeholders to care for an upscaling strategy before starting a Living Lab. Nguyen and Marques [2021, p. 4] address aspects of expectations describing that while “anticipated advantages of Living Labs are undeniable” findings suggest that in the activation phase initial “expectations reasonably explain the engagement of stakeholders”.

The Living Labs outcomes as described for the third dimension represent the open nature of the networks, despite perceived and documented success, problems, issues, and challenges are experienced. Living Lab configurations lead neither directly nor without doubt explicit positive outcomes as might have been assumed at the start of its conceptualization. Such perceptions of outcomes align with Enserink et al. [2022] regarding the complexity of recognising institutional and normative values. To further interpret aspects of resource investments by actor groups this may be included in the framework linked to an elaboration of roles.

A life cycle perspective gives insights in the dynamics and nonlinear process of a Living Lab as it is generous to the dynamics of the performance and the outcomes, it allows the partnership to reformulate and reactivate original ambitions upon stagnation. This generosity does not apply to respecting paramount qualities of a Living Lab approach such as the explicit focus on participation, engagement and voicing.

The findings suggest that Living Labs are often in the stage of reframing, re-orientation, and re-creation (LLs Upper Citarum and LLs Delta East), which makes it difficult for participants, especially citizens, to stay engaged. As the life cycle perspective creates space for change and adaptive dynamics it also requires considering an eventual drop out by non-institutional participants as they may leave with contra productive sentiments against public or institutional partners without distinguishing differences e.g., between waterboards, provinces or environmental agencies based on their negative perception of the Living Lab experience.

Higher Education Institutes (HEIs) play diverse roles in a Living Lab in relation to the substance and facilitation of a Living Lab. This may carry implications for the inclusion of Living Lab competences in the curriculum of HEI [L. Witteveen, Eweg, Smits & Voskamp-Harkema, 2016] to prepare students for participating in Living Lab environments in their professional future.

HEIs may feel accountable accordingly to contribute to knowledge creation and exchange regarding the Living Lab performance. Promoting the plural form ‘knowledges’ may induce respect and recognition of diverse knowledges and disclose dominant knowledge systems such as written textualities and expert language rather than opting for conventional epistemological concepts such an indigenous or local knowledge. Working in a context-sensitive way and facilitating public engagement and participation are responsibilities in the enabling space, which may not be entangled with enabling funding aspects for reasons of autonomy.

The constructed framework with the dimensions and life cycle as analytical perspectives allowed reflecting on four Living Labs and will gain accuracy and relevance with further use. It is yet to be seen if the influence of the COVID-19 pandemic had a temporal impact on the Living Labs performance or refers to an overall element of limited resilience.

Future work. Knowledge providers in Living Labs have a kind of neutral position HEIs must question that position when combining a knowledge brokering role with other roles such as a facilitation role. Transparency is required on normative aspects as knowledge may be neutral, but paradigms are not. Kronsell and Mukhtar-Landgren [2018] argue to construct and rethink guiding principles for diverse actor roles in Living Labs.

The focus on participation fatigue and social capital apparent in these Living Labs inspires elaborating an accountability perspective for the enabling space considering time as a scarce resource of civil actors. Also mediated participation and innovative methodological options require to be designed to overcome the absence of overlooked or distant voices [Roosmini et al., 2023].

In line with the statement in Dunnigan’s paper on Fairy tales, Myth and Happy Endings⁹ that “For most mortals however life without stories is too bleak a prospect: we need to have something to look forward to: to anticipate possible future outcomes based on past and present activity. We want to believe in a hopeful, shareable world where we are understood”. A longing for Living Lab as approach for radical sustainability change, social action, and happy endings may be discernible but is not in accordance with the findings and blown away with the realisation that inadequate public engagement and participation strongly affect the collective social capital.

The quest towards a more responsible way of acting in all aspects of the Living Lab life cycle remains imperative. As referred to in the introduction, the call by youth to act systemic and radical needs to be heard and deserves responding as the performance of Living Lab approaches do not fulfil the potential of public engagement and participation while providing and enabling parties underestimate the complexity of the process. Developing guiding principles for actor roles will further foster a more responsible approach to Living Labs.

Elaborating further on roles requires mentioning that all Living Lab partners contribute and receive [Hossain, Leminen & Westerlund, 2019]; a balance which requires transparency to prevent falling back on conventional interpretations of participation with words like beneficiaries, recipients and victims or other disrespectful discourses. Similarly to the critique on COP that a focus on a deeper or radical system transformation is not really on the institutional agendas, alternative routes need to be outlined for a Living Lab approach to be as transformative as envisioned.

Acknowledgments

The authors are grateful to Koen de Reus, Twan Ponse, Marnicq de Regt from Van Hall Larenstein University of Applied Sciences and Nissa from Institut Teknologi Bandung for their contribution to the partial data collection of Living Lab Delta East and Living Lab Upper Citarum. In addition, the authors would like to thank all Living Lab partners for their participation in the Living Lab sessions and meetings with enablers.

Declaration of conflicting interests. The authors declare no conflicts of interest.

Funding. This project has received funding from the European Union’s Horizon 2020 research and innovation programme Invest for Excellence in Regional Sustainability under grant agreement No 101035815.

References

Beaudoin, C., Joncoux, S., Jasmin, J.-F., Berberi, A., McPhee, C., Schillo, R. S. & Nguyen, V. M. (2022). A research agenda for evaluating living labs as an open innovation model for environmental and agricultural sustainability. Environmental Challenges 7, 100505. doi:10.1016/j.envc.2022.100505

Bergvall-Kareborn, B. & Stahlbrost, A. (2009). Living Lab: an open and citizen-centric approach for innovation. International Journal of Innovation and Regional Development 1 (4), 356. doi:10.1504/ijird.2009.022727

Bronson, K., Devkota, R. & Nguyen, V. (2021). Moving toward generalizability? A scoping review on measuring the impact of living labs. Sustainability 13 (2), 502. doi:10.3390/su13020502

den Haan, R. J., Fliervoet, J. M., van der Voort, M. C., Cortes Arevalo, V. J. & Hulscher, S. J. M. H. (2018). Understanding actor perspectives regarding challenges for integrated river basin management. International Journal of River Basin Management 17 (2), 229–242. doi:10.1080/15715124.2018.1503186

Dvarioniene, J., Gurauskiene, I., Gecevicius, G., Trummer, D. R., Selada, C., Marques, I. & Cosmi, C. (2015). Stakeholders involvement for energy conscious communities: the energy labs experience in 10 European communities. Renewable Energy 75, 512–518. doi:10.1016/j.renene.2014.10.017

Eggens, L. & Chavez-Tafur, J. (2019). Facilitating experience capitalization: a guidebook. Wageningen, The Netherlands: Technical Centre for Agricultural and Rural Cooperation (CTA). Retrieved from https://cgspace.cgiar.org/bitstream/handle/10568/99247/2058_PDF.pdf

Enserink, B., Bots, P., van Daalen, E., Hermans, L., Koppenjan, J., Kortmann, R., … Thissen, W. (2022). Policy analysis of multi-actor systems. doi:10.5074/T.2022.004

Enserink, B., Hermans, L., Kwakkel, J., Thissen, W., Koppenjan, J. & Bots, P. (2010). Policy analysis of multi-actor systems. The Hague, The Netherlands: Lemma.

Fliervoet, J. M., Van den Born, R. J. G., Smits, A. J. M. & Knippenberg, L. (2013). Combining safety and nature: a multi-stakeholder perspective on integrated floodplain management. Journal of Environmental Management 128, 1033–1042. doi:10.1016/j.jenvman.2013.06.023

Foth, M. (2018). Participatory urban informatics: towards citizen-ability. Smart and Sustainable Built Environment 7 (1), 4–19. doi:10.1108/sasbe-10-2017-0051

Fuglsang, L., Vorre Hansen, A., Mergel, I. & Taivalsaari Røhnebæk, M. (2021). Living labs for public sector innovation: an integrative literature review. Administrative Sciences 11 (2), 58. doi:10.3390/admsci11020058

Genskow, K. D. & Born, S. M. (2006). Organizational dynamics of watershed partnerships: a key to integrated water resources management. Journal of Contemporary Water Research & Education 135 (1), 56–64. doi:10.1111/j.1936-704x.2006.mp135001007.x

Gomez, T. & Derr, V. (2021). Landscapes as living laboratories for sustainable campus planning and stewardship: a scoping review of approaches and practices. Landscape and Urban Planning 216, 104259. doi:10.1016/j.landurbplan.2021.104259

Havinga, H. (2020). Towards sustainable river management of the Dutch Rhine river. Water 12 (6), 1827. doi:10.3390/w12061827

Hossain, M., Leminen, S. & Westerlund, M. (2019). A systematic review of living lab literature. Journal of Cleaner Production 213, 976–988. doi:10.1016/j.jclepro.2018.12.257

Hughes, H., Foth, M., Dezuanni, M., Mallan, K. & Allan, C. (2018). Fostering digital participation and communication through social living labs: a qualitative case study from regional Australia. Communication Research and Practice 4 (2), 183–206. doi:10.1080/22041451.2017.1287032

Imperial, M. T., Johnston, E., Pruett-Jones, M., Leong, K. & Thomsen, J. (2016). Sustaining the useful life of network governance: life cycles and developmental challenges. Frontiers in Ecology and the Environment 14 (3), 135–144. doi:10.1002/fee.1249

Kamols, N., Foth, M. & Guaralda, M. (2021). Beyond engagement theatre: challenging institutional constraints of participatory planning practice. Australian Planner 57 (1), 23–35. doi:10.1080/07293682.2021.1920993

Kronsell, A. & Mukhtar-Landgren, D. (2018). Experimental governance: the role of municipalities in urban living labs. European Planning Studies 26 (5), 988–1007. doi:10.1080/09654313.2018.1435631

Latour, B. (2020). Het parlement van de dingen. Over Gaia en de representatie van niet-mensen. Amsterdam, The Netherlands: Boom uitgeverij.

Leminen, S., DeFillippi, R. & Westerlund, M. (2015). Paradoxical tensions in living labs [Conference paper], In XXVI ISPIM conference — Shaping the Frontiers of Innovation Management, 14th–17th June 2015, Budapest, Hungary.

Lie, R., van Paassen, A. & Witteveen, L. (2023). Living labs and innovation platforms: a literature review. In print. Penang, Malaysia: WorldFish.

McPhee, C., Bancerz, M., Mambrini-Doudet, M., Chrétien, F., Huyghe, C. & Gracia-Garza, J. (2021). The defining characteristics of agroecosystem living labs. Sustainability 13 (4), 1718. doi:10.3390/su13041718

Monno, V. & Khakee, A. (2012). Tokenism or political activism? Some reflections on participatory planning. International Planning Studies 17 (1), 85–101. doi:10.1080/13563475.2011.638181

Nguyen, H. T. & Marques, P. (2021). The promise of living labs to the quadruple helix stakeholders: exploring the sources of (dis)satisfaction. European Planning Studies 30 (6), 1124–1143. doi:10.1080/09654313.2021.1968798

Nyström, A.-G., Leminen, S., Westerlund, M. & Kortelainen, M. (2014). Actor roles and role patterns influencing innovation in living labs. Industrial Marketing Management 43 (3), 483–495. doi:10.1016/j.indmarman.2013.12.016

Reed, M. S. & Rudman, H. (2022). Re-thinking research impact: voice, context and power at the interface of science, policy and practice. Sustainability Science 18 (2), 967–981. doi:10.1007/s11625-022-01216-w

Resosudarmo, B. P. (2003). River water pollution in Indonesia: an input-output analysis. International Journal of Environment and Sustainable Development 2 (1), 62. doi:10.1504/ijesd.2003.002363

Roosmini, D., Septiono, M. A., Putri, N. E., Shabrina, H. M., Salami, I. R. S. & Ariesyady, H. D. (2018). River water pollution condition in upper part of Brantas River and Bengawan Solo River. IOP Conference Series: Earth and Environmental Science 106, 012059. doi:10.1088/1755-1315/106/1/012059

Roosmini, D., Witteveen, L., Dwi Mayangsari, I., Nastiti, A. & Botden, T. (2023). Facilitating social imaginaries methods in living lab Upper Citarum. Journal of Nonformal Education. Under review.

Smits, A. J. M., Straatsma, M. W. & Fliervoet, J. M. (2019). Co-creative river landscapes. Landschap 2, 115–123. Retrieved from https://www.landschap.nl/wp-content/uploads/2019-2_LandschapSmitsp114-123losse.pdf

Ståhlbröst, A. & Holst, M. (2017). The living lab methodology handbook. Sweden: Social Informatics at Luleå University of Technology and CDT — Centre for Distance-spanning Technology. Retrieved from https://www.ltu.se/cms_fs/1.101555!/file/LivingLabsMethodologyBook_web.pdf

Van den Broek, J., van Elzakker, I., Maas, T. & Deuten, J. (2020). Voorbij lokaal enthousiasme — Lessen voor de opschaling van living labs. The Hague, The Netherlands: Rahenau Instituut. Retrieved from https://www.rathenau.nl/sites/default/files/2020-02/Voorbij%20lokaal%20enthousiasme%20%E2%80%93%20Rathenau%20Instituut_2.pdf

Van Ginkel, C. H., Ozerol, G. & Lufiandi (2015). Water quality monitoring in the Upper Citarum River Basin: rethinking the role of stakeholders. In 5th Environmental Technology and Management Conference “Green Technology towards Sustainable Environment”, 23rd–24th November 2015. Institut Teknologi Bandung, West Java, Indonesia.

Veeckman, C., Schuurman, D., Leminen, S. & Westerlund, M. (2013). Linking living lab characteristics and their outcomes: towards a conceptual framework. Technology Innovation Management Review 3 (12), 6–15. doi:10.22215/timreview/748

Wals, A. (2015). Beyond unreasonable doubt. Education and learning for socio-ecological sustainability in the Anthropocene. Wageningen, The Netherlands: Wageningen University. Retrieved from https://edepot.wur.nl/365312

Wennemers, S., Spek, T. & van Eijk, P. (2021). Handreikingen voor een toekomstbestendig watersysteem op de Veluwe: Gebiedsspecifieke bestrijding van verdroging op basis van een landschapsbiografische aanpak. Water Governance 2, 92–103. Retrieved from https://edepot.wur.nl/555503

Westerlund, M. & Leminen, S. (2011). Managing the challenges of becoming an open innovation company: experiences from living labs. Technology Innovation Management Review 1 (1), 19–25. doi:10.22215/timreview/489

Witteveen, L., Eweg, R., Smits, A. & Voskamp-Harkema, W. (2016). Design principles for Living Lab’s aiming at sustainable development. The role of higher education in Living Lab’s [Conference paper], In Conference ‘Competence 2016’, October 2016, Wageningen, The Netherlands.

Witteveen, L., Fliervoet, J., Eweg, R., Arabska, E., Van der Maas, P., Lazarov, A., … Puhakka-Tarvainen, H. (2022). The transition towards regional sustainable futures: a position paper on exploring a transdisciplinary view on the INVEST living lab approach. INVEST4EXCELLENCE project (101035815).

Witteveen, L. M., Enserink, B. & Lie, R. (2009). Mediated participation: using filmed narratives in complex multi-stakeholder settings. International Journal of Public Participation 3 (1), 32–62.

Authors

Loes Witteveen is professor of the research group Communication, Participation and Social Ecological Learning (CoPSEL) at Van Hall Larenstein University of Applied Sciences and works with the Knowledge, Technology, and Innovation (KTI) group, Wageningen University, the Netherlands. Her work relates to strategies for social change, participatory and deliberative governance and is situated at environmental and social science interfaces. Current research links to social imaginaries and sustainable water management, inclusion and affordance in user centred design and visual research methodologies.
@@WitteveenLoes E-mail: loes.witteveen@hvhl.nl

Jan Fliervoet is a senior researcher of the research group Communication, Participation and Social Ecological Learning (CoPSEL) and coordinator of the master River Delta Development at Van Hall Larenstein University of Applied Sciences, the Netherlands. He did research on collaborative governance in the context of integrated river management. More recently, his research is focusing on mediated participation strategies and multi-actor collaboration, such as Living Labs, in the water domain.
E-mail: jan.fliervoet@hvhl.nl

Dwina Roosmini is an associate professor in the Environmental Engineering Department, Faculty of Civil and Environmental Engineering, Institut Teknologi Bandung (ITB). Indonesia. She is appointed as the ITB Centre of Environmental Studies (CES) head, focusing on research, community services, and environmental management capacity-building training. CES is developing research involving social and economic aspects that need multidisciplinary collaboration. Research collaboration with social-engineering expertise gives an essential perspective for the environmental engineer on completing environmental problems in multidisciplinary collaboration.
E-mail: dwina@ftsl.itb.ac.id

Paul van Eijk is professor of the research group Sustainable Water in the Planning and Environment Act, Van Hall Larenstein University of Applied Sciences and program leader for implementation of the Act. Environment Act, Water Board Vallei & Veluwe, the Netherlands.
E-mail: paul.vaneijk@hvhl.nl

Nurdahlia Lairing is lecturer and researcher at the Research Center for Communication, Social Ecological Learning and Sustainable Environment (CoSeLSe), University of Muhammadiyah Enrekang, Indonesia. Her work focuses on design and development of the Digital Farmer Field School (DFFS) in Enrekang district and is otherwise related to Rural Communication Services and User Centred Design.
E-mail: nurdahlia2112@gmail.com

Endnotes