# Development of a Learning Ecosystem for Effective Learning in Socio-Technical Complex Systems

Maira Callupe, Monica Rossi, Brendan P. Sullivan, Sergio Terzi

► To cite this version:

Maira Callupe, Monica Rossi, Brendan P. Sullivan, Sergio Terzi. Development of a Learning Ecosystem for Effective Learning in Socio-Technical Complex Systems. 19th IFIP International Conference on Product Lifecycle Management (PLM), Jul 2022, Grenoble, France. pp.587-596, 10.1007/978-3-031-25182-5_57. hal-05420894

HAL Id: hal-05420894

https://inria.hal.science/hal-05420894v1

Submitted on 17 Dec 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Development of a Learning Ecosystem for Effective Learning in Socio-Technical Complex Systems

Maira Callupe✉, Monica Rossi✉, Brendan Sullivan✉, and Sergio Terzi✉

Politecnico di Milano, Via Lambruschini 4B, 20156 Milan, Italy
maira.callupe@polimi.it

**Abstract.** This paper presents the development of a Learning Ecosystem (LE) to cope with the challenges brought by the characteristic complexity and uncertainty of research and innovation projects. These projects are made up of consortium partners who constitute a complex socio-technical system (STT), which works towards the development of cutting-edge technology by creating new and valuable knowledge. Through the creation of a LE grounded in complexity, learning and knowledge management theories, the project consortium aims to enable an efficient knowledge exchange and effective learning processes within the system, which are ultimately conducive to the achievement of the project goals. After a review of the literature about complex STTs and LEs, the development of the LE for a specific project taken as a pilot is discussed in detail, elaborating on its components, knowledge flows, learning processes, learning tools, and the methods to assess its effectiveness.

**Keywords:** Complex socio-technical system · Learning ecosystem · Effective learning · Knowledge gap · Research and innovation project

## 1 Introduction

Complexity in engineering has been studied from different perspectives and different foci due to the absence of a unique concept or an overarching complexity theory, with works discussing its source, typology, scope, concepts, etc. [1–3]. The study of complexity has been associated with challenges arising from uncertainty, iteration, multidisciplinary, and dynamic behavior, which are all characteristic of research and innovation (R&I) projects [4]. In this context, the complexity arises not only from the technology embedded in the subject of the project itself, but also the socio-technical interactions between actors involved in the project, as well as the social and technological pressures from external stakeholders which represent the target of the projects’ output [3]. This degree of complexity is often found in R&I projects financed by the European Union, and calls for solutions that enable the management of complex socio-technical systems (STS) while allowing for effective innovation and supporting organizations in being more adaptive in their response to change and uncertainty [5]. Furthermore, modern views
on STS aim to promote not only innovation, but also knowledge sharing and learning within, as creating these opportunities is conducive to an informed decision-making process along the system [6, 7].

The present work uses complexity theory to describe the organizational context of the R&I project “Ecosystemic knowledge in Standards for Hydrogen Implementation on Passenger Ship” (e-SHyIPS) as a complex STT, and to develop a methodology that aims to address, in particular, the efficient exchange of knowledge within the system. Section 2 discusses the characterization of the e-SHyIPS project as a complex STT, identifying its elements, their interactions, and highlighting the importance of knowledge and learning. Section 3 reviews the literature on LEs, and Sect. 4 builds on the previous sections to develop the “e-SHyIPS LE”, detailing its components, learning tools, and measures of effectiveness. The work finishes with the conclusions and next steps presented in Sect. 5.

## 2 Complex Socio-technical Systems in Research and Innovation Projects: The e-SHyIPS Case

When it comes to R&I projects, adopting a socio-technical view is based on the notion that the actors involved in the project (project consortium members) interact and negotiate a solution that can be considered “satisfactory” in terms of the actors performance and the project outcomes, and that the activities conducted by the actors are not only affected by the technical properties of the project’s subject, but also by the manner in which they act and interact, ultimately influencing their performance and the project outcomes [4]. Considering the external forces also exerting their influence, this perspective is more pragmatic and richer as it can provide a better understanding of the reality of a project, which is of great value for research and practice.

Several succinct definitions of complexity can be found in literature, but, due to the nature of the concept itself, they manage to capture its dimensions only partially. Thus, a number of studies are concerned, instead, with describing its characteristics as a way to grasp the complexity of a certain system [8]. Saurin and Sosa [8] describe a complex STS as having a particular set of characteristics which include: a large number of dynamically interacting elements, wide diversity of elements, unanticipated variability, and resilience.

The e-SHyIPS project is a Horizon Europe project that aims to define a pre- standardization plan for the update of the International Code of Safety for Ship Using Gases or other Low-flashpoint Fuels (IGF code) pertaining to hydrogen-based fuels passenger ships, and a roadmap for the boost of Hydrogen economy in the maritime sector. The scientific and technical contents addressed in the context of the project are divided into 4 categories called “experimental pillars”. The project consortium is made up of 14 partners, each with a specific expertise that determines their role and partic- ipation in the project experimental pillars, which include: vessel designers, regulatory entities, fuel cells R&D centers, ferry and containership companies, computing centers, hydrogen suppliers, port operators, and engineering consultants. Several approaches and scientific research methodologies are used to gather relevant state of the art and to guide experimental activities, such as literature reviews, theoretical studies, surveys and inter- views. The project also envisions the involvement of a wide list of external stakeholders in order to collect baseline information, best practices, expertise, feedback, and more.
These stakeholders include Advisory Board members and consortia of projects working in similar topics or “Cluster Projects”. They are expected to be continuously involved in the project development and to provide constant and rigorous knowledge contributing to the activities of the consortium. Therefore, the interactions between all actors (external stakeholders and consortium partners) involve the flow of knowledge within and outside of the organization as well as the associated learning activities and processes. Table 1 gathers all elements of the organizational context and presents them as evidence to characterize the e-SHyIPS project as a complex STS as per Saurin and Sosa (2013) [8].

<table><caption>**Table 1.** Examples of evidence that characterizes the e-SHyIPS project as a complex STS.</caption><thead><tr><td>Characteristics of complex STS [8]</td><td>Examples of evidence in the e-SHyIPS project</td></tr></thead><tbody><tr><td>1. A large number of dynamically interacting elements</td><td>Number of consortium partners, number of Advisory Board members, number of Cluster Projects</td></tr><tr><td>2. Wide diversity of elements</td><td>Types of knowledge exchange, actors and stakeholders' types of expertise, types of learning processes</td></tr><tr><td>3. Unanticipated variability</td><td>Types of decisions taken by the consortium partners, doubts arising during decision-making, sources of uncertainty</td></tr><tr><td>4. Resilience</td><td>Adaptations of internal procedures on the basis of feedback received from external sources</td></tr></tbody></table>

## 3 Learning Ecosystems

Knowledge sharing and learning within the system are subjects of modern studies about complex STS due to their impact in the decision-making and the overall system output [6]. Furthermore, in recent years the learning environment has been approached as a system, with works highlighting the importance of mapping and understanding all complex relationships arising between the elements within [9]. The high complexity of modern learning setups calls for frameworks that can appropriately represent the dynamics between all involved stakeholders and the impact of external influences [10]. In this context, a LE is defined as consisting of stakeholders incorporating learning processes and learning utilities within specific environmental borders [11]. The individual behavior of stakeholders contributes positively or negatively to the success of the LE, while the relationships and interactions between them are related to the flow of information as well as the transfer and transformation of knowledge [12].
# 4 The e-SHyIPS Learning Ecosystem

The present work adopts the LE methodology to accurately represent the complexity identified in the e-SHyIPS project in order to address the challenges noted in Sect. 2 from a learning point of view, namely, the multiple knowledge flows and learning processes taking place between a variety of stakeholders. While the hydrogen maritime economy is an issue of direct application based off the project, the method described is intended to focus on areas where there is a high degree of uncertainty and multiple interconnected elements that are both known and unknown. The development of the e-SHyIPS LE is guided by 3 main principles: i) giving visibility to processes and outcomes, ii) encouraging the diversity of perspectives in the decision-making, and iii) having an efficient exchange of knowledge. A brief introduction to the e-SHyIPS LE is given in Sect. 4.1, while the theoretical foundation of the LE is laid out in Sect. 4.2. Finally, Sect. 4.3 describes the proposed methods to assess the effectiveness of the LE, this is, to assess the learning effectiveness.

## 4.1 Components

The e-SHyIPS LE (Fig. 1) is created with the involvement of the project consortium and stakeholders from maritime, technological, and hydrogen sectors. In the highly innovative and dynamic context of the project, this methodology will enable the ecosystem to react quickly to the changeability needs of hydrogen and fuel-cell fast developing technologies, as well as to bring together a diverse set of perspectives. In order to achieve the project objectives, the LE continuously monitors, controls, and pulls the knowledge gaps identification and resolution, integrates the results from experimental activities, and promotes the capture of new knowledge. Based on the contents described in Sect. 3, the components of the e-SHyIPS LE are as follows:

* *Learning stakeholders.* The 14 members of the consortium represent the learning stakeholders. Throughout the duration of the project they will engage with each other as well as external actors.

* *Learning contents.* The knowledge directly related to the 4 experimental pillars. This knowledge can be originated from within the ecosystem (internal sources) or from sources beyond the learning environmental borders (external sources).

* *Learning processes.* The activities facilitated through the use of specific tools for the acquisition and creation of knowledge and, as a consequence, the closure of knowledge gaps. There are 4 main learning processes taking place in the e-SHyIPS LE: i) learn by experimenting, ii) learn by searching, iii) learn by imitating, and iv) learn by interacting.

* *Learning environmental borders.* The boundaries of the ecosystem enclosing the learning stakeholders (i.e., the consortium members). The learning processes involving exclusively the learning stakeholders are referred to as endogenous learning, whereas the learning processes involving the learning stakeholders and interactions with external forces are referred to as exogenous learning.

Furthermore, the e-SHyIPS LE is affected by the following external forces:
<figure><img src="image_3.png" /><figcaption>Fig. 1. The e-SHyIPS learning ecosystem</figcaption></figure>

* *The Advisory Board and the Cluster Projects.* Actors with whom the learning stakeholders interact synchronously or asynchronously on a regular basis. They are considered as not fully external actors, since the consortium has a direct line of communication with them; but also not fully internal actors, since they are not part of the learning stakeholders, do not engage in their regular activities, and do not share the same objectives and attitudes.

* *External sources.* The external sources from which knowledge can be acquired. These include academic literature, standards and regulations currently in place, case studies, and the activities carried out by organizations of interest to the consortium (enterprises, similar projects' consortia, etc.) The feedback received from the European Commission about the project performance is also a valuable external source.

## 4.2 Learning Framework

The learning framework lays out the theoretical foundation of the LE as well as the learning tools developed on the basis of said theory. The interactions and relationships between the learning stakeholders are described in terms of knowledge flow and learning processes. The knowledge flow refers to the processes that the learning contents go through as the learning stakeholders make sense of it, while the learning processes refer to the activities that facilitate the flow of knowledge [13]. The developed tools are intended to support the learning stakeholders in the various learning processes. The
main objective of the learning framework is to support the identification and closure of knowledge gaps.

## Knowledge Flow

The knowledge flow refers to the processes that the learning contents go through as the learning stakeholders make sense of it. The processes involved in the flow of knowledge are encapsulated in the concept “Knowledge Management”, which refers to the management of knowledge within an organization “by steering the strategy, structure, culture and systems and the capacities and attitudes of people with regard to their knowledge”, ultimately achieving an organization’s goals by making the factor knowledge productive [14]. Several models of Knowledge Management covering a wide spectrum of perspectives are abundantly described in literature. Therefore, there is some degree of diversity in the knowledge flows described by these models depending on the disciplinary context [15]. Those with relevance to context of the e-SHyIPS LE can be grouped into the following three main stages: i) Knowledge Gap Identification, ii) Knowledge Gap Resolution, and iii) Knowledge Capitalization. Each stage is made up of the phases shown and described in Fig. 2.

<figure><img src="image_4.png" /><figcaption>Fig. 2. Knowledge flow in the e-SHyIPS learning ecosystem</figcaption></figure>
## Learning Processes and Learning Tools

Once a knowledge gap is identified it can be bridged through the acquisition of existing knowledge, often from outside the organization, or the creation of new needed knowledge, often within the organization. In line with the boundaries described in Sect. 4.1, four learning processes, also referred to as “Learn by X”, are put in place in order to acquire or create knowledge, each one with an associated tool and source of knowledge (Table 2) which are described as follows:

<table><caption>**Table 2.** Learning processes with their corresponding tool and source of knowledge</caption><thead><tr><td>Learn by X</td><td>Learning tool</td><td>Source of knowledge</td></tr></thead><tbody><tr><td>Learn by searching</td><td>Document library</td><td>Academic literature, state of practice and standards</td></tr><tr><td>Learn by imitating</td><td>Repository of observed actions</td><td>External actors, Cluster Projects, Advisory Board members</td></tr><tr><td>Learn by interacting</td><td>Repository of interactions</td><td>Consortium partners, Cluster Projects, Advisory Board members</td></tr><tr><td>Learn by experimenting</td><td>A3 Experiment cards</td><td>Experiments</td></tr></tbody></table>

1. *Learn by searching.* Exogenous learning process involving the comparison, integration, and synthetization of information from external sources [16]. To support the acquisition of knowledge from the academic literature, state of practice and standards, a document library has been created as the supporting learning tool. The documents are added by the consortium partners along with information meant to facilitate their consultation: general information, keywords, associated experimental pillars, and associated knowledge gaps.

2. *Learn by imitating.* Learning process involving the acquisition of knowledge through passive or active observation of an act followed by its imitation [17]. The dedicated learning tool is meant to ensure that all relevant observations are recorded in a single repository, storing those corresponding to external actors (exogenous learning) and Cluster Projects and Advisory Board members (exo/endogenous learning). These observations also include information about the associated experimental pillars and knowledge gaps.

3. *Learn by interacting.* Type of learning that relies on the experiences of the consortium partners and their interactions beyond the environmental boundary of the ecosystem [18]. Similarly to learn by imitating, the dedicated learning tool is meant to ensure that all relevant interactions are recorded in a single repository, primarily those with Cluster Projects and the Advisory Board (exo/endogenous learning). Internal interactions are already recorded during meetings. These interactions also include information about the associated experimental pillars and knowledge gaps.
4. *Learn by experimenting.* Learning process that involves the creation of new knowledge from the development of experiments within the consortium. In order to setup the experimental plans, the A3 problem-solving approach is used to document, communicate, and transfer the created knowledge [19]. The dedicated tool is called the “A3 experiment card”, which has been adapted for the project and also includes information about the associated experimental pillars and knowledge gaps.

## 4.3 Assessing the Effectiveness of the Learning Ecosystem

Being able to assess the effectiveness of the overall LE is as important as laying the theoretical foundation and developing the corresponding learning tools. The attributes through which the effectiveness of the ecosystem can be measured are knowledge and learning; however, these are subjective and intangible concepts whose measurement is not simple as it depends on several contextual factors arising from the characteristics of the LE. The measurement of the effectiveness of knowledge management and learning are current topics of research focused not only on the LEs found in the educational system, but also in those in the scope of organizational learning.

In the context of the e-SHyIPS LE, the individual knowledge gap is selected as the unit of analysis to be used in the assessment of its effectiveness. Therefore, the effectiveness of the ecosystem will be determined primarily by the number of knowledge gaps that are successfully closed within a certain time period. Thus, effectiveness is defined as the degree to which something contributes towards the closure of knowledge gaps. Considering this, there are two manners in which the effectiveness of the LE can be measured: i) the performance of the learning tools, and ii) the feedback from the learning stakeholders making use of the learning tools.

### Learning Tools Performance

The performance of the learning tools is measured through their contribution towards the resolution of knowledge gaps. Thus, a set of KPIs both absolute and relative were defined for each of the learning tools. These KPIs will be tracked throughout the development of the experimental pillars. A sample of the KPIs corresponding to the tool “Repository of observed actions” corresponding to the process “Learn by Imitating” is shown in Table 3 below. The KPIs will be measured and updated regularly, with eventual adjustments to better track the performance of the learning tools.

<table><caption>**Table 3.** KPIs to measure the effectiveness of the Learn by Imitating tool</caption><thead><tr><td>KPIs</td><td>Description</td></tr></thead><tbody><tr><td>Absolute</td><td><ul><li>Total number of organizations identified as Cluster Projects</li><li>Number of members belonging to each sector in the Advisory Board</li></ul></td></tr><tr><td>Relative</td><td><ul><li>Number of organizations contacted/Total number of organizations in the cluster</li><li>Number of organizations contacted that contribute towards the closing of knowledge gaps/Total number of organizations in the cluster</li></ul></td></tr></tbody></table>
## Feedback from Learning Stakeholders

The effectiveness of the learning tools will also be measured as they are perceived by the learning stakeholders i.e., measuring the quality of the experience of learning stakeholders as they use the tools for the closing of knowledge gaps. The proposed media to collect the feedback are surveys, questionnaires and/or interviews scheduled every number of months.

* Surveys. The surveys aim to assess the following features:

- Contents. Learning stakeholders' perception of the contents offered and whether it facilitates its expected purpose.

- Ease of use. The difficulty of using the tools by the learning stakeholders in order to fulfill its purpose.

- Availability. The guaranteed access to the tools at any time.

* Interviews. The interviews are meant to have immediate access to the experience of learning stakeholders as they are using the tools. The interviews also have the purpose of enabling the tracking of the status and evolution of knowledge gaps.

# 5 Conclusion and Next Steps

The objective of this work was to capitalize on complexity, learning, and knowledge management theories to develop a methodology to enable an efficient exchange of knowledge. The e-SHyIPS LE, thus, not only supports the project consortium to work towards achieving the objectives of the project, but also the overall objective of R&I projects, such as those funded by the European Union, to promote learning and to advance the scientific and technological community through the creation and sharing of knowledge. This methodology represents a pivotal work to create solid learning environments to guarantee achieving the project outcomes despite the inherent complexity and challenges presented by these type of R&D projects.

As for the next steps, the LE is planned to be deployed in the context of the project and real data is expected to be collected according to Sects. 4.2 (Learning Framework) and 4.3 (Assessment of the LE effectiveness). General findings will be exploited to implement the LE in R&I projects dealing with similar complexity or similar complex STS.

**Acknowledgement.** This project has received funding from the Clean Hydrogen Partnership under grant agreement No 101007226. This Joint Undertaking receives support from the European Union's Horizon 2020 Research and Innovation program, Hydrogen Europe and Hydrogen Europe Research.

## References

1. Elmaraghy, W., Elmaraghy, H., Tomiyama, T., Monostori, L.: Complexity in engineering design and manufacturing. CIRP Ann. Manuf. Technol. 61(2), 793–814 (2012). https://doi.org/10.1016/j.cirp.2012.05.001
2. Sheard, S.A., Mostashari, A.: A complexity typology for systems engineering. In: 20th Annual International Symposium of the International Council on Systems Engineering, INCOSE 2010, vol. 2 (2010). https://doi.org/10.1002/j.2334-5837.2010.tb01115.x

3. Sheard, S.A.: Systems engineering complexity in context. In: 23rd Annual International Symposium of the International Council on Systems Engineering, INCOSE 2013, vol. 2 (2013). https://doi.org/10.1002/j.2334-5837.2013.tb03077.x

4. Hassannezhad, M., Cantamessa, M., Montagna, F., Clarkson, P.J.: Managing sociotechnical complexity in engineering design projects. J. Mech. Des. Trans. ASME **141**(8) (2019). https://doi.org/10.1115/1.4042614

5. Kiridena, S., Sense, A.: Profiling project complexity: insights from complexity science and project management literature. Proj. Manag. J. **47**(6), 56–74 (2016). https://doi.org/10.1177/875697281604700605

6. Walker, G.H., Stanton, N.A., Salmon, P.M., Jenkins, D.P.: A review of sociotechnical systems theory: a classic concept for new command and control paradigms. Theor. Issues Ergon. Sci. **9**(6), 479–499 (2008). https://doi.org/10.1080/14639220701635470

7. Oosthuizen, R., Pretorius, L.: Establishing a methodology to develop complex sociotechnical systems (2013). https://doi.org/10.1109/ICIT.2013.6505890

8. Saurin, T.A., Gonzalez, S.S.: Assessing the compatibility of the management of standardized procedures with the complexity of a sociotechnical system: case study of a control room in an oil refinery. Appl. Ergon. **44**(5), 811–823 (2013). https://doi.org/10.1016/j.apergo.2013.02.003

9. Goodyear, P., Carvalho, L.: The analysis of complex learning environments. In: Rethinking Pedagogy for a Digital Age: Designing for 21st Century Learning (2013)

10. Gütl, C., Chang, V.: Ecosystem-based theoretical models for learning in environments of the 21st century. Int. J. Emerg. Technol. Learn. **3**(1) (2008). https://doi.org/10.3991/ijet.v3i1.742

11. Chang, V., Guetl, C.: E-learning ecosystem (ELES) - a holistic approach for the development of more effective learning environment for small-and-medium sized enterprises (SMEs) (2007). https://doi.org/10.1109/DEST.2007.372010

12. Giannakos, M.N., Krogstie, J., Aalberg, T.: Toward a learning ecosystem to support flipped classroom: a conceptual framework and early results. Lect. Notes Educ. Technol., 9789812878663 (2016).: https://doi.org/10.1007/978-981-287-868-7_12

13. Easterby-Smith, M., Lyles, M.: The Blackwell handbook of organizational learning and knowledge management. Oxford (2003)

14. Beijerse, R.P.U.: Knowledge management in small and medium-sized companies: knowledge management for entrepreneurs. J. Knowl. Manag. **4**(2), 162–179 (2000). https://doi.org/10.1108/13673270010372297

15. Mcadam, R., Mccreedy, S.: A critical review of knowledge management models. Learn. Organ. **6**(3), 91–101 (1999). https://doi.org/10.1108/09696479910270416

16. Kammerer, Y., Brand-Gruwel, S., Jarodzka, H.: The future of learning by searching the web: mobile, social, and multimodal. Front. Learn. Res. **6**(2), 81–91 (2018). https://doi.org/10.14786/flr.v6i2.343

17. Albino, V., Petruzzelli, A.M., Rotolo, D.: Exploring and exploiting through external sources: the effect of learning and technological proximity. Int. J. Innov. Learn. **8**(1), 1–11 (2010). https://doi.org/10.1504/IJIL.2010.034011

18. Bresman, H., Zellmer-Bruhn, M.: The structural context of team learning: effects of organizational and team structure on internal and external learning. Organ. Sci. **24**(4), 1120–1139 (2013). https://doi.org/10.1287/orsc.1120.0783

19. Morgan, J.M., Liker, J.K.: The toyota product development system: integrating people, process and technology. J. Prod. Innov. Manag. **24**(3), 276–278 (2006)