# Orchestration and Situation Awareness in an Assistance System for Assembly Tasks

Wernher Behrendt, Felix Strohmeier

<figure><img src="image_1.png" /></figure>

## To cite this version:

Wernher Behrendt, Felix Strohmeier. Orchestration and Situation Awareness in an Assistance System for Assembly Tasks. 9th International Precision Assembly Seminar (IPAS), Dec 2020, Held virtually, Unknown Region. pp.332-343, 10.1007/978-3-030-72632-4_25. hal-03520407

HAL Id: hal-03520407

https://inria.hal.science/hal-03520407

Submitted on 11 Jan 2022

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Orchestration and Situation Awareness in an Assistance System for Assembly Tasks

Wernher Behrendt, Felix Strohmeier¹

¹ Salzburg Research, Austria

**Abstract.** We report on the design, specification and implementation of a situation awareness module used for assistive systems in manufacturing, in the context of Industry 4.0. A recent survey of research done in Germany and Europe, concerning assistive technology in industry shows a very high potential for “intelligent assistance” by combining smart sensors, networking and AI. While the state of the art concerning actual technology in industrial use points more towards user-friendly, speech-based interaction with personal assistants for information retrieval (typically of in-house documentation), the research presented here addresses an enterprise-level assistance system that is supported by a number of specialized *Assistance Units* that can be customized to the end users’ specifications and that range from tutoring systems to tele-robotics. Key to the approach is situation awareness, which is achieved through a combination of a-priori, task knowledge modelling and dynamic situation assessment on the basis of observation streams coming from sensors, cameras and microphones. The paper describes a working fragment of the industrial task description language and its extensions to cover also the triggering of assistive interventions when the observation modules have sent data that warrants such interventions.

**Keywords:** Assembly tasks, Situation awareness, Assistance systems, Process language.

## 1 The Concept of Assistance in Industry 4.0

One of the tenets of Industrie 4.0 is the digitization of all processes, assets and artefacts in order to be able to virtualize and simulate production and to achieve maximum flexibility and productivity. While much of the innovation effort is aimed at outright automation, it is also obvious that for the foreseeable future, human-to-machine interaction will remain an important element of any production. With an aging workforce and products that are becoming less standardized (lot-size 1) the notion of assistance has been getting significant attention. In an Austrian “lighthouse-project” the idea of “Assistance-Units” is put forward. This paper presents a taxonomy and internal structure of these units and a proposed formal language to specify their desired interaction with humans when an assistance need arises. It should be noted that the proposed language acts as a mediation layer between higher level business process descriptions (e.g., an order to manufacture a product) and machine-specific programming constructs that are
still needed to operate e.g. a welding robot. The purpose of our task description lan-
guage is to orchestrate human workers, robotic assistance as well as informational as-
sistance, in order to keep the factory IT in sync with progress on the shop-floor.

1.1 Motivation and State of the Art

While there is plenty of research work on collaborative robotics, exoskeletons, and
informational assistance through e.g. virtual reality headsets, less work has been done
on bringing these diverse forms of assistance under a common roof. A recent study [1]
also took a broad view that included assistance systems for both services and industry.
There, the authors distinguish three kinds of (cognitive) assistance: (1) Helper systems
providing digital representations of e.g. manuals, teaching videos, repair guides, or
other elements of knowledge management. (2) Adaptive Assistance systems that are
able to take into account some situational context, e.g. through sensors, and that can
then provide relevant information that is adapted to the specific situation. (3) Tutoring
Assistance systems are also adaptive, but address explicitly, the need for learning in the
work context.

The study forecasts a very high market potential for the use of AI in assistive tech-
nology and this is of relevance to the category of “machines and networked systems”
which gives the closest fit with manufacturing. The study also lists a number of German
research projects broadly addressing digital assistance, in the years 2013 – 2017.

Looking at the current state of the art in industrial use of assistive technology, there
is a prevalence of personal digital assistants based on speech recognition engines such
as Apple Siri, Microsoft Cortana, Amazon Alexa, or Google's Assistant with its cloud
speech tool. These are often combined with VR or AR headsets, or with tablets or smart
phones depending on the use case. It should be noted that any user-activated assistance
(e.g. by speech recognition) is *reactive*, i.e. it has no option of pro-actively helping the
user. A pro-active approach requires the system to somehow know where worker and
assistive machinery are, in the overall task. The following example comes from one of
our industrial use cases for which a demonstrator is being built.

**Motivating Example – Assembly.** The use case comes from a manufacturer of elec-
trical equipment for heavy duty power supply units as used for welding at construction
sites. There are detailed documents available describing the assembly process and the
parts involved. The actual assembly process is done by referring to a paper copy of the
assembly manual. There are also CAD drawings available that can be used to derive
the physical dimensions of the main parts. There are a number of important steps in the
assembly process where the worker has to verify e.g. the correct connection of cables,
or where he or she has to hold and insert, a part in a certain way. Furthermore, the
assembly steps are associated with typical timings in order to ascertain a certain
throughput per hour. The purpose of the associated Assistance Unit (we will use the
abbreviation A/U) is to help inexperienced workers learn the process quickly and to
remind experienced workers to double-check the critical steps. It should also be possi-
ble for the worker to ask for assistance at any point, through speech interaction and the
A/U should be able to detect delays in a process step, as well as errors in the order of steps when the order of steps is important for the assembly process.

Further use cases not discussed in this paper address maintenance, repair, re-tooling, working on several machines simultaneously, and physical (robotic) assistance with lifting of heavy parts.

## 1.2 Structure, Usage and Purpose of Assistance Units

The research into assistance units started with a hypothetical structure that was deemed useful to define a methodology for creating and customizing different kinds of assistance units:

<table><caption>**Table 1.** Initial Structure of Assistance Units.</caption><thead><tr><td>Characteristic Feature</td><td>Example</td></tr></thead><tbody><tr><td>Description of the Assistance Unit</td><td>E.g. Assembly-Assistant</td></tr><tr><td>Type of Assistance</td><td>Cognitive, physical or hybrid</td></tr><tr><td>Knowledge Resources</td><td>E.g. data or media files required</td></tr><tr><td>Input format</td><td>E.g. speech interface, user interface</td></tr><tr><td>Input device(s)</td><td>E.g. microphone.</td></tr><tr><td>Output format</td><td>E.g. video explaining the assembly process</td></tr><tr><td>Output device(s)</td><td>E.g. tablet and in-built speaker</td></tr></tbody></table>

Looking at the concept from a technical perspective, however, it is difficult to refine this structure sufficiently, in order to use it as a taxonomic discriminator: the *Description* points at a potentially significant set of user requirements that are not covered in the specification. Likewise, distinguishing between *cognitive* and *physical* assistance is a binary decision that can be rewritten as “content management” (for cognitive needs of the user/worker) or “robotics” (addressing physical needs of the end user/worker). Input format and device, and analogously, output format and device are somewhat dependent entities because speech interfaces require microphones and video output requires a screen of some sort. The “*knowledge resources*” are worth a more detailed investigation with two possible technical approaches: firstly, one could think of the resources as media content that is searchable and through appropriate tagging, can be triggered in certain circumstances. The second approach would be to endow the assistance unit with a detailed model of the task and thus, make the unit capable of reasoning over the state of the task in order to assess progress or any deviations. This latter approach involves Artificial Intelligence techniques for knowledge representation and reasoning.

**Usage as a distinguishing feature?** It is clear that in principle, there may be as many kinds of assistance units as there are kinds of manufacturing tasks. So one approach to
classification could be high level distinctions of usage. Initially, we classified our use cases as being either assembly or maintenance or re-tooling, but in each of the cases, the distinguishing features only show themselves in the content of the knowledge resources or in the knowledge based model of the actual activity, without having a structural manifestation in the assistance unit itself.

**Purpose (of interaction) to the rescue?** The main reason for wanting a distinction between different kinds of assistance units is to have clearly separable methodologies for their development and deployment. Ideally, one should be able to offer pre-structured assistance “skeletons” that can be parameterized and customized by experts in the application domain, rather than requiring ICT specialists for bespoke programming. Having been dissatisfied with the previous attempts at structuring, we then looked at the way in which human and machine (assistance unit) were supposed to interact and this led to a proposed model that distinguishes eight forms of interaction or purpose, for using an assistance unit, with distinctive capabilities specified for each kind of unit.

<table><caption>**Table 2.** Taxonomy of Assistance Units according to Purpose of Interaction</caption><thead><tr><td>Purpose</td><td>Capabilities (A/U = Assistance Unit)</td></tr></thead><tbody><tr><td>Mediator</td><td>A/U enables connection to/with a remote expert</td></tr><tr><td>Tutor</td><td>A/U explains a certain procedure</td></tr><tr><td>Trouble-shooter</td><td>A/U helps to localize a specific problem</td></tr><tr><td>Guard</td><td>A/U observes a situation and reports observations</td></tr><tr><td>Savior</td><td>A/U is called to clear up a known failure condition</td></tr><tr><td>Helper</td><td>A/U takes over physical sub-tasks in collaboration</td></tr><tr><td>Interpreter</td><td>A/U finds situation-relevant information from diverse sources, in an ill-defined problem space</td></tr><tr><td>Avatar (for tele-presence)</td><td>A/U enables an expert to act in a remote location (requires e.g. tele-robotics and situation awareness)</td></tr></tbody></table>

The above taxonomy has the advantage that it requires an increasing set of capabilities going from mediator to avatar. In other words, the distinguishing features allow one to define capabilities on an ordinal scale (Mediator < Tutor < Trouble-shooter < etc.). It is of course questionable whether the *mediator* role qualifies at all, as an assistance unit in its own right, but we may accept that a combination of on-line manuals, plus the ability to call an expert, all packaged in some smart-phone app, is sufficient functionality for a low-hanging fruit w.r.t. assistance units.

## 1.3 Assistance Units as Actors on the Shop-Floor

One of the defining features of any assistance unit is the ability to recognize when it is needed. This presupposes some form of situational awareness which can be obtained either by explicitly being called by the worker or by some sensory input triggering the A/U. Situational awareness requires explicit knowledge about the environment,
knowledge about desirable sequences of actions (e.g. the steps of an assembly task) and
it also requires some planning capability unless we can afford the system to learn by
trial and error – an unlikely option in a competitive production environment.

Formal and semi-formal production planning methods are already being used in
many manufacturing firms. Hagemann [4] gives the example of Event-Process-Chains
for planning assembly lines in the automotive sector, but for assistance, one would need
a more granular level of description. At the level of robotic task execution, there are
examples such as the Canonical Robot Command Language [6] that provides vendor-
independent language primitives to describe robotic tasks.

In the field of production optimization, many firms rely on some variety of measure-
ment-time-method (MTM). There are different variants of MTM offering different
granularity [5], [3], [7], [9] but the main problem is that the method is not well suited
to formally describing purposeful, planned processes with inputs and outputs, but in-
stead, focusses on the description of isolated actions and their time- and effort-related
parameters. The method is also propagated rather exclusively, by a community of con-
sultants belonging to the MTM association which keeps control over the intellectual
property of practicing the art of MTM analysis.

So, while there are clearly potential connecting points with process planning at large,
and with action-based productivity measurement at the more detailed level, one still
requires an orchestration formalism that can distinguish between actors, can name in-
puts and outputs of processes, and can describe the manufacturing process as well as
any assistive measure that one may want to add to the manufacturing process. To
achieve this goal, we used – as a starting point – the agent model of Russell and Norvig
[2] as shown below.

<figure><img src="image_3.png" /><figcaption>Fig. 1. Simple reflex agent according to Russell and Norvig (2012).</figcaption></figure>

The important issue to note is that we distinguish *Environment* and *Agent*, the latter consisting of *sensors* to perceive observations from the environment, *effectors* to manipulate entities in the environment and some form of rationality expressed in a world
model and in a rule system that predetermines what actions the agent will take, under
certain conditions. For a simple reflex agent as shown above, there is a direct relation-
ship between singular observations and according actions. This is definitely not a suf-
ficient model for what we would call “intelligent” behavior, and so we should extend
the basic model, as shown in the next figure.

<figure><img src="image_4.png" /><figcaption>**Fig. 2.** Agent with explicit goals and reasoning, according to Russell and Norvig (2012).</figcaption></figure>

As can be seen, the second model introduces a *memory* for the agent to store a *state* of the perceived world, and a *planning mechanism* to reason about the effects of possible actions. This means that the agent can now make decisions for its actions, on the basis of some utility function included in the planning mechanism. This agent-model gives us now a better handle on the original structural interpretation of the assistance unit: the input devices are a subset of all sensory input to the A/U and the associated formats are the data structures of those observations (also known as “percepts”). The output devices are a subset of all effectors of the A/U and quite clearly, for these to work in an intelligent way, a reasoning engine of some sort is required between the input and the output mechanisms of the A/U – as demonstrated in **Fig. 2**.

## 2 Situational Awareness and Assistance

The next step is to apply the agent-based Assistance Units to situations where timely
interventions are required by the A/Us that are monitoring the shop-floor processes. We
now bring observations, knowledge resources and collaborative activities into play, as
proposed in the following conceptual model: in order for the A/U and the worker to be
able to collaborate they must be in agreement with respect to the process in which they
are involved. They also need to have some mutual understanding what knowledge is
relevant for the process at the current step, and at least the A/U should have a basic
understanding which knowledge can be assumed as given, for the worker. A concrete
assistive intervention would then be to ask the worker whether he/she needs additional
information for this step, or the next step, or whether they need any other form of as-
sistance. **Fig. 3** below illustrates such a situation.

<figure><img src="image_5.png" /><figcaption>**Fig. 3.** Establishing a mutual focus of attention between human worker and assistance unit</figcaption></figure>

The diagram of Fig. 3 shows knowledge items that may come from observations or
from prior knowledge, as triangles. The triangles outside the focus of attention (in red)
could be any input that is recognized by the sensors of the actors, but which is not
having any relevance to the current situation. There could also be red triangles inside
the current situation, but outside the mutual focus of attention – these would then rep-
resent knowledge items that are not yet, or no longer, of immediate relevance. The ar-
row going from the yellow triangle of the A/U to the (green) frame of a triangle in the
mutual focus of attention, indicates that an assistive intervention is occurring, i.e. the
assistance unit is transferring knowledge to the worker, e.g., by reminding him/her to
check the polarity of some electrical connection. The objects named “Resource” refer
to repositories accessible to the workers and/or the A/Us. In the case of the worker, a
printed manual would qualify as a resource, whereas in the case of an A/U we would
require digital assets preferably accessible via a URL.

Note that the diagram represents a snapshot at a particular time step in the manufac-
turing process. The intersection of Assistance Unit, Worker and current overall situa-
tion constitutes the mutual focus of attention. It shows one green triangle representing
relevant knowledge that the worker has and a green frame of a triangle representing a
piece of knowledge that the worker is missing and that the Assistance Unit can bring
into play (yellow triangle with arrow to green frame). Going back to the assembly use
case introduced earlier, this could be the situation where the worker is reminded to
check the polarity of an electrical connection and wants to reassure him-/herself by
asking the assistance unit to display the connection diagram.

3 Process Description as Frame for Situational Awareness

Having a model of the overall manufacturing task is necessary for the A/U to be syn-
chronized with the human worker w.r.t. the manufacturing process in which the assis-
tance is supposed to happen. This may look like paying a high price (i.e. heavy model-
ling) for the sole purpose of getting a reminder to check some quality feature. However,
behind the simple example is a more ambitious objective, namely to develop a process
modelling methodology that can be used not only for simple assistive interventions, but
also for highly synchronized collaborative work, as would be required when a collabo-
rative robot helps a worker to place a truck engine in a chassis frame.

In the use case of the power unit assembly, four sensory inputs are used and related
to the manufacturing process in question: (1) a video camera is analyzing the worker's
movement in order to recognize certain actions, such as picking a screw and tightening
it. (2) a second video stream recognizes artefacts on the work table, such as the side
panels of the power unit. (3) A third video stream comes from a Hololens used by the
worker, detecting the worker's gaze (direction of viewing) and also recognizing those
artefacts that are directly in view of the worker. (4) A microphone picks up utterances
of the worker, e.g. when he/she requests additional information, such as the procedure
for assembling a specific part of the power unit. The detection systems of these four
sensor streams translate the raw data into discrete observations that are then be used by
the situational awareness module. In the next section we specify the modelling language
that is able to merge process knowledge with observational knowledge, in order to ar-
rive at the required situational awareness.

3.1 Pseudo-natural language for process steps, observations and interventions

The following is an example of the orchestration language for collaboration and assis-
tance on the manufacturing shop floor. It has some syntactic sugar to make it look like
a structured natural language, but it has a direct translation into a logic based program-
ming formalism:

<Assembly>
Assembling product [PowerUnit-1] involves steps:
place () material [Base] on equipment [Worktable] with step-range <10-15s>,
place () material [Bottom_cover] on top of [Base] with step-range <10-15s>,
place (from the right), component [Powersupply] on top of [Bottom_cover]
on trigger (show-how) show user (tutorial "Powerunit-1/Powersupply.mp3");
on `trigger` (delay) ask user (need-help-in-task?)
on `trigger` (behaviour) ask user (are-you-sure?)
else do either:
`place` (from the left), `material` [Side_panels left] on side
of assembly then
`place` (from the right), `material` [Side_panels right];
or
`place` (from the right), `material` [Side_panels right] then
`place` (from the left), `material` [Side_panels left];
`place` () `material` [Housing] on top of assembly [PowerUnit-1];
`inform` about [Bottom_cover]: myfirm/PowerUnit-1/Connectors_bottom_cover.jpg;
`inform` about [Powersupply]: ]: myfirm/PowerUnit-1/Powersupply.mp3;
`inform` about [PowerUnit-1]: myfirm/PowerUnit-1/Assembly.mp3;
`assist` on demand [PowerUnit-1]: Lifting/+robot/+Lifty;
`in_panic_do` [PowerUnit-1]: emergency_plan_call_the_docctor.help
</Assembly>

A similar description is conceivable for another task, e.g. the packaging of the power
supply unit after assembly:

<Packaging>
Packing product [PowerUnit-1] involves steps:
Pick () material [cardboard-VPK1] and place it<fnref n="1" /> on equipment
[worktable],
Transform [cardboard-VPK1] through [make-PU1-box] into mate-
rial [PU1-box],
Place (bottom down) product [PowerUnit-1] into [PU1-box],
[...]
</Packaging>

Here, the construct `transform [X] through [P]` allows us to declare a sub-task `make-PU1-box` which needs to be further specified elsewhere.

At this stage, our task description language is not complete yet, but the fragment
above has been formalized and used for the demonstration of the assembly use case. In
the research implementation, we have focused on the formal constructs and it would be
the task of productization, to add a compiler/interpreter that transforms the structured
natural language into the respective formal constructs.

The following table summarizes our current set of constructs. The middle column
shows the language construct, on the right we give examples or comments and on the
left we explain the function of the construct for the purpose of expressing either manu-
facturing task steps or interventions by the assistance unit.

<footnote n="1">¹ We allow simple anaphoric references: „and place it“ refers to the cardboard.</footnote>
<table><caption>**Table 3.** Constructs of the Task Description Language for Assistance Units</caption><thead><tr><th>Meta-language<br>Explanation</th><th>Language Construct</th><th>Examples/<br>Comments</th></tr></thead><tbody><tr><td>Assistance trigger</td><td>trigger<br>(show-how | delay |<br>behavior | panic)</td><td>Show-how is initiated by the worker; delay<br>gets triggered when the time is used up; be-<br>havior is used when a deviation is observed;<br>panic triggers a defined emergency procedure</td></tr><tr><td>A/U Intervention<br>control</td><td>On trigger T &lt;action&gt;<br>... else</td><td>If trigger T is raised then do [...], else do [...]</td></tr><tr><td>object</td><td>product [ID]</td><td>We associate assembly tasks with products,<br>e.g. PowerUnit-1 with some catalogue-ID</td></tr><tr><td>object</td><td>equipment [ID]</td><td>Instrumental objects such as tools, e.g. a<br>screwdriver or a workbench, or an info-screen</td></tr><tr><td>object</td><td>material [ID]</td><td>A part made of homogeneous material, e.g. a<br>metal base of some product.</td></tr><tr><td>object</td><td>component [ID]</td><td>A specific functional part of a product that<br>may itself be a subassembly or product</td></tr><tr><td>object</td><td>assembly [ID]</td><td>A non-functional intermediate state of a prod-<br>uct during the manufacturing process.</td></tr><tr><td>object</td><td>user [ID]</td><td>A human worker involved in the task</td></tr><tr><td>action</td><td>place (+ qualifiers)<br>(more actions will<br>need to be defined)</td><td>e.g. place (from the left) ...<br>e.g. taking a screw from a container</td></tr><tr><td></td><td>(do) show X [to user]</td><td>What the A/U should display to the user</td></tr><tr><td></td><td>(do) ask [user]</td><td>What the A/U should ask the user</td></tr><tr><td></td><td>(do) help [user]</td><td>Refers to physical or reasoning help</td></tr><tr><td>A high-level action<br>pointing to another<br>task specification</td><td>Transform [X]<br>through [P] into &lt;ob-<br>ject&gt; newX</td><td>References a sub-process P, which transforms<br>X into a new shape or form „NewX“ e.g. cre-<br>ating a box out of some sheet of cardboard</td></tr><tr><td>Spatial qualifiers</td><td>on top of</td><td>These are qualifiers that have a local interpre-<br>tation from the worker's point of view.</td></tr><tr><td></td><td>from the right<br>from the left<br>below, from below</td><td>They must be used carefully because they are<br>relative to the worker's assumed position</td></tr><tr><td>Domain-specific<br>terms</td><td>[Dom-Spec-Term]</td><td>e.g. material [Bottom-cover] – the referenced<br>objects must be recognizable for the assis-<br>tance system through some identification<br>mechanism (e.g. RFID, image recognition).</td></tr></tbody></table>

At present, the domain-specific language only uses the action constructs “pick”, “place” and “insert” for assembly tasks. This corresponds to the fact that the observational modules cannot distinguish any further activities, e.g. “gluing” or “screwing” etc. As soon as we extend the scope of manufacturing activities to e.g. machine re-tooling
or to machine maintenance, we will have to extend the vocabulary, e.g. to express that
some machine part may have to be disassembled. However, such distinctions only make
sense if they can be detected by the observational modules. One simple “fallback”-
detection mechanism is of course, to require the worker to confirm that step X has been
done. What should become clear though is, that we already have a sufficient set of
language constructs to combine actors’ task specifications, observations and triggered
interventions, as the minimally required vocabulary.

4 Summary, work in progress and beyond

We have presented a conceptual model and an orchestration language for specifying
tasks on a manufacturing shop floor. The language allows the user to also specify as-
sistive interventions in relation to observations that come as information streams from
different sensory channels. At the time of writing, a software demonstrator for the de-
scribed assembly use case is being implemented by the research partners and will be
validated by the industrial use case partner.

The implementation of the situation awareness module is inspired by the Situation Calculus, a modelling approach for cognitive robotics pioneered in the works of Reiter [14], Levesque, Lesperance et al. [13]. We use the word “inspired” because at present, we do not make full use of e.g. the IndiGolog programming environment developed by that group at the University of Toronto [15]. Instead, we remain with the more straightforward task specification and use the unification algorithm of Prolog to activate the assistance triggers when certain conditions in the observation streams are met. In a future project, we plan to map our task specifications to equivalent actions and preconditions/post conditions as required by the Situation Calculus.

**Acknowledgement.** This work has been co-funded by the Austrian Research Promotion Agency (FFG) and by the Austrian Ministry for Transport, Innovation and Technology (bmvit), through the project “MMAssist II” (FFG No.: 858623) within the research program “Production of the Future”. We are grateful to the anonymous reviewers for pointing out weaknesses and giving advice on related work.

References

1. (WeApWi 2018) Wenke Apt, Michael Schubert und Steffen Wischmann (2018), Digitale Assistenzsysteme - Perspektiven und Herausforderungen für den Einsatz in Industrie und Dienstleistungen, Hrsg.: Institut für Innovation und Technik (iit)in der VDI/VDE Innovation + Technik GmbH, 80 S.

2. Russell S, Norvig P, (2012), Artificial Intelligence – A modern Approach, 3rd ed., Prentice Hall.

3. Hold, Philipp & Ranz, Fabian & Sihn, Wilfried. (2016). Konzeption eines MTM-basierten Bewertungsmodells für digitalen Assistenzbedarf in der cyber-physischen Montage.
4. Hagemann S, (2016), Optimierung des Fertigungsplanungsprozesses im Karosseriebau durch die Entwicklung eines Konzepts zur automatisierten Generierung von Fertigungsanlagen mithilfe eines wissensbasierten Systems, Master-Arbeit, Otto von Guericke Univ. Magdeburg, 202 S.

5. Bokranz, R., Landau, K., (2012). Handbuch Industrial Engineering: Produktivitätsmanagement mit MTM. Schäfer-Peschel, 2012

6. Proctor F. M., Balakirsky S.B., Kootbally Z., Kramer T.R., Schlenoff C.I., Shackleford W.P., (2016), The Canonical Robot Command Language (CRCL), Industrial Robotics. 2016 ; 43(5): 495–502. doi:10.1108/IR-01-2016-0037

7. Kelterborn, David (2016), Erweiterung eines Systems vorbestimmter Zeiten zur Bewertung der körperlichen Belastung in der Produktionslogistik, Dissertation TU München, http://www.fml.mw.tum.de/fml/images/Publikationen/Kelterborn.pdf

8. Mertens & Schlick, Vorlesungsfolien 2016/17, RWTH Aachen, IAW, https://www.iaw-aachen.de/files/iaw/vorlesungen/Winter/2017/IEE/IEE_LE05_WiSe17_Slides.pdf

9. Martin Benter (2018), Analyse von Arbeitsabläufen mit 3D-Kameras, Dissertation, https://tore.tuhh.de/bitstream/11420/1920/1/Dissertation_BenterMartin.pdf

10. André Hengstebeck, Kirsten Weisner, Maike Klöckner, Jochen Deuse, Bernd Kuhlenkötter, Jürgen Roßmann (2016), Formal modelling of manual work processes for the application of industrial service robotics, 48th CIRP Conference on MANUFACTURING SYSTEMS - CIRP CMS 2015, Procedia CIRP 41 (2016) 364 – 369.

11. Bauer, Sebastian (2015), Prozesssprachenbasiertes System zur Ansteuerung digitaler Menüschemmodelle als Teilkomponente einer Software zur Planung und Visualisierung menschlicher Arbeit in der Digitalen Fabrik, Dissertation. https://monarch.qucosa.de/api/qucosa%3A20388/attachment/ATT-0/

12. Oliver Korn (2014), Context-aware Assistive Systems for Augmented Work - A Framework Using Gamification and Projection
http://www.oliver-korn.de/wp-content/uploads/2014/05/Context-Aware-Assistive-Systems-for-Augmented-Work.-A-Framework-Using-Gamification-and-Projection_WEB.pdf

13. Levesque et al., 1997, H. Levesque, R. Reiter, Y. Lespérance, F. Lin, and R. Scherl. GOLOG: A logic programming language for dynamic domains. Journal of Logic Programming, 31:59–84, 1997.

14. Raymond Reiter (2001). Knowledge in Action. Logical Foundations for Specifying and Implementing Dynamical Systems. MIT Press, 2001.

15. De Giacomo, Giuseppe & Lespérance, Yves & Levesque, Hector & Sardina, Sebastian. (2009). IndiGolog: A High-Level Programming Language for Embedded Reasoning Agents. 10.1007/978-0-387-89299-3_2.