Brain-Computer Integration: A Framework for the Design of Brain-Computer Interfaces
from an Integrations Perspective
NATHAN SEMERTZIDISExertion Games Lab, Department of Human-Centered Computing, Monash University, Melbourne, Australia
nathan@exertiongameslab.org

FABIO ZAMBETTA
School of Software Engineering, RMIT University, Fabio.zambetta@rmit.edu.au

FLORIAN ―FLOYD‖ MUELLER
Exertion Games Lab, Department of Human-Centered Computing, Monash University, Melbourne, Australia,
floyd@exertiongameslab.org
Brain-computer interface (BCI) systems hold the potential to foster human flourishing and self-actualization. However, we
believe contemporary BCI system design approaches unnecessarily limit these potentialities as they are approached from a
traditional interaction perspective, producing command-response experiences. This article proposes to go beyond interaction
and toward a paradigm of human-computer integration. The potential of this paradigm is demonstrated through three
prototypes: Inter-Dream, a system that integrates with the brain’s autonomic physiological processes to drive users toward
healthy sleep states; Neo-Noumena, a system that integrates with the user’s affective neurophysiology to augment theinterpersonal communication of emotion; and PsiNet, a system that integrates interpersonal brain activity to amplify humanconnection. Studies of these prototypes demonstrate the benefits of the integration paradigm in realizing the multifaceted
benefits of BCI systems, and this work presents the brain-computer integration framework to help guide designers of future BCI
integrations.
CCS CONCEPTS • Human-centered computing • )nteraction design • )nteraction design theory, concepts andparadigms

Additional Keywords and Phrases: Brain-Computer Interfaces, Human-Computer Integration, EEG

1

INTRODUCTION

The term brain-computer interface refers to technologies that facilitate the direct transfer of
information between brains and computers [121]. With recent developments in technology, BCIs haveemerged as a consumer product with an exponentially growing market size [80,180]. In the last 40
years, BCI has gone from a laboratory novelty to an assistive technology empowering sufferers of
diseases such as paralysis, to what is now becoming a trendy consumer device [145,175,180]. We notethat such consumer BCI devices have often been presented as mind-operated remote controls,
intended for gaming and interacting with digital content through user brain activity [61,88,163]. While
such devices indeed offer novel experiences, they are often just that - novel -, with BCI-driven control
often tediously difficult to learn, slow to respond, and largely inaccurate in reading the user’s
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that
copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first
page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit ispermitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from permissions@acm.org.
© 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.
1073-0516/2023/1-ART1 $15.00
http://dx.doi.org/10.1145/3603621
ACM Trans. Comput.-Hum. Interact.

intentions [37,52,112,146]. Considering these issues, more contemporary BCI companies are beginning
to realize that the strength of the technology is not in specific and intentional control, but rather in thesensing of more ambiguous and experiential brain phenomena, such as states of cognition andconsciousness [1].
In the context of such trends, many have lamented a severe lack of applications [1,25,37,52,112,146].
Furthermore, major industry voices, such as Thomas Reardon - head of Facebook's CTRL-Labs - havestated that BCI is a technological dead-end that will be superseded by electromyography (EMG) - a
technology that extracts and infers information from muscle signals [52]. While this notion should
appear laughable, considering brain activity supersedes EMG by containing within it the informationalsource of muscle activity read by EMG, if the way we design BCI systems does not change, this negative
assessment of the future of BCI technology leading to a dead-end may very well be correct. But why isthis the case? Do we just lack the imagination and creativity necessary to go beyond brain-based
remote controls [9,91,94,96,109,131,142,143,185]? We find this doubtful, especially when reconsidering
the bountiful array of example applications provided to us through science-fiction, such as mind
control, mind uploading, consciousness cloning, dream exploration, instant communication, telepathy,
cognitive enhancement, superintelligence, infinite knowledge, and even immortality [55].
Through this article, we argue that many of the challenges that BCIs face as an emerging technology
does not only concern engineering and technical implementation, but rather, we suggest that it is theabsence of any formally articulated design knowledge to guide the development of BCI systems thatmaybe be contributing to stagnation in BCI development. With the most recent general design
framework for BCI design being published in 2003 [107], the state of the art for conceptualizing BCI
design has been limited to medical models with a dominant interaction design paradigm oftechnologies as tools whose relationships with humans are limited to command and response.
Furthermore, we also acknowledge that BCI technology is still very much in its infancy when
considering other domains of technological progress outside of design knowledge, such as its statusfrom an industrial, business, legal, or regulatory perspective. For the stable maturation of BCI
technology, it is important that developments in these domains are made in hand with application
developments. Nonetheless, the scope of the present manuscript focuses on developing a high-level
understanding of the hereto underexplored experiential affordances of BCI technology, and in turn
guide the design of BCI systems. To drive this development, we employ the emerging paradigm of(uman-Computer Integration'' [32,117,150] to formulate a new framework to describe the design of
BCI systems. It is through this framework that we ultimately argue that the future of BCI is not one ofinteraction between brains and computers, but one of Brain-Computer )ntegration .

In this article, we therefore begin answering the research question: (ow do we design braincomputer integration? )ntegration refers to human-computer integration - an HCI paradigm that
acknowledges that computers can be agential actors, allowing for the conceptualization of humanmachine systems which merge to form one cohesive whole [117]. To answer the question, we followeda research-through-design lead process, including the development of three prototypal systems
iterating over different aspects of brain-computer integration. Designing and studying these systems
allowed for the exploration of brain-computer integration from a range of perspectives. Through the

ACM Trans. Comput.-Hum. Interact.

analytical reflection of each of these perspectives, the qualities that emerged from subsequent
experiences these prototypes afforded were broken down and compared across each iteration,
thereby leading to the creation of the brain-computer integration framework. Ultimately, it is the aimof this work that future designers are inspired to consider and understand how their systems caninteract, and ultimately integrate, with brain activity through the brain-computer integration
framework. Furthermore, it is intended that this work illustrates how the design opportunities
afforded by brain-computer integration can be realized through actionable design strategies fordesigning new systems.
To summarize, this work makes the following contributions:This research contributes to design theory by extending the existing paradigm of human-computer integration
to consider how technology can be integrated with the human brain to participate in, mediate, and modulate its
underlying neurocognitive processes through brain-computer interfaces.
This research presents the brain-computer integration framework. It is the first theoretical conceptualization of
how to design for the integration of neurocognitive processes from humans-to-computers, and humans-tohumans. This framework describes the design space of brain-computer integration systems, providing HCI
researchers with a language to describe integrated BCI systems and their associated user experiences. It isenvisioned this will help progress BCI research through establishing a point of reference to unpack,interoperate and compare between integrated BCI systems and their user experiences.

This research also contributes a set of design strategies for practitioners designing integrated BCI
systems. It is envisioned these design strategies will guide practitioners in designing systems capableof producing their intended user experiences. In the following sections, we first review related work,
followed by an articulation of the three prototypes that informed this framework. We then present thebrain-computer integration framework, including a visualization of the design space, a description of
its composite dimensions, and provide prescriptive strategies to guide HCI researchers and design
partitioners in applying our theoretical contribution. Finally, we discuss future work and limitations,
before concluding this work.

2

RELATED WORKThis section delivers a review of the existing work preceding, informing, and leading up to the present
investigation. We begin by describing the current state of the art in contemporary HCI-based BCI
research. This is followed by a description of existing frameworks, and the observation of theopportunity for new developments in BCI design knowledge through the adoption of a humancomputer integration perspective. Finally, these points are considered to ultimately produce theresearch question.

2.1 Brain-Computer Interface Research in HCIWhile the majority of BCI research has been conducted in the context of neuroscientific, medical, andbiomedical engineering research, BCI devices have also recently begun to be discussed in the context
of HCI research.

ACM Trans. Comput.-Hum. Interact.

2.1.1

Neuroresponsive BCI.

Considering the recent emphasis HCI research has placed on enabling reflection and supporting
meditative practices [89,90,124,158,167], a large majority of BCI research in HCI has explored therepresentation of brain activity through various interactive technologies. These systems are typicallydesigned to offer some form of neurofeedback, in which the system interprets an individual's neuralactivity and provides a representation of their mental state in real-time, which users can observe andlearn to self-monitor or regulate, mostly for mindfulness training.
One example of this is )nner Garden'', where a living world is projected onto a desktop-sized
sandbox using augmented reality [141]. This world is populated in accordance to how frustrated or
how meditative the participant is. Through monitoring any changes to this world, the participant
receives information about their degree of focus in a neurofeedback loop, with the goal of encouraging
mindfulness. Similarly, another example of this is PsychicVR , which pairs BC) with VR to produce animmersive playful experience, allowing users to make changes to a virtual 3D environment when in a
focused state of mind, and thereby encouraging mindfulness [2]. A system that also attempts to enable
the regulation of brain activity is Lucid loop , a neurofeedback system designed to simulate theexperience of lucid dreaming and train participants in achieving lucid dream states [84].
Alternatively, rather than regulating brain activity, another class of neuroresponsive systems
include BC) systems that adjust parameters of a task or application based on the user’s brain activity,
typically to accommodate the user’s concurrent cognitive state. One example of this is the use of EEG in
facilitating collaborative robot industrial applications [19]. Specifically, studies have demonstrated
EEG to be a reliable way to detect potential emergencies in cases where robots work in proximity withhumans, using markers from a worker’s EEG, such as perceived safety, to reactively guide the behaviorof their robotic collaborators [19]. Similarly, works done by Jacob et al. [67,159,179] havedemonstrated fNIRS-based BC) can be used to monitor a user’s cognitive load, allowing the system tothus adjust the difficulty of a task to compensate in the event the user is mentally overtaxed.
Taken together, these studies demonstrate the potential to enrich the human experience andpromote a strong coupling between the processes of the technology and the user's underlying neuralactivity. With this considered, these BCI systems represent an alternative perspective in theapplication of BCI beyond its longstanding conception of a brain-based controller interface. However,neurofeedback is not the only way that HCI research conceptualizes BCI systems outside of a control
interface paradigm, as we discuss below.
2.1.2

Social BCI.

Beyond the use of BCI for neurofeedback, BCI research in HCI has begun to consider social affordances
facilitated by the technology, and the unique user experiences they evoke. One early exploration of thisnotion uses EEG data to attempt to extract an individual’s experience of emotion from their neuralactivity [97]. This information is then used to animate the facial expression of a virtual avatar to matchthe emotional state of the participant. Furthermore, some of these related systems demonstrated theefficacy of BCI technology in augmenting interpersonal connections. For example, Breeze , a wearable
pendant that measures and displays breathing patterns, was found to increase connectedness and

ACM Trans. Comput.-Hum. Interact.

empathy with loved ones, as well as aid the user to control their breathing via BCI-driven
neurofeedback [46].
Several studies have taken the BCI-mediation of interpersonal connections further by involving
interpersonal neural synchrony as part of the system’s functioning. These have mostly been artistic
installations. For example, (ive Mind'' is an installation in which two performers on a stage generate
light pulses and sound in synchrony with the oscillations of their brains. SocioPathways
demonstrates how to apply inter-brain synchrony to game design. Players are represented as dots on ascreen and their dots become closer to other players as they become more synchronous with eachother [122]. NeuralDrum is an inter-brain-synchrony-focused drumming game where the goal of theplayer is to hit objects in time with a musical rhythm [128]. By situating the experience within
extended reality and employing players’ EEG activity, the game expands traditional drumming games
by adding visual and audio distortion as players become more synchronous. Through this mechanism,
the game becomes easier while players are out of sync, and harder as they become more synchronous.
Taken together, these studies demonstrate the potential for BCI to play an influential role ininterpersonal interactions and relationships. Thus, these examples further challenge the longstanding
conception of BCI as a brain-based controller interface, instead demonstrating how these technologies
can be used as novel mediums for communication and empathy. Furthermore, with these prior works
considered, we see that HCI-based BCI research has demonstrated the potential for BCI to be morethan a control interface. Yet despite this, we find that HCI-based BCI research has been currently
conducted on a case-by-case basis, yielding application specific findings while falling short ingenerating overarching frameworks describing the design and user experience of BCI systems as awhole. With this considered, we look to earlier BCI frameworks outside of BCI research to guide theformulation of our own novel framework moving forward.

2.2 Brain-Computer Interface Frameworks
Despite the long history of BCI research, there has been little attempt to formally establish aframework for designing such technologies. The most current framework for BCI design was proposedin, titled the general framework for BC) design [107]. Through their framework, the authors
define a generic BCI system as a system in which a user controls a device through brain activity in an
operating environment, through a series of functional components. These functional components
ultimately represent the steps of processing undergone by BCI-extracted brain information; ultimately
being interpreted by a computer to control a device as intended. The process is described as involving
the user, who consciously modulates their own brain activity in an attempt to control a device (such asa wheelchair). This brain activity is sensed through electrodes, producing a signal that is amplified andthen subjected to a feature extraction process, transforming raw data into values that operationalize
the underlying neural mechanisms as a feature vector . The feature vector is translated into a logicalcontrol signal interpretable by the target device, processes this information and responds with acorresponding output that the user observes as feedback on their performance in controlling thedevice.

ACM Trans. Comput.-Hum. Interact.

The authors justify their method of partitioning processes into functional components, stating that
their choice of boundaries between these components facilitates objective comparisons between
systems. This approach creates a common language that can describe the information exchange
between a user and a device [107]. They further suggest that this breakdown of the BCI process allows
for specific objective measure and study of BCI systems, as well as their individual functional
components, enabling the development of standardized testing of components and benchmarking
control interfaces [107]. Ultimately, through their framework, the authors present a clear, detailed andmodular description of the flow of information from a user's brain to a BCI-controlled device.
However, while the basic steps in BCI signal processing as described by the authors remain relatively
unchanged even in contemporary BCI technologies, there are many crucial aspects of BCI design thatthe framework fails to consider, perhaps due to the framework's antiquity relative to contemporary
progress in BCI systems, being proposed in 2003. Recalling that, until recently, BCI research almost
exclusively focused on medical and assistive technology applications, it is apparent that there are
inherent limitations in the scope and generalizability of this apparently general framework as aresult of the historical context of its creation.
Specifically, the exclusively medical context surrounding the construction of this framework
appears to restrict the application of any possible BCI design to be solely directed to the conscious,
intentional and purposeful control of an external device (i.e., the BCI as a control interface). As a result,the framework fails to fully describe many of the BCI systems from HCI discussed above in section 2.2,
which move beyond mere control by considering how the feedback of system output may recursively
influence the system as a whole, but also by affording implicit interactions, in which the signal is notconsciously or purposely controlled, but rather processes and interpreted without the user’s
intentional input. For example, consider the use of BCI activity by the aforementioned study by Liu etal. [97] to detect emotion and animate an avatar's face in virtual reality, providing a passive channel ofexpression through the system’s implicit interpretation of brain activity, rather than an active control
channel. The framework also fails to consider BCI design intended for more than one user, which is
becoming increasingly important with the rise of systems such as SocioPathways and (ive Mind
discussed above in 2.1.2 [122]. Similarly, while Mason and Birch state how extraneous variables in theoperation environment may confound the desired operation of a BCI system, studies of more recent
BCI systems such as Inner Garden and SocioPathways hint at the possibility that the situational
context of the BCI system may be a powerful design resource for enriching the affordances offered by
the system [122,141].
An additional aspect that we believe to be important to BCI design but is seldom described in Mason
and Birch’s framework is neurofeedback. While the framework acknowledges in passing thepropensity for feedback between the system and the user, the framework disproportionately focuses
on user input and control or, the encoding process , in turn failing to articulate in any sufficientdetail the potential influence the system has on the user. This is clearly an oversight for describing
systems like PsychicVR and LucidLoop , which facilitate cognitive feedback loops [2,84]. Similarly,
the framework makes no reference or allusion to the user experiences produced by BCI systems, nor

ACM Trans. Comput.-Hum. Interact.

design choices that may shape the user experience, thereby drastically limiting the design space ofpossible BCI systems an HCI designer may develop.
Considering these shortcomings of the framework, we acknowledge how well it describes theextraction of information from brain activity into a codified, computer-interpretable format, yet we
also point out how it falls short in explaining the processes involved in the subsequent reception orinterpretation of the resultant output signal. Typically, theories and models which describe the flow of
information between bodies (i.e., information theory in electrical engineering and theencoding/decoding model of communication in semiotics) refer to these processes as encoding anddecoding respectively [40,51,134,181]. Thus, we can say that while the framework does well indescribing encoding processes in BCI design, the framework falls short in describing decoding
processes in BCI design.
While newer BC) design frameworks have emerged since Mason and Birch’s model, they focus
heavily on the technology of BCI (versus the experience) [87], or focus on specific BCI applications
such as medical risk management [47] and games [60], or concentrate heavily on the encoding of thesystem, ignoring peripheral components such as other users, situational contexts, or the feedback
effects afforded by the interaction between the user’s brain interpreting the system’s output, and therecursive resulting input being driven by a BCI-altered brain [47,60].
For example, in contrast to Mason and Birchs’s framework, a more recent attempt to provide ataxonomy of different variations of BC) systems is Kosmyna and Lécuyer’s work A Conceptual Space
for EEG-Based Brain-Computer )nterfaces [87]. In this work, the authors strive to create an abstracted
conceptual space providing a taxonomy of different variations an EEG-based BCI system can possibly
take. The authors take a similar stance in that BCI systems appear to be underutilized and suggest thismay be due to the observation that BCI interaction commands are not self-revealing, meaning that it is
not inherently obvious what kinds of affordances a BCI might have, both from the perspective of theuser and the designer. In response, the authors develop a conceptual space that is composed of nineaxes: interface adaption; decision about execution; command initiative; neural mechanism; input type;
pragmatism; interaction task; multimodality; and representation space. Overall, the authors present aframework that provides a comprehensive breakdown and analysis of the composite components ofbrain-computer interface systems. However, when translating this framework toward integration
systems, especially when considering designing for experiences of the system as an extension of theuser, the framework meets some limitations. Firstly, the framework is relatively focused on thetechnology first, comprehensively detailing the differing ways in which information may be collected,
processed, and mediated at a mechanistic level, as considering how differing contexts and modalities
may influence these mechanisms. While this puts the framework in a strong position to characterize,
compare and design BCI systems, the focus placed on technological design decisions detracts frombuilding an understanding of the phenomenology of the associated system. As a result, the framework
falls short in describing, categorizing, and predicting the possible user experiences afforded by BCI.
Furthermore, while there is opportunity to extend the framework beyond EEG, the framework's
current state is EEG-specific, which limits the framework's ability to anticipate user experiences

ACM Trans. Comput.-Hum. Interact.

afforded by future BC) technologies that may allow for deeper integration between the user and thesystem.
As such, we consider these prior research projects to be good starting points for describing theinformation processing within a BCI system. However, they do not yet provide a full answer to our
focus on how to design BCI systems that integrate with the user, and hence we now look to otherbiofeedback works to further inform our framework describing the design of BCI systems that aim to
go beyond an outdated emphasis on input.

2.3 Learning from Biofeedback Frameworks
While BCI research has seemingly overlooked the importance of the decoding process, the moregeneral investigation of biofeedback systems within HCI has acknowledged the importance ofunderstanding this process. The attention-regulation process [123] appreciates how the design offeedback in terms of its modality , instructional cues'', and judgement-free aesthetics'' isinstrumental in the system’s integration with the user’s cognitive experience specifically facilitating
focused attention for mindfulness meditation in their case). Similarly, in their review of biofeedback
systems for stress, Yu et al. suggest that the presentation of a biofeedback display can influence how
the user interprets the information embedded in its encoding, but also the experience itself, stressingthat the encoding is not only a carrier of information but also a stimulus that can alter physiology[182]. Although these frameworks acknowledge the propensity for an encoding to integrate with therecipient's neurocognitive processes when decoded, these frameworks often conceptualize feedback
as something restricted to traditional screen-based interactions.
Moving forward, Lux et al. [103] proposed an integrative framework for live biofeedback, in which
the authors translate the Shannon-Weaver model of communication toward the description of
biofeedback systems [103]. In doing so, their framework includes an information source; a transmitterthat encodes information; a receiver that decodes the encoded message; and a destination, which
processes the message. Furthermore, they move beyond screen-based interactions, suggesting that a
feedback channel can address more than sight, such as hearing, touch, etc. However, Lux et al. state BCI
systems are beyond the scope of their framework. Additionally, while the authors acknowledge
different channels through which a system can integrate with the physiology of the user, theunderlying mechanisms of how this is completed are not described, nor do the authors investigate
how changes in the code’s expression which they call a feedback channel influences the resulting
experience.

2.4 Need for Paradigmatic Shift
Considering the current state of BCI research and design knowledge, it is arguable that while thefunctions and mechanisms of the encoding processes of BCI systems have been steadily progressing,
little is known about the decoding process. A comprehensive description of the decoding process - thesense-making step in BC) interaction - would entail how system output relates to the user, influences
agency, interacts with its situational context, and also, opens (or does not open) channels of
information exchange between other users or systems. Furthermore, the subjective experience of

ACM Trans. Comput.-Hum. Interact.

these components at play together would ultimately come from the user experience, something that, tothe best of our knowledge, has not been discussed in any of the preceding frameworks.
We argue that the traditional interaction paradigm in which previous frameworks were created has
led to the overemphasis on input and encoding, cursory examinations of decoding, and the extraneous
variables interacting with codes themselves. That is, we argue these frameworks have been built with
an episteme that understands the relationship between human and computer as ontologically distinct
and therefore of command and response. As a result, these frameworks place an emphasis on encoding
human intentions into computer-interpretable commands, with the human individual being the soleactor or agent, and the machine being a predictable static tool. However, we believe that with current
advances in technologies such as artificial intelligence and devices that overlap with the human bodyas wearables or implantables, the validity of a command-response human-tool relationship is
becoming an increasingly antiquated paradigm.
As such, it is being argued that we are beginning to see human-technology relationships in which
the two no longer interact, but rather, integrate. This distinction is likely why previous frameworks
have been unsuccessful in fully describing BCI systems and their design, as the relationship between a
BCI and its user may be better described as integration rather than merely interaction. With thisconsidered, the present work looks to the theory of Human-Computer Integration [5,6,31,43,44,115–
118,126,147,148,150], the growing paradigm which studies and describes these new forms of humantechnology relationships, to form a foundational perspective on which to build a new and more
complete framework of brain-computer interface design.

2.5 Human-Computer Integration)n, a panel at the ACM Conference on (uman Factors in Computing Systems C() titled (umanComputer )ntegration versus Powerful Tools [43] articulated what the authors deemed to be a newparadigm within (C), (uman-Computer )ntegration . The panel proposed a move in technology awayfrom the stimulus-response paradigm we commonly think of when we talk about interaction andtoward a symbiotic partnership between humans and computers, in which both parties areintegrated and must be considered holistically.
Carrying forward the sentiment put forward in this panel, a Dagstuhl symposium was held in 2018,
in which 29 leading experts came together over a five-day workshop to develop and discuss the futureof Human-Computer Integration, or HInt [118]. The discussions during this workshop ultimately
spawned an overarching work titled Next steps in Human-Computer )ntegration [117]. The paper
defined ()nt as a new paradigm with the key property that computers become closely integrated withthe user , which included examples in which "humans and digital technology work together, either
towards a shared goal or towards complementary goals (symbiosis)" and "integration in which
devices extend the experienced human body or in which the human body extends devices (fusion)".
Learning from this work and applying its insights to the development of a new perspective for BCI
design, we deduce two fundamental axioms from which the framework should be built. One is that weshould assume both the human and the system as agents, or as the authors describe, partners , rather
than ontologically distinct entities. The second suggests we should also consider the integration

ACM Trans. Comput.-Hum. Interact.

between humans and BCI as scalable, suggesting that integration can occur beyond one machine agent
and one human agent; as networks or assemblages can include many of each, all integrated with eachother.
Because the previously discussed BCI systems illustrate how technologies can recursively influence,and become influenced by, the user’s physiology specifically their brain , we find that the fusion
aspect of HInt is particularly relevant to BCI design. Moving forward, more recent developments in()nt have built on fusion’s focus on the integration between humans and technology at the level of thehuman body. Specifically, Mueller et al. contribute the framework of designing bodily integration[116]. Through this framework, the authors elucidate how integration systems can be designed for
closed coupling with the user’s physiological form and processes, ultimately enabling users toexperience technology as an extension of their own body, or conversely, experience themselves as anextension of the technology, depending on how the system is designed. Their thesis is that in designingsystems capable of bidirectional agential actuation between the user and the system, they facilitate atight coupling in which the two can be experienced as a unified whole. Furthermore, while thecloseness of hardware to the human body is indeed an element or subset of human-computer
integration, our interest and interpretation of human computer integration in the context of braincomputer integration takes a more functional and experiential approach in its focus than entertaining
the centrality of physical hardware. As such, we are interested in systems that integrate with the bodyin the sense that they bidirectionally interact with the user’s physiology and can be experienced as anextension of the user, which has been described as an important element of human-computer
integration in several publications [31,116,118,147,151]. This allows for BCI systems that have asignificant influence on user neurophysiology, such as many neurofeedback systems, to be considered
as a form of integration.

2.6 Research Opportunity
Through examination of past research, we find that BCI has had a long and varied history. Research
has largely focused on extracting information from brain activity, ultimately guided by the epistemic
foundation that BCIs are control interfaces. Furthermore, there has seldom been effort to articulate aframework for the design of BCI systems, and the examples that do exist are antiquated and only
describe the encoding of brain activity to digitized signals, while failing to elucidate how humans
decode these signals, as well as the phenomenological experience of the decoding process. As such,there exists no formally articulated design knowledge detailing what kinds of BCI systems can be made(beyond control interfaces) and what kinds of experiences their users can expect to have. We, once
again, argue that the etiology of the contemporary lack of BCI design knowledge stems from previous
BCI frameworks being based in a traditional interaction paradigm that considers human-computer
relationships as one of command and response, ultimately limiting BCI to a control interface. As such,
we argue that to progress beyond this conceptual dead end, it is required that a more contemporary
BCI design framework be contextualized in a new paradigm for describing human-computer
relationships. This proposed paradigm is Human-Computer Integration, and as such we name this newframework the Brain-Computer Integration framework.

ACM Trans. Comput.-Hum. Interact.

In learning from the most recent canonic works produced by HInt theorists [116,117], we havemodified the implications for design practice the authors have offered into axioms on which to basethe development of the Brain-Computer Integration framework we undertake throughout this work.
The synthesis of these axioms was undertaken to serve as a guide for our design process andinvestigative exploration away from traditional interaction focused BC)’s and toward integration
focused BC)’s. These axioms were developed through considering the definitions, insights, andstrategies denoted by prior HInt works [31,116–118,147,151] regarding what properties of a system
specifically and uniquely make it an integration system, which were presented as design strategies
and design space dimensions in these previous works. We then distilled these strategies anddimensions into a minimalist set of properties that together allow systems to be defined as
integrated by compiling system properties each publication listed as being unique to integration, andthen coding each of these properties into themes through the process of thematic analysis [161] toproduce a minimal set of properties with no overlap. This process ultimately resulted in four axioms,
being:


Humans and technology in a BCI system must both be considered agents, imbued with agency, existing
on a flat ontology (existing on an equal ontological level). Thus, both human and artificial agents areparts of a BCI system, working together as partners toward a common goal.
Integration in a BCI system must be scalable. Thus, a BCI system can be assumed to contain few or many
human and artificial agents, all integrated with each other.
Agency must be variable between agents within a BCI system. Thus, it is important to understand how
agency is distributed across the agents constituting the system as a whole.
Ownership must be variable between agents within a BCI system. Thus, it is important to understandhow ownership is distributed across the agents constituting the system as a whole.

Equipped with the axioms provided by this new paradigmatic perspective, this work seeks todevelop a more complete framework for BC) design that fully describes not only BC)’s encoding
processes, but also the decoding processes, the system’s interaction with extraneous factors, andultimately, the user experience provided by these elements in concert.

2.7 Research Question
With the above considered, it can be stated that the present work seeks to develop a novel frameworkfor formally articulating the design of brain-computer interfaces from an integration perspective,
rather than an interaction perspective. Doing so will provide the design knowledge necessary for thedevelopment of Brain-Computer Integration and BCI systems that integrate with the brain and its
cognitive processes, rather than functioning as mere control interfaces. Thus, the present work seeks
to answer the research question:
How do we design Brain-Computer Integration?

3

METHODOLOGY

The following section details the methods employed in this work in order to arrive at the BrainComputer Integration framework.

ACM Trans. Comput.-Hum. Interact.

3.1 Research through Design
To understand the design space of integration BCI systems and ultimately formulate the BrainComputer Integration framework, a variety of methods have been adopted from the researchdisciplines this work touches, including psychology, neuroscience, philosophy, design and HCI. At ahigher level, the structuring of the studies and general approach to the completion of this work has
been largely informed by the research paradigms of HCI. The rationale for this is that while it ispossible the research methods conventional to other approaches (such as systems engineering or
psychophysics) might be applicable in the present exploration, the questions this work seeks toanswer specifically focus on the interaction (or integration) between the human subject, and computer
systems, rather than understanding the two entities dichotomously, or in isolation. As HCI can be
defined as a field of study focusing on the design of computer technology and interaction between
humans and computers [63,188], the method of this work aligns foremost with the methodologicalpractices of HCI.
Considering the wide range of methodological approaches available within the field of HCI, thiswork engages with (C) centrally through the approach of research through design RtD , which canbe defined as the adoption of methods and processes from design practice applied toward the inquiryof new knowledge [186]. The strengths of such an approach can be seen in that it is effective insynthesizing many ideas together through processes of composition and integration due to its origin indesign theory [50,186]. As such, these properties have rendered RtD well suited for the formation oftheory in novel and emerging contexts, while also being robust enough to support the later
development of more mature and comprehensive theoretical constructs [50,85]. With these properties
of RtD considered, it is notable that this approach aligns well with the topics central to the present
exploration. This can be initially seen in that brain-computer integration as a theoretical construct is asynthesis of ideas from diverse fields of research, including computer science, information theory,
psychology, and neurocognitive science. Furthermore, this approach was taken considering that RtD
permits researchers to focus on research of the future [187], allowing understanding of braincomputer integration as or before it emerges. This is important when considering that the present
work focuses on a novel path of inquiry emerging from the design synthesis of cutting-edge
technologies.
Considering the latter sentiment, a critique of RtD is that the field is dominated by the sentimentthat being first or designing something new takes precedence in value, recognition, and motivation
over in-depth analysis and critique [187]. As a result, it has been suggested that practitioners of RtD
often squander the potential strengths of RtD in theory formation by instead shifting attention to thedevelopment of the next design prototype. This has been considered and avoided in that the primarycontribution of this work is a completed theoretical framework. Furthermore, the approach of thepresent work is iterative and reflective, necessitating that the process of prototyping becomes thesource of research outcomes. In turn, artefacts become a conduit for transforming the world from its
current state to a preferred state [187]. Such a future-oriented focus consequently leads to anemphasis on the phenomenological experience, motivations, and mechanisms of interaction (or evenintegration), rather than realizing a fully developed system or product.

ACM Trans. Comput.-Hum. Interact.

3.2 Research-in-the-wild
To complement the exploration of the future enabled by RtD, the present work also employs aresearch-in-the-wild R)TW approach to the design of its constituent studies [27]. Research-in-thewild can be described as a research design in which studies take place outside of the lab, often instead
being situated within communities or homes for extended periods of engagement. The strength of suchan approach allows researchers to develop a deep understanding of the impacts and affordances
technologies have on day-to-day life and in the real world [18]. Its proponents argue that setting
studies within home and community life presents a rich context for understanding challenges andpossibilities of the technology of interest, as researchers can examine reactions to everyday activities[10,18,23,27,82]. Furthermore, participants are offered novel opportunities for participation as theyalso act to understand the technologically facilitated interactions between people afforded to them by
the novel technologies deployed in their home or community [23]. Since RITW includes naturalisticsocial interaction in its research design, it benefits this work by providing a rich contextual
environment to understand the experience of integrated consciousness from its necessaryinterpersonal perspective [10,18,23,27].
Furthermore, it could be argued that the first case study, Inter-Dream, breaks from this RITW
approach in that the system was not deployed for participants’ longitudinal personal use, but insteadin an installation space during an allotted time. However, we maintain that Inter-Dream still follows a
RITW approach as it employs a subset of the approach known as performance-led-research-in-thewild [15]. In this approach, the design and presentation of an artefact is led by an artist following
artistic processes. In turn, research findings emerge from reflection on the artefact and theparticipants’ experience. Considering )nter-Dream specifically, the artefact was originally designed by
the artists with the intention of producing an interactive public art installation that explored thespeculative future concept of interpersonally sharing dreams through BCI. Nonetheless, the findings ofthis study prompted the adoption of a more traditional RITW approach in which systems are deployed
to participants for longitudinal use that persisted for the remaining two case studies. Similar examplesof this approach to BCI RITW include the work of Kosmyna [86], who employed a controlled in-thewild research design to develop a framework of BC) interaction based on the behavioral patterns of1563 people who engaged with their BCI system as part of several public demonstrations. The author
motivates their use of this approach, stating that BC)’s rarely leave the lab and that only a small
number of in-the-wild BCI studies exist (citing Guger [59], and Pfurtscheller [132]). The author argued
that researching BCI systems in-the-wild provides the opportunity to unveil different patterns of usage
which may not manifest during lab-based interactions, with the interaction language of BC) being
particularly unique, novel and unlike any other existing interaction modality.
Developing the framework
Through employing the methodological approaches discussed above, we explored the design spaceof brain-computer integration through the design, development, deployment, and evaluation of threenovel brain-computer interface prototypes. Each prototype served as a case study in which we could
learn about the user experiences afforded by different BCI designs in different application domains,

ACM Trans. Comput.-Hum. Interact.

with the findings of each preceding case study guiding the direction of the next. Each prototype was
evaluated through a mixed methods study design, taking psychometric, physiological, and qualitativeinterview data from participants in order to build a holistic understanding of the mechanisms andphenomena which comprise the design space of brain-computer integration. The insights gained fromthe evaluation of all three prototypes were then ultimately considered together to synthesize thebrain-computer integration framework. This was completed through a process in which the themes
yielded from the study of each prototype via thematic analysis of user interviews were compiled andfurther thematically analyzed as a single corpus as a qualitative meta-analysis [169]. Furthermore, wenote that the analysis of findings across a set of prototypes to arrive at design knowledge is aconventional method in HCI, typically referred to as annotated portfolio research [49]. As such, ourapproach can be seen as a qualitative meta-analysis using our annotated portfolio showcasing threebrain-computer integration prototypes.

4

DESIGN PROTOTYPESThe following section details the three design prototypes which were developed as case studies to
explore and study the design space of brain-computer integration. In addition to their descriptions
below, each prototype has been detailed and evaluated individually in their own separate publications[42,149,152,153]. These prototypes will also be used for the remainder of the articles as concrete
examples to ground the unpacking and articulation of the brain-computer integration design space
and are summarized here for the reader’s benefit.

4.1 Inter-Dream

Figure 1. Inter-Dream, a system that integrates with the brain’s autonomic physiological processes to drive users toward healthy
sleep states.

ACM Trans. Comput.-Hum. Interact.

The first case study aimed to answer the sub-research question: (ow do we design integrated braincomputer interfaces for regulating brain activity? . To do this, this case study explored how BC)s canregulate brain activity by studying the system )nter-Dream figure 1), a multisensory, neurofeedbackdriven, interactive art installation in which participants rest in a haptic bed whilst their brain activity
is fed back to them in virtual reality. In a study of Inter-Dream [152], twelve participants individually
rested, augmented by Inter-Dream. In evaluating the system and its associated user experience, amixed method research design was employed that utilized a combination of physiological recording
through EEG, self-report psychometric scales to measure pre-sleep affect, arousal, and emotion, andthematic analysis of participant interviews recounting their experience with the system. Results
demonstrated: statistically significant decreases in pre-sleep cognitive arousal and negative emotion,and negative affect. EEG readings were also indicative of restorative restfulness and cognitive stillness,while interview responses described experiences of mindfulness and playful self-exploration. These
results lead to forming the foundation of the brain-computer integration framework. Namely, theinsights gained from the exploration of this first case study were that feedback and agency are two
critically influential factors of a BCI system when considering the user experience.

4.2 Neo-Noumena

Figure 2. Neo-Noumena, a system that integrates with the user’s affective neurophysiology to augment the interpersonalcommunication of emotion

The second case study aimed to answer the sub-research question: (ow do we design integrated
brain-computer interfaces for communicating brain activity? To answer this question, this case study
explored how BC)s can communicate brain activity in this case emotion by studying the system NeoNoumena figure 2). Neo-Noumena is a communicative neuro-responsive system that uses brain-

ACM Trans. Comput.-Hum. Interact.

computer interfacing and machine learning to read one’s emotional states and dynamically represent
them to the user and others in mixed-reality through head-mounted displays. In the study [153], five
participant pairs were given Neo-Noumena for three days, using the system freely. In evaluating thesystem and its associated user experience, a mixed-methods research design was employed thatutilized a combination of self-report psychometric scales to measure the system’s influence on changeof participants’ emotional competence before and after system use, and thematic analysis ofparticipant interviews recounting their experience with the system. Measures of emotional
competence demonstrated a statistically significant increase in participants’ ability to interpersonallyregulate emotions. Furthermore, participant interviews revealed themes regarding spatiotemporal
actualization , objective representation , and preternatural transmission . Through Neo-Noumena,
the framework was extended through the realization that brain activity could not merely beconceptualized as feedback but rather as abstract information. Thus, BC) processes can be likened to
encoding-decoding processes through evoking Shannon )nformation Theory and Verbeek’s postphenomenological framework of human-technology mediation [144,176–178]. Furthermore, thefindings of Neo-Noumena suggested that BCI-related agency has a variable distribution between theagents and actors participating in the flows of information mediated by the system, informing theframework with the knowledge that agency was not only something possessed by users’ brains, butalso by other agents acting on or within the system, such as the environment.

4.3 PsiNet

Figure 3. PsiNet, a system that integrates interpersonal brain activity through brain-to-brain interfacing to amplify humanconnection.

The third case study aimed to answer the sub-research question: (ow do we design integrated braincomputer interfaces for synchronizing brain activity interpersonally? . This case study explored theinterpersonal integration of consciousness through brain-to-brain integration of participants via thesystem PsiNet figure 3). PsiNet is a networked wearable brain-to-brain system designed to amplify
inter-brain synchrony across its users by sensing brain activity through electroencephalography (EEG)and by modulating brain activity through transcranial electrical stimulation (tES). The system

ACM Trans. Comput.-Hum. Interact.

classifies the dominant cognitive activity of each user in a group, and then, based on this information,
attempts to synchronize the dominant group’s cognitive activity through strategically selecting uniquetES stimulation protocols for each group member, selected by a reinforcement learning agent running
remotely on a server. Regarding physiological measures, group inter-brain synchrony was assessed
through measure of the circular correlation coefficient taken from the group’s concurrent EEG
recordings, with sample comparisons of group EEG circular correlation coefficient before and after
synchronizing brain stimulation. We also employed thematic analysis of participant interviews. The
outcomes of this in-the-wild study [42,149] suggested that brain-to-brain interfaces are feasible forsupporting interpersonal connections. The analysis of EEG data revealed a statistically significant
increase in inter-brain synchrony, and interviews revealed three themes that describe a user
experience, inclusive of hyper-awareness ; relational interaction ; and the dissolution of self . The
findings of this case study extend the framework through the realization that brain signals can bereceived without the need for user interpretation, permitting the user to experience them as if they
were generated by their own body. This finding, distinguished from the findings of the previous casestudies regarding how information is presented, ultimately lead to the formulation of the framework’s
axis neural congruence . Furthermore, the case study demonstrated that sense of agency and sense ofownership are things that can be distributed between brains, which when considered with the resultsof the previous case studies, synthesized into the distribution of agency axis of the brain-computer
integration framework.

5

THE BRAIN-COMPUTER INTEGRATION FRAMEWORKThis section introduces the brain-computer integration framework (figure 4). The framework depicts
the design space of Brain-Computer Integration as a two-dimensional cartesian plane. This framework
is a synthesis of the knowledge gained through reflecting on the design of the three prototypes, inconjunction with their results. The emergent framework describes the design space of brain-computer
integration systems, as well as prescribes strategies for designing brain-computer integration systems
in order to reach an intended user experience. It is intended this framework will ultimately help
designers and practitioners navigate this design space to generate the desired user experiences when
designing future integrated BC)’s. This was inspired by previous works in (C) that have used two
dimensions to describe design spaces, prescribing names to each quadrant which resultantly
represent unique types of user experiences [3,21,22,113–117].

ACM Trans. Comput.-Hum. Interact.

Figure 4. The brain-computer integration framework.

5.1 The Framework Axes
The first dimension concerns neural congruence , spanning from semiotic to engrammic. The second
dimension concerns the distribution of agency , spanning from an egocentric distribution to an
allocentric distribution. The following section defines, describes, and differentiates these dimensions
and the quadrants that result from their interaction.
5.1.1

Neural Congruence.

The first dimension of the design space is neural congruence . This dimension is concerned with theextent of congruence between the source experience the BCI is encoding from neurocognitive activity,
and the resulting user experience of the recipient when decoding BCI data. For example, if a user feelssad, and the underlying neurocognitive processes behind this feeling are encoded and represented ona screen as a sad face, what is the similarity between the feeling that generated that sad face, and the
feeling one would get when looking at it? That extent of similarity, or neural congruence, is the focus of
this axis.
Systems that exhibit a low extent of congruence generally encode brain data via semiosis, meaningthat they translate the source experience into abstracted signs, symbols, or representative metaphors

ACM Trans. Comput.-Hum. Interact.

[39,130]. Recipients of such signals usually need to decode the signal by engaging in the more active
cognitive processes of perceiving and schematizing these symbols in order to extract meaning. This
process ultimately renders the user responsible for sense-making, as they are required to consciously
act as a decoder.
Conversely, we find that systems that exhibit a high extent of neural congruence generally encode
brain data as an engram , meaning that they relay the source experience through stimulating neuralactivity in the recipient which is congruent with the neural activity that was previously encoded [75].
Recipients of signals with a high extent of neural congruence decode these signals more passively, as
their brain is entrained to produce a distribution of neural activity similar to that underlying thesource experience. This process ultimately distances the user from being involved in sense-making, as
information is instead decoded unconsciously by the user’s brain as the system’s output merges withthe user’s neural activity.

The dimension of neural congruence speaks to postphenomenological conceptualizations of humancomputer relations. Specifically, we refer to Verbeek’s theory of technological mediation [176] thatwas formulated to analyze how different technologies can mediate the relations between users and theexternal world. )n applying Verbeek’s theory of mediation to the context of a brain-computer
integration framework, our work supports this previous work by demonstrating how different forms
of Verbeek’s human-technology relations are manifested by systems at different points of the neuralcongruence spectrum. Furthermore, our work extends this work by describing how Verbeek’s humantechnology relations are mutated into modified variants of the theory's original relations due tounique properties of BCI systems, as we will discuss in the following paragraphs.
Verbeek uses the term hermeneutic relations [144,176] to refer to human-technology relations inwhich a system is used via the act of perceiving and interpreting a system’s semiotic output. Throughthis process, the user experience is a transformed encounter of what is being represented through thedirect experience and interpretation of the technology itself. An example of this in the context of BCI
would be a representation of brain activity through a graphical visualization. In this example, the user
experiences symbolically translated access to the cognitive processes of the brain being encoded. This
resulting experience can be considered to have a low extent of neural congruence, as the human
decoder is perceiving and actively interpreting the semiotics of the technology itself, rather than thesource experience it is translating. Thus, it can be said that BCIs transmit semiotic signals throughhermeneutic relations. However, due to the nature of BCI, the subject that is being hermeneuticallyrelated as semiotic information via technology is itself the subject (i.e., a user) which interprets it.
Specifically, neurocognitive activity is what is both being represented via hermeneutic relation, butalso, what is being used to interpret the hermeneutic relation. This process creates a hermeneutic
feedback of semiotic brain data that ultimately puts the user’s conscious cognitive processes at thecenter of sense-making in the experience, both in terms of encoding and decoding information.
Alternatively, embodiment relations [144,176] are human-technology relations that transform auser’s behavior and perception of the world. )n the context of BC), embodiment relations facilitate

ACM Trans. Comput.-Hum. Interact.

experiences in which the user's neurocognitive processes are mediated through the system, rather
than being represented by the system. That is, the user's experience of reality is modulated throughthe device, with the device in some ways taken into the user’s bodily awareness. An example of this is asystem that modulates the user's brain to produce an altered state of consciousness, like how PsiNet
was able to modulate users’ brains to have them enter a state of focus. This thereby changes the way
the user perceives the world. Such systems are able to facilitate a high extent of neural congruence as
they are able to shift a decoder’s experience of the world to resemble that of the encoder. (owever,while Verbeek describes embodied relations as relations in which the human uses a system as amediational lens through which to see the world, embodiment relations in the context of BCI are
unique in that the mediational lens through which the human interacts with the world is their own
brain. That is, while the human sees the world through technology in typical embodiment relations,the human sees the world through their technologically altered brain in BCI embodiment relations.
Here, humans are not perceiving through the technology, but rather the technology is part of theperception process itself. Through this insight it can thus be seen that in the context of BCI
embodiment relations, the separation of human and technology becomes incredibly difficult,suggesting that embodiment relations align closer to what Rosenberger and Verbeek describes as
fusion or cyborg relations [144].

Fusion relations describe embodiment relations taken to a deeper form in which it is no longer
enough to say that the user experiences through the device, as no clear distinction can be madebetween the human and nonhuman elements in these relations, referring to neural-implants for deepbrain and cochlear stimulation as examples of technologies enabling fusion relations [144]. The
experiences afforded by fusion relations can be considered to possess a higher extent of neuralcongruence than embodied relations, as they enable the activation of specific neural distributions
necessary to reproduce the originally encoded source experience in the decoder. That is, the signal
takes the form of an engram. With this considered, such systems find themselves at the highest end of
the neural congruence spectrum.
To further unpack the dimension of neural congruence, we also consider Zander and Kothe’s
distinction between active, reactive and passive BCI systems [183]. The authors first define active BCI
as an interaction paradigm in which the user must directly and consciously produce an input signal
with their brain with intention to control the system, independent of external events. Conversely, theydescribe reactive BCI systems as those whose input signals are consciously controlled by the user,
derived from the brain’s response to external stimulation. Finally, the authors describe passive BC)
systems as those in which system input is derived from brain activity that arises arbitrarily without
intentional purpose or voluntary control (i.e., autonomic neurophysiological and cognitive processes),
which the authors state enrich human-machine interactions with implicit information on user states[172]. This paradigm thus allows for implicit interactions where input is not actively chosen by theuser but rather, where the system infers information about the user’s physiological, psychological, or
contextual state to consequentially guide the heuristics of a system to ultimately produce a desireduser experience based on those parameters [172].

ACM Trans. Comput.-Hum. Interact.

With these distinctions considered, we argue that the prototype systems that were investigated in
the synthesis of this framework are all passive BCI systems, and thus the brain-computer integration
can only be said to describe passive brain-computer integration systems with confidence. Nonetheless,
we also believe that the framework could be successfully translated or generalized toward active orreactive BCI systems, with the active/reactive/passive distinctions possibly serving as additional
dimensions for the framework, which can be further developed through future research. For instance,
it is likely the distinction between active/reactive/and passive BCI is not dichotomous but rather
continuous. For example, the prototypes that were of lower neural congruence tended to facilitate selfregulation through a form of neurofeedback. Typically, in these instances, while the system began as
passive , they would become reactive as the output generated by the user’s brain was fed back tothem, thereby influencing the system’s input signal. This was oftentimes intentional on the part of theuser, as they consciously attempted to change their own physiological, cognitive, or affective state toinfluence the system’s functioning. This in turn initiated what has previously been described as anaffective loop experience where in a user interacts with a system intentionally and physiologically, towhich the system responds through affective expression (e.g., through hermeneutic relations of using
semiotic encodings of affect). This is thereby interpreted by the user, which in turn makes them
respond, and this process iterates ad infinitum. Thus, we can see there is some continuous spectrum
between passive and reactive BCI systems, with temporality, context, and user intention being factors
deciding whether the system is more passive or reactive. Furthermore, it is likely that in thishypothetical spectrum, reactive systems exist in the center, with active and passive systems existing ateither extreme.
5.1.2

Distribution of Agency.

The second dimension of the design space is Distribution of Agency . This dimension is concernedwith causal influence distribution amongst agents participating in the system. This dimension
describes how the brains of users of the system control its processes, how equally that control is
distributed amongst users, and how much other causal factors, such as situational context, influence
the user experience. It illustrates this by placing egocentric BCI systems, in which agency is centralized
in a single actor, on one side of the dimension, and allocentric systems, systems in which agency isdecentralized amongst actors, at the opposite end. This dimension considers interactions between
causal agents within the system holistically, imagining them as a network which can be described as awhole. For example, imagine a user is part of a BCI experience in which the neurocognitive activity ofthemselves and a group of other users are influencing the flocking behavior of a swarm of drones. In
analyzing this experience, this axis would pose the questions how equally is control distributed
across users’ brains? and how much does situational context influence the experience? , e.g., nearby
drafts and airflow, or the social norms around using drones (such as air traffic laws). Furthermore, theformation of this network of users, technologies, environmental conditions, and abstracted sociosemiotic factors can then be taken together to be described as a singular agent in itself throughconceptualizing it as a network.
While it may be easy to quickly assume this axis refers to how social a system is, this is notentirely correct. The distribution of agency within a BCI system is not merely a measure of how many

ACM Trans. Comput.-Hum. Interact.

users are in the system, but rather a description of the distribution of the causal agents acting within
the system. For example, a neurofeedback system in which a single user’s brain drives the experience(i.e., the system completely stops if the user removes the BCI) is highly egocentric, even if other people
were involved in this experience. Furthermore, this axis does not just concern how many people are
wearing BC)s. For example, a system in which a group of users’ brains are stimulated in accordancewith the activity of a single user’s brain would be considered highly egocentric.

The dimension of distribution of agency speaks to Latour’s Actor-Network Theory ANT [93],which seeks to define and describe the relational ties between human and non-human actors bydescribing them as nodes within a network, with each node being called an actant . This network isplaced on a flat ontology , meaning that all actants within the network can equally be assumed tohave implicit value or agency. This is the case regardless of what the actant is, be it human or not, andas such it treats humans and technology equally in terms of possessing agency and value within the
network, thus favoring neither social nor technological determinism. Taken together, theamalgamation of actants within a given network can be conceptualized as an assemblage , the sum
total of individual actants forming the whole. With these points considered, we can thus state that ANT
explains how material–semiotic networks (networks of physical artefacts and living organisms(material) and transmissible information (semiotic)) come together to act as a whole.
For example, consider the network of PsiNet. The actants in this network include not just the users,
but also the algorithms driving the system, the hardware mediating the exchange of information
between the system and its users, and the contextual factors present around the users. Together, theseactants connect to create a joint agency from which a brain of brains emerges, imbued with its own
novel ontological experience. Furthermore, in describing the actants within this brain of brains as
networked nodes, its composition can be further described through node centrality , which will
ultimately allow us to determine the distribution of agency of this brain assemblage [17]. Nodecentrality [17] is a descriptor of the importance of a given node in a network (i.e., how central it is tothe network). This importance can be established in various ways, such as by considering the numberof connections to a given actant, or also the number of important actants connected to a given actant.
Important nodes are integral to the identity of the network and have the most agency over the flow of
information throughout the network; thus, dictating its ontology. Highly centralized networks would
tend to have a single or small number of important nodes which boast a disproportionately large
number of connections relative to other nodes. Conversely, the connectivity of nodes in decentralized
networks would tend to be homogenous, with many nodes sharing a similar number of connections
resulting in no nodes being of particular importance.
Returning to BC)’s, we find that human-system assemblages with an egocentric distribution of
agency can be described as centralized networks. In such assemblages, a small number of importantnodes, or actants, exist, most often a single user and the interface itself, with all other actants (e.g.,observers, other participants, etc.) participating in the network through connections to these actants(e.g., rather than to each other). As such, the experience is highly contingent on these specific actants
and their actions, giving them a disproportionate amount of agency over the experience. In turn, other

ACM Trans. Comput.-Hum. Interact.

factors such as context and situatedness present little influence over the system. Consequently, theremoval of said important highly connected central actants would ultimately destroy the assemblage,
in turn ending the experience.
Conversely, we find that human-system assemblages with an allocentric distribution of agency canbe described as decentralized networks. )n such assemblages, there are no important actants but
rather all actants have homogenous degrees of connectivity within the assemblage, resultantly
supplying similar amounts of influence over the network. As a result, this creates a joint agency inwhich agency is not centralized within a given actant but rather in the gestalt of the assemblage. As
such, the system is not contingent on a single actant. This comes with the benefit of rendering such
systems scalable, specifically in that actants can be added and removed from the assemblage (e.g.,more and more users can join) without disrupting the experience or needing to re-engineer thesystem. Furthermore, such distributions place a larger emphasis on situational factors, as each actant
is not more or less connected to their semiotic (informational) or material (physical) space.
Additionally, it is likely in such networks that BCI interaction paradigms are bi-directional, able to both
read and write neural information [66].
With the assumption that in a BCI system, important nodes will tend to be brains considering they
are the system’s subject of interest, a system’s distribution of agency can simply be summarized in thenotion of whether there is a central brain or not , with this outcome being resultant on whether thedesigner configures flows of agency within the system to be either allocentric or egocentric.
In addition to the shape and connectivity of the network, an additional important factor in the
distribution of agency is the directness of control between actants within the network (i.e., in what
direction causal influence flows between nodes). This can be further unpacked with the help of
Benford et al.’s concept of contesting control [16]. Specifically, the authors discuss how in embodied
interactive experiences, control is something that is contested between the human and the machine,with the user battling for direct control of the interface. The antagonist in this struggle for control
could even sometimes be the user’s own body, as the authors describe autonomic physiologicalprocesses that serve as the input signal to the system (e.g., cognitive state) are often the subject toregulation by the user in order from them to gain control over the interaction. The authors discuss
how this contest of control can be further broken down into three dimensions: self-awareness ofcontrol; extent of awareness of control; and looseness of control. Surrender of control describes the
feeling of battling for or losing control, which can be experienced to varying extents, from feeling as
though the user is in control of the system and their body is part of an interactive loop; to the usercompletely relinquishing control of the system and their body to the processes of the loop. Selfawareness of control describes to what extent the user is aware of how their input is influencing thesystem, with experiences ranging from a complete awareness of how the user is influencing thesystem, to complete unawareness as to how they are influencing the system. Finally, the dimension oflooseness of control is concerned with the reliability of the control signal from the human to thesystem, and the predictability of the system’s response, ranging from tight to loose control. Whentranslating this framework to help unpack our dimension of distribution of agency, it could be said

ACM Trans. Comput.-Hum. Interact.

that the relationship between each actant in the network (or each edge joining to nodes in the network
graph) is characterized by varying degrees of surrender, self-awareness, and looseness of control.
Furthermore, as distribution of agency is ontologically flat, and agnostic toward whether a node oractant is a human or non-human agent, control may also be contested between humans, extending thescope of Benford et al.’s original envisioning of their framework toward human-to-human contests ofcontrol [16].

5.2 Introducing the Quadrants
With the axes taken together, four quadrants emerge, each representing unique ways BCI systems that
integrate with the user’s neurophysiological processes can be experienced based on design decisions
relating to neural congruence and distribution of agency. The design space helps designers to identify
the quadrant for which they are designing, further guiding the design process towards attaining thedesired user experiences. The following section describes the opportunities and challenges thatdesigners may face when designing for each user experience which we encountered through theevaluation of our prototypes, summarized below in table 1. We also note that there may me more
opportunities and challenges that have yet been uncovered, but nonetheless present the following as astarting point.
Table 1. Opportunities and challenges for each quadrant of the design space.
Quadrant
Psychonaut
Swarm
Hivemind

Superintelligence

5.2.1

Opportunity
Ability to actively explore, understand, andregulate one’s own mind
Ability to actively contribute an individual
subject to a gestalt object
Ability to passively enhance interpersonalconnections
Ability to passively enhance or modulate
neurocognitive processes

Challenge
Providing a sufficiently deep level of exploration
to keep users engaged
Finding the optimal tradeoff between simplicity
and complexity of symbols
Managing the complex input/output
interactions that arise from a highly connected
network
Dealing with the loss of habituated increased
mental capacity

Lower Left: Psychonaut.

In the bottom left quadrant sit systems that are characterized by a low extent of neural congruence
and an egocentric distribution of agency. We call this user experience psychonaut greek for sailorof the mind in reference to psychonautics : the method of inducing altered states of consciousness
through the use of meditation, psychoactive substances or biofeedback, to explore the self andconsciousness [20]. We find that through employing systems in this quadrant, users generally become
psychonauts as the system facilitates the exploration of their own consciousness in the form of
semiotic sensory feedback.
The majority of contemporary BCI systems find themselves in this quadrant, namely those labeledneurofeedback systems. An example of such a system would be the first prototype, Inter-Dream.
Psychonautic systems often facilitate solitary experiences in which the user’s brain is a centralized
focal point of agency in the human-computer assemblage, although other people can be included in theexperience, e.g., see the moment [140], a cinematic experience in which an audience watches a film

ACM Trans. Comput.-Hum. Interact.

where cuts are decided by the brain activity of a single audience member. Through these systems, theuser’s neurocognitive activity is interpreted by the BCI and fed back to the user symbolically, typicallyin the form of sensory metaphors that represent cognitive processes or affective states.
For example, Inter-Dream provides the user with visual feedback in the form of a spherical
distribution of motes of light, which change in color and movement in response to the band powers of
the user’s power spectral density. Through this mechanism, unique visual displays are dynamicallygenerated based on the concurrent cognitive processes of the user, communicating this back to themin VR through a kaleidoscopic display of movement and light. Similarly, in Lucid loop [84], lucid
dreaming skills are trained through the visual metaphor of becoming lucid . Specifically, users begin
in a VR environment with blurry visuals. These visuals become clearer as the user becomes morelucid as indicated by their EEG, ultimately training the user to move their mind into a lucid statethrough the system’s feedback.

In psychonaut-type experiences, users learn to connect the presented metaphorical symbols totheir underlying meaning as they reflect on how they feel while the system dynamically represents
their introspective journey. In doing so, an emergent lexicon is formed that users can then use to makesense of future thoughts (e.g., some users of Inter-Dream noticed they could produce specific colors bytrying to move toward specific states of mind [152]). Through this affordance, users are enabled to
explore their feelings in novel ways, as the system provides the ability to ask, how do ) feel about thisthought? , or how do ) feel now? , with the question being met with an informational response in theform of neurofeedback. Again, this was exemplified in the study of Inter-Dream, with one participant
describing how the system encouraged introspection, jumping to different thoughts more than usual
because it made me a bit more excited about those thoughts […] I was more active in them and engaged
with them more quickly and explaining how I was thinking about my math assignment, and then the
introspective nature changed my thoughts on the math assignment, why do I feel the way I do about thatassignment? And they were generally more positive .
Through this process, users are ultimately provided a platform through which they can explore andlearn about their brain, a channel to observe and monitor their mind’s reactions to certain thoughts orperceptions. This also opens the potential for users to experience their body as play [114]. For
example, the study of Inter-Dream concluded that the affordance of playful self-exploration was
instrumental in producing positive affective and arousal states indicative of healthy pre-sleep
physiology [152]. We believe that the more immersive and complex the metaphor being provided, thedeeper the user can sink into their introspective journey, opening more opportunities for selfdiscovery, self-mastery, and self-play. Notable examples push these systems toward digitally
facilitated lucid dreams in which users can craft digital worlds reflective of the contents of their own
mind for them to explore, for example see [8,26,78,120,135].
Self-exploration provides the user with a heightened ability to self-regulate neurocognitive
processes, as they are provided with feedback on how their mental actions bring them closer orfurther away from their target mental state [157]. These regulatory abilities can be quite profound

ACM Trans. Comput.-Hum. Interact.

[24,157], with cases demonstrating the ability to regulate oneself into altered states of consciousness,e.g., promoting pre-sleep states in Inter-Dream [152]. These regulatory abilities can be learned andstrengthened with repetitive use, providing users with a translational skill that they can continue toleverage even without the use of the system [157].
The opportunity for designers creating systems in this quadrant is to help people actively explore,
learn about, and understand their own minds, whilst ultimately providing them with mental selfregulatory skills that can hopefully be translated beyond the use of the system. We acknowledge thatthe self-regulatory abilities provided by these systems have been widely documented [157]. However,
due to their historical origins in clinical neuropsychological practice, the affordances of engaging withthese systems with rich sensory metaphorical symbolism have been given less attention. We,
therefore, extend this prior work by contributing the knowledge that these systems provide a platform
for the user to experience their body as play [114], as players observe and explore with their own
neurocognitive processes that can be fed back to them in the form of engaging sensory metaphors (e.g.,
procedurally generating a game level based on their neuronal activity allowing them to play theirbrain. This may be particularly useful in instances in which users are trying to explore sensitive or
difficult thoughts or feelings, perhaps providing a means for this challenge to be fun, less daunting, or
empowering, making these systems possibly even better suited for psychotherapy than former clinicalincarnations.
One challenge is the design of a suitably deep level of exploration with regard to the ways inwhich the system can metaphorically communicate the user’s neurocognitive processes. For example,
while the commonly employed simple visual metaphors of colors or symbolic objects [136] are a goodstarting point for the user to familiarize themselves with their own neurocognitive processes, they
may quickly exhaust the educational and exploratory affordances offered by this medium overmultiple uses. Future work may consider the generation of more complex, multisensory metaphoricalrepresentations such as narratives or characters, or open-ended environments to keep exploring. An
example from science fiction of such a system taken to its extreme would be the Aleph from William
Gibson’s novel Mona Lisa Overdrive [54], a BC) system in which the user’s mind projects an artificial
reality in which they can learn, grow, and act independently.
5.2.2

Lower Right: Swarm.

In the lower right quadrant sit systems that are characterized by low neural congruence and an
allocentric distribution of agency. We call the user experience swarm , as the experience and its
tendency to form emergent properties are analogous to the process of semiochemical signaling thatmany eusocial swarming insects are capable of. One example of this is ant pheromone trails, whichhave been described as a set of chemical symbols [65] that ants use to autonomically signal
information to other members of its colony in reaction to that ant’s experience of the world. When thecolony collectively contributes to this signaling behavior, this leads to self-organizing patterns (such astrails, rafts, structures made of bodies), ultimately forming a set of chemical symbols and interpersonal
interactions that provide a gestalt body for the embodiment of information about the colony and thesummed experiences of its members [33,56,168].

ACM Trans. Comput.-Hum. Interact.

Systems that sense and symbolically represent brain activity as a sensory metaphor that caninteract with the environment or brain activity representations of other users find themselves in thiscategory. An example of such a system would be the second prototype, Neo-Noumena, as the systemutilizes affectively generated fractals that interact with the physical environment (e.g., avoid and landon surfaces) as well as the fractals generated by other users (e.g., they form a single swarm when bothusers are experiencing the same emotion . Another example is the game Socio-pathways [122] inwhich users (usually five at a time) are represented on a screen as dots. As one user's brain activity
becomes more synchronous with another, their dots move together, with the goal of the game being
the assimilation of all dots into a single ball. This then gives rise to emergent behaviors in the players
as they attempt to synchronize with each other, e.g., such as doing the same repetitive movement.
In these examples, the system offers a shared experience in which each individual contributes to thepooled symbolic representations of brain data, in turn altering the gestalt interpretation of that set ofdata. As a result, the meaning of an individual’s brain data evolves when interpreted alongside thebrain data of others, as opposed to if it was presented in isolation. Furthermore, such systems canallow for representations of brain data to dynamically interact with each other to further provide
information regarding the gestalt of the group. For example, when users of Neo-Noumena wereexperiencing the same emotion, their procedurally generated fractals would join to create a single
flock, signifying affective unity. Similarly, in Socio-pathways the dots representing the brains of theusers would join as they became more synchronous, signifying a convergence of mental state. In boththese examples, the system provides representations of brain data that can be both interpreted to
inform one about an individual or interpreted to understand the group dynamic as a whole.
Taken together, these systems allow users to collectively contribute to a gestalt of brain data that
can interact with itself and combine to form new emergent meanings. This ultimately creates asemiosphere of brain data, with semiosphere being defined by Lotman as the sphere of semiosis inwhich sign processes operate in the set of all interconnected Umwelten from the German Umwelt
meaning environment or surroundings [100]. Specifically, Lotman’s Umwelt theory states that themind and the world are inseparable, because it is the mind that interprets the world for the individual.
Consequently, the Umwelten of each individual differ due to the uniqueness in the biology, history, andlived experience of each individual. When two Umwelten interact, this creates a semiosphere. Thus, forswarm systems, the contribution of one’s Umwelt through their brain data ultimately generates asemiosphere of neurocognitive information in which brain data interacts to generate a narrative of thegroup’s Umwelten.
This can be further unpacked through the Körper - Leib and Erfahrung - Erlebnis distinctions given
by Mueller et al. [113]. Here, the authors use the German lexicon to describe the user experiences
afforded by bodily systems. Specifically, they evoke Körper to refer to the objectified body that
performs its individual functions like maintaining homeostasis and sensing the environment but holds
no lived experience. In contrast, Leib is used in reference to the subjective body, imbued with anontological sense of being that is experienced (i.e., having lived experience). In further unpacking theexperiences of the Leib, we can evoke Erlebnis to signify declarative or procedural knowledge which

ACM Trans. Comput.-Hum. Interact.

can be gained and consciously processed; and Erfahrung to describe pre-reflective knowledge or lived
experience, which only becomes accessible in the process of Erlebnis. Using this lexicon, we can thendescribe the user experience of swarm systems by stating that through using these systems, theneurocognitive data, produced by the Leib of the group is given a collective Körper, embodied by thegestalt sum of the group’s neurogenerated semiosphere. )n turn, this translates the Erlebnis of thegroup into a Körper that provides group Erfahrung which can be accessed by the observers or thegroup itself.
With the above considered, we see that the design opportunity presented by swarm systems is theenabling of individual agencies to contribute their lived experience to the generation of a gestalt bodythat can provide information about the emergent group as a whole. In turn, designers can use theseresultant experiences to engage users and observers to cognitively appraise and extract information
about how a group feels as if it is a superorganism a term used to describe how ants and othereusocial species act together as a single body). Furthermore, these systems still retain the opportunity
to allow for further investigation of the feelings of individuals within the group if they so wish. Fromthe perspective of a group member, this can allow the individual to ask questions such as how do )feel , while concurrently enabling the inquiry of how do we feel? and being able to receive an
Erfahrung (objective) answer to both these Erlebnis (subjective) questions. This provides anadditional dimension in engaging with a group, as in allowing users to perceive and assess how it
feels they have access to information we might not otherwise have.

A key challenge designers might face when designing for this quadrant is how to appropriately
design semiotic signals that balance complexity, simplicity, and informativeness in representing the
gestalt. Specifically, more complex generations of symbols to represent neurocognitive processes canprovide the recipient with more details about the feelings that generated them. However, as
complexity increases, so too does the cognitive effort to interpret them. This is compounded in swarm
systems as each user is contributing their own feelings toward the semiosphere. While designers could
aggregate the representations of all users into a single representation, this would disable any ability tomake inferences about specific individuals in the group. Instead, designers should consider how tocommunicate the group gestalt, while still facilitating the opportunity to interpret neurocognitive
activity on an individual basis as well. For example, Neo-Noumena dealt with this using boid behavior(the logic underlying the behavior of flocking birds, schooling fish and swarming insects [64]), where
signifiers of similar emotions flocked together across users, but contrasting emotions avoided eachother. While this worked with two users, it is anticipated that this may not translate as well for larger
user bases.
5.2.3

Upper Right: Hivemind.

In the upper right quadrant sit systems that are characterized by a high extent of neural congruence
and an allocentric distribution of agency. We call the associated user experience hivemind , as theexperience is likened to being part of a decentralized telepathic collective consciousness common inscience fiction literature [30,92,127,137], such as the Borg from the television series Star Trek [125]- cybernetic organisms whose minds are linked to form a gestalt consciousness called "the Collective".

ACM Trans. Comput.-Hum. Interact.

Systems in the hivemind quadrant often harness brain sensing and stimulation technologies tofacilitate brain-to-brain neural entrainment or amplify inter-brain synchrony. An example of such asystem is the third case study, PsiNet . Another example is the performative art installation titled(ivemind [122], in which two performers have the oscillatory electrical activity of their brain
converted to strobing light, which in turn entrains neural oscillations in the opposite performer. This
process of oscillatory strobing and neural entrainment is continued in a turn-taking manner (like a
conversation) until the neural oscillations of the performers synchronize, ultimately achieving interbrain synchrony localized in the visual cortex.
The users of such systems form a decentralized network of minds, with each user being a locus ofagency that contributes democratically to the gestalt brain activity of all users in the network. This isultimately experienced by users as phenomenological unity , a notion defined by Danaher andPeterson as when there is some unity of phenomenological experience across individuals, i.e. where insome sense they are seeing, feeling, hearing, touching, or tasting the same thing [30]. This wasdemonstrated in the study of PsiNet, where one participant reported I’d feel like I just had heaps ofcaffeine or coffee or energy drinks or something as a result of other users concentrating or being
engaged in work. Similarly, when one participant was rationalizing why another participant may havebeen stimulated in the way they did, they stated: I think maybe if someone was quite agitated or
aggravated by the work that they were doing or whatever the topic was, that might explain why ourhousemate got a phosphene . These examples demonstrate how (ivemind experiences involve aninterpersonal integration of the neurocognitive processes through the sensing and distributed
stimulation of brain activity throughout the network. This ultimately leads to a unity ofphenomenological experience in which users feel similarly to other group members.
As the phenomenological unity of Hivemind experiences is achieved through neurally congruent
signaling, the similarity in how the group feels is experienced implicitly and passively, as users are
not required to divert their attention to the interpretation of symbols to receive this feeling. For
example, participants of PsiNet reported a feeling of connection and being able to affect each otherwithout having to really act and do something and that it automatically sent stuff out, picking up on
your emotions and brain states and sending that out for you . As such, users experience the output ofother brains directly as if it were their own conscious experience, exemplified in the study of PsiNet
where participants stated that you kind of don’t know why you are doing things or to what degree
you’re doing things or influencing each other. You don’t really know where things are coming from .

This ambiguity ultimately allows the output of other brains to be experienced with a high sense ofownership, meaning these individuals feel their cognitive experience to be their own (i.e., generated bytheir brains’ own endogenous neurocognitive processes . As users can find it difficult to separate theirown unique cognitions from the collective cognitions of the group, this suggests users can at times
experience exogenous feelings which come from the stimulation of the system as their own naturally
occurring endogenous feelings. This is further benefited by the notion that in a Hivemind experience,
users feel that agency is homogeneously distributed across all users, as each brain has an equal ability
to change the functioning of the system. This allows Hivemind systems to facilitate experiences of

ACM Trans. Comput.-Hum. Interact.

collective agency, characterized by the feeling of we did that rather than ) did that . An example ofthis can be demonstrated when PsiNet participants were questioned about who had control in the
network, responding it was with us ; it seemed pretty equally distributed ; and we had control via our
inputs and how we responded to the outputs of the system as well [...] so it was everyone .

The design opportunity provided by Hivemind systems is for designers to facilitate experiences ofphenomenological unity to amplify interpersonal connections and neurocognitive cohesion within agroup. Given that humans are highly social, this could provide benefits in many aspects of life. It has
been demonstrated that inter-brain synchrony is much greater when measured between people withclose relationships, such as family members and romantic partners [28,71,71,83,83,154,174]. Thissuggests that Hivemind systems could potentially amplify otherwise weak social connections (e.g., coworkers) toward a more empathetic and familiar standing that would ultimately generate a sense of
comradery. This comes with functional and performative benefits too, as higher levels of interbrain
synchrony have been demonstrated to assist in improved group performance, decision making,cohesion, agreeableness, and empathy [28,71,71,83,83,154,174])..
The main design challenge of the Hivemind quadrant is the logistical complexity of identifying thebest information exchange protocol for the system. In simple terms, this is the issue of knowing when
to send what and to who. Based on input x from user a, which other users should receive output y andwhen? Should all inputs and outputs be averaged? Or considered on a case-by-case basis? This wasperhaps the biggest design challenge in the design of PsiNet, and we answered it by outsourcing thesolution to a reinforcement learning algorithm motivated to increase the inter-brain synchrony of thegroup. While this method worked, we imagine future work would benefit from exploring alternative
solutions to this issue. This challenge also raises ethical questions. What if user x does not want toreceive a specific input, or does not want to feel how a specific other group member is feeling? What if
the collective’s phenomenological unity is moving toward a direction one user is uncomfortable with?
Should we exclude people with psychopathologies from the Hivemind to prevent the spread ofmaladaptive cognitions to others?
5.2.4

Upper Left: Superhumachine.

In the upper left quadrant sit systems that are characterized by high neural congruence and an
egocentric distribution of agency. We call the associated user experience Superhumachine , referring
to Mann’s description of the humachine as a closed-loop feedback system between human andmachine from which a symbiotic cyborg emerges, which has superhuman intelligence [105]. In thiswork, however, we extend the definition to include humachine systems that yield any superhumanability, intelligence or otherwise, which also do not exclusively have to be of positive benefit for thehuman component of the feedback loop. One example of such a system is Machine_in_the_middle [34],a system that classifies the concurrent emotion of the user through EEG, and uses EMS to force thefacial expression of the user to match that emotional classification. As consequence of the system’s
design, the humanchine closed-loop feedback system forms a kind of symbiosis which is parasitic,
rather than mutualistic, forcing the human to sacrifice their ability of expressive deception andaffective privacy in order for the machine to achieve its purpose in expressing emotion. The result is a

ACM Trans. Comput.-Hum. Interact.

Superhumachine assemblage that has enhanced abilities of emotion expression (in contrast to otheraffective systems through borrowing the user’s body as a display, at the cost of the expressive agencyof the human. This raises the question as to whether such as relationship is inducive to being a cyborg
or not, given that the vironment with vironment being Mann’s articulation of the interface between
the users inner body and the outside world is no longer an extension of the user’s agency, but rather
the user is an extension of the viornment’s agency, similar to Mann’s question of whether a slave galley- a ship powered by slave labor - can be considered a cyborg [105].
Systems that sense neurocognitive processes and then reflexively entrain desired neural activity
through stimulation find themselves in this quadrant. These systems provide an experience in which
the user’s neurocognitive processes are passively modified by the system. This modulation may be
imperceptible as users experience their exogenously altered neurocognitive processes as their brain’s
own endogenous activity, providing a strong sense of ownership in the user over the changes thesystem makes to their brain (such as in the case of PsiNet). This sense of ownership is closely tied withthe sense of agency, and in this instance specifically relates to the degree in which an individual feels
their cognitions are their own. This is mostly completed through brain stimulation but can also include
sensory stimuli that can cause neural entrainment (e.g., slowly blowing air through the nasal passage
can slow cortical oscillations, leading to altered states of consciousness [133]). We also note, however,that this is not the only type of experience afforded by systems in this quadrant, and that the locus ofagency and ownership can be flipped to be possessed by the machine rather than the human - asdemonstrated by Machine_in_the_middle . Furthermore, this example also demonstrates that if themachine is the source of agency, the actuation of the human body, while still passive, is no longer
required to be imperceptible to the human, as the human’s sense of agency is sacrificed to thefunctioning of the machine. We note that the experiences of superhumachine systems are typicallysolitary experiences, although they could also be designed to include others (e.g., a system in which
one person's brain controls the brain activity of many, yet the many have no control over the system).
Beyond Machine_in_the_middle, contemporary superhumachine systems typically exist as medical
devices designed to treat clinical populations. These systems are often referred to as brainpacemakers [166], an umbrella term that encompasses devices that sense neural activity andstimulate specific neural structures to correct pathological neural activity. Some example use cases are
the treatment of tremors in Parkinson’s patients [102,111], and the treatment of seizures in epilepsy
patients [184]. In translating this method toward the stimulation of key neural structures in healthy
brains, these systems could be designed to not only maintain the user’s homeostatic neurocognitive
functions but modulate or even enhance them. An example of such a system would be one that notices
a user’s brain activity that the user intends to move, and thus preemptively stimulates the motor
cortex, allowing to perform that movement quicker and with greater control (similar to studies
employing preemptive muscle stimulation to increase reaction time [79], or having your brain
connected to an ebike, helping you slow quicker in the event of danger [4]). Similarly, a user may be
performing a cognitively intensive task and the system might detect the user is concentrating. In turn,the system stimulates the frontal cortex to give an intellectual boost.

ACM Trans. Comput.-Hum. Interact.

As demonstrated in studies of long-term use of neural stimulation, through regular use thesesystems may induce long lasting effects on the synaptic plasticity of the individual, thereby quickening
the rate they acquire new skills [77,98,160]. However, as the user is not involved in this process, this is
not an ability they themselves can regulate (unless of course the designer has given them such control
over the system’s functions . Furthermore, the augmentation of the individual is entirely dependent on
the system, as users are not taught how to regulate cognitive activity though system use due to its
regulation being a passive ongoing process. As such, if the system were to be removed, the benefits it
provides would slowly fade away, rendering the user lesser without the system in contrast to with thesystem. This contrasts with psychonautic systems, in which the system teaches the user cognitive
regulation skills they can then perform without the system.
The opportunity provided to designers creating systems in this quadrant is to help users extend orenhance their neurocognitive capabilities. With the potential for these technologies to become not only
wearable, but implantable in the very near future, this implication goes beyond simply providing users
with empowering tools. Rather, users are provided with a potentially permanent passive enhancement
to abilities and skills such as learning, reaction times, attention, information processing, and memory
for as long as they use the system [77,98,160]. While this has obvious benefits for clinical applications
such as the treatment of epilepsy, dementia, and Parkinson’s disease, these systems can go beyond
therapy by enabling healthy individuals to passively become their better selves with little to no
training, all the while perceiving this enhancement to be their own endogenous abilities.
The challenge faced by superhumachine systems is that the extended abilities provided by such
systems might become part of the user’s perceived self. As such, if the system is removed, the abilities
it provided will eventually subside. Consequently, users may feel lesser, or no longer feel likethemselves with the system’s absence. The design challenge in this quadrant is similar to the design
challenges that face the super body user experience in Mueller et al.’s bodily integration
framework [116].

5.3 Applying the Brain-Computer Integration Framework

This section describes how the brain-computer integration framework can be applied to describing
and modifying the three prototypes presented in this work to demonstrate how the framework can beused in design practice. The three systems support a variety of application domains and employ
different technologies (all the while maintaining the commonality of a focus on BCI as part of thesystem) (summarized in table 2). This demonstrates the general applicability of the framework to mosttypes of BCI systems.
Table 2. The three systems and their characteristics
SystemTechnology

Application

Aim

Inter-Dream

BCI + VR

Sleep

Facilitate healthy pre-sleep

Neo-Noumena

BCI + AR

Communication

Augment emotion communication

PsiNetBCI + tES

Synchrony

Amplify inter-brain synchrony

ACM Trans. Comput.-Hum. Interact.

5.3.1

Design 1 - Inter-Dream.

Inter-Dream is now examined through the brain-computer integration framework to clarify theadvantages of the design while also articulating opportunities to extend Inter-Dream.
Inter-Dream is situated in the Psychonaut quadrant of the design space. Inter-Dream allows users
to experience their own brain activity through a hermeneutic relation as semiotic information. This
provides the user with objective feedback about their subjective states that they can attenuate to inorder to infer knowledge about themselves they would otherwise not have access to. Furthermore,this feedback dynamically changes as their subjective state changes in response to their appraisals of
the semiotic representations of their brain activity, creating an ever-shifting feedback loop. This puts
the user in a position in which they can objectively explore and learn about their own subjectivity, andeven regulate their brain activity if they choose to do so. Furthermore, the Inter-Dream user
experience is highly egocentric. The user is almost completely cut off from the outside world: VR
obscuring vision beyond their neurofeedback, auditory sensation occupied by the ambient score, andeven proprioception being obscured to some degree by the weightless sensation provided by the bed.
The only causal influence on the system other than the user’s brain is the occasional shifting of the
bed’s position by the artists. )nterestingly, participants found that elements of the experience thatwere under control from external influences to be intrusive to the experience overall (specifically the
bed and the score), which at the time led to infer that all dynamic elements of the experience should bedesigned to be neuro-responsive.
In using the brain-computer integration framework, one can envision moving Inter-Dream from thePsychonaut quadrant to the swarm quadrant, where the system has a more allocentric distribution of
agency. The conceptualization of this movement allowed to develop Inter-Dream’s follow-up project,
Neo-Noumena, which takes the artistic generation of semiotic information from brain activity andturns this outward to other users through AR. As such, rather than being disconnected from outside
interference like Inter-Dream, Neo-Noumena was more open to situational influences, with the digital
representation of brain activity being accessed in and interacting with the material world. This
allowed users to access gestalt information about the group, the environment, and the influence of theenvironment on their group. While Neo-Noumena’s application domain was emotion communication,
we can imagine an allocentric Inter-Dream in which the gestalt brain activity of themselves, andperhaps a partner they are sharing the bed with, is visualized and projected onto the roof. This would
allow them to interpret their joint brain activity as they fall asleep hermeneutically, while also being
situated enough to interpret how environmental factors might interact with this physiological process(e.g., a notification on their phone triggering a beeping noise might be followed by a change in thevisualization).
Alternatively, one can envision Inter-Dream being designed for the superintelligence quadrant. In
keeping with the application domain of sleep, an example of a superintelligence-type Inter-Dreamversion might take the form of a wearable BCI system. This wearable would detect if the user was
becoming sleepy based on an increase in delta wave amplitude. In detecting this, the system would
then employ some form of stimulation to increase the entrainment of slow wave brain oscillations,
thereby making the user even more sleepy, helping them fall asleep quicker. This could be done

ACM Trans. Comput.-Hum. Interact.

through a neuromodulatory technology such as tACS, or even through mechanically stimulating theolfactory epithelium with slow bursts of air [133]. Alternatively, the system might also be able to tell if
the user does not want to go to sleep (e.g., there is a growing delta wave amplitude, but the user's
brain activity also demonstrates high levels of cognitive load, suggesting they are working). In such an
event, the system might stimulate their frontal lobe with a high frequency stimulation to help themstay more alert.
5.3.2

Design 1 - Neo-Noumena.

Neo-Noumena is now examined through the brain-computer integration framework to clarify theadvantages of the design while also articulating opportunities to extend Neo-Noumena.
Neo-Noumena sits in the swarm quadrant of the framework, yet its degree of allocentricity
oscillates depending on the actions of the user. For example, Neo-Noumena could be used individually,
with the system providing a situated visualization of the emotional state. However, as the visualization
is situated, interacting with the environment to a similar degree as it interacts or is changed by its
user, the system is still allocentric enough to not be considered psychonautic (or perhaps just on theborder between the two). However, when another user enters the experience, the distribution of
agency shifts greatly toward the allocentric end of the spectrum, as the visualizations now not onlyinteract with the environment but interact with each other and the perceptions of the group
witnessing its gestalt brain activity. This provides users with the opportunity to interpret information
about the group that is not readily accessible when the semiosphere is populated by a single individual(with semiosphere being an abstract epiphenomenal space in which physical, energetic and material
phenomena interact as informational signals).
In using the brain-computer integration framework, one can envision moving Neo-Noumena fromthe swarm quadrant to the hivemind quadrant, where the system transmits informationengrammically rather than semiotically, allowing for a higher degree of neural congruence between
users in the group. The conceptualization of this movement allowed to develop Neo-Noumena’s follow
up project, PsiNet, which uses neurostimulation to synchronize the brain activity of users in the group,
rather than generating visualizations to communicate activity. The original design of Neo-Noumena
afforded a hermeneutic relation with the system, in which users had to actively engage in the cognitive
task of attenuating to semiotic information, and then applying their cognitive schemas to extract
meaning from it and make sense of it, allowing to appraise it objectively. However, in PsiNet, changing
the information from semiotic to engrammic afforded a fusion relation, in which users no longer
actively engaged in the extraction and interpretation of information objectively, but rather subjectively
and passively experienced the brain activity of other users with a high sense of ownership, as if it were
their own brain activity.
Alternatively, we can envision moving the system back toward the psychonaut quadrant, albeit
keeping the application domain of emotion communication. Such a system might take the form of avirtual garden or ecosystem, which represents the user’s emotions semiotically in the form ofparameters within that ecosystem. As the user experiences certain emotions, elements in theecosystem might change (e.g., more sun when they are happy, rain when they are sad, increased

ACM Trans. Comput.-Hum. Interact.

predation when angry, etc.). Thus, if the user maintains a healthy emotional balance, they would
expect to see a healthy and thriving ecosystem. They may also be able to share this environment withothers, allowing them to explore that user’s emotional state as a visitor, but not change it thus
keeping it in the psychonaut quadrant).
5.3.3

Design 3 - PsiNet.

PsiNet sits in the hivemind quadrant of the framework. Agency is distributed allocentricallythroughout the group, with each group member having an equal opportunity to influence the brain
activity of the group. Furthermore, due to the absence of centralization, the system is scalable, withnew users being able to leave and join the group without disrupting equilibrium in the distribution of
agency. Furthermore, as the system works passively (i.e., the user is not required to expend cognitiveeffort or attention to receive information from it), users are free to engage with the environment,
adding another channel of influence over the group’s collective brain activity. The system also exhibitsa high degree of neural congruence, with an oscillation pattern in one person’s brain leading to theactuation of that same oscillation pattern in another person’s brain. As this information is transmitted
engramically, it produces a fusion relation between users, ultimately allowing them to experience eachother's brain activity subjectively with a high sense of ownership.
The design space can now be used to help envision alternative versions of the system. For example,
we can envision an alteration placing the system in the superintelligence quadrant, in which a single
master user has their brain activity sensed by an EEG, and all other users are synchronized to that
individual via tES. In turn, the agency of the system is centralized in the master user, being the only
one able to have genuine cognitive experiences. In contrast, the brains of other users will be enslaved
to feel what the master user is feeling, in a sense becoming possessed by the master user.
Frighteningly, given that the insights from the study of PsiNet suggest that individuals experience
altered brain activity with a high sense of ownership, users with enslaved brains might not evenrealize they are being manipulated (besides the obvious fact that they are wearing a mind-altering
wearable . This would be particularly problematic in the cases in which the system’s neuromodulatory
capabilities were obfuscated by being incorporated invisibly into a hat or bike helmet for example.
Furthermore, with sufficiently advanced brain stimulation, it is possible that such systems might resultin the master user imprinting their self onto the enslaved users, with the enslaved users
experiencing a high sense of ownership to the master’s brain activity wherein they ultimately believe
they are them. This would in effect clone the master’s consciousness, making copies of themselves,
similar to agent Smith in the Matrix movie trilogy.
Alternatively, there are several existing examples of systems which represent what PsiNet might belike after being moved toward the swarm quadrant. One of these is the game SocioPathways [122],
which we described earlier in section 2.1.2.

6

DESIGN STRATEGIES

While the framework can be used to descriptively provide a taxonomy of the possible user experiences
afforded by brain-computer integration systems, it must also be considered how the framework can be

ACM Trans. Comput.-Hum. Interact.

used prescriptively to elucidate how designers can evoke these experiences. Therefore, we now
present a set of strategies that designers might benefit from when developing brain-computer
integration systems. These strategies are informed by our own experience in designing, developing,
deploying, and trialing brain-computer integration systems. Furthermore, these strategies are also
grounded in our own studies evaluating these systems. Considering that these design strategies havebeen synthesized through our interpretation of the results yielded by the study of our prototype
systems, the mapping of the brain-computer integration design space, and through our craft
knowledge gained through the act of developing these systems, we further stress that these do not
represent an exhaustive completed set of strategies fully dictating the design of brain-computer
integration systems. Rather, we suggest that these be seen as an initial set of strategies to guide further
investigation into the design of brain-computer integration whilst also providing actionable advice for
BCI design practitioners, and we expect with further research we will come to a more complete andconcrete articulation of brain-computer integration design strategies. Taken together, our research
insights and craft knowledge have been synthesized into the following strategies (table 3). Threestrategies focus on neural congruence, an additional three on distribution of agency, and an additional
strategy concerned with the design of BCI integration systems in a more general light.

Table 3. Seven Design Strategies
Dimension
Neural Congruence

Title
Exploration
Continuity
Perceptual transparency

Distribution of Agency

Centrality
Spatiotemporality

Social Context

Integrated BCI in General

Learning

Strategy
Consider procedural generation tofacilitate exploration
Consider continuous metrics for morenuanced output
Consider perceptual transparency tosupport high neural congruence
Consider maximizing centrality foregocentric experiences
Consider how data is actualized
spatiotemporally to better facilitate theintended distribution of agency
Consider how social context canenhance playful BCI experiences ingames and play
Consider fostering ongoing integration
through learning

6.1 Exploration: Consider Procedural Generation to Facilitate Exploration
BCIs provide users with a powerful means to learn about themselves and develop a more nuanced
understanding of their brain activity, and the complex thoughts and feelings it drives. Semiotic
encodings of brain activity fed back to the user provide a medium to explore the mind throughinformative or metaphorical codings, and the stories these codings generate through the brain’s
dynamically reactive and ephemeral processes. However, as these representations, metaphors, andnarratives typically require a designer to design them, the array of forms these representations cantake are limited to how many hours designers are able to spend creating content for each given state.

ACM Trans. Comput.-Hum. Interact.

This in turn either limits the permutations of outputs a system is capable of generating or leads todesigners creating very simple representations in order to communicate varied information efficiently(e.g., associating emotion with color or using graphs and charts). As a result, the explorative
affordances of these systems can often suffer from a deficiency in depth and breadth of explorable
content, limiting the user’s engagement and learning potential.
To avoid this limitation, designers should consider incorporating procedural generation in thedesign of BCI output to facilitate exploration. Procedural generation is the method of creating digitalcontent algorithmically as opposed to manually, typically involving the use of mathematicalparameters and some degree of stochasticity to guide the modification of designer-generated content
into entirely new and unexpected forms [45,57,58,138,139,156]. This is a common strategy in video
games in which exploration is a core gameplay mechanic, as new and varied content invites
exploration from users, procedural generation provides a breadth of experience far more expansive
than what can be hand-crafted by a designer [73,139,156]. For example, the game No Man’s Sky
places players in a universe containing 18 quintillion fully explorable planets which are generated as
the player discovers them, each with their own unique terrain, weather, flora, fauna, and even alien
civilizations for the player to explore [165]. Taking inspiration from such applications of procedural
generation, future BCI designs could facilitate deep self-exploration through the generation of detailed
and expansive content generated by dynamic brain-data-fed algorithms.
However, procedural generation does not necessarily require the complexity of a universe
simulator to benefit the design of BCI systems. For example, consider Neo-Noumena, whose
application domain of emotion communication imposes the tradeoff of a need for complexity toadequately express the user’s emotion, while also requiring simplicity enough for the recipient toeffectively interpret it without overloading their senses. Here, rather than generating universes,
procedural generation was employed to subtly assist the semiosis of emotion between individuals. For
example, the procedurally generated behavior was given to the fractals to give the fractals an added
extent of emotional expressiveness through their movement. Specifically, the fractals were
programmed with boids behavior [64] that procedurally generated movement in a group of agentsultimately simulating flocking behavior of birds. Here, the brain activity of the user was fed into theparameters of the boids in order to change the fractals’ movement behavior based on the user’s
emotional states. These movement patterns were not manually animated, but rather procedurally
generated from user’s brain activity. Similarly, the fractals representative of user emotions weregenerated by information extracted from the user’s brain activity, in turn representing the user’s
emotion through the symmetry and geometry of the fractal. Thus, it is demonstrated that brain-driven
procedural generation need not be limited to the creation of expansive vast universes for theindividual to explore, but also in the creation of simple but unique semiotic signifiers. These areexamples of procedural generation that can serve as a method to drive the exploration of both the self
and others, and as such we suggest designers consider procedural content generation to facilitate
exploration.

ACM Trans. Comput.-Hum. Interact.

6.2 Continuity: Consider Continuous Metrics for More Nuanced Output
Coding brain activity into categorical classifications e.g., designating a given set of brain data as sad ,relaxed , awake , etc. comes with some advantages when designing a BC) system. Categoricalcodings allow BCI designers to develop a fixed amount of discrete and hence predictable outcomes,
making it relatively easy to curate the resulting experience of every system state permutation. For
users, this comes with the added benefit that the output of the system is easily interpretable andassociable with single word categories that can compress a lot of information into a single code.
However, there are also significant tradeoffs. We found through the studies of our systems thatparticipants interpreted categorical classifications as authoritative and objectively correct. For
example, even though Neo-Noumena’s classification accuracy was around %, participants always
interpreted the system’s output to be the objective truth, even rationalizing classifications they foundinconsistent with how they were feeling by reasoning that it was they who were wrong, that they wereperhaps not in touch with their inner selves, and that the system was correct.
This aligns with the fact that others within HCI have stressed the importance of refraining fromdesigning categorical feedback for biofeedback systems due to their oversimplification, arguing that it
may lead to the calculability of human subjectivity quantizing the individual into information for
psychographic models through which individuals can be digitally categorized against their bestinterests [161,162]. Considering this, it has been suggested that rather than designing for discretelyclassified presentations of BCI activity, designers should instead consider ambiguous representations
that allow the user to form their own meaning [69,70]. However, the proponents of this argument push
this direction perhaps too far in suggesting that it is incorrect to consider psychological phenomena
and the physiological mechanisms underlying it stateful. The postulation of the denial of states inbiological systems runs contrary to the contemporary understanding of human physiology [74,129].
For example, recent discoveries in neuroscience point to clear states, boundaries, and transitory
tipping points in between, which characterize the dynamics of networks of brain structures and theirfunctions [48,95,110]. Proponents against a stateful approach to physiological activity sometimes also
advocate against the computational processing of biodata, suggesting this should be left to the human
decoder [70]. This approach is particularly non-progressive in the context of BCI, where it is often thecase that much of the informative content of a given physiological signal is embedded in its frequency
component, or in some other extra dimensional geometry of the signal that is not accessible in its timeseries form [173]. Thus, such an approach would limit BCI to pre- 9 ’s capabilities, where the heightof neurotechnology were machines that spat out batches of paper with scribbled lines which took
teams of trained neuroscientists weeks to decode by visual inspection [155].
With these points considered, we suggest designers adopt a more nuanced approach to dealing withbrain state classification, rather than refraining from classification all together. Specifically, we
encourage designers to consider that the brain is a highly dynamic networked system, and that therecognition of any given state is highly dependent on the frame of reference and the question being
asked. In practice, we suggest that designers consider translating categorical classifications tocontinuous metric predictions (a brain-derived metric that is represented as a continuous variable
rather than a categorical variable . Rather than classifying if someone was sad or happy , the system

ACM Trans. Comput.-Hum. Interact.

could provide a normalized dynamic happiness quotient that rises, and falls, based on the valence of
the user (rather than switching a binary category). Similarly, in the case of a system designed tointerpret someone’s state of consciousness, it may be more helpful to derive a continuous metricdescribing their state of consciousness on some sort of scale (e.g., like using the metric of integrated
information phi [170]) and using that to drive representation generation or neurostimulation, rather
than using the outputs of awake or asleep . Such an approach would increase the ambiguity of theoutput, whilst also increasing its informativeness, thereby avoiding the absolutism, oversimplificationand technological determinism that comes with a categorical approach. Thus, to facilitate morenuanced output in BCI integration systems, we suggest designers consider adopting continuous
metrics.

6.3 Perceptual Transparency: Consider Perceptual Transparency to Support High Neural
Congruence
Low neural congruence does indeed have advantages. Abstracted semiotic representations of brain
data promote hermeneutic human-technology relations in which the user can access subjective
experience as objective information. This allows users to extract easily interpretable and actional
information from coded brain data. However, this comes with the cost that using such a system
requires the user’s attentional resources, adding to the ever-growing ecosystem of displays, apps, andnotifications that compete for our attention. This is particularly the case for semiotic information
communicated visually, as vision is a channel already heavily occupied through our interactions withthe world [7,12,13]. Other HCI works suggest that people engage in bodily activities such as sports andphysical exercise to unplug from or escape the always on and constant connectedness afforded bycontemporary pervasive media [119]. However, as integration systems lend themselves to being
designed to always be on the user [116], integration BC)’s with low neural congruence would make thisescape impossible, as the body itself becomes a channel for pervasive media to manifest in the user'slife.
In contrast, high neural congruency affords the design of BCI systems that allow for unplugging,whilst still being connected, as brain-computer technology integrates into the brain’s pre-reflective
endogenous processes, freeing up attentional bandwidth and facilitate fusion human-technology
relations. To do this, we suggest considering designing interfaces with perceptual transparency[117]. This involves the communication of information through artificial sensory experiences [117],borrowing parts of the user’s body for input and output [99], and exploiting psychological phenomena
such as intentional binding – strategically timing the reaction of the system to a user’s biophysicaloutput to create the feeling the action was congruent with their intention [31], to ultimately intertwine
the user and the technology for seamless bilateral information exchange without the necessity forattenuating to the system’s output. Ultimately, the use of a BC) system with high neural congruence
should feel as if the user is not using technology at all, but rather performing mundane bodily
processes that are as unconscious as, for example, breathing and digestion. Such processes do not sit at
the forefront of perception, competing for the user’s attention, but rather are so integrated with the
body that they are in essence part of the process of life.

ACM Trans. Comput.-Hum. Interact.

One example of this in action are people with magnetic implants, who’s brains have adapted toperceive magnetic fields as though it is an endogenic ability rather than a technologically afforded
novelty [36]. Studies of the nervous systems of these individuals demonstrate that the body physicallyincorporates the input of these implants directly into its self-schema, undergoing synaptogenesis andinnervation to accommodate for this new sense [36,164]. Another example is the work ofneuroscientist David Eagleman, who designed a haptic vest that leverages the largely unused realestate of the user’s back to communicate auditory information through haptic vibration, allowing deaf
people to understand speech [38]. Similarly, the cyborg Neil Harbisson is implanted with an artificial
sensory device that allows him to perceive color through intracranial neurostimulation even though
he is color blind [62]. Neurological studies have demonstrated that his brain has learned to integrate
this information into his physiological processes as if this were the endogenous input of color
information from auditory receptors, which phenomenologically allows Harbisson to experience color
as sound, rather than vibration [76]. Therefore, taken together, we suggest designers of integrated
BCIs striving for a high degree of neural congruence consider perceptual transparency.

6.4 Centrality: Consider Maximizing Centrality for Egocentric Experiences
Moving a BCI system toward a higher degree of egocentricity is not solely a matter of designing
solitary experiences or single user systems. While this is indeed a factor which may influence how
egocentric a given BCI system is, this is not enough on its own. Rather, designers should consider how
strongly connected elements of the system are to the user intended to be the ego in the egocentric
experience. That is, designers should consider maximizing centrality when designing egocentric
systems. To do this, designers should strive to minimize the influence of externalities, confounding, or
extraneous variables on the processes of the system, especially in the coding and generation of thesystem’s output. At the same time, designers should also strive to make all other components of thesystem highly reactive to the brain activity of the important node of the network - the central user.
This should be apparent to the user or observers to the degree in which the system obviously fails tofunction if it is not being fed the brain activity of the central user.
In the study of Inter-Dream, participants reported feeling that elements of the experience that did
not respond to brain activity detracted from the experience, specifically noting that they found the
movement of the bed and the music discordant or intrusive as those elements commonly stole theirattention while they puzzled over whether they could control these elements or not. At the same time,
the user experience of Inter-Dream was designed to draw attention to the neuroreponsivity of thesystem through the careful design of the user’s journey in interacting with it. When participants wereintroduced to Inter-Dream, the system was running, albeit in a static state due to the absence of input.
The visualization was still and lifeless, an unmoving sphere. However, as soon as the EEG was fitted tothe participant, the visualizations came to life instantly and explosively, often followed by an
exclamation of excitement from the participant. Here, the neuroresponsivity of the system was madeobvious though demonstration of how much the presence of the participant’s brain activity influenced
the system.

ACM Trans. Comput.-Hum. Interact.

Initially, these findings from Inter-Dream led to consider that it was of utmost importance for BCI
systems to provide users with a high sense of individual control, with demonstrable neuroreactivity
and a high degree of responsivity. However, in later studies, we found this property to be increasingly
less important the more allocentric the distribution of agency was. For instance, during thedevelopment of Neo-Noumena, we encountered a bug in which the fractals swarms would leave theirusers and instead join together in the middle of the room when a pair of users were experiencing thesame emotion. We opted to keep this as a feature, appreciating the visual metaphor of unity andoneness the display communicated, which participants also found equally appropriate despite the lackof control over the flocks that came with it. Similarly, individual control was further relinquished in
PsiNet: control over the analysis of inputs and choice of output was ultimately given to thereinforcement learning agent managing the group’s stimulations. This dynamic ultimately allowed
participants to feel like each had an equal contribution of agency in the group, rather than a sense ofunease or discordance, as we would have originally assumed from the findings of Inter-Dream alone,demonstrating that emphasizing centrality becomes less desirable as the system becomes increasingly
allocentric. Thus, we suggest that designers consider maximizing centrality for egocentric experiences.

6.5 Spatiotemporality: Consider How Data is Actualized Spatiotemporally to Better
Facilitate the Intended Distribution of Agency
Regardless of the distribution of agency in a BCI integration system, the spatiotemporal actualizationof brain data - how information changes (or does not) over time and how it is manifested in the world was found to be consistently important to how users interacted with brain data throughout thestudies. In actualizing brain activity into the world either semiotically or engrammically, it becomessubject to interaction with other properties such as location. Furthermore, if this information can berevisited, either by its creator or an observer, this thereby alters the way it is interpreted, thus
highlighting the influence of time. Taken together, these properties of space and time can ultimately
dictate or modify how a user interprets the actualized brain data, while also providing uniqueaffordances for the embedding of additional information in actualized brain data specific to thosespatiotemporal conditions. In the study of immersive analytics, this property has been referred to as
situated data [41]. However, spatiotemporal actualization of integrated BCI data tends to go beyond
the boundaries of situatedness as described by immersive analytics, in that BCI data can also be
situated within the biology, or physiological processes and rhythms of a user’s brain, particularly when
information is actualized engrammically.
With this said, we suggest that designers consider the spatiotemporal affordances of the application
domain when designing an integrated BCI. For example, Neo-Noumena could be rebuilt to be a BCIpowered, neuroreactive, automated version of the review system Yelp. In this variant, the brain data of
the users, specifically measures of valence, could be sensed and then communicated to other users inand around the restaurant they are dining in, which then other users can interpret to help choose arestaurant to eat at while they browse the city by interpreting the BCI output of other users while also
noticing what restaurant they are eating at. Furthermore, the BCI output of diners could remain in thespace they were dining at after they leave, and remain there over time, similar to how Yelp reviews are
not taken down after a given length of time but are instead persistent and added to a pooled aggregate

ACM Trans. Comput.-Hum. Interact.

of reviews. Taken together, the strategic spatiotemporal actualization of brain data in this context has
ultimately reproduced enhanced decision making processes found in nature, like how ants use thepheromones of explorers from the same colony to better inform themselves of where good sources offood are.
Designers could also take advantage of the contextual information of the application domain itself,
and its interaction with neurophysiological processes, to further capitalize on this. For example, it has
been well documented that smell is strongly connected to the formation of memory, emotion, and also
the experience of taste. With this considered, in returning to the aforementioned Yelp-like BCI system,
it would make sense to use olfactory stimulation as the information communication medium of thisspecific BCI integration system. As smell is strongly related to emotion, the BCI valence readings ofdiners would cause the interface to produce positively or negatively valanced smells. Due to theconnection between smell and memory, other users passing by will have either positive or negative
experiences autonomically recalled when passing by the olfactory output of other users dining,producing a strong reaction in the brains of the observers. Furthermore, as smell interacts with theperception of taste, diners could have their eating experience augmented positively or negatively by
the output of co-diners, leading to a powerful feedback loop in which the valence of the food items
being served are strongly reinforced in proportion to the number of people eating in that given space.
With this in mind, it is evident the amount of interplay between the brain and the environment thesystem permits greatly influences the resulting user experience the system can produce. Controllingthe impact the environment can have on the user's brain (e.g. by blocking out sound and vision) would
limit the amount of impact the environment can thereby have on the resulting user experience,pushing the system toward an egocentric distribution of agency. Conversely, opening up the system toenvironmental influences (e.g., brain data controls the behavior of a robot which also interacts withthe environment autonomously) pushes the system toward an allocentric distribution of agency. Thus,
we suggest that designers consider how data is actualized spatiotemporally to better facilitate theintended distribution of agency.

6.6 Social Context: Consider How Social Context Can Enhance Playful BCI Experiences inGames and Play
Similar to how time and space are influential on the interpretation of BCI data, social context is also
particularly impactful. As the human brain is constructed to extrapolate inferences about the state of
the world through interpersonally oriented neural processes like empathy and social comparison,interpretation of a BCI output is highly influenced by the social context it is appraised within [53]. For
example, users of Neo-Nomena believed that they were able to determine if other players were dealt
good or bad hands while playing a card game based on their emotional output; and users of PsiNet
reported feelings of cooperatively distributing cognitive processing abilities across group members
when playing a videogame together. With this in mind, we suggest that designers consider how social
context can enhance playful BCI experiences in games and play. Furthermore, while play presents apromising social context based on the findings, it is also anticipated that these affordances will be

ACM Trans. Comput.-Hum. Interact.

translatable to many social applications in general. One notable context that represents a promising
opportunity for further exploration is that of VR social spaces (such as VR chat).
One application domain that would particularly benefit from this is that of games and play. While
the potential for BCI as a gaming technology has been explored to some depth [81], we find that therole of the BCI in such explorations is usually limited to that of a game controller, most typically in asingle player experience. However, rather than their traditional role as a controller, we propose that
BCIs in a social gaming context could instead fulfil their potential as channels of communication
between players, or in-game systems (such as game-world game-states). From this perspective, one
could imagine a game of charades where the subject matter is hinted exclusively though BCI output, ora team strategy game where team members are neurostimulated with enhanced concentration if allother team members are concentrating, encouraging participation. Alternatively, one could also
imagine a variant of Monopoly, a game notorious for triggering fits of rage. In this variant, players
could be given the impossible task of trying to not upset any of the other players (detected by the BCI),
as this would trigger a game over for all, leading to play strategies that oddly contradict the central
aim of the game. Thus, we suggest designers consider how social context can enhance playful BCI
experiences in games and play.

6.7 Learning: Consider Fostering Ongoing Integration Through Learning
A key part of the integration between the human body and technology is the process the body takes in
integrating the technology into the user's body schema [106]. This is particularly prevalent in BCI, as
BCI use is often a skill that the user must acquire over time to make full use of the system [108]. This is
a process that necessitates that the user learns to think differently to gain full leverage of the system’s
affordances, which ultimately culminates in notable adaptive changes in neurophysiology andsynaptogenesis. However, we suggest that this is only half the story, and encourage designers toconsider how their BCI integration system can be designed to learn from and adapt to their users overtime, reaching an understanding on how to better integrate with them. This is particularly importantin the context of BCI as every brain is unique, making it difficult to design general purpose algorithms
that are consistent between users [108]. With that said, designers should be aware that BCI use is askill that the user and the system must learn together.
One implementation of this is the use of machine learning to improve the system’s interpretation ofbrain activity. This is often completed as some form of supervised machine learning task and has been
demonstrated to be very effective in applications in which the system has complete information, suchas games [68,101]. However, this becomes challenging when the system has incomplete information, or
no way to verify if what it is learning is in fact correct. One way to address this challenge would be todesign BCI systems with contextually aware computing capabilities in mind, allowing the system toassociate brain activity of its user with their concurrent context and extrapolate patterns andinferences from that [72]. However, another less explored approach that could be applied in tandem
would be to have the system learn as an agent through reinforcement, being rewarded for helping theuser achieve personal goals. For example, PsiNet exhibited this form of learning in that it was
rewarded every time the group’s neural synchrony increased, helping the system learn how it could
best help its users become more synchronous. Similarly, an app by the neurotechnology company

ACM Trans. Comput.-Hum. Interact.

Neurosity builds Spotify playlists based on how long they can keep users in a flow state. Thus, inallowing the system to learn, both the user and the system can work together as an integrated entity tohelp each other achieve their goals synergistically.

7

LIMITATIONS AND FUTURE WORK

One limitation of this work is that the brain-computer integration framework has been articulated
from a mostly qualitative perspective. While some quantitative methods were employed in theanalysis of each of the case studies, this was mainly applied toward the validation of specificparameters of each corresponding prototype (e.g., sleep or inter-brain synchrony) rather than toward
the establishment of the higher-level concepts of the framework itself. The framework wassynthesized through the combined thematic analyses of qualitative user interviews describing theexperiential properties of brain-computer integration systems. Considering that this is a new andunderexplored area of research, we argue that a qualitative approach was the correct path to take, as it
has been acknowledged that qualitative research is particularly useful for theory building, especially inareas where little exploration has been made. Nonetheless, given that this work articulates the braincomputer integration framework, there is now the opportunity to operationalize the framework suchthat brain-computer integration systems can, in the future, be evaluated though objective quantitative
methods [14,29]. For example, future work could explore how the axis neural congruence can be
operationalized through information theory analyses [35,171], rather than this work’s approach of
posthenomenology [144]. Similarly, the axis distribution of agency could be operationalized throughdynamic network analyses [11], rather than this work’s approach of actor network theory [93].
Neuroscience and engineering research fields in particular are well equipped with methodologicalconventions and procedures that have been tried and tested in understanding informational flows
within a system, and the network dynamics of systems as a whole, whether it is electronic or
biological. Connecting both of these disciplines is a heavy use of quantitative analyses and researchmethods stemming from Shannon information theory, particularly when understanding information
flow within a system, which makes it a strong approach for measuring neural congruence. Similarly,
these fields also commonly employ complexity theory, chaos theory, and graph theory-based analyses
and methods when understanding the dynamics of either network machines or networks of neurons,
and thus could also be employed in quantitatively measuring distributions of agency. Furthermore,this transition of a quantitative rendition of the framework also brings the opportunity to use alreadyexisting terminology (e.g., such as terms from information theory, graph theory, etc.) to discuss theconcepts presented. The density of novel terminology present in the framework may be a barrier toentry; the adoption of existing terminology from well-established fields may be helpful incommunicating these ideas beyond HCI.
A further limitation of the framework is that it condenses the complex phenomena of braincomputer integration into two dimensions. While this allows for a parsimonious and arguably neatarticulation of the user experiences afforded by brain-computer integration systems, particularly in
the generation of four UX quadrants, this may also be a hindrance to the framework's ability to explain
more nuanced experiences or interactions. As such, future work could also further contribute tounderstanding the design of brain-computer integration by expanding or extending the framework. It

ACM Trans. Comput.-Hum. Interact.

should be noted that distribution of agency and neural congruence are not the only factors present in
the experience of brain-computer integration. For example, future studies may do well inunderstanding how crowd size influences the experience of brain-computer integration systems.
While such a factor has some overlap with the already present dimension of distribution of agency ,distribution of agency is chiefly concerned with how causal actors interact within the system rather
than the sheer number of humans. Furthermore, it is also possible to deconstruct the distribution of
agency into two factors: crowd size and free will , the degree to which actions originate from theuser’s own mind without outside influence [104], allowing for more nuanced expressions ofcombinations of agency and number of agents. For example, this would allow for the design space toillustrate the experiential distinction between an individual acting on their own free will or not
independent from their immediate social surroundings.
Finally, an additional shortcoming of the framework as it currently stands is that it has only beenused to evaluate and describe systems designed by the authors. To evaluate the validity of theframework's ability to generalise and describe and guide the design of brain-computer integration
systems beyond those of the authors, future work involving validation studies of the framework
through workshops or fitting novel BCI systems to the design space would help mitigate thisshortcoming.

8

CONCLUSION

Through this work, we sought to answer the research question: How do we design Brain-Computer
Integration systems? We have answered this question through the exploration of three prototypes andthe development, presentation, and analysis of three integration BCI systems and their resulting user
experiences: Inter-Dream, Neo-Noumena, and PsiNet. In synthesizing the results yielded from theevaluation of these prototypes, we constructed the Brain-Computer Integration Framework. This
framework descriptively explains the user experiences afforded by BCIs that have been designed toboth interpret and manipulate user neurophysiology, and prescriptively demonstrates how designers
can develop systems to produce an intended user experience. Ultimately, it is intended that thisframework contributes a theoretical basis through which theorists and researchers can discuss
integration BCIs, while also providing practical guidance in the design of future integration BCI
systems.

Acknowledgements
We would like to thank everyone in the Exertion Games Lab for your feedback, insights and support during thecompletion of this research. We would also specifically like to thank Rakesh Patibanda, Josh Andres, and Michaela
Vranic-Peters for their feedback that helped in conceptualizing the framework and the support they provided in theexecution of the three case studies. Florian ‗Floyd‘ Mueller thanks the Australian Research Council, especially
DP190102068, DP200102612 and LP210200656.

ACM Trans. Comput.-Hum. Interact.

REFERENCES[1]

Alexandre Gonfalornieri. 2021. Consumer Brain-Computer Interface: Challenges & Opportunities | by Alexandre Gonfalonieri | Medium. Medium.
Retrieved October 28, 2021 from https://alexandregonfalonieri.medium.com/consumer-brain-computer-interface-challenges-opportunitiese8204190d828[2]

Judith Amores, Xavier Benavides, and Pattie Maes. 2016. PsychicVR: Increasing mindfulness by using Virtual Reality and Brain Computer
Interfaces. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’16, ACM Press,
New York, New York, USA, 2–2. DOI:https://doi.org/10.1145/2851581.2889442[3]

J Andres. 2021. Designing Human–Computer Integration in an Exertion Context. Extended Abstracts of the 2021 CHI Conference on Human Factors
in Computing Systems (2021), 1.

[4]

J Andres, MC Schraefel, N Semertzidis, B Dwivedi, YC Kulwe, J von Kaenel, and FF Mueller. 2020. Introducing Peripheral Awareness as a
Neurological State for Human-Computer Integration. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (2020), 1.

[5]

Josh Andres and others. 2022. Interactive Technology Integrating with the Physically Active Human Body: Learnings from Rider and Ebike
Integration. In Interactive Sports Technologies. Routledge, 14–26.

[6]

Josh Andres, Nathan Semertzidis, Zhuying Li, Yan Wang, and Florian Floyd Mueller. 2023. Integrated Exertion—Understanding the Design of
Human–Computer Integration in an Exertion Context. ACM Transactions on Computer-Human Interaction 29, 6 (2023), 1–28.

[7]

R Arakawa and H Yakura. 2021. Mindless Attractor: A False-Positive Resistant Intervention for Drawing Attention Using Auditory Perturbation.
Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (2021), 1.

[8]

Hardik Arora, Arun Prakash Agrawal, and Ankur Choudhary. 2019. Conceptualizing BCI and AI in video games. In 2019 International Conference
on Computing, Communication, and Intelligent Systems (ICCCIS), IEEE, 404–408. DOI:https://doi.org/10.1109/ICCCIS48478.2019.8974549[9]

Alkinoos Athanasiou, Ioannis Xygonakis, Niki Pandria, Panagiotis Kartsidis, George Arfaras, Kyriaki Rafailia Kavazidi, Nicolas Foroglou,
Alexander Astaras, and Panagiotis D Bamidis. 2017. Towards Rehabilitation Robotics: Off-the-Shelf BCI Control of Anthropomorphic Robotic
Arms. BioMed research international 2017, (August 2017), 5708937. DOI:https://doi.org/10.1155/2017/5708937[10] Mara Balestrini, Sarah Gallacher, and Yvonne Rogers. 2020. Moving HCI Outdoors: Lessons Learned from Conducting Research in the Wild. In
HCI outdoors: theory, design, methods and applications, D. Scott McCrickard, Michael Jones and Timothy L. Stelter (eds.). Springer International
Publishing, Cham, 83–98. DOI:https://doi.org/10.1007/978-3-030-45289-6_4[11] Danielle S Bassett and Olaf Sporns.
DOI:https://doi.org/10.1038/nn.4502

2017.

Network

neuroscience.

NatureNeuroscience20,

3(February2017),

353–364.

[12] A Beattie. 2020. Move Slow and Contemplate Things. Making Time for Digital Lives: Beyond Chronotopia (2020). Retrieved August 31, 2021 fromhttps://books.google.com/books?hl=en&lr=&id=T8b4DwAAQBAJ&oi=fnd&pg=PA137&dq=Beattie,+A.+(2020).+Move+Slow+and+Contemplate+Things.+Making+Time+for+Digital+Lives:+Beyond+Chronotopia,+137.&ots=5pAupTjYTh&sig=c09R9GRwOAK7AVe5GMthm8vjr5o[13] A Beattie. 2020. The Manufacture of Disconnection. (2020). Retrieved August 31, 2021 from http://researcharchive.vuw.ac.nz/handle/10063/9362[14] Pedro F. Bendassolli. 2013. Theory Building in Qualitative Research: Reconsidering the Problem of Induction. Forum Qualitative Sozialforschung /Forum: Qualitative Social Research (2013). DOI:https://doi.org/10.17169/fqs-14.1.1851[15] Steve Benford, Chris Greenhalgh, Andy Crabtree, Martin Flintham, Brendan Walker, Joe Marshall, Boriana Koleva, Stefan Rennick Egglestone,
Gabriella Giannachi, Matt Adams, Nick Tandavanitj, and Ju Row Farr. 2013. Performance-Led Research in the Wild. ACM Transactions on
Computer-Human Interaction 20, 3 (July 2013), 1–22. DOI:https://doi.org/10.1145/2491500.2491502[16] Steve Benford, Richard Ramchurn, Joe Marshall, Max L Wilson, Matthew Pike, Sarah Martindale, Adrian Hazzard, Chris Greenhalgh, Maria
Kallionpää, Paul Tennent, and others. 2021. Contesting control: journeys through surrender, self-awareness and looseness of control in embodied
interaction. Human–Computer Interaction 36, 5–6 (2021), 361–389.
[17] Stephen
P.
Borgatti.
2005.
Centrality
DOI:https://doi.org/10.1016/j.socnet.2004.11.008

andnetworkflow.

Socialnetworks

27,

1(January2005),

55–71.

[18] Barry Brown, Stuart Reeves, and Scott Sherwood. 2011. Into the wild: Challenges and opportunities for field trial methods. In Proceedings of the2011 annual conference on Human factors in computing systems - CHI ’11, ACM Press, New York, New York, USA, 1657.
DOI:https://doi.org/10.1145/1978942.1979185[19] Achim Buerkle, Thomas Bamber, Niels Lohse, and Pedro Ferreira. 2021. Feasibility of detecting potential emergencies in symbiotic human-robot
collaboration with a mobile EEG. Robotics and Computer-Integrated Manufacturing 72, (2021), 102179.
[20] R Butler. 2019. The Way of the Psychonaut: Encyclopedia for Inner Journeys: Two-Volume Compendium. Journal of Transpersonal Psychology 51,
2 (2019), 283–286.
[21] Richard Byrne. 2016. Designing digital vertigo games. In Proceedings of the 2016 ACM Conference Companion Publication on Designing
Interactive Systems - DIS ’16 Companion, ACM Press, New York, New York, USA, 25–26. DOI:https://doi.org/10.1145/2908805.2909419[22] Richard Byrne, Joe Marshall, and Florian ‗Floyd‘ Mueller. 2020. Designing digital vertigo experiences. ACM Trans. Comput.-Hum. Interact. 27, 3(June 2020), 1–30. DOI:https://doi.org/10.1145/3387167[23] M. Callon and V. Rabeharisoa. 2003. Research ―in the wild‖ and the shaping of new social identities. Technology in society 25, 2 (April 2003), 193–
204. DOI:https://doi.org/10.1016/S0160-791X(03)00021-6[24] Kaitlyn Casimo, Kurt E Weaver, Jeremiah Wander, and Jeffrey G Ojemann. 2017. BCI use and its relation to adaptation in cortical networks. IEEE
Trans Neural Syst Rehabil Eng 25, 10 (October 2017), 1697–1704. DOI:https://doi.org/10.1109/TNSRE.2017.2681963[25] Grégoire Cattan. 2021. The use of brain–computer interfaces in games is not ready for the general public. Frontiers of Computer Science 3, (MarchACM Trans. Comput.-Hum. Interact.

2021). DOI:https://doi.org/10.3389/fcomp.2021.628773[26] Marc Cavazza, Gabor Aranyi, Fred Charles, Julie Porteous, Stephen Gilroy, Ilana Klovatch, Gilan Jackont, Eyal Soreq, Nimrod Jakob Keynan,
Avihay Cohen, Gal Raz, and Talma Hendler. 2014. Towards Empathic Neurofeedback for Interactive Storytelling. Schloss Dagstuhl - LeibnizZentrum fuer Informatik GmbH, Wadern/Saarbruecken, Germany (2014). DOI:https://doi.org/10.4230/oasics.cmn.2014.42[27] Alan Chamberlain, Andy Crabtree, Tom Rodden, Matt Jones, and Yvonne Rogers. 2012. Research in the wild: Understanding ―in the wild‖
approaches to design and development. In Proceedings of the Designing Interactive Systems Conference on - DIS ’12, ACM Press, New York, New
York, USA, 795. DOI:https://doi.org/10.1145/2317956.2318078[28] Artur Czeszumski, Sara Eustergerling, Anne Lang, David Menrath, Michael Gerstenberger, Susanne Schuberth, Felix Schreiber, Zadkiel Zuluaga
Rendon, and Peter König. 2020. Hyperscanning: A Valid Method to Study Neural Inter-brain Underpinnings of Social Interaction. Frontiers in
Human Neuroscience 14, (February 2020), 39. DOI:https://doi.org/10.3389/fnhum.2020.00039[29] Peter Dalsgaard and Christian Dindler. 2014. Between theory and practice: Bridging concepts in HCI research. In Proceedings of the 32nd annual
ACM conference on Human factors in computing systems - CHI ’14, ACM Press, New York, New York, USA, 1635–1644.
DOI:https://doi.org/10.1145/2556288.2557342[30] John Danaher and Steve Petersen. 2020. In defence of the hivemind society. Neuroethics (October 2020). DOI:https://doi.org/10.1007/s12152-02009451-7[31] V Danry, P Pataranutaporn, A Haar Horowitz, P Strohmeier, J Andres, R Patibanda, Z Li, T Nakamura, J Nishida, P Lopes, and F León. 2021. Do
Cyborgs dream of Electric Limbs? Experiential Factors in Human-Computer Integration Design and Evaluation. Extended Abstracts of the 2021 CHI
Conference on Human Factors in Computing Systems (2021), 1.
[32] Valdemar Danry, Pat Pataranutaporn, Florian Mueller, Pattie Maes, and Sang-won Leigh. 2022. On Eliciting a Sense of Self when Integrating with
Computers. In Augmented Humans 2022. 68–81.
[33] J. -L. Deneubourg, S. Aron, S. Goss, and J. M. Pasteels. 1990. The self-organizing exploratory pattern of the argentine ant. Journal of insect behavior
3, 2 (March 1990), 159–168. DOI:https://doi.org/10.1007/BF01417909[34] Rod Dickinson, Nathan Semertzidis, and Florian Floyd Mueller. 2022. Machine In The Middle: Exploring Dark Patterns of Emotional HumanComputer Integration Through Media Art. In CHI Conference on Human Factors in Computing Systems Extended Abstracts, 1–7.
[35] Alexander G Dimitrov, Aurel A Lazar, and Jonathan D Victor. 2011. Information theory in neuroscience. Journal of Computational Neuroscience30, 1 (February 2011), 1–5. DOI:https://doi.org/10.1007/s10827-011-0314-3[36] Mark D. Doerksen. 2017. Electromagnetism and the N th sense: augmenting senses in the grinder subculture. The Senses and Society 12, 3(September 2017), 344–349. DOI:https://doi.org/10.1080/17458927.2017.1367487[37] Khalida Douibi, Solène Le Bars, Alice Lemontey, Lipsa Nag, Rodrigo Balp, and Gabrièle Breda. 2021. Toward EEG-Based BCI Applications for
Industry 4.0: Challenges and Possible Applications. Frontiers in Human Neuroscience 15, (August 2021), 705064.
DOI:https://doi.org/10.3389/fnhum.2021.705064[38] D Eagleman. 2020. Livewired: The inside story of the ever-changing brain. Canongate Books. Retrieved August 31, 2021 fromhttps://books.google.com/books?hl=en&lr=&id=FtjBDwAAQBAJ&oi=fnd&pg=PT9&dq=Eagleman,+D.+(2020).+Livewired:+The+inside+story+of+the+ever-changing+brain.+Canongate+Books.&ots=D5G7qLJ3rT&sig=rsuov9-BQRuRAjaZJudRx8F_ZmY[39] Umberto Eco. 1976. A theory of semiotics. Macmillan Education UK, London. DOI:https://doi.org/10.1007/978-1-349-15849-2[40] A. El Gamal and T.M. Cover. 1980.
DOI:https://doi.org/10.1109/PROC.1980.11897

Multipleuser

information

theory.

Proc.

IEEE

68,

12

(1980),

1466–1483.

[41] B Ens, B Bach, M Cordeil, U Engelke, M Serrano, W Willett, A Prouzeau, C Anthes, W Büschel, C Dunne, and T Dwyer. 2021. Grand challenges inimmersive analytics. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (2021), 1.
[42] Xiao Fang, Nathan Semertzidis, Michaela Scary, Xinyi Wang, Josh Andres, Fabio Zambetta, and Florian‘Floyd‘ Mueller. 2021. Telepathic Play:
Towards Playful Experiences Based on Brain-to-brain Interfacing. In Extended Abstracts of the 2021 Annual Symposium on Computer-Human
Interaction in Play, 268–273.
[43] U Farooq, J Grudin, B Shneiderman, P Maes, and X Ren. 2017. Human computer integration versus powerful tools. Proceedings of the 2017 CHI
Conference Extended Abstracts on Human Factors in Computing Systems (2017), 1277.
[44] Umer Farooq and Jonathan Grudin.
DOI:https://doi.org/10.1145/3001896

2016.

Human-computer

integration.

interactions

23,

6(October2016),

26–32.

[45] Jonas Freiknecht and Wolfgang Effelsberg. 2017. A survey on the procedural generation of virtual worlds. MTI 1, 4 (October 2017), 27.
DOI:https://doi.org/10.3390/mti1040027[46] Jérémy Frey, May Grabli, Ronit Slyper, and Jessica R. Cauchard. 2018. Breeze: Sharing Biofeedback through Wearable Technologies. In
Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’1κ, ACM Press, New York, New York, USA, 1–12.
DOI:https://doi.org/10.1145/3173574.3174219[47] F Garro and Z McKinney. 2020. Toward a standard user-centered design framework for medical applications of brain-computer interfaces. 2020
IEEE International Conference on Human-Machine Systems (ICHMS) (2020), 1.
[48] Shree Hari Gautam, Thanh T Hoang, Kylie McClanahan, Stephen K Grady, and Woodrow L Shew. 2015. Maximizing sensory dynamic range bytuningthecorticalstate
tocriticality.
PLoS
Computational
Biology
11,
12(December2015),
e1004576.
DOI:https://doi.org/10.1371/journal.pcbi.1004576[49] Bill Gaver and John Bowers. 2012. Annotated portfolios. interactions 19, 4 (2012), 40–49.

ACM Trans. Comput.-Hum. Interact.

[50] William Gaver. 2012. What should we expect from research through design? In Proceedings of the 2012 ACM annual conference on Human Factors
in Computing Systems - CHI ’12, ACM Press, New York, New York, USA, 937. DOI:https://doi.org/10.1145/2207676.2208538[51] GARY Genosko. 2015. A-SIGNIFYING
DOI:https://doi.org/10.2307/j.ctt183p6gn.8

SEMIOTICS.

In

Félix

guattari:Acriticalintroduction.

Pluto

Press,

89–109.

[52] O George, R Smith, P Madiraju, N Yahyasoltani, and SI Ahamed. 2021. Motor Imagery: A Review of Existing Techniques, Challenges andPotentials. 2021 IEEE 45th Annual Computers, Software, and Applications Conference (COMPSAC) (2021), 1893.
[53] JP Gerber. 2020. Social comparison theory. Encyclopedia of personality and individual differences (2020), 5004–5011.
[54] W Gibson. 1989. Mona Lisa Overdrive. 1988. New York: Bantam (1989).
[55] F Gilbert, C Pham, Jnm Viaña, and W. Gillam. 2019. Increasing brain-computer interface media depictions: pressing ethical concerns. BrainComputer Interfaces (August 2019), 1–22. DOI:https://doi.org/10.1080/2326263X.2019.1655837[56] Arnaud Glad, Olivier Buffet, Olivier Simonin, and François Charpillet. 2009. Self-Organization of Patrolling-Ant Algorithms. In 2009 Third IEEE
International Conference on Self-Adaptive and Self-Organizing Systems, IEEE, 61–70. DOI:https://doi.org/10.1109/SASO.2009.39[57] S Greuter. 2008. Undiscovered worlds, real-time procedural generation of virtual three-dimensional spaces. THESIS.DEGREE. Retrieved August 27,
2021 from https://researchrepository.rmit.edu.au/esploro/outputs/doctoral/Undiscovered-worlds-real-time-procedural-generation-of-virtual-threedimensional-spaces/9921858966301341[58] S Greuter, J Parker, N Stewart, and G Leach. 2003. Real-time procedural generation ofpseudo infinite‘cities. Proceedings of the 1st international
conference on Computer graphics and interactive techniques in Australasia and South East Asia (2003), 87.
[59] Christoph Guger, Gunter Edlinger, W Harkam, I Niedermayer, and Gert Pfurtscheller. 2003. How many people are able to operate an EEG-based
brain-computer interface (BCI)? IEEE transactions on neural systems and rehabilitation engineering 11, 2 (2003), 145–147.
[60] Hayrettin Gürkök, Anton Nijholt, and Mannes Poel. 2012. Brain-Computer Interface Games: Towards a Framework. In Entertainment Computing ICEC 2012, Marc Herrlich, Rainer Malaka and Maic Masuch (eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 373–380.
DOI:https://doi.org/10.1007/978-3-642-33542-6_33[61] D. Corydon Hammond. 2011. What is Neurofeedback: An Update. Journal of neurotherapy 15, 4 (October 2011), 305–336.
DOI:https://doi.org/10.1080/10874208.2011.623090[62] Neil Harbisson. 2018. HEARING COLORS: MY LIFE EXPERIENCE AS A CYBORG. In Creativity, imagination and innovation: perspectives andinspirational stories. WORLD SCIENTIFIC, 117–125. DOI:https://doi.org/10.1142/9789813273009_0015[63] ER Harper, T Rodden, Y Rogers, A Sellen, and B Human. 2008. Human-Computer Interaction in the year 2020. (2008). Retrieved October 29, 2021
from http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.153.4252[64] Christopher Hartman and Bed??ich Bene?? 2006. Autonomous boids. Comput Animat Virtual Worlds 17, 3–4 (July 2006), 199–206.
DOI:https://doi.org/10.1002/cav.123[65] JA Helms. 1998. Dictionary of forestry. (1998). Retrieved August 31, 2021 from http://bibliotecadigital.infor.cl/handle/20.500.12220/104[66] Samuel W Hincks. 2019. A Physical Paradigm for Bidirectional Brain-computer Interfaces. PhD Thesis. Tufts University.
[67] Samuel W Hincks, Sarah Bratt, Sujit Poudel, Vir V Phoha, Robert JK Jacob, Daniel C Dennett, and Leanne M Hirshfield. 2017. Entropic Braincomputer Interfaces-Using fNIRS and EEG to Measure Attentional States in a Bayesian Framework. In PhyCS, 23–34.
[68] Mohammad-Parsa Hosseini, Amin Hosseini, and Kiarash Ahi. 2021. A review on machine learning for EEG signal processing in bioengineering.
IEEE Rev Biomed Eng 14, (January 2021), 204–218. DOI:https://doi.org/10.1109/RBME.2020.2969915[69] Noura Howell, John Chuang, Abigail De Kosnik, Greg Niemeyer, and Kimiko Ryokai. 2018. Emotional Biosensing. Proc. ACM Hum.-Comput.
Interact. 2, CSCW (November 2018), 1–25. DOI:https://doi.org/10.1145/3274338[70] Noura Howell, Laura Devendorf, Tomás Alfonso Vega Gálvez, Rundong Tian, and Kimiko Ryokai. 2018. Tensions of Data-Driven Reflection: A
Case Study of Real-Time Emotional Biosensing. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’1κ,
ACM Press, New York, New York, USA, 1–13. DOI:https://doi.org/10.1145/3173574.3174005[71] Yi Hu, Yafeng Pan, Xinwei Shi, Qing Cai, Xianchun Li, and Xiaojun Cheng. 2018. Inter-brain synchrony and cooperation context in interactive
decision making. Biological Psychology 133, (March 2018), 54–62. DOI:https://doi.org/10.1016/j.biopsycho.2017.12.005[72] David Hübner, Albrecht Schall, and Michael Tangermann. 2020. Unsupervised learning in a BCI chess application using label proportions andexpectation-maximization. Brain-Computer Interfaces 7, 1–2 (April 2020), 22–35. DOI:https://doi.org/10.1080/2326263X.2020.1741072[73] S Ito, M Ishihara, M Tamassia, T Harada, R Thawonmas, and F Zambetta. 2017. Procedural Play Generation According to Play Arcs Using MonteCarlo Tree Search. Proc. 18th International Conference on Intelligent Games and Simulation (2017), 67.
[74] III J DiStefano. 2015. Dynamic systems biology modeling and simulation. Academic Press. Retrieved August 31, 2021 fromhttps://books.google.com/books?hl=en&lr=&id=nWoYAgAAQBAJ&oi=fnd&pg=PP1&dq=DiStefano+III,+J.+(2015).+Dynamic+systems+biology+
modeling+and+simulation.+Academic+Press.&ots=eENjAc5r4W&sig=ZA5uFEUiwNmvxjzBTcXhv7I2XLc[75] Sheena A Josselyn, Stefan Köhler, and Paul W Frankland. 2015. Finding the engram. Nature Reviews. Neuroscience 16, 9 (September 2015), 521–
534. DOI:https://doi.org/10.1038/nrn4000[76] J Kadlecová and J Krbec. 2020. Umwelt Extended: Toward New Approaches in the Study of the Technologically Modified Body. Journal of
Posthuman Studies 4, 2 (2020), 178–194.
[77] RC Kadosh. 2014. The stimulated brain: cognitive enhancement using non-invasive brain stimulation. Elsevier. Retrieved August 31, 2021 fromhttps://books.google.com/books?hl=en&lr=&id=sV1zAwAAQBAJ&oi=fnd&pg=PP1&dq=Kadosh,+R.+C.+(Ed.).+(2014).+The+stimulated+brain:+cognitive+enhancement+using+non-invasive+brain+stimulation.+Elsevier.&ots=J8Ug-1FUNC&sig=97k3VzDWSqeZB9TIDwCa0iW8Dic

ACM Trans. Comput.-Hum. Interact.

[78] Kostas Karpouzis and Georgios N. Yannakakis
DOI:https://doi.org/10.1007/978-3-319-41316-7(Eds.).

2016.

EmotioninGames.

SpringerInternational

Publishing,Cham.

[79] Shunichi Kasahara, Jun Nishida, and Pedro Lopes. 2019. Preemptive Action: Accelerating Human Reaction using Electrical Muscle Stimulation
Without Compromising Agency. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI ’19, ACM Press, New
York, New York, USA, 1–15. DOI:https://doi.org/10.1145/3290605.3300873[80] Aleksandra Kawala-Sterniuk, Natalia Browarska, Amir Al-Bakri, Mariusz Pelc, Jaroslaw Zygarlicki, Michaela Sidikova, Radek Martinek, and
Edward Jacek Gorzelanczyk. 2021. Summary of over Fifty Years with Brain-Computer Interfaces-A Review. Brain sciences 11, 1 (January 2021).
DOI:https://doi.org/10.3390/brainsci11010043[81] Bojan Kerous, Filip Skola, and Fotis Liarokapis. 2017. EEG-based BCI and video games: a progress report. Virtual reality 22, 2 (October 2017), 1–
17. DOI:https://doi.org/10.1007/s10055-017-0328-x[82] Rohit Ashok Khot, Ryan Pennings, and Florian‘Floyd‘ Mueller. 2015. EdiPulse: supporting physical activity with chocolate printed messages. In
Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, 1391–1396.
[83] Sivan Kinreich, Amir Djalovski, Lior Kraus, Yoram Louzoun, and Ruth Feldman. 2017. Brain-to-Brain Synchrony during Naturalistic Social
Interactions. Scientific Reports 7, 1 (December 2017), 17060. DOI:https://doi.org/10.1038/s41598-017-17339-5[84] Alexandra Kitson, Steve DiPaola, and Bernhard E. Riecke. 2019. Lucid loop: A virtual deep learning biofeedback system for lucid dreaming practice.
In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems - CHI EA ’19, ACM Press, New York, New York,USA, 1–6. DOI:https://doi.org/10.1145/3290607.3312952[85] Ilpo Koskinen, John Zimmerman, Thomas Binder, Johan Redstrom, and Stephan Wensveen. 2013. Design research through practice: from the lab,field,
andshowroom.
IEEE
Transactionson
Professional
Communication
56,
3(September2013),
262–263.
DOI:https://doi.org/10.1109/TPC.2013.2274109[86] Nataliya Kosmyna. 2019. Brain-computer interfaces in the wild: lessons learned from a large-scale deployment. In 2019 IEEE International
Conference on Systems, Man and Cybernetics (SMC), IEEE, 4161–4168.
[87] Nataliya Kosmyna and Anatole Lécuyer. 2019. A conceptual space for EEG-based brain-computer interfaces. PloS one 14, 1 (2019), e0210145.
[88] Olave E Krigolson, Chad C Williams, Angela Norton, Cameron D Hassall, and Francisco L Colino. 2017. Choosing MUSE: Validation of a low-cost,portable EEG system for ERP research. Frontiers in Neuroscience 11, (March 2017), 109. DOI:https://doi.org/10.3389/fnins.2017.00109[89] Joseph La Delfa, Mehmet Aydin Baytas, Olivia Wichtowski, Rohit Ashok Khot, and Florian Floyd Mueller. 2019. Are Drones Meditative? In
Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems - CHI EA ’19, ACM Press, New York, New York, USA,
1–4. DOI:https://doi.org/10.1145/3290607.3313274[90] Joseph La Delfa, Robert Jarvis, Rohit Ashok Khot, and Florian ―Floyd‖ Mueller. 2018. Tai chi in the clouds: using micro uav‘s to support tai chipractice. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts, ACM, New York,NY, USA, 513–519. DOI:https://doi.org/10.1145/3270316.3271511[91] Bram van de Laar, Hayrettin Gurkok, Danny Plass-Oude Bos, Mannes Poel, and Anton Nijholt. 2013. Experiencing BCI control in a popular
computer game. IEEE Transactions on Computational Intelligence and AI in Games 5, 2 (June 2013), 176–184.
DOI:https://doi.org/10.1109/TCIAIG.2013.2253778[92] HR Langsdorf. 2020. Tracing the Cultural Influence and Linguistic Journey of 4 Mind-Related Science Fiction Words. (2020). Retrieved August 31,
2021 from https://digitalcommons.wayne.edu/honorstheses/62/[93] B Latour. 1996. On actor-network theory: A few clarifications. Soziale welt (1996), 369–381.
[94] Robert Leeb, Doron Friedman, Gernot R Müller-Putz, Reinhold Scherer, Mel Slater, and Gert Pfurtscheller. 2007. Self-paced (asynchronous) BCI
control of a wheelchair in virtual environments: a case study with a tetraplegic. Computational intelligence and neuroscience (2007), 79642.
DOI:https://doi.org/10.1155/2007/79642[95] Ingrid A van de Leemput, Marieke Wichers, Angélique O J Cramer, Denny Borsboom, Francis Tuerlinckx, Peter Kuppens, Egbert H van Nes,
Wolfgang Viechtbauer, Erik J Giltay, Steven H Aggen, Catherine Derom, Nele Jacobs, Kenneth S Kendler, Han L J van der Maas, Michael C Neale,
Frenk Peeters, Evert Thiery, Peter Zachar, and Marten Scheffer. 2014. Critical slowing down as early warning for the onset and termination ofdepression. Proc Natl Acad Sci USA 111, 1 (January 2014), 87–92. DOI:https://doi.org/10.1073/pnas.1312114110[96] Man Li, Feng Li, Jiahui Pan, Dengyong Zhang, Suna Zhao, Jingcong Li, and Fei Wang. 2021. The mindgomoku: an online P300 BCI game based on
bayesian deep learning. Sensors (Basel, Switzerland) 21, 5 (February 2021). DOI:https://doi.org/10.3390/s21051613[97] Yisi Liu, Olga Sourina, and Minh Khoa Nguyen. 2010. Real-Time EEG-Based Human Emotion Recognition and Visualization. In 2010 International
Conference on Cyberworlds, IEEE, 262–269. DOI:https://doi.org/10.1109/CW.2010.37[98] Chung Yen Looi, Mihaela Duta, Anna-Katharine Brem, Stefan Huber, Hans-Christoph Nuerk, and Roi Cohen Kadosh. 2016. Combining brain
stimulation and video game to promote long-term transfer of learning and cognitive enhancement. Scientific Reports 6, (February 2016), 22003.
DOI:https://doi.org/10.1038/srep22003[99] Pedro Lopes. 2018. The next generation of interactive devices Human Computer Interaction Lab, Hasso Plattner Institute. XRDS: Crossroads, The
ACM Magazine for Students 24, 3 (April 2018), 62–63. DOI:https://doi.org/10.1145/3186701[100] M Lotman. 2002. Umwelt and semiosphere. Σημε ωτ ή-Sign Systems Studies 30, 1 (2002), 33–40.
[101] F Lotte, L Bougrain, A Cichocki, M Clerc, M Congedo, A Rakotomamonjy, and F Yger. 2018. A review of classification algorithms for EEG-based
brain-computer interfaces: a 10 year update. Journal of Neural Engineering 15, 3 (June 2018), 031005. DOI:https://doi.org/10.1088/17412552/aab2f2[102] Andres M Lozano, Jonathan Dostrovsky, Robert Chen, and Peter Ashby. 2002. Deep brain stimulation for Parkinson‘s disease: disrupting the

ACM Trans. Comput.-Hum. Interact.

disruption. Lancet Neurology 1, 4 (August 2002), 225–231. DOI:https://doi.org/10.1016/s1474-4422(02)00101-1[103] Ewa Lux, Marc T. P. Adam, Verena Dorner, Sina Helming, Michael T. Knierim, and Christof Weinhardt. 2018. Live biofeedback as a user interface
design element: A review of the literature. CAIS (2018), 257–296. DOI:https://doi.org/10.17705/1CAIS.04318[104] S. Mann. 2001. Wearable computing:
DOI:https://doi.org/10.1109/5254.940020

towardhumanistic

intelligence.

IEEE

intelligent

systems

16,

3

(May

2001),

10–15.

[105] Steve Mann. 2021. Can Humans Being Machines Make Machines Be Human? In International conference, Medical University of \Lódz.
[106] Angelo Maravita and Atsushi Iriki. 2004. Tools for the body (schema). Trends in Cognitive Sciences 8, 2 (February 2004), 79–86.
DOI:https://doi.org/10.1016/j.tics.2003.12.008[107] Steven G Mason and Gary E Birch. 2003. A general framework for brain-computer interface design. IEEE Transactions on Neural Systems andRehabilitation Engineering 11, 1 (March 2003), 70–85. DOI:https://doi.org/10.1109/TNSRE.2003.810426[108] Dennis J McFarland and Jonathan R Wolpaw. 2018. Brain-computer interface use is a skill that user and system acquire together. PLoS Biology 16, 7(July 2018), e2006719. DOI:https://doi.org/10.1371/journal.pbio.2006719[109] Michael McMahon and Michael Schukat. 2018. A low-Cost, Open-Source, BCI- VR Game Control Development Environment Prototype for Game
Based
Neurorehabilitation.
In
2018
IEEE
Games,
Entertainment,
Media
Conference(GEM),IEEE,
1–9.
DOI:https://doi.org/10.1109/GEM.2018.8516468[110] Christian Meisel, Andreas Klaus, Christian Kuehn, and Dietmar Plenz. 2015. Critical slowing down governs the transition to neuron spiking. PLoS
Computational Biology 11, 2 (February 2015), e1004097. DOI:https://doi.org/10.1371/journal.pcbi.1004097[111] Ameer Mohammed, Richard Bayford, and Andreas Demosthenous. 2018. Toward adaptive deep brain stimulation in Parkinson‘s disease: a review.
Neurodegenerative disease management 8, 2 (April 2018), 115–136. DOI:https://doi.org/10.2217/nmt-2017-0050[112] M F Mridha, Sujoy Chandra Das, Muhammad Mohsin Kabir, Aklima Akter Lima, Md Rashedul Islam, and Yutaka Watanobe. 2021. Brain-Computer
Interface: Advancement and Challenges. Sensors (Basel, Switzerland) 21, 17 (August 2021). DOI:https://doi.org/10.3390/s21175746[113] FF Mueller, L Matjeka, Y Wang, J Andres, Z Li, J Marquez, B Jarvis, S Pijnappel, R Patibanda, and RA Khot. 2020. ― Erfahrung & Erleb nis‖
Understanding the Bodily Play Experience through German Lexicon. Proceedings of the Fourteenth International Conference on Tangible,
Embedded, and Embodied Interaction (2020), 337.
[114] Florian ―Floyd‖ Mueller, Richard Byrne, Josh Andres, and Rakesh Patibanda. 2018. Experiencing the body as play. In Proceedings of the 2018 CHI
Conference on Human Factors in Computing Systems - CHI ’1κ, ACM Press, New York, New York, USA, 1–13.
DOI:https://doi.org/10.1145/3173574.3173784[115] Florian ―Floyd‖ Mueller, Tuomas Kari, Zhuying Li, Yan Wang, Yash Dhanpal Mehta, Josh Andres, Jonathan Marquez, and Rakesh Patibanda. 2020.
Towards designing bodily integrated play. In Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied
Interaction, ACM, New York, NY, USA, 207–218. DOI:https://doi.org/10.1145/3374920.3374931[116] Florian ‗Floyd‘ Mueller, Pedro Lopes, Josh Andres, Richard Byrne, Nathan Semertzidis, Zhuying Li, Jarrod Knibbe, and Stefan Greuter. 2021.
Towards understanding the design of bodily integration. Int J Hum Comput Stud 152, (August 2021), 102643.
DOI:https://doi.org/10.1016/j.ijhcs.2021.102643[117] Florian Floyd Mueller, Pedro Lopes, Paul Strohmeier, Wendy Ju, Caitlyn Seim, Martin Weigel, Suranga Nanayakkara, Marianna Obrist, Zhuying Li,
Joseph Delfa, Jun Nishida, Elizabeth M. Gerber, Dag Svanaes, Jonathan Grudin, Stefan Greuter, Kai Kunze, Thomas Erickson, Steven Greenspan,
Masahiko Inami, Joe Marshall, Harald Reiterer, Katrin Wolf, Jochen Meyer, Thecla Schiphorst, Dakuo Wang, and Pattie Maes. 2020. Next Steps for
Human-Computer Integration. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA,
1–15. DOI:https://doi.org/10.1145/3313831.3376242[118] Florian Mueller, Pattie Maes, and Jonathan Grudin. 2019. Human-Computer Integration (Dagstuhl Seminar 18322). Schloss Dagstuhl - LeibnizZentrum fuer Informatik GmbH, Wadern/Saarbruecken, Germany (2019). DOI:https://doi.org/10.4230/dagrep.8.8.18[119] Florian Mueller and Damon Young. 2018. 10 Lenses to Design Sports-HCI. now Publishers Inc. DOI:https://doi.org/10.1561/9781680835298[120] Ryan Murdoch. 2019. An Experiential Learning-Based Approach to Neurofeedback Visualisation in Serious Games. Advances in Experimental
Medicine and Biology 1156, (2019), 97–109. DOI:https://doi.org/10.1007/978-3-030-19385-0_7[121] Chang S. Nam, Anton Nijholt, and Fabien Lotte (Eds.). 2018. Brain–computer interfaces handbook: technological and theoretical advances. CRC
Press, Boca Raton : Taylor & Francis, CRC Press, 2018. DOI:https://doi.org/10.1201/9781351231954[122] Anton Nijholt (Ed.). 2019. Brain Art: Brain-Computer Interfaces for Artistic Expression. Springer International Publishing, Cham.
DOI:https://doi.org/10.1007/978-3-030-14323-7[123] Kavous Salehzadeh Niksirat, Chaklam Silpasuwanchai, Peng Cheng, and Xiangshi Ren. 2019. Attention Regulation Framework. ACM Trans.
Comput.-Hum. Interact. 26, 6 (November 2019), 1–44. DOI:https://doi.org/10.1145/3359593[124] Francisco Nunes, Nervo Verdezoto, Geraldine Fitzpatrick, Morten Kyng, Erik Grönvall, and Cristiano Storni. 2015. Self-Care Technologies in HCI.
ACM Transactions on Computer-Human Interaction 22, 6 (December 2015), 1–45. DOI:https://doi.org/10.1145/2803173[125] M Okuda, D Okuda, and D Mirek. 2011. The Star Trek Encyclopedia. Simon and Schuster. Retrieved August 31, 2021 fromhttps://books.google.com/books?hl=en&lr=&id=cbYf2l7gczUC&oi=fnd&pg=PT3&dq=Okuda,+M.,+Okuda,+D.,+%26+Mirek,+D.+(2011).+The+St
ar+Trek+Encyclopedia.+Simon+and+Schuster.&ots=lHoIFaiYRv&sig=Uf8QkXdIr_dhPds8YhedYcKK9kU[126] Jorge Andres Moros Ortiz. 2020. Integrated Exertion–Understanding the Design of Human–Computer Integration in an Exertion Context. PhD
Thesis. RMIT University.
[127] J O‘Sullivan. 2010. Collective Consciousness in Science Fiction. Foundation 39, 110 (2010). Retrieved August 31, 2021 fromACM Trans. Comput.-Hum. Interact.

https://search.proquest.com/openview/036b39d2d8caf299d7e01c8ed5ada97b/1?pq-origsite=gscholar&cbl=636386[128] YS Pai, R Hajika, K Gupta, P Sasikumar, and M Billinghurst. 2020. NeuralDrum: Perceiving Brain Synchronicity in XR Drumming. SIGGRAPH
Asia 2020 Technical Communications (2020), 1–4.
[129] Bernhard O. Palsson and Marc Abrams. 2011. Systems biology: simulation of dynamic network states. Cambridge University Press, Cambridge.
DOI:https://doi.org/10.1017/CBO9780511736179[130] CS Peirce. 1991. Peirce on signs: Writings on semiotic. UNC Press Books. Retrieved August 31, 2021 fromhttps://books.google.com/books?hl=en&lr=&id=OgRazXzttrwC&oi=fnd&pg=PP9&dq=Peirce,+C.+S.+(1991).+Peirce+on+signs:+Writings+on+sem
iotic.+UNC+Press+Books.&ots=S772vwt0xm&sig=92ATmUeyeDJ9LpHydhaF0JPtT84[131] Serafeim Perdikis, Luca Tonin, Sareh Saeedi, Christoph Schneider, and José Del R Millán. 2018. The Cybathlon BCI race: Successful longitudinalmutual learning with two tetraplegic users. PLoS Biology 16, 5 (May 2018), e2003787. DOI:https://doi.org/10.1371/journal.pbio.2003787[132] Gert Pfurtscheller and Christa Neuper. 2001. Motor imagery and direct brain-computer communication. Proceedings of the IEEE 89, 7 (2001), 1123–
1134.
[133] A Piarulli, A Zaccaro, M Laurino, D Menicucci, A De Vito, L Bruschini, S Berrettini, M Bergamasco, S Laureys, and A Gemignani. 2018. Ultraslow mechanical stimulation of olfactory epithelium modulates consciousness by slowing cerebral rhythms in humans. Scientific Reports 8, 1 (April2018), 6581. DOI:https://doi.org/10.1038/s41598-018-24924-9[134] Poonam Pillai. 1992. Rereading stuart hall‘s encoding/decoding
DOI:https://doi.org/10.1111/j.1468-2885.1992.tb00040.x

model.

CommunTheory2,

3(August1992),

221–233.

[135] A Pinilla, J Garcia, W Raffe, JN Voigt-Antons, and S Möller. 2021. Visual representation of emotions in Virtual Reality. (2021). Retrieved August
31, 2021 from https://psyarxiv.com/9jguh/[136] Dominic Potts, Kate Loveys, HyunYoung Ha, Shaoyan Huang, Mark Billinghurst, and Elizabeth Broadbent. 2019. Zeng: AR neurofeedback formeditative mixed reality. In Proceedings of the 2019 on Creativity and Cognition - C&C ’19, ACM Press, New York, New York, USA, 583–590.
DOI:https://doi.org/10.1145/3325480.3326584[137] J Prucher. 2007. Brave New Words: The Oxford Dictionary of Science Fiction. Oxford University Press. Retrieved August 31, 2021 fromhttps://books.google.com/books?hl=en&lr=&id=lJCS0reqmFUC&oi=fnd&pg=PP2&dq=Prucher,+J.+(2007).+Brave+New+Words:+The+Oxford+Dictionary+of+Science+Fiction.+Oxford+University+Press.&ots=KE5uKY3ib-&sig=KUhmY4nB7FCS6LaAy0f1fHLs8YY[138] William L. Raffe, Fabio Zambetta, and Xiaodong Li. 2012. A survey of procedural terrain generation techniques using evolutionary algorithms. In
2012 IEEE Congress on Evolutionary Computation, IEEE, 1–8. DOI:https://doi.org/10.1109/CEC.2012.6256610[139] William L. Raffe, Fabio Zambetta, Xiaodong Li, and Kenneth O. Stanley. 2015. Integrated approach to personalized procedural map generation using
evolutionary algorithms. IEEE Trans. Comput. Intell. AI Games 7, 2 (June 2015), 139–155. DOI:https://doi.org/10.1109/TCIAIG.2014.2341665[140] Richard Ramchurn, Sarah Martindale, Max L. Wilson, and Steve Benford. 2019. From Director‘s Cut to User‘s Cut: To Watch a Brain-Controlled
Film is to Edit it. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI ’19, ACM Press, New York, New
York, USA, 1–14. DOI:https://doi.org/10.1145/3290605.3300378[141] Joan Sol Roo, Renaud Gervais, Jeremy Frey, and Martin Hachet. 2017. Inner garden: connecting inner states to a mixed reality sandbox for
mindfulness. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, 1459–1470.
DOI:https://doi.org/10.1145/3025453.3025743[142] S Rosca, M Leba, A Ionica, and O Gamulescu. 2018. Quadcopter control using a BCI. IOP Conference Series: Materials Science and Engineering
294, (January 2018), 012048. DOI:https://doi.org/10.1088/1757-899X/294/1/012048[143] Sebastian – Daniel Rosca and Monica Leba. 2019. Design of a Brain-Controlled Video Game based on a BCI System. MATEC Web of Conferences
290, (2019), 01019. DOI:https://doi.org/10.1051/matecconf/201929001019[144] R Rosenberger and PP Verbeek. 2015. Postphenomenological investigations: essays on human-technology relations. Lexington Books. Retrieved
August 12, 2021 from https://research.utwente.nl/en/publications/postphenomenological-investigations-essays-on-human-technology-re[145] Rüdiger Rupp, Sonja C. Kleih, Robert Leeb, José del R. Millan, Andrea Kübler, and Gernot R. Müller-Putz. 2014. Brain–computer interfaces andassistive technology. In Brain-Computer-Interfaces in their ethical, social and cultural contexts, Gerd Grübler and Elisabeth Hildt (eds.). Springer
Netherlands, Dordrecht, 7–38. DOI:https://doi.org/10.1007/978-94-017-8996-7_2[146] Simanto Saha, Khondaker A Mamun, Khawza Ahmed, Raqibul Mostafa, Ganesh R Naik, Sam Darvishi, Ahsan H Khandoker, and Mathias Baumert.
2021. Progress in brain computer interface: challenges and opportunities. Frontiers in Systems Neuroscience 15, (February 2021), 578875.
DOI:https://doi.org/10.3389/fnsys.2021.578875[147] NA Semertzidis, M Scary, X Fang, X Wang, R Patibanda, J Andres, P Strohmeier, K Kunze, P Lopes, F Zambetta, and FF Mueller. 2021. SIGHInt:
Special Interest Group for Human-Computer Integration. Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems(2021), 1.
[148] Nathan Semertzidis. 2021. Brain-Computer Integration. PhD Thesis. Monash University.
[149] Nathan Semertzidis. 2022. PsiNet: Understanding the Design of Brain-to-Brain Interfaces for Enhancing Inter-Brain Synchrony. In to appear.
[150] Nathan Semertzidis, Josh Andres, Martin Weigel, Suranga Nanayakkara, Rakesh Patibanda, Zhuying Li, Paul Strohmeier, Jarrod Knibbe, Stefan
Greuter, Marianna Obrist, and others. 2022. Human–Computer Integration: Towards Integrating the Human Body with the Computational Machine.
Foundations and Trends® in Human-Computer Interaction 16, 1 (2022), 1–64.
[151] Nathan Arthur Semertzidis, Zoe Xiao Fang, Pedro Lopes, Kai Kunze, Paul Pangaro, Florian Floyd Mueller, and Pattie Maes. 2022. What We Talk
About When We Talk About Human-Computer Integration. In CHI Conference on Human Factors in Computing Systems Extended Abstracts, 1–4.

ACM Trans. Comput.-Hum. Interact.

[152] Nathan Arthur Semertzidis, Betty Sargeant, Justin Dwyer, Florian Floyd Mueller, and Fabio Zambetta. 2019. Towards Understanding the Design ofPositive Pre-sleep Through a Neurofeedback Artistic Experience. In Proceedings of the 2019 CHI Conference on Human Factors in Computing
Systems - CHI ’19, ACM Press, New York, New York, USA, 1–14. DOI:https://doi.org/10.1145/3290605.3300804[153] Nathan Semertzidis, Michaela Scary, Josh Andres, Brahmi Dwivedi, Yutika Chandrashekhar Kulwe, Fabio Zambetta, and Florian Floyd Mueller.
2020. Neo-Noumena: Augmenting Emotion Communication. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems,
ACM, New York, NY, USA, 1–13. DOI:https://doi.org/10.1145/3313831.3376599[154] Mohammad Shehata, Miao Cheng, Angus Leung, Naotsugu Tsuchiya, Daw-An Wu, Chia-huei Tseng, Shigeki Nakauchi, and Shinsuke Shimojo.
2020. Team flow is a unique brain state associated with enhanced information integration and neural synchrony. BioRxiv (June 2020).
DOI:https://doi.org/10.1101/2020.06.17.157990[155] H W Shipton. 1975. EEG analysis: A history and
DOI:https://doi.org/10.1146/annurev.bb.04.060175.000245

aprospectus.

Annu

Rev

Biophys

Bioeng

4,

1

(June

1975),

1–13.

[156] Tanya X. Short and Tarn Adams. 2017. Procedural generation in game design. A K Peters/CRC Press, Boca Raton : Taylor & Francis, CRC Press,2017. DOI:https://doi.org/10.1201/9781315156378[157] Ranganatha Sitaram, Tomas Ros, Luke Stoeckel, Sven Haller, Frank Scharnowski, Jarrod Lewis-Peacock, Nikolaus Weiskopf, Maria Laura Blefari,Mohit Rana, Ethan Oblak, Niels Birbaumer, and James Sulzer. 2017. Closed-loop brain training: The science of neurofeedback. Nature Reviews.
Neuroscience 18, 2 (2017), 86–100. DOI:https://doi.org/10.1038/nrn.2016.164[158] Jacek Sliwinski. 2019. Mindfulness and HCI. In Handbook of Research on Human-Computer Interfaces and New Modes of Interactivity, Katherine
Blashki and Pedro Isaías (eds.). IGI Global, 314–332. DOI:https://doi.org/10.4018/978-1-5225-9069-9.ch018[159] Erin Solovey, Paul Schermerhorn, Matthias Scheutz, Angelo Sassaroli, Sergio Fantini, and Robert Jacob. 2012. Brainput: enhancing interactive
systems with streaming fnirs brain input. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems, 2193–2202.
[160] Vishnu Sreekumar, John H Wittig, Timothy C Sheehan, and Kareem A Zaghloul. 2017. Principled approaches to direct brain stimulation for
cognitive enhancement. Front Neurosci 11, (November 2017), 650. DOI:https://doi.org/10.3389/fnins.2017.00650[161] Luke Stark. 2018. Algorithmic psychometrics
DOI:https://doi.org/10.1177/0306312718772094

and

the

scalable

subject.

Soc

Stud

Sci

48,

2

(April2018),

204–231.

[162] Luke Stark and Kate Crawford. 2015. The conservatism of emoji: work, affect, and communication. Social Media + Society 1, 2 (September 2015),
205630511560485. DOI:https://doi.org/10.1177/2056305115604853[163] Pierce Stegman, Chris S. Crawford, Marvin Andujar, Anton Nijholt, and Juan E. Gilbert. 2020. Brain–computer interface software: A review anddiscussion. IEEE Trans Hum Mach Syst 50, 2 (April 2020), 101–115. DOI:https://doi.org/10.1109/THMS.2020.2968411[164] P Strohmeier and J McIntosh. 2020. Novel Input and Output opportunities using an Implanted Magnet. Proceedings of the Augmented Humans
International Conference (2020), 1.
[165] Emma R Tait and Ingrid L Nelson. 2021. Nonscalability and generating digital outer space natures in No Man‘s Sky. Environment and Planning E:
Nature and Space (March 2021), 251484862110007. DOI:https://doi.org/10.1177/25148486211000746[166] PA Tass, C Hauptmann, and OV Popovych. 2020. Brain pacemaker. Synergetics (2020), 235–262.
[167] Nađa Terzimehić, Renate Häuslschmid, Heinrich Hussmann, and m.c. schraefel. 2019. A review & analysis of mindfulness research in HCI: framing
current lines of research and future opportunities. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI ’19,
ACM Press, New York, New York, USA, 1–13. DOI:https://doi.org/10.1145/3290605.3300687[168] Guy Theraulaz, Eric Bonabeau, Stamatios C Nicolis, Ricard V Solé, Vincent Fourcassié, Stéphane Blanco, Richard Fournier, Jean-Louis Joly, Pau
Fernández, Anne Grimal, Patrice Dalle, and Jean-Louis Deneubourg. 2002. Spatial patterns in ant colonies. Proceedings of the National Academy of
Sciences of the United States of America 99, 15 (July 2002), 9645–9649. DOI:https://doi.org/10.1073/pnas.152302199[169] Ladislav Timulak. 2014. Qualitative meta-analysis. The SAGE handbook of qualitative data analysis 481, (2014).
[170] G Tononi. 2010. Information integration: its relevance to brain function and consciousness. Archives Italiennes de Biologie 148, 3 (September 2010),
299–322.
[171] Giulio Tononi, Melanie Boly, Marcello Massimini, and Christof Koch. 2016. Integrated information theory: from consciousness to its physicalsubstrate. Nature Reviews. Neuroscience 17, 7 (July 2016), 450–461. DOI:https://doi.org/10.1038/nrn.2016.44[172] Erin Treacy Solovey, Daniel Afergan, Evan M Peck, Samuel W Hincks, and Robert JK Jacob. 2015. Designing implicit interfaces for physiologicalcomputing: Guidelines and lessons learned using fNIRS. ACM Transactions on Computer-Human Interaction (TOCHI) 21, 6 (2015), 1–27.
[173] Swati Vaid, Preeti Singh, and Chamandeep Kaur. 2015. EEG signal analysis for BCI interface: A review. In 2015 Fifth International Conference on
Advanced Computing & Communication Technologies, IEEE, 143–147. DOI:https://doi.org/10.1109/ACCT.2015.72[174] Ana Lucía Valencia and Tom Froese. 2020. What binds us? Inter-brain neural synchronization and its implications for theories of humanconsciousness. Neuroscience of consciousness 2020, 1 (June 2020), niaa010. DOI:https://doi.org/10.1093/nc/niaa010[175] Gabriel Alves Mendes Vasiljevic and Leonardo Cunha de Miranda. 2019. Brain–Computer Interface Games Based on Consumer-Grade EEG
Devices: A Systematic Literature Review. International Journal of Human–Computer Interaction (June 2019), 1–38.
DOI:https://doi.org/10.1080/10447318.2019.1612213[176] PETER-PAUL Verbeek. 2005. What things do: philosophical reflections on technology, agency, and design. Penn State University Press.
DOI:https://doi.org/10.5325/j.ctv14gp4w7[177] PP Verbeek. 2015. Toward a theory of technological mediation. Technoscience and postphenomenology: The Manhattan papers (2015). Retrieved
September
3,
2021
from

ACM Trans. Comput.-Hum. Interact.

https://books.google.com.au/books?hl=en&lr=&id=oJcpCwAAQBAJ&oi=fnd&pg=PA189&dq=verbeek+technology+mediation&ots=rh8aZJRUyT&sig=1GP46cF_rcTQMK89fYkzbAJgK5U[178] S. Verdu. 1998. Fifty years of
DOI:https://doi.org/10.1109/18.720531

Shannontheory.

IEEE

Transactions

on

Information

Theory44,

6(1998),

2057–2078.

[179] Liang Wang, Zhe Huang, Ziyu Zhou, Devon McKeon, Giles Blaney, Michael C Hughes, and Robert JK Jacob. 2021. Taming fNIRS-based BCI Input
for Better Calibration and Broader Use. In The 34th Annual ACM Symposium on User Interface Software and Technology, 179–197.
[180] Richmond Y. Wong, Nick Merrill, and John Chuang. 2018. When BCIs have APIs: Design Fictions of Everyday Brain-Computer Interface
Adoption. In Proceedings of the 2018 on Designing Interactive Systems Conference 2018 - DIS ’1κ, ACM Press, New York, New York, USA, 1359–
1371. DOI:https://doi.org/10.1145/3196709.3196746[181] Justin Wren-Lewis. 1983. The encoding / decoding model: criticisms and redevelopments for research on decoding. Media, Culture & Society 5, 2(April 1983), 179–197. DOI:https://doi.org/10.1177/016344378300500205[182] Bin Yu, Mathias Funk, Jun Hu, Qi Wang, and Loe Feijs. 2018. Biofeedback for everyday stress management: A systematic review. Front. ICT 5,(September 2018). DOI:https://doi.org/10.3389/fict.2018.00023[183] Thorsten O Zander and Christian Kothe. 2011. Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general. Journal of neural engineering 8, 2 (2011), 025005.
[184] Nasser Zangiabadi, Lady Diana Ladino, Farzad Sina, Juan Pablo Orozco-Hernández, Alexandra Carter, and José Francisco Téllez-Zenteno. 2019.
Deep Brain Stimulation and Drug-Resistant Epilepsy: A Review of the Literature. Front Neurol 10, (June 2019), 601.
DOI:https://doi.org/10.3389/fneur.2019.00601[185] QiBin Zhao, LiQing Zhang, and Andrzej Cichocki. 2009. EEG-based asynchronous BCI control of a car in 3D virtual reality environments. Chinese
Science Bulletin 54, 1 (January 2009), 78–87. DOI:https://doi.org/10.1007/s11434-008-0547-3[186] John Zimmerman, Jodi Forlizzi, and Shelley Evenson. 2007. Research through design as a method for interaction design research in HCI. In
Proceedings of the SIGCHI conference on Human factors in computing systems - CHI ’0ι, ACM Press, New York, New York, USA, 493.
DOI:https://doi.org/10.1145/1240624.1240704[187] John Zimmerman, Erik Stolterman, and Jodi Forlizzi. 2010. An analysis and critique of Research through Design Towards a formalization of a
research approach. In Proceedings of the 8th ACM Conference on Designing Interactive Systems - DIS ’10, ACM Press, New York, New York, USA,
310. DOI:https://doi.org/10.1145/1858171.1858228[188] 1988. Handbook of Human-Computer Interaction. Elsevier. DOI:https://doi.org/10.1016/C2009-0-12113-X

ACM Trans. Comput.-Hum. Interact.