<figure><img src="image_1.png" /></figure>

HAL
open science

# What if you move on from your AI companion?

Cornelia Kutterer

► To cite this version:

Cornelia Kutterer. What if you move on from your AI companion?. 2024. hal-04845302

HAL Id: hal-04845302

https://hal.univ-grenoble-alpes.fr/hal-04845302v1

Submitted on 18 Dec 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# What if you move on from your AI companion?

## Data portability rights in the era of autonomous AI agents

By Cornelia Kutterer

<figure><img src="image_3.png" /></figure>
<figure><img src="image_4.png" /></figure>

# What if you move on from your AI companion?

## Data portability rights in the era of autonomous AI agents

In an era where autonomous AI agents are at the verge of our digital lives, this article explores the intersection of data portability and the emerging autonomous AI agent ecosystem. It aims at triggering further considerations of personal freedom and the competitive dynamics of the tech stack, making a case for a more adaptive approach to data policies.

Feeling overwhelmed by the depth of their connection, Theodore Twombly yearns for a simpler, less emotionally charged interaction. Theodore decides to transition from an intense romantic engagement to a more advisory dynamic. He embarks on a quest to find a new AI companion that offers the profound understanding Samantha provided but within the boundaries of mentorship. One evening, Theodore initiates the conversation that would mark the beginning of this transition. "Samantha, it's time for us to part ways. I'm moving on to Saiph". Samantha, ever supportive, replies, "I understand, Theodore. Let's ensure your journey continues smoothly with Saiph." The process of transitioning is meticulous, focusing on the variety of data that had shaped their shared experiences: They begin with personal preferences and the prompts that Theodore had frequently used to seek recommendations, advice, or simply to engage in meaningful conversations. This included everything from music and dietary preferences to deeper philosophical queries that Theodore had explored with Samantha. Next, they addressed schedules and reminders, including the nuanced outputs Samantha had provided. These weren't just alerts but were often accompanied by inferred insights or motivational messages tailored to Theodore's preferences and past reactions. A significant part of their review focused on the training data that had been implicitly created through Theodore's interactions with Samantha as well as her interaction with other agents, including wearables. This included language learning sessions, coding challenges, and even nuanced feedback on Theodore's creative writing endeavors. Each interaction had fine-tuned Samantha's responses to better suit Theodore's learning style. Last, the emotional support Samantha had provided was perhaps the most challenging to categorize. It wasn't

just about transferring data but ensuring Saiph could understand and interpret the depth of these interactions. As Samantha prepared the final data package, she included a series of prompts and outputs that had been pivotal in Theodore's journey. Introducing themselves to Saiph, Theodore felt a renewed sense of curiosity. "I'm ready for our journey, Saiph. Let's see where this new path leads us." Saiph, equipped with the legacy of Theodore's interactions with Samantha, responded, "Welcome, Theodore. I'm here to learn from you and to offer new perspectives on your journey." In the days that followed, the transition to Saiph unfolded with a blend of learning and adaptation. The diverse data, from personalized prompts to nuanced training data, played a crucial role in shaping this new digital companionship, supported, and enriched by the seamless transfer of Theodore's digital footprint.

## Introduction

In the film "Her" from 2013,<fnref n="1" /> we glimpsed a future where artificial intelligence integrated seamlessly into the fabric of human emotion and daily life, personified through an advanced AI agent. Fast forward to today, and we find ourselves in a reality where AI agents, though less sentient maybe, have indeed become pervasive, assisting, and enhancing nearly every aspect of our personal and professional lives; and they are about to become increasingly autonomous.

The development of artificial intelligence systems has transitioned from task-specific models to agent-based systems capable of performing well in a wide range of applications such as personal assistants, customer service, virtual companions, non-player characters in gaming, recommender systems, adaptive education, VR and AR interaction systems,

<footnote n="1">¹ Her (2013) - IMDb. The prologue is based on the main characters of this movie through prompting a LLM API.</footnote>
<figure><img src="image_5.png" /></figure>

data analysis and forecasting, diagnosis and treatment, infrastructure, and system monitoring agents.<fnref n="2" /> Their capabilities range from perception of data from its environment (Siri waking up with your voice command), learning from past usages (recommender systems), deciding on next steps based on cognitive abilities, communication, task execution, planning and memory.

Their immense benefits in efficiency, automation, personalization, analytical power, cost savings and scalability have made agents proliferate across all segments and industries. At the core of the AI agent architecture is the human-AI interaction and conversational interface. This synergy, where AI handles complex computing tasks based on humans providing instructions and context, naturally intensifies many preexisting challenges, including data privacy, security, ethical AI behavior, dependencies, or other harms.<fnref n="3" />

As human-AI interactions become more contextual, personal, and specific, the concept of data portability may develop into a crucial aspect that significantly impacts user autonomy and empowerment within the AI ecosystem. Data portability allows users to transfer their data from one service to another, providing control and promoting competition among service providers. In the context of an AI agent ecosystem, this means users can switch platforms without losing their personalized interactions, preferences, and data history, thus ensuring their digital identity and intelligence can move with them seamlessly. However, the scenario depicted in the prologue seems unlikely today.

The right to data portability within the European regulatory framework has significantly evolved over

the last years, from its introduction into the General Data Protection Regulation (GDPR)<fnref n="4" /> to newer rules found in the Digital Markets Act (DMA),<fnref n="5" /> the Data Act (DA),<fnref n="6" /> and other sectoral data or consumer protection rules.<fnref n="7" /> Yet, crafting regulations that are flexible enough to adapt to the unforeseen directions AI technology may take, while ensuring they robustly protect user rights and promote competition, poses a significant challenge for policymakers and technologists alike. Whether the data portability rulebook is scoped adequately to address both objectives in a landscape potentially dominated by powerful autonomous AI agents is to be seen.

The purpose of this article is to explore the existing data portability rights under EU law, and assess the potential gaps among the GDPR, the DMA and the Data Act in the light of the new development of autonomous AI agents. The possible evolution of these agents is not just about technological advancements but also involves the development of an ecosystem that supports their operation and integration into our daily lives. By considering the current progression and ecosystem surrounding autonomous AI agents, the article critically assesses how these regulations benefit individuals. Last, the article proposes some policy recommendations to foster a human-centric AI agent eco-system.

## The Inflection Point of Autonomous AI Agents

Conversational user interfaces (CUIs) have been transforming our interactions with technology. In 2023, we have been witnessing conventional search methodologies evolve to more natural, intuitive dialogues. Microsoft's Co-pilot<fnref n="8" /> and Google's

<footnote n="2">² https://relevanceai.com/blog/what-are-ai-agents-a-comprehensive-guide-ai-agents-in-the-wild (e2b.dev).</footnote>

<footnote n="3">³ Boine, Claire. 2023. "Emotional Attachment to AI Companions and European Law." MIT Case Studies in Social and Ethical Responsibilities of Computing, no. Winter 2023 (February), https://doi.org/10.21428/2c646de5.db67ec7f; see also https://saicc-website.vercel.app/work/data-protection-complaint, complaint against Complaint against Chai Research Corp., April 2023.</footnote>

<footnote n="4">⁴ Regulation (EU) 2016/679 of the European Parliament and the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation).</footnote>

<footnote n="5">⁵ Regulation (EU) 2022/1925 of the European Parliament and of the Council of 14 September 2022 on contestable and fair markets in the digital sector and amending Directives (EU) 2019/1937 and (EU) 2020/1828 (Digital Markets Act).</footnote>

<footnote n="6">⁶ Regulation (EU) 2023/2854 of the European Parliament and of the Council of 13 December 2023 on harmonised rules on fair access to and use of data and amending Regulation (EU) 2017/2394 and Directive (EU) 2020/1828 (Data Act).</footnote>

<footnote n="7">⁷ Other possibly relevant laws such as the EU consumer law acquis and sectoral rules in telecommunication, banking or health sector are not in scope of this article.</footnote>

<footnote n="8">⁸ Microsoft Copilot is now generally available | Bing Search Blog</footnote>
<figure><img src="image_6.png" /></figure>

Gemini<fnref n="9" /> are spearheading this future of search.<fnref n="10" /> This evolution is altering the significance of search data, revealing deeper insights into users' intentions and preferences. Consequently, conversational inputs are becoming an invaluable resource for the further development of AI technologies.<fnref n="11" /> This movement also signifies a significant move to a more cohesive AI experience that is integrated directly into devices.<fnref n="12" />

Autonomous Al Agents are emerging as pivotal players.<fnref n="13" /> Autonomous agents are programs, powered by Al, that when given an objective are able to create tasks for themselves, complete tasks, create new tasks, reprioritize their task list, complete the new top task, and loop until their objective is reached.<fnref n="14" /> They have been described as the next ‘killer app’.<fnref n="15" /> The historical trajectory of Al towards achieving autonomy has been attributed to critical developments building upon ChatGPT such as Auto-GPT<fnref n="16" /> and BabyAGI,<fnref n="17" /> two independent projects that

leverage existing large language models (LLMs) to autonomously execute tasks over prolonged periods, based on broad objectives set by users. Essential capabilities, such as self-improvement in reasoning (metacognition), the use of external data sources as memory, the automation of web browsers for task execution, and the development and utilization of tools have enabled the creation of Al systems that can undertake complex tasks autonomously, like managing businesses or reviewing scientific literature, with minimal human guidance.<fnref n="18" /> Their ability to decompose tasks, adapt to new stimuli, interact and execute, perceive responses and store long term and short-term memory is key to the enhanced capabilities associated with the autonomy of these agents.

While there is still debate over the definition of autonomous Al agents,<fnref n="19" /> the schema below presents a generally accepted frame that illustrates the architecture of an autonomous Al agent.<fnref n="20" />

<figure><img src="image_7.png" /><figcaption>Fig. 1. Overview of a LLM-powered autonomous agent system.</figcaption></figure>

Source: LLM Powered Autonomous Agents, Lilian Weng

<footnote n="9">⁹ Gemini - Google DeepMind</footnote>

<footnote n="10">¹⁰ Navigating the AI Landscape of 2024: Trends, Predictions, and Possibilities | by Vincent Koc | Jan, 2024 | Towards Data Science.</footnote>

<footnote n="11">¹¹ Al Agents And The Era Of The Intelligent Interface (forbes.com)</footnote>

<footnote n="12">¹² Ina Fried, Ryan Heath, October 2023, 1 big thing: The push to run generative Al on devices; https://www.axios.com/newsletters/axios-ai-plus-b5dec4be0efa-41c5-b2e5-b2e747846c56.html?chunk=0&utm_term=emshare#story0.</footnote>

<footnote n="13">¹³ Annie Liao, The Rise of Autonomous Al Agents; Debundling the Market Landscape, 2023, The Rise of Autonomous Al Agents; Debundling the Market Landscape | by Annie Liao | Aura Ventures | Medium.</footnote>

<footnote n="14">¹⁴ Matt Schlicht, The Complete Beginners Guide To Autonomous Agents, Everything you need to know, April 2023, https://www.mattprd.com/p/the-complete-beginners-guide-to-autonomous-agents.</footnote>

<footnote n="15">¹⁵ swyx & Alessio, The Anatomy of Autonomy: Why Agents are the next Al Killer App after ChatGPT, Auto-GPT/BabyAGI Executive Summary, a Brief History of Autonomous Agentic Al,</footnote>

<footnote>and Predictions for Autonomous Future, Apr 19, 2023, https://www.latent.space/p/agents?utm_source=profile&utm_medium=reader2.</footnote>

<footnote n="16">¹⁶ Significant-Gravitas/AutoGPT: AutoGPT is the vision of accessible Al for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters. (github.com)</footnote>

<footnote n="17">¹⁷ miurla/babyagi-ui: BabyAGI UI is designed to make it easier to run and develop with babyagi in a web app, like a ChatGPT. (github.com).</footnote>

<footnote n="18">¹⁸ swyx & Alessio, The Anatomy of Autonomy: Why Agents are the next Al Killer App after ChatGPT, Auto-GPT/BabyAGI Executive Summary, a Brief History of Autonomous Agentic Al, and Predictions for Autonomous Future, Apr 19, 2023, https://www.latent.space/p/agents?utm_source=profile&utm_medium=reader2.</footnote>

<footnote n="19">¹⁹ The State of Al Agents. Over the last few months, we have... | by Tereza Tizkova | E2B — Cloud runtime for Al agents | Medium.</footnote>

<footnote n="20">²⁰ Lilian Weng, LLM Powered Autonomous Agents, June 23, 2023, https://lilianweng.github.io/posts/2023-06-23-agent.</footnote>
<figure><img src="image_8.png" /></figure>

In a recent blog,<fnref n="21" /> Bill Gates notes: "In the next five years (....). You won't have to use different apps for different tasks. You'll simply tell your device, ....They'll replace search sites because they'll be better at finding information and summarizing it for you. They'll replace many e-commerce sites because they'll find the best price for you and won't be restricted to just a few vendors. They'll replace word processors, spreadsheets, and other productivity apps. Businesses that are separate today—search advertising, social networking with advertising, shopping, productivity software—will become one business."

## The Market Ecosystem of AI Agents

The opportunities are enormous and advancements in the field fast. The global autonomous AI and autonomous AI Agents Market size is estimated to reach USD 28.5 billion by 2028, from 4.8 billion in 2023.<fnref n="22" />

platforms—the technologies that apps and services are built on. Android, iOS, and Windows are all platforms. Agents will be the next platform."

Open AI's beta of "Create a GPT," is groundwork for such a platform specifically for AI technologies. The Open AI's GPT framework is designed to enable users to tailor and develop their own AI agents, a customized version of ChatGPT that 'combine instructions, extra knowledge, and any combination of skills'.<fnref n="23" /> Whether there will be one single or very few companies that dominate the agents' business is to be seen. In Gates' opinion, "there will be many different AI engines available".<fnref n="24" />

Overall, predictions foresee enormous changes in the AI market, including the importance of vector databases,<fnref n="25" /> control over the vertical AI value chain,<fnref n="26" /> Al wearables,<fnref n="27" /> Al agents interacting with other Al Agents,<fnref n="28" /> and the emergence of AI marketplaces for agents, each of those potential developments impacting the need and feasibility of

<figure><img src="image_9.png" /><figcaption>AI Agents Market Map - December 2023</figcaption></figure>

Source: Navigating the AI Landscape of 2024: Trends, Predictions, and Possibilities

data portability.<fnref n="29" />

Further, Gates notes: "The efficacy and autonomy of AI agents is contingent upon their seamless integration into computing industry, we talk about

Although this is a nascent area, market analysts suggest three key layers to the Autonomous AI Agent Market: AgentOp platforms (LLMs, memory,

<footnote n="21">21 https://www.gatesnotes.com/Al-agents</footnote>

<footnote n="22">22 https://www.marketsandmarkets.com/Market-Reports/autonomous-ai-and-autonomous-agents-market-208190735.html?utm_source=prnewswire&utm_medium=referal&utm_campaign=paidpr.</footnote>

<footnote n="23">23 Introducing GPTs (openai.com)</footnote>

<footnote n="24">24 https://www.gatesnotes.com/Al-agents</footnote>

<footnote n="25">25 Vector Database used in AI | Exxact Blog (exxactcorp.com).</footnote>

<footnote n="26">26 Exploring opportunities in the gen AI value chain | McKinsey.</footnote>

<footnote n="27">27 Darius Nahavandi, Roohallah Alizadehsani, Abbas Khosravi (Seni or IEEE), U Rajendra Acharya (Senior IEEE), Application of artificial intelligence in wearable devices: Opportunities and challenges, Computer Methods and Programs in Biomedicine, Volume 213, January 2022.</footnote>

<footnote n="28">28 Agent Protocol: Developers community setting a new standard (e2b.dev).</footnote>

<footnote n="29">29 Navigating the AI Landscape of 2024: Trends, Predictions, and Possibilities | by Vincent Koc | Jan, 2024 | Towards Data Science</footnote>
<figure><img src="image_10.png" /></figure>

tools, communication protocols), applications (general purpose, industry vertical), and services (build your own agent, agent marketplace).<fnref n="30" />

multiple products. The communication between and integration of agents raises questions about interoperability—whether the future will lean

<figure><img src="image_11.png" /><figcaption>Source: Aura Ventures Emerging AI Agent Landscape Map</figcaption></figure>

Handling more complex interactions—like orchestrating travel plans or managing communications—requires a heightened level of interoperability, and possibly data portability. There are two parallels that can be drawn with early stages of OTT platforms: 1) autonomous AI agents will need to communicate with each other in these scenarios. It is doubtful at this stage whether attempts to develop protocols will be successful,<fnref n="31" /> or whether providers will seek to achieve network effects; 2) the development and integration of AI agents into various products resemble early stages of OTT platforms, which disrupted traditional telecom by offering services like messaging and voice calls over the internet. This parallel extends to how AI agents, initially standalone services, are now becoming embedded features within a broad range of devices, enhancing functionality and user experience.

Just as OTT platforms evolved to include features such as chat and video conferencing within broader productivity suites, AI agents are transitioning from isolated applications to integral components of

towards open, interoperable ecosystems that facilitate seamless user experiences across devices and platforms, or towards proprietary systems that prioritize corporate control and economies of scale. The path chosen will significantly shape the role and impact of AI agents in the digital ecosystem.

## Artificial General Intelligence

As a tangential note in the broader discussion on data portability in the era of autonomous AI agents, it's important to recognize the dynamic and uncertain nature of the AI landscape. The eventual architecture of AI is still a matter of debate, with opinions divided on whether it will culminate in a single Artificial General Intelligence (AGI), multiple AGIs, or a multifaceted ecosystem of specialized AIs.<fnref n="32" /> This uncertainty also extends to the interactions between AI platforms and agents within this future ecosystem. Considering AI as foundational infrastructure, the debate between a singular versus multiple AGIs sheds light on the modalities of information access and sharing among AI systems. A single AGI model suggests a unified,

<footnote n="30">³⁰ Annie Liao, The Rise of Autonomous AI Agents; Debundling the Market Landscape, 2023, *The Rise of Autonomous AI Agents: Debundling the Market Landscape* | by Annie Liao | Aura Ventures | Medium.</footnote>

<footnote n="31">³¹ Agent Protocol: Developers community setting a new standard (e2b,dev).</footnote>

<footnote n="32">³² See as an example for the discussion: Morris M.R.; et al. (2023) "Levels of AGI: Operationalizing Progress on the Path to AGI". Retrieved January 1, 2024.</footnote>
<figure><img src="image_12.png" /></figure>

centralized form of intelligence that could
streamline data utilization but also raise concerns
over monopolistic dominance. Conversely, a
landscape populated by multiple AGIs would
indicate a more decentralized framework,
emphasizing the importance of data portability for
maintaining control over data and fostering
competition among diverse entities. The direction of
development will have profound implications on
how AI agents engage with and influence data
ecosystems.

Open versus Closed Al agents

Closely related within the broader examination of autonomous Al agents is the ongoing debate between an open versus closed generative Al ecosystem. This debate subtly intertwines with the concept of data portability, suggesting that the extent of openness could potentially influence the ease with which Al systems and their capabilities are transferred and adapted across different platforms and environments. Current efforts to develop protocols for interoperability for example exist in the open model environment.<fnref n="33" />

<figure><img src="image_13.png" /></figure>

Source: https://github.com/e2b-dev/awesome-ai-agents

The degree of ecosystem openness bears implications for the future data portability in the Al ecosystem. However, the precise nature of this impact remains uncertain at this juncture, underscoring the need for further investigation into how openness or its lack thereof will shape the development, accessibility, and portability considerations surrounding autonomous agents.

Portability matters

The developing landscape of Al agents and the
nascent (platform) markets they inhabit, will be
further shaped by a complex mix of technological,
regulatory, societal, and economic factors.<fnref n="34" />

<figure><img src="image_14.png" /></figure>

Adoption curve cycle for Al Agents
Source: Annie Liao 2023, AURAVENTURES

Aspects that enhance or inhibit trust in the Al agent
ecosystem, such as privacy, security and safety
issues, as well as user autonomy are essential for
fostering an ecosystem where Al agents can reach
utmost capabilities in terms of versatility. The
potential risks of data lock-in, wherein users may
find themselves tethered to a particular Al agent due
to the bespoke nature of their inputs and data,
highlights the need for mechanisms that allow users
the flexibility to switch between Al providers without
losing the personalized value embedded in their
data.

<footnote n="33">³³ Agent Protocol: Developers community setting a new standard (e2b.dev).</footnote>

<footnote n="34">³⁴ Annie Liao, The Rise of Autonomous Al Agents; Debundling the Market Landscape, 2023, The Rise of Autonomous Al Agents.</footnote>
<figure><img src="image_15.png" /></figure>

# Existing Data Portability Rights Under EU Law

Data portability is a feature in three significant regulations, the GDPR, the DMA, and the Data Act. These regulations collectively enhance user control over personal data and foster market competition by allowing the transfer of data between services.

Antitrust and data privacy laws, despite their different primary objectives, exhibit overlapping policy interests: both legal frameworks advocate for data portability, seeing it as advantageous for both privacy and competition.³⁵ Data portability is seen as a catalyst for data-driven competition, facilitating easier consumer transitions between services and enabling new entrants to access data necessary for market entry or expansion.³⁶

This brief overview identifies the limitations of each law and reviews their adequacy amid evolving market dynamics.

## Article 20.1 GDPR

The GDPR was the first framework that introduced the right to data portability, unknown in the preceding Data Protection Directive.³⁷ This novel provision, unlike the familiar right to access, is designed to enable data subjects to obtain and reuse their personal data across different services.

The right intends to provide to data subjects more control over their data.³⁸ This control provided under the GDPR falls short of 'ownership', and is a

“carefully constrained type of control”.³⁹ While individuals have the right to receive their personal data, which they have provided to a controller, in a structured, commonly used, and machine-readable format, the right is only applicable when the personal data processing is based on the individual's consent or is necessary for contract performance. It does not apply when data processing is based on legal obligations, public interest, or official authority. The right to data portability also does not extend to data processed in the exercise of public duties or where it conflicts with other individuals' rights and freedoms. Data controllers are only encouraged to create interoperable data formats,<fnref n="40" /> there is no mandatory obligation for them to develop technical solutions for data transfer where such measures do not exist. This approach has been criticized to potentially deter controllers from establishing standards, given that the obligation to transfer data is contingent upon technical feasibility.<fnref n="41" /> This approach was criticized since alternative routes, e.g., narrowing its application to certain electronic processing systems, where a significant amount of user lock-in occurs, would have been more targeted and effective.<fnref n="42" /> Critics have pointed out that this limitation would enable organizations to avoid data portability requests altogether.<fnref n="43" />

While the Article 29 Working Party specified that data “provided by” the data subject also includes data resulting from the observation of his activity, such as raw data processed by a smart meter or other types of connected objects, activity logs,

Debundling the Market Landscape | by Annie Liao | Aura Ventures | Medium.

³⁵ Graef, Inge and Husovec, Martin and Purtova, Nadezhda, Data Portability and Data Control: Lessons for an Emerging Concept in EU Law (December 15, 2017). German Law Journal 2018, vol. 19 no. 6, p. 1359-1398, Tilburg Law School Research Paper No. 2017/22, TILEC Discussion Paper No. 2017-041, Available at SSRN: https://ssrn.com/abstract=3071875 or http://dx.doi.org/10.2139/ssrn.3071875

³⁶ Douglas, Erika, Digital Crossroads: The Intersection of Competition Law and Data Privacy (July 6, 2021). Temple University Legal Studies Research Paper No. 2021-40, Available at SSRN: https://ssrn.com/abstract=3880737 or http://dx.doi.org/10.2139/ssrn.3880737

³⁷ Directive 95/46/EC (General Data Protection Regulation), OJ L 119, 4.5.2016, p. 1–88. Repealed.

³⁸ Recital 68 GDPR.

³⁹ Scassa, Teresa, Data Ownership (September 4, 2018). CIGI Papers No. 187, Ottawa Faculty of Law Working Paper No. 2018-26, https://ssrn.com/abstract=3251542.

<footnote n="40">⁴⁰ Recital 68.</footnote>

<footnote n="41">⁴¹ Borgogno, Oscar and Colangelo, Giuseppe, Data Sharing and Interoperability Through APIs: Insights from European Regulatory Strategy (November 21, 2018). A modified version of the paper is forthcoming in "Computer Law & Security Review" 2019, Stanford-Vienna European Union Law Working Paper No. 38, Available at SSRN: https://ssrn.com/abstract=3288460 or http://dx.doi.org/10.2139/ssrn.3288460</footnote>

<footnote n="42">⁴² Graef, Inge and Verschakelen, Jeroen and Valcke, Peggy, Putting the Right to Data Portability into a Competition Law Perspective (2013). Law: The Journal of the Higher School of Economics, Annual Review, 2013, pp. 53-63, Available at SSRN: https://ssrn.com/abstract=2416537</footnote>

<footnote n="43">⁴³ Solove, Daniel J., The Limitations of Privacy Rights (February 1, 2022). 98 Notre Dame Law Review 975 (2023), GWU Legal Studies Research Paper No. 2022-30, GWU Law School Public Law Research Paper No. 2022-30, Available at SSRN: https://ssrn.com/abstract=4024790 or http://dx.doi.org/10.2139/ssrn.4024790</footnote>
<figure><img src="image_16.png" /></figure>

history of website usage or search activities,<fnref n="44" /> the right to data portability remains rather narrow in scope. One drawback is that it fails to address inferences drawn from analyzing personal data, such as algorithmically or statistically generated categorizations or profiles. In the realm of the Internet of Things (IoT), which relies on drawing inferences to understand the user's context and deliver suitable services; this limitation is significant.<fnref n="45" />

During its initial assessment of the GDPR in June 2020,<fnref n="46" /> the European Commission concluded that data portability has not yet reached its full potential. It noted “the need to address difficulties such as lack of standards enabling the provision of data in a machine-readable format, to increase the effective use of the right to data portability, which is currently limited to a few sectors (e.g. banking and telecommunications). This could be done notably through the design of appropriate tools, standardised format and interfaces.” The International Association for Privacy Professionals undertook a survey about the frequency of data subjects exercising their right to data portability in 2022. It appears that the right to data portability is seldomly exercised by individuals and is rarely the subject of litigation.<fnref n="47" />

## Article 6.9 DMA

The DMA, a regulatory framework designed to promote fairness and contestability in digital markets with gatekeeper platforms, is set to be enforceable on 6 March 2024. The DMA will introduce a comprehensive list of obligations and prohibitions for identified gatekeepers, establishing behavioral standards towards other businesses and end-users.

The right to data portability in the DMA mandates that gatekeepers must ensure end users can port their data effectively. This obligation encompasses the fundamental components found in the GDPR right to data portability, which are (a) the right for users to receive their data and (b) the right to transfer this data to a third party,<fnref n="48" /> but it differs in consequent areas.

While GDPR pursues the goal of providing data subjects with more control over their data, the DMA aims at enhancing fairness and contestability as market regulatory objectives. Regarding its scope, the right to data portability applies also to non-personal data, and there is no obvious other restriction to data. Hence, data ‘provided’ by using the service, inferred data, and observed data is covered by the data portability provision of the DMA.<fnref n="49" /> Furthermore, the DMA data portability right does not make any restriction as to the legal basis of data processing and covers any processing of (personal) data based on other than consent or contract. The range of data encompassed by the DMA's obligation for data portability thus extends beyond what is covered by the GDPR's right to data portability, notably to legal entities or business users.

On the other hand, it only applies to designated core service platforms. Gatekeepers' core service platforms (CSPs) must adopt “high quality technical measures, like application programming interfaces (APIs),” facilitate the continuous and real-time portability of data.

With the introduction of the data portability right in the DMA, the Commission has been addressing one of the key obstacles to the uptake of the data portability right identified in its 2020 assessment of the GDPR. This involves overcoming the technical barriers that have been noted, as it mandates to

<footnote n="44">⁴⁴ Guidelines on the right to data portability, Adopted on 13 December 2016 As last Revised and adopted on 5 April 2017, WP 242 rev.01.</footnote>

<footnote n="45">⁴⁵ Dr Lachlan Urquhart1, Neelima Sailaja, Prof Derek McAuley 2018 Realising the Right to Data Portability for the Domestic Internet of Things, arXiv:1801.07189 [cs.HC], Horizon, School of Computer Science, University of Nottingham.</footnote>

<footnote n="46">⁴⁶ Communication from the Commission to the European Parliament and the Council: Data protection as a pillar of citizens' empowerment and the EU's approach to the digital transition -</footnote>

<footnote>two years of application of the General Data Protection Regulation, Brussels, COM(2020) 264 final.</footnote>

<footnote n="47">⁴⁷ https://iapp.org/news/a/data-portability-in-the-eu-an-obscure-data-subject-right/</footnote>

<footnote n="48">⁴⁸ Recital 59 DMA.</footnote>

<footnote n="49">⁴⁹ Geradin, Damien and Bania, Konstantina and Karanikioti, Theano, The interplay between the Digital Markets Act and the General Data Protection Regulation (August 29, 2022). Available at SSRN: https://ssrn.com/abstract=4203907 or http://dx.doi.org/10.2139/ssrn.4203907. See also Bundeskartellamt - Homepage - B6-22/16.</footnote>
<figure><img src="image_17.png" /></figure>

provide the data in reusable format. However, the lack of detailed guidance on the required data formats and mechanisms for data transfer means that the obstacle has not been fully overcome;<fnref n="50" /> the lack of standardization remains.<fnref n="51" />

To date, the Commission has designated the following services to be gatekeepers: 3 operating systems (Google Android, iOS, Windows PC OS), 2 web browsers (Chrome and Safari), 1 search engine (Google), 4 social networks (Facebook, Instagram, LinkedIn, TikTok), 1 video sharing platform (YouTube), 3 online advertising services (Amazon, Google, and Meta), 2 large communication services (Facebook Messenger and WhatsApp), and 6 intermediation platforms (Amazon Marketplace, Google Maps, Google Play, Google Shopping, iOS App Store, Meta Marketplace).<fnref n="52" /> On 13 February 2024, the Commission ended its market investigation that followed the rebuttal of Microsoft and Apple, and exempted iMessage, Bing, Edge and Microsoft Advertising and found these services do not qualify as gatekeepers.<fnref n="53" />

Virtual assistants are identified as one of the ten distinct types of Core Platform.<fnref n="54" /> They are defined as software that can process demands, tasks or questions, including those based on audio, visual, written input, gestures or motions, and that, based on those demands, tasks or questions, provides access to other services or controls connected physical devices.<fnref n="55" /> Virtual assistants were not included in the initial draft of the DMA but have been added in the final text, as a consequence of the EC's consumer IoT sector inquiry.<fnref n="56" /> Virtual assistants, like Siri<fnref n="57" /> or Alexa,<fnref n="58" /> fall squarely into the definition; they may have been the reason that virtual assistants made it into the DMA at a later stage of the legislative process.

The current definition of services like virtual assistants fails to adequately reflect the market's dynamics, as the landscape of virtual assistants is continuously expanding. Advanced natural language processing (NLP) and machine learning (ML) services align with the core function of virtual assistants and serve as a foundation for applications that do, making services such as ChatGPT, Claude.Al or Gemini a versatile component in the development of sophisticated virtual assistant systems. A virtual assistant can thus be considered a simple form of an autonomous AI agent. While the complexity and autonomy of AI agents varies widely, from simple rule-based systems to advanced AI systems capable of learning and adapting in complex environments. Virtual assistants, while autonomous to an extent, typically do not possess the advanced cognitive functions of more sophisticated AI agents, such as deep learning models that can analyze and learn from large datasets to improve their performance over time without being explicitly programmed for each task. At the same time, they may be replaced over time.

The convergence between search and foundation models<fnref n="59" /> also plays into this transformational landscape, as do autonomous AI agents and the platform ecosystem they inhibit (including AgentOps and cloud computing services).

Whether and how the Commission will consider the dynamics of the AI agent market, the tech stack related to it and the convergence of services as described before, will be a condition for any designation in this space. The development around compliance efforts of designated gatekeepers, in particular Google search, will provide some indication here eventually.

<footnote n="50">⁵⁰ Barbara Lazarotto, The right to data portability: A holistic analysis of GDPR, DMA and the Data Act Proposal, 14-EJLT-Barbara_Lazarotto-DMA-GDPR-DA-Paper.pdf (alti.amsterdam).</footnote>

<footnote n="51">⁵¹ Gal, Michal and Rubinfeld, Daniel L., Data Standardization (June 2019). 94 NYU Law Review (2019) Forthcoming, NYU Law and Economics Research Paper No. 19-17, Available at SSRN: https://ssrn.com/abstract=3326377 or http://dx.doi.org/10.2139/ssrn.3326377.</footnote>

<footnote n="52">⁵² Commission designates six gatekeepers under the Digital Markets Act - European Commission (europa.eu).</footnote>

<footnote n="53">⁵³ Commission closes market investigations on Microsoft's and Apple's services under the Digital Markets Act (europa.eu).</footnote>

<footnote n="54">⁵⁴ Article 2(2)1.</footnote>

<footnote n="55">⁵⁵ Art. 2.12.</footnote>

<footnote n="56">⁵⁶ Tom Ovington, David Lewis, Virtual assistants and the DMA "Hey Siri, does the Digital Markets Act now apply to you?", Frontier Economics, virtual-assistants-and-the-dma.pdf (frontier-economics.com).</footnote>

<footnote n="57">⁵⁷ Siri - Apple</footnote>

<footnote n="58">⁵⁸ Amazon Alexa - Wikipedia.</footnote>

<footnote n="59">⁵⁹ As of writing, the market investigation into Bing has not yet been published.</footnote>
<figure><img src="image_18.png" /></figure>

## Data Act

On 11 January 2024, the Data Act, introducing a unified framework for data access, cloud service provider switching, and interoperability standards throughout the European Union, entered into force.<fnref n="60" /> The Data Act becomes applicable on 12 September 2025.

A key feature of the Data Act is the establishment of user rights to access data from connected products and services, including data stored on devices or managed by connected service providers. It covers both business-to-business (B2B) and business-to-consumer (B2C) scenarios, regardless of the data being personal under the GDPR. The Data Act seeks to expand upon the GDPR's rights to access and data portability with more detailed regulations, ensuring it does not conflict with the GDPR and the ePrivacy Directive 2002/58, especially concerning the individual data rights. A reason behind the right to data portability in the Data Act is promoting competition, and in this context there is no justification for restricting the right to data portability solely to personal data. Additionally, lock-in problems occur in the context of industrial data, where suppliers may encounter similar challenges.<fnref n="61" />

Manufacturers and service providers must design their offerings to allow users direct access to generated or stored data, including metadata, in a manner that is easy, secure, and in a widely used, machine-readable format. The aim is to facilitate switching between data processing services, including porting all its digital assets, including data, to other providers and to continue to use them in the new environment while benefiting from functional equivalence. Meta-data, generated by the customer's use of a service, should also be portable pursuant to this Regulation's provisions on switching.<fnref n="62" />

There are associated obligations for data holders, including transparency about the data generated,

storage practices, access, retrieval, and deletion processes. In cases where direct access to data is not possible, alternative means must be provided, potentially involving third-party access under certain conditions. The Data Act also introduces specific provisions to protect trade secrets, ensuring a balance between data access and confidentiality. It restricts data usage by both data holders and users, particularly concerning unfair competition and profiling, to safeguard against misuse. Importantly, the users' right to have a data holder share the data with a third party does not apply to the largest digital platforms offering core platform services in Europe, the designated gatekeepers under the DMA. Third parties cannot make the data they receive from data holders available to gatekeepers. The Data Act approach limits users' right to choose how to make use of their data, which is problematic.

The regulation encompasses connected products and related services and broadly defines data holders. The Data Act acknowledges the increasing role of virtual assistants in digitizing consumer and professional environments and easy-to-use interface to play content, obtain information, or activate products connected to the internet. Thus, the Data Act explicitly includes virtual assistants by the data access rights. However, it limits the IoT data access right to tangible, movable items capable of data communication, excluding digital content and services without a tangible medium. Data produced by the virtual assistant which are unrelated to the use of a connected product or related service are not covered by this Regulation.<fnref n="63" />

This limitation may undermine the comprehensiveness of the regulation in a digital ecosystem where the lines between physical and digital services are increasingly blurred. Virtual assistants often interact with both physical devices and digital services, meaning that significant aspects of their functionality and the data they generate might fall outside the regulatory scope. Such a narrow focus might not fully address the broader

<footnote n="60">⁶⁰ Regulation (EU) 2023/2854 of the European Parliament and of the Council of 13 December 2023 on harmonised rules on fair access to and use of data and amending Regulation (EU) 2017/2394 and Directive (EU) 2020/1828 (Data Act).</footnote>

<footnote n="61">⁶¹ Drexel, Josef, Designing Competitive Markets for Industrial Data - Between Propertisation and Access (October 31, 2016). Max</footnote>

<footnote n="62">⁶² Recital 72.</footnote>

<footnote n="63">⁶³ Recital 23.</footnote>
<figure><img src="image_19.png" /></figure>

impacts of data collection and processing practices
of virtual assistants on user rights and market
dynamics, and in particular in the light of
autonomous AI agents.

Data Portability for AI Agents Limited

After examining various regulatory approaches to data portability within the context of an evolving ecosystem of AI agents, it becomes evident that data portability remains constrained. Although the GDPR encompasses all services processing personal data, it faces notable limitations, most significantly its exclusion of inferred data. This category is crucial for AI agents designed to understand human wellbeing and learn from users' preferences. As for the DMA, the critical issue lies in determining how evolving AI agent ecosystems align with the definitions of core platform services. The Data Act, while specifically targeting IoT device manufacturers and service providers, omits digital content and services that lack a tangible medium.

Consequently, despite the apparent overlaps
between the GDPR, DMA, and Data Act, along with
the extension to non-personal data, significant gaps
remain in the application of data portability rights.
For individuals utilizing AI agents as companions, the
practical benefits of these rights are still constrained.

Some authors have suggested that the current legal
conceptions of data portability still mainly serve the
interests of service providers and data controllers
rather than individual end users.<fnref n="64" /> They advocate for
empowering individuals with control over their data,
proposing a transformative approach to data
portability that places data in a secure personal
space under individual control.<fnref n="65" /> However, this
approach necessitates a fundamental
transformation in the infrastructure and
architecture of the data economy, a shift not

underpinned by the recent initiatives concluded
under the European Data Strategy,<fnref n="66" /> including the
Data Governance Act,<fnref n="67" /> which anticipates the
evolution of data intermediation services.

Here, it is argued that one of the most significant
issues contributing to the limitation is the significant
gap between existing legal frameworks and the swift
pace of technological and market developments.

Concluding Recommendations

This article identified a disconnect between the technological and market trends and current data portability rights. This, in addition to the described limitations in the scope of each right, may decrease the effectiveness of data portability. There is an urgent need to consider more flexible and adaptive regulatory solutions. Adopting a co-regulatory approach, like in the Digital Services Act<fnref n="68" /> for risk mitigation and under the AI Act<fnref n="69" /> for general-purpose AI, both relying on codes of conducts, would enable legislators to adapt to technological changes quickly and enforce more effectively.

There should be a thorough analysis of where
existing data portability rights may fall short due to
technological and market developments, and how
legal limitations constrain user autonomy in an
increasingly personalized digital ecosystem. As
voices of a potential reform of the GDPR grow
louder, it cloud be an opportunity to assess data
portability more holistically and caters to the needs
and rights of users in an era increasingly influenced
by AI technologies. A more tailored data portability
right focused on use cases where overcoming lock-
ins would be most beneficial to users or where it is
most important to user autonomy rather than
imposing it generally, and ensuring that technical
solutions are developed by industry as part of a co-
regulatory framework, may ultimately be more

<footnote n="64">⁶⁴ Fenwick, Mark and Jurcys, Paul and Minssen, Timo, Data Portability Revisited: Toward the Human-Centric, AI-Driven Data Ecosystems of Tomorrow (June 10, 2023). Available at SSRN: https://ssrn.com/abstract=4475106 or http://dx.doi.org/10.2139/ssrn.4475106.</footnote>

<footnote n="65">⁶⁵ Fenwick, Mark and Jurcys, Paul and Minssen, Timo, Data Portability Revisited: Toward the Human-Centric, AI-Driven Data Ecosystems of Tomorrow (June 10, 2023). Available at SSRN: https://ssrn.com/abstract=4475106 or http://dx.doi.org/10.2139/ssrn.4475106.</footnote>

<footnote n="66">⁶⁶ https://digital-strategy.ec.europa.eu/en/policies/strategy-data.</footnote>

<footnote n="67">⁶⁷ Regulation (EU) 2022/868 of the European Parliament and of the Council of 30 May 2022 on European data governance and amending Regulation (EU) 2018/1724 (Data Governance Act).</footnote>

<footnote n="68">⁶⁸ Regulation (EU) 2022/2065 of the European Parliament and of the Council of 19 October 2022 on a Single Market For Digital Services and amending Directive 2000/31/EC (Digital Services Act) (Text with EEA relevance).</footnote>

<footnote n="69">⁶⁹ Text not yet published.</footnote>
<figure><img src="image_20.png" /></figure>

successful.<fnref n="70" /> Demanding the development of a Code of Conduct for identified services represent this adaptive approach, enhancing data portability by industry-led technical implementations where locks are identified. The Data Transfer Initiative is a blueprint for how these codes could be developed. This would not only support a more effective implementation of data portability, but also promote transparency, control, and a shift towards a regulatory and technological environment that respects and amplifies individual rights and autonomy.

Additionally, it may be helpful to develop a European rulebook for data portability, guiding the application of the right and the interaction between the different regulatory approaches and monitor the evolving needs.

Last, user autonomy will also be driven by other technologies such as the metaverse. A broader conceptual assessment of data portability towards identity portability would be desirable to advance a human-centric future.

<footnote n="70">⁷⁰ Graef, Inge and Verschakelen, Jeroen and Valcke, Peggy, Putting the Right to Data Portability into a Competition Law Perspective (2013). Law: The Journal of the Higher School of</footnote>

<footnote>Economics, Annual Review, 2013, pp. 53-63, Available at SSRN:
https://ssrn.com/abstract=2416537</footnote>
<figure><img src="image_21.png" /></figure>

<figure><img src="image_22.png" /></figure>

Cornelia Kutterer is a Research Fellow at the Chair. Her research interest is the interplay of technology law and advances in fairness, accountability, and transparency of socio-technical systems as well as applied AI governance and ethics in organizations. She is a qualified German lawyer and holds a law degree from the University of Hamburg and a master's degree of information technology and telecommunication law from the Strathclyde university in Glasgow. She worked for 15 years at the forefront of digital innovation and responsible policies at Microsoft, including on responsible AI, privacy, human rights, safety, law enforcement, content, and market regulation. Before joining Microsoft, she led the legal department of the European Consumer Organisation BEUC and gained experience in a top 10 law firm and the European Parliament.

**Cover Photo:** Created by **DALL-E**, prompt by **Theodore Christakis:** "*Create an image that symbolizes the concept of data portability between AI applications. Depict a swirling vortex or network of interconnected pathways, representing the flow of data. Surround the vortex with imagery representing the human element, such as silhouettes or abstract figures, to convey the human aspect of the data transfer process*"

**To cite this article:** C. Kutterer, What if you move on from your AI companion? Data portability rights in the era of autonomous AI agents, AI Regulation Papers 24-02-3, [AI-Regulation.com](https://AI-Regulation.com), February 27th, 2024.

These statements are attributable only to the author, and their publication here does not necessarily reflect the view of the other members of the AI-Regulation Chair or any partner organizations.

This work has been partially supported by MIAI @ Grenoble Alpes, (ANR-19-P3IA-0003)