# The Rise of Cybernetic Citizenship

Wessel Reijers, Liav Orgad, Primavera de Filippi

► To cite this version:

Wessel Reijers, Liav Orgad, Primavera de Filippi. The Rise of Cybernetic Citizenship. Citizenship Studies, 2022. hal-03882012

HAL Id: hal-03882012

https://hal.science/hal-03882012

Submitted on 2 Dec 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# The Rise of Cybernetic Citizenship

**Wessel Reijers**

University of Vienna; European University Institute; Technion – Israel
Institute of Technology

**Liav Orgad**

European University Institute; WZB Berlin Social Science Center;
Reichman University; Peking University School of Transnational Law
Contact: Liav.orgad@wzb.eu

**Primavera de Filippi**

CNRS; European University Institute

## Abstract

The global COVID-19 pandemic demonstrates how states and companies mobilise new sociotechnical systems to track, trace, evaluate, and modulate the behaviour of citizens. This development illustrates an already-existing transformation of citizenship governance, which this article captures as the move to “cybernetic citizenship.” Part I explores the concept of cybernetic citizenship, providing an overview of the concepts of “cybernetic” and “citizenship” and synthesising these in a discussion of the cybernetic modulation of citizenship. Part II presents the rise of cybernetic citizenship in the urban realm, zooming in on the case of the Chinese Social Credit System and the way it affects civic life in the urban realm. Part III turns into the normative implications of cybernetic citizenship, arguing that it confronts the idea of citizens as equal, free, and vigilant. It challenges equality by turning rights into ends, freedom by turning status into process, and civic virtue by turning excellence into effectiveness.

## Introduction

Systemic crises like wars and natural disasters tend to propel new waves of technological innovation. This observation is demonstrated by the global COVID-19 pandemic, which has led to massive data collection on citizens, track- and trace applications, and automated means of intervention. A fascinating example comes from China, where the pandemic initially started, which has deployed its emerging Social Credit System (SCS) to fight the pandemic (Knight and Creemers 2021). The SCS uses technological
innovations to rate (and often score) citizens to improve public order and trust. For instance, it has made it more socially costly to engage in behaviours that might lead to spreading the disease, such as hiding virus symptoms or evading medical treatment. At the same time, the SCS has been used to relax some restrictions on behaviours that would otherwise be more costly, for example, enforcing financial obligations that are harder to meet due to the economic downfall. The costs and benefits imposed by the SCS are not only monetary, but rather attached to one's status, rights, and duties as a citizen.

This article investigates the emerging form of cybernetic citizenship as illustrated by the SCS. It builds on a growing body of literature that addresses the transformation brought about by technology to the concept of citizenship (digital citizenship, Mossberger, Tolbert, & McNeal, 2008; e-citizenship, Dumbrava, 2015; algorithmic citizenship, Cheney-Lippold, 2016). At first, literature focused on the way technologies shaped existing, stable aspects of citizenship, for instance, by looking at the digitalisation of state services and online civic education. More recently, there has been a turn toward questioning the foundations of liberal citizenship, as conceptualised in the tradition that follows T.H. Marshall (1950). Most notably, Marion Fourcade has coined the term "ordinal citizenship," which implies a transformation of citizenship due to the emergence of large sociotechnical systems that measure, quantify, and evaluate citizens (2021). Some utopian and dystopian scenarios related to the shift in the essence of citizenship due to cybernetic systems were already discussed decades ago by philosophers of technology like Lewis Mumford (1965) and Langdon Winner (1969, 1978). This article continues this strain of thought and draws from recent work in the humanities (Pickering 2010; Hui 2019) that signals the revival of cybernetic thinking.

The article's central claim is that, *empirically*, we are witnessing the rise of cybernetic citizenship as demonstrated by the Chinese SCS and similar sociotechnical systems around the globe; *normatively*, this development has urgent ethical, legal, and political implications. First, cybernetics professes to be a general epistemology that takes the networked computer as its model (Winner 1969). While citizenship in the current international order of states is still primarily based on the mechanised worldview of enlightenment philosophy, cybernetics challenges this worldview and thereby the sense of citizenship based on it. Second, cybernetics accounts for how sociotechnical systems learn and adapt their behaviour according to contingent inputs (Krivý 2018). Hence, it explains how citizens' seemingly accidental and contingent behaviours are connected and harnessed for citizenship governance. And third, initially emerging in the wake of computerised systems, cybernetics speaks to a much longer history of philosophical thinking that mobilises the notion of a living machine to understand the institution of citizenship (Mumford 1965).

The article proceeds as follows. Part I develops the concept of cybernetic citizenship, synthesising the outlines of “cybernetics” and establishing perspectives on “citizenship”. Part II describes the rise of cybernetic
citizenship through the lens of the application of the SCS in the urban realm
in China. Part III develops a normative analysis of cybernetic citizenship,
arguing that it erodes core distinctions that inform modern citizenship:
between rights and ends, status and process, and excellence and effectiveness.

# I. Concept

This part develops the concept of cybernetic citizenship. First, it provides an overview of the cybernetic perspective and how it is used to understand different phenomena. Second, it outlines different established perspectives on citizenship. Third, it shows how cybernetic citizenship modulates the relation between citizens and institutions in terms of space, time, and social interaction.

## 1. Cybernetics

Cybernetics can be understood in two ways: a scientific discipline and a
general worldview. As a scientific discipline, cybernetics emerged at the end
of World War II. A varied, interdisciplinary group of scholars, including the
mathematician Norbert Wiener, psychiatrist Ross Ashby, and anthropologists
Margaret Mead and Gregory Bateson, convened around a shared scientific
view of the world that encompassed human and non-human elements in a
communication network. This view arose in the wake of the development of
early computer systems, mainly for military purposes, which gathered
information from the environment and translated the data into behavioural
outputs by means of feedback mechanisms. Due to the deterministic qualities
of technologies at its inception, the initial focus of cybernetics was on
(relatively) closed systems with negative feedback loops. This view does not
fit current cybernetic systems, which are open, constitute positive feedback
loops, and can control but also adapt to environmental changes (Krivý 2018).

As a general worldview, early cyberneticians successfully identified and conceptualised a way of thinking that is found in today's biology, computer science, statistics, and political science. Cybernetics is a general understanding of the world that emerges from studying three seemingly distinct phenomena: life processes, machines, and communities. It combines these perspectives through the single metaphor of the *kybernetes*, the ancient Greek word for a pilot. The *kybernetes* had a practical understanding of navigating the ship and adapting to unknown circumstances, like the weather. The influence of cybernetic thinking on modern technoscience can be demonstrated by the increasing use of core cybernetic concepts—such as systems, networks, feedback, and information (Francois 1999)—and their application to the world.

As a scientific discipline or a way of thinking, cybernetics is too complex
to be comprehensively captured in this article. Yet, a brief explanation
through three core concepts—networked system, recursive feedback, and
informatisation—will suffice to make a relevant connection with citizenship.
At first, cybernetics is about the networked system. This notion refers to the way cybernetics understands *space*, which has horizontal and vertical dimensions. The horizontal dimension is captured by the idea of a *network*— a spatial arrangement of interconnected nodes connecting and exchanging information (Castells 2004, 3). Networks are relatively flat and distributed: they have no centre of command. A prime example is the Internet, the main driver of the “network society.” The vertical dimension is captured by the idea of a *system*. This idea adds control or regulation to the communication network. A system is a goal-directed assemblage of functional components. It is not simply a machine that repeats a mechanical movement, like a clock. Instead, it has a direction and can pursue an indeterminate number of ends. For this reason, systems are “quasi-living” entities (Wiener 1989); they respond to their environment in light of certain ends. Systems are open, and they interact with their environments. The system dimension introduces a hierarchy to communication networks, which have layers or components of networks that hierarchically relate to one another. An example is DNS (Domain Name System), which is nested within the Internet architecture (Galloway 2004). The hierarchy inherent to DNS is not necessarily top-down, based on a command-and-control structure. On the contrary, most systems that operate on the Internet today produce bottom-up, emergent hierarchies, such as the hierarchy of websites displayed after a Google Search query.

Next, cybernetics is about recursive feedback. This relates to the way cybernetics understands *time*, how networked systems operate and give rise to a particular temporality (Hui 2019). Recursive feedback allows systems to have a “memory” of the past and to “predict” a future. This process is often likened to forms of reflective learning, similar to how a child learns a language. The notion of recursive feedback refers to a movement between a system and its environment, whereby inputs received from the environment are processed and fed back into the system as new inputs. This is not a mechanical process of mere repetition, like the mechanical movement of a clock, but a movement of adaptation. This means that contingent inputs do not lead to noise or disruption of the system but are reflexively interpreted to anticipate future events. From a cybernetic perspective, recursive feedback can be found everywhere in our world. Animals use it to process information from their environment and adapt their behaviour to new circumstances. The human brain uses it to learn new skills. And digital systems use it to engage in processes of machine learning, in which new information is analysed based on a vast corpus of existing data. In other words, recursive feedback relies on sensing the environment, processing information, and generating behavioural change.

Finally, cybernetics relates to informatisation. It captures how systems *interact*. Interaction is premised on intelligibility, i.e., systems must understand each other, and they do so through information. In other words, cybernetic systems can only understand reality once it is accessible as
"information." The practical notion of informatisation explains how phenomena are modulated to become informative in a cybernetic sense. This modulation shows a kinship with the much-discussed idea of datafication (Lycett, 2013), which broadly means (i) eliciting the informational aspect from physical objects and events and their context, (ii) processing data in such a way that it can easily be manipulated and transferred, and (iii) recombining informational resources for particular contexts and problems. According to de Mul (1999), informatisation presupposes phenomena to be programmable (subject to calculation), synthesisable (subject to self-organisation), and manipulable (subject to artificial change of limits that govern the natural world). In short, cybernetics modulates entities so that they can be quantified, organised, and manipulated. Consider the technology of genetic modification. It captures genes as a form of code that can be programmed; it synthesises this code with a certain purpose in mind, for instance, eradicating a genetic disease; and, by doing so, artificially overcomes the limits of the natural world.

## 2. Citizenship

Like cybernetics, citizenship is a complex concept with various meanings. In the most basic sense, citizenship denotes (equal) *membership* in a self-governing political community (Joppke 2010, 160). This definition spells out citizenship as a basic aspect of the human condition, which considers humans as “political animals” (Depew 1995) capable of co-governing communities. Considering this basic notion of citizenship, even stateless refugees could potentially be “citizens” of virtual/cloud-based political refugee communities (Orgad, 2018). Yet, since the world is still dominated by territorial-based nation-states, citizenship has been chiefly conceptualised by this spatial reality (Hirschl & Shachar, 2019) and can be seen by using two perspectives.

The first is a *human rights* perspective. It focuses on the internal function of citizenship—the relation between the state and the citizens. T.H. Marshall (1950) was one of the leading proponents of the view that citizenship denotes a gradual increase in citizens’ rights, starting from civil rights, via political rights, to social rights. This historical thesis accounts for the way citizens have gained, amongst others, rights to private property, voting rights, and social rights to education and healthcare. After World War II, this idea of citizenship as giving access to rights internal to states has been extended to the idea that it could also give access to the more general category of human rights. As a citizen of France, for example, one could appeal to international or European human rights while residing in Germany. This view is paradoxical since the enjoyment of universal human rights depends on having the legal status attached to the membership of a particular nation-state. For this reason, Hannah Arendt (while not ascribing to the Marshallian view of citizenship) proclaimed that citizenship is essentially a “right to have rights” (1973, 298).
The second is a *sociological* and *anthropological* perspective. Citizenship consists of different, interrelated elements, such as rights, obligations, and identities (Joppke 2010), to which might be added civic virtue and political participation. This view is more attentive to how the relationship between states and people, both as citizens and as migrants, is constituted. It considers citizenship not only as a legal status but also as a sociocultural reality of belonging. Being a German citizen does not only mean having a German passport or rights associated with it but also sharing a historical and cultural sense of being German and acting according to some German way of doing things. It is about “citizenship in practice” (Barnes, Auburnn, and Lea, 2004), everyday social life, how people do things in a political community. This view is often used to distinguish “real” or “true” citizens from the “other” citizens.

Citizenship, thus, can be understood both as a formal legal status and de facto membership in a political community (citizenship as everyday practice).

### 3. Cybernetic Citizenship

A cybernetic understanding of citizenship is not new. It originates from what might have been the most sophisticated “living” technology in ancient Greece, the trireme—a mighty warship that played a decisive role in the Peloponnesian Wars. Plato is frequently cited as the Western thinker who considered citizenship governance in the ancient polis in analogy with (or as a metaphor for) the steering of a ship (Winner 1980, 129). In *The Republic* (488a-489e), Plato argued that unlike the owner of a trireme, the trierarch, who lacks expertise in navigation, the *kybernetes*, an experienced crew member, should be in charge of the ship. This is because the *kybernetes* would have practical skills and technical knowledge of navigating the ship. Using this analogy, Plato advocated a mode of an expert governor, who would use technoscientific knowledge and practical skills to govern the citizenry. While Plato initiated the debate, it was Aristotle who used the cybernetic analogy more decidedly to theorise the governance of the ancient Greek city, the polis (Mumford 1965, 275). In *The Politics* (1276b19-30), Aristotle spelt out the first comprehensive concept of citizenship governance in cybernetic terms:

> Just as a sailor is one of a number of members of a community, so, we say, is a citizen. And though sailors differ in their capacities (for one is an oarsman, another a captain, another a lookout, and others have other sorts of titles), it is clear both that the most exact account of the virtue of each sort of sailor will be peculiar to him, and similarly that there will also be some common account that fits them all. For the safety of the voyage is a task of all of them, since this is what each of the sailors strives for. In the same way, then, the citizens too, even though they are dissimilar, have the safety of the community as their task. But the community is the constitution. Hence the virtue of a citizen must be suited to his constitution.
Cybernetic thinking has migrated from the philosophical discussion of citizenship in Ancient Greece, via the Enlightenment philosophy of life and the organic (Kant, Schelling), through biology (Mendel, Darwin) and mathematical physics (Schrödinger, Gödel), to computer science (Turing, Wiener), dissipating into a variety of disciplines (cf. Hui 2019), including politics. And yet, despite its influence on political philosophy, cybernetic thinking has had limited practical impact on citizenship governance for most of history. The reason is that, as a technical notion of governance, it requires sophisticated conditions of human organisation, techniques, and technologies to be effectively applied. Early forms of cybernetic governance applied to relatively small-scale units. A paradigmatic example is Jeremy Bentham's Panopticon (August 2021, 16), which establishes a form of recursive feedback (the prisoners regulate their own behaviour because they might be observed) within a system that governs the network of communication (the prison architecture). Other examples applied during the Industrial Revolution, both in education through the grading system and in factory management through Frederick Taylor's scientific management theory. But while institutions like schools and factories have relatively stable boundaries and are easy to regulate, the scale and complexity of political communities have prevented, thus far, cybernetic governance from being extended to society as a whole.

The proliferation of sociotechnical systems in every aspect of life makes the cybernetic governance of citizenship a practical possibility today. Three transformations are contributing to that. First, cybernetic systems transform the *space* of citizenship, promoting both connectivity (horizontal) and systematisation (vertical). Paradoxically, the more connectivity is achieved, the more room for control and regulation is made. In terms of a network, as Varnelis and Friedberg (2013, 32) argue, the space of citizenship is shaped by the merging of the “geospatial Web” and “ubiquitous computing.” It relies on the expansion of cyberspace—the digital space through which citizens connect and their institutions, and simultaneously on a vast physical network of connected devices and sensors: phones, cameras, radio-frequency identification chips. These two aspects allow for the placing of citizens and institutions as connecting nodes in a network that functions like a geometric overlay on the physical space. In this context, citizens have to digitally check-in when they enter physical spaces so that health authorities can trace their behaviour in case a COVID-19 outbreak happens at a location they visited. Interconnectivity under such a citizenship regime is less of a free choice and more of a public imperative: being a node means being able to send and receive information. In terms of a system, cybernetic governance generates a built environment that connects people and institutions to a certain hierarchy of ends (Zandbergen and Uitermark 2019); citizens are placed into a hierarchical relation vis-à-vis the ends of political institutions, which in turn relate to a higher-order end that justifies the overall system. To be sure, citizenship has always been organised around an overarching end, at least in spirit. Yet, with the advent of cybernetic sociotechnical systems, individual
behaviours can systematically be brought into a manageable relationship with
the ends of the political community, similar to how the behaviours of the
worker can be brought into a relationship with the ends of a company.
Consider how political institutions are now able to set quantifiable metrics for
individual behaviours to align with institutional aims. Such systematisation
sets citizenship into a concrete rather than imagined relation to an overarching
end of the political community, for the sake of which citizenship is governed.

Second, cybernetic systems transform the *time* of citizenship, turning it
into a historicised process that draws from recorded memory traces and
calculated predictions. Practically, this happens through the introduction of
recursive feedback into the relations between citizens and institutions; more
information is processed between citizens and officials through government
institutions. A tax officer might scan a citizen's tax record and send it back
for corrections. This helps the tax officer "learn" about faulty behaviour and
the citizen to "learn" about the correct way to file taxes; it is a process of
mutual adaptation. Today, such recursive feedback is increasingly automated
by means of algorithmic processes. An algorithm might interpret the
behaviour of citizens based on historical patterns and assign them a risk score,
which indicates the probability that taxes will be filed correctly (Alm and
Torgler 2011). Automated processes of recursive feedback rely on vast
collections of historical data to deal with scale and complexity (Fourcade and
Johns 2020). They can use statistical methods to learn from the populations'
behaviour and use this data to predict and modulate individual behaviours. In
this way, cybernetic systems mediate the temporality of human activities (Hui
2019). To be sure, the impact of technological change on time has been a
theme of reflection since the rise of mechanisms of time and, in particular,
the use of the clock in industrial operations. Yet, earlier time machines differ
from contemporary time machines because they did not adapt to contingencies.
The clock keeps ticking at a regular pace, regardless of whether the worker
goes on strike. Cybernetic systems, in contrast, might anticipate the strike
and, as a response, allocate more free time to workers to modulate their
sentiments. This temporal logic penetrates now into the realm of citizenship.

Third, cybernetic systems transform the *interaction* between citizens and
institutions by informatising it. It becomes subject to calculation, which
means that standards for being a “good” citizen or institution are determined
through quantifiable metrics. Primary instruments for this are techniques for
ranking, rating, and scoring. Public officials like politicians are rated and
ranked, as are public institutions like universities and hospitals. Government
agencies are regularly subjected to impact assessment exercises by which they
are scored on performance indicators. Similarly, citizens get scores for
increasingly many activities: not only do they get grades for their educational
accomplishments, but they are scored as consumers, as social media users,
and even as citizens qua citizens. Such metrics rely on standardisation and
interoperability. It means that standards of measurement are brought from a
local to a global understanding, like the way different standards of time have
merged toward the global Greenwich Mean Time measurement. This shows
itself through global standards for qualification and certification, like ECTS
(European Credit Transfer and Accumulation System) points in European
education (Backer and McQuilla 2020). Interoperability, in turn, ensures that
different metrics can “speak” to one another. To illustrate, the metric of one’s
social media activity might interact with one’s credit score. Finally, these
forms of quantified assessment rely on citizens and institutions to be liable to
modulation by means of incentives. Thus, metrics can only have effects when
they are linked up with incentive structures that modulate behaviour. Citizens
can be rewarded when they engage in public service and sanctioned when
they commit traffic infractions, with the goal that they regulate themselves.

To sum up, cybernetic citizenship means that citizenship is increasingly
shaped by the cybernetic understanding of space, time, and interaction. In
terms of space, citizens are turned into communicating nodes that are put into
a systemic relation with defined ends of their political community. In terms
of time, citizenship becomes a process that draws from recorded history,
leading to a prediction of future behaviour. In terms of interaction, citizens
and institutions are informatised: subjected to quantifiable assessment, which
is standardised and made interoperable, and premised on the practical notion
that behaviour can be modulated through incentive mechanisms. This
challenges the normative basis of citizenship in terms of status, rights, and
identity, as discussed in Part III. Before we turn to the normative assessment
of cybernetic citizenship, we present how it already exists in the urban realm.

II. Case Study

This part provides empirical evidence for the rise of cybernetic citizenship,
with a focus on the urban realm. First, it gives a high-level description of how
cybernetic systems are integrated into the fabric of citizenship at the urban,
national, and transnational levels. And second, it discusses the case of the
Social Credit System, emphasising how it affects citizenship in Chinese cities.

1. *Citizenship in the Digital Age*

With the proliferation of personal computers and the Internet, the
transformation of political institutions has been mainly linked to increased
data collection and the networked exchanges of digital documents. Certain
rights and duties have become embedded in digital communication systems.
Thus, with the advent of cloud services, large internet platforms, artificial
intelligence, and advances in cryptography, new sociotechnical systems have
emerged; they can collect different types of information and process them
through statistical methods that enable these systems to learn from contingent
inputs and execute outputs accordingly. Some social rights, for instance, is
now conditioned on having a digital identity, navigating an online platform,
and digitally signing or verifying information. This is particularly true in the wake of the COVID-19 pandemic, as governments around the world have implemented new digital passes (i.e., proofs of vaccination), which citizens are required to display in order to benefit from specific rights and services (e.g., free testing), enter into specific locations, and use public transportation.

These new sociotechnical systems operate both in digital environments and in the tangible world, where a large number of physical devices are increasingly linked together under the heading of the Internet of Things. Together, they constitute an emerging ecosystem that has an increasingly powerful hold on the everyday lives of citizens and the regular functioning of political institutions, be it at the municipal, national, and transnational levels.

At the *municipal* level, sociotechnical systems are integrated into smart cities, sensing cities, and smart urbanisation initiatives. Urban institutions deploy systems that collect data to monitor and assess citizens' behaviours and interpret the data to enhance the responsiveness of public service infrastructures and increase the citizens' well-being. The same systems are used to influence or modify citizens' behaviours, nudging them to act in such a way as to improve the fit between the built environment and the public life it affords (Krivý 2018). Perhaps the clearest illustration of this trend is the multiple attempts by municipalities to encourage civic participation by rewarding citizens points whenever they engage in a particular set of desirable activities. Some cities in Europe are experimenting with a social credit system based on “good deeds” to encourage civic engagement, social solidarity, and political participation. For instance, the municipality of Cascais in Portugal has established a credit system (InnoWave Citypoints) that awards anyone who achieves “good deeds” included in the municipal catalogue (e.g., buying local products, adopting an animal, donating clothes, helping the elders). The points can then be used to receive a particular reward selected from the catalogue of municipal benefits (e.g., free entry to museums, books, concert tickets, animal care services, and public transport) (Orgad and Reijers, 2022).

Public institutions formulate certain aims to which the behaviour of citizens is made responsive through nudges, persuasion, and sanction. This generates a tendency to incorporate the citizen within the system of governance through a mutual interest: institutions are interested in understanding the needs of citizens, while citizens are interested in modulating their behaviour in accordance with the institutions' aims. This requires a networked infrastructure in all aspects of public and often private life.

Similar developments exist at the national and transnational levels. At the *national* level, states are deploying systems that monitor the behaviours of citizens, evaluate them, and intervene when needed. Tracking how citizens act or interact with a governmental authority can help identify suspicious and potentially problematic behaviours, thereby triggering a state intervention. Liberal democracies have already implemented such systems in a variety of
contexts, ranging from predictive policing (Kaufmann & al 2019) to tax fraud
detection (Meuwese 2020) and national welfare abuse. At the *transnational*
level, global corporations like Facebook and Uber increasingly participate
providing public services, which are key to citizenship, such as maintaining
a platform for public discourse or providing mobility services. The operators
of these platforms act as what Frank Pasquale (2018) describes as “functional
sovereigns”—actors unilaterally dictating the rules of their platforms with
broad discretionary powers, no checks and balances, and little to no
democratic accountability. In this capacity, global corporations deploy some
of the most powerful sociotechnical systems that combine vast data
collection, often combined with machine learning and profiling techniques.
Using these systems enables corporations to adapt to the behaviour of Internet
users, possibly influencing and even manipulating their behaviour, as became
clear with the Cambridge Analytica scandal (Maschewski and Nosthoff 2016).

These types of sociotechnical systems operate at different levels yet
remain interconnected. Indeed, the municipal realm in which citizens live
their daily lives is mediated by local, national, and transnational systems that
often feed on the same information sources and potentially share information.
In the not-so-distant future, it could become commonplace for the many
sensors that are already embedded in our everyday environment to be
connected to a variety of larger nationwide systems and transnational
platforms relying on artificial intelligence to profile citizens into specific
categories based on their current or future activities, moods, and behaviours.

While the trend we are witnessing is global, its most salient and alarming
example is the Chinese Social Credit System and its implementation in the
urban realm in China. The SCS is a sociotechnical system that explicitly
targets citizenship (and citizens) and is arguably the most advanced system of
its kind. It targets all three levels described above, involving city pilots,
national government agencies, and commercial entities. We turn to it now.

2. *Social Credit in the Urban Realm*

The SCS was initially conceived to fill the gap of credit reporting in China at
the backdrop of scandals and a pervasive distrust in commercial agents during
the late 1990s. Setting up an American-style credit reporting system like
FICO, it was believed, would make information about commercial entities
more robust and, hence, increase public trust. The initiative was soon taken
up in the broad governance strategy of the Chinese Communist Party (CCP),
extended beyond mere financial creditworthiness toward social and moral
trustworthiness. The SCS initiative was aligned with General Secretary Jiang
Zemin's doctrine, articulated in 2001, of “governing by virtue.” At the same
time, the SCS initiative got entangled in the broader Chinese governance
strategy of “social management,” which finds its origins in Leninist-Maoist
cybernetics and complex systems engineering (Hoffman 2017). After its
initial development, the CCP launched a detailed Planning Outline of the Construction of a Social Credit System (2014). This document lays out the SCS as having three dimensions: *economic*, establishing the idea of financial creditworthiness; *legal*, ensuring compliance; and *moral*, installing a notion of trustworthiness based on following “professional ethics and behavioural norms” (Orgad and Reijers, 2019, 2022). While the initial emphasis of the rollout has been on the commercial sector—companies have a social credit attached to them—the SCS covers all of society, including citizens. It is arguably the first cybernetic system that targets citizens and, thus, citizenship.

The CCP characterises the overall aim of the SCS as generating an environment in which “keeping trust is glorious and breaking trust is disgraceful” (CCP 2014, Para I (3)). Rather than constituting a single system with one master score for each citizen, the SCS comprises a dynamic ecosystem that connects various systems with governance experimentation initiatives. At the basic level, the SCS connects blacklists (for trust breakers) and redlists (for trust enhancers) that different state agencies maintain through information sharing and a joint sanction system. Practically, if a citizen ends up on one blacklist, s/he might also end up on another one and undergo the sanctions attached to both. Information sharing is centralised by the National Credit Information Sharing Platform (NCISP) and facilitated through memoranda of understanding between government agencies. Social credit information includes a variety of domains, from compliance with court orders, via tax payment history, up to reported behaviour in public transport.

The urban realm constitutes a particularly important nexus in the rollout of the SCS. This takes the form of an innovative and decentralised mode of experimentation with SCSs through incentivising municipal governments to compete for being “model cities.” This has generated a variation in SCS configurations in different urban contexts. Today, the SCS is heterogeneous and fragmented, existing not as a single, centralised system, but rather as a decentralised network of connected initiatives. Its effectiveness varies significantly across sectors and implementations. As such, it does not fit the dystopian *Black Mirror* characterisations it was faced in some of the Western media.<fnref n="1" /> At the same time, all the SCS initiatives share a certain overarching logic regarding their relationship with citizens’ space, time, and interactions.

First, the SCS is premised on the idea that citizens should interact with one another and their institutions in a particular manner that promotes public order. Reliance on advanced data collection, sensing technologies, and monitoring techniques enables municipalities to ensure that citizens comply with specific rules and do not misbehave in public spaces. A few smart city governments, like Shenzhen, have been experimenting with facial recognition

<footnote n="1">¹ The episode Nosedive of *Black Mirror* (Netflix) is often compared to the Chinese Social Credit System in the popular media. In this episode, citizens use digital devices to rate each-other, which leads to a social ranking with vital benefits and sanctions attached to it.</footnote>
technology in traffic situations that directly shames jaywalkers. Other cities
are experimenting with different point systems, combined with specific
processing methods attached to catalogues of praiseworthy and blameworthy
behaviours. The catalogues include behaviours like leaving trash in hallways,
which makes one lose points, or helping the elderly, which contributes to
additional points. These points are the basis for a more sophisticated system
of rewards, such as preferential access to employment opportunities, and
punishments, such as increased official scrutiny when partaking in an
examination. This shows how the SCS reinforces the idea that citizens are
integrated into a hierarchical system, where encouraged behaviours are linked
to the ends of the sovereign government. For instance, donating to a charity
might help a citizen enter a red list because it aligns with those behaviours
designated by party officials as being in line with their overarching end. As
such, the behaviours of citizens are subsumed under more general categories
of “good” and “bad” conduct, which, in turn, are subsumed under even more
general governance strategies/principles (e.g., social harmony). The Chinese
government has initially followed a strategy of incentivising the development
of social credits by large private companies like Alibaba and Tencent. This
yielded numerous scoring systems, the most famous was Alibaba’s Sesame
Credit that gathered information on one’s payment history, personal
character, and social relations. Although not linked with the NCISP, these
systems can affect daily lives. Low scores can restrict services, ending with
reduced Internet speed or limited access to public toilets, and high scores can
lead to commercial privileges like discounts on mobile phone rates (Orgad
and Reijers, 2019, 2022). While commercial actors were eventually denied
credit reporting licenses, the CCP set up the public-private partnership
Baihang Credit in 2018, which is likely to become the central scoring agency.

Second, the SCS transforms the traditional understanding of citizenship
as a set of rights granted to a particular individual to a new idea of citizenship
based on memory and anticipation. It creates a digital “memory” of citizens’
behaviours to anticipate future events, generating a score that determines the
conferring of rights or the imposition of restrictions based on whether past
behaviours conform with the system’s ends. This is well illustrated by how
the SCS has been used during the pandemic to provide specific affordances
to virtuous individuals who have been negatively affected by the lock-down
and introduce restrictions and punishments to the less virtuous individuals
who did not properly comply with the rules. For instance, in the municipality
of Shandong, an individual’s social credits will be reduced for refusal to wear
a mask in public; participating in gatherings, parties, and crowded activities;
or failing to implement the quarantine policy. All this empowers the
“smartification” of the urban environment where citizens live through the
proliferation of sensors and information gathering systems. This goes hand in
hand with China’s initiatives to enhance its surveillance capacities through
installing security cameras linked to facial recognition software in many
urban areas. Specifically, the SCS understands citizens in accordance with
behaviours that can be measured and quantified. It relies on the presence or
absence of payment data, records about types of conduct, or machine-readable
recordings of infractions. This understanding can be predefined according to
a list or catalogue, or emerge from big data analytics, processing information
about citizens' behaviours and using this to anticipate future events.
Particularly relevant in this context is the City Brain initiative in Hangzhou,
one of China's most prominent projects in building a smart city. Originally
intended to deal with the problem of heavy traffic in the city (Liu 2020), the
initiative has progressively expanded in scope, engaging in a much broader
range of data collection to promote a variety of public services—such as
facilitating car parking, increasing public security, and providing efficient
government subsidies (Zhao et al. 2020). The data gathered by the Hangzhou
City Brain project comes from public departments datasets, private
companies that partnered with the project, and data voluntarily provided by
citizens. Today, the scope of the Hangzhou City Brain extends over nearly
every aspect of city management. The system is designed to track, evaluate
and provide constant feedback on social management, economic regulation,
market supervision, public service, environmental protection, and government
operation. Because of its focus on prediction and anticipation, the City Brain
has allegedly successfully resolved the traffic problem in Hangzhou (Chen
2019) and further enhanced the efficiency of the police force (Qiu et al. 2019).

Third, when it comes to the informatisation aspect of the SCS, the system
evaluates citizens for the extrinsic actions that they execute, which are
benchmarked according to quantifiable metrics and data points. The centrality
of a quantitative assessment of both citizens is well illustrated by the approach
adopted by Suzhou, one of the first cities in China to explore the SCS. The
municipality of Suzhou established a dual citizen scoring system, comprising
the Fragrans Score and the Suzhou Civilization Code (quickly abandoned due
to fierce critiques). The Fragrans Score is a scoring service operated by the
Suzhou Public Credit Information Centre, which is mainly used to reward
trustworthy people. While the algorithm for the score was never disclosed to
the public, it seems to rely on a variety of data points, including age, marital
status, educational background, professional title, volunteer service, health
status, and a series of criminal and administrative records. Depending on the
score, citizens can benefit from municipal rewards (i.e., travel, culture, health,
education, parking, medicine). At the same time, the municipality promoted
the Suzhou Civilization Code to evaluate people based on a broader scale of
factors, mostly oriented toward the respect of transportation rules (e.g., one's
score would be deducted by 50 points if one drove a red light, and 100 points
if one drove a car drunk). A high civilization score would provide significant
benefits, such as better access to education and greater allocation of public
resources. Yet, shortly after the announcement, the Suzhou Civilization Code
incurred numerous critiques across China, as the treatment of what is
considered a “civilised” behaviour would depend on arbitrary data points.
Facing the pressure, Suzhou abandoned the notion of a “Civilization Code.”
These three points illustrate how the implementations of the SCS, despite their divergences, contribute to creating cybernetic citizenship. The SCS changes the *space* of citizenship by introducing layers of connectivity and incorporating parts of the digital world, where citizens interact with one another and with public institutions, into the physical space through devices, sensors, and digital cameras. The SCS also transforms the *time* of citizenship, which is no longer established at a particular point in time, but rather is constantly redefined in an ongoing manner from the historical reconstruction of past behaviours. Finally, the SCS also modifies the *interaction* between citizens and public institutions because of the process of datafication or informatisation. The identity of citizens is defined by metrics; and their behaviours are rated, ranked, and scored according to performance indicators.

### III. Normative Assessment

This part presents a normative assessment of cybernetic citizenship. It shows that the dimensions of citizenship are shifting, as conditioned by its cybernetic modulation. This happens through three tendencies: (1) the primacy of ends over rights, (2) the transformation of citizenship of status into process, and (3) the primacy of civic effectiveness over excellence. Together, these tendencies can be understood as normative consequences of what Alain Supiot (2019) calls the rise of “governance by numbers.” Yet, rather than seeing quantification as the basis of a new form of citizenship governance, it may be seen as an inherent aspect of a broader perspective of cybernetic governance.

#### 1. *From Rights to Ends*

Modern architectures underpinning citizenship, both as a legal status and a societal practice, give rise to rights and duties that citizens might have. Those rights are categorical and are not premised on the fulfilment of government ends as a prerequisite. Furthermore, the public sphere is based on the principle of equality before the law. When citizens act in the public sphere, they act as equals, for example, by voting in elections. Arendt likened the architecture of the public sphere to the space enclosed by the city wall (1958): like a wall, it sets categorical conditions for human action. Yet it does not actively interfere with the activities of free, equal citizens, and it does not introduce its agency.

Cybernetic citizenship challenges equality before the law because it links equality to human activities and the extent that they fulfil political ends, thereby introducing an *architecture of ends*. This notion resonates with recent debates in legal and political theory that identify alternative modes of non-legal regulation of human behaviour, such as “nudging” (Sunstein 2015), market forces, social pressures, and technological codes (Lessig 2006). Yet, unlike these modes of regulation and governance, the architecture of ends, as promoted by cybernetic citizenship, is not one mode of governance out of
many, but rather the basis (and sometimes a prerequisite) for realising human rights. In other words, it is *not* a complementary mode of regulation (alongside laws, codes, and economic forces) but often the only mode enabling access to certain rights and privileges. The realisation of rights is ends-dependent; citizens are data collectors pushed by a goal-directed system. An architecture of ends means that citizenship may still be a “right to have rights” under the human rights perspective (Part I) yet, in and of itself, it provides access to a narrower list of rights and privileges. This turns citizenship, which is already a thin legal concept in its exclusive entitled rights, to even a lighter concept.

Linking human rights to the pursuit of political ends as a prerequisite for their realisation changes the *space* of citizenship in three senses. First, citizens are no longer treated equally as citizens or as a subgroup of citizens (e.g., women, students, elderly), but as individuals who are constantly judged and evaluated by the extent that they align their actions with political ends. It applies a logic like that of “personalised pricing,” which attempts to capture the entire social surplus in a transaction for setting up the price, to the citizenry. Personalised pricing is a method that looks carefully at the customer and calculates various known factors (computer type, IP location, frequency of searches, etc.) to offer a price for a good or a service that is compatible with the customer’s profile. The method of personalised pricing, some argue, may be more equal than categorial equality because everyone is treated, to paraphrase Marx, according to their abilities and needs. Yet, applying this method to the citizenry is a radical change in the essence of equality, making it dependent on individual actions and thereby limiting its spatiality. Second, cybernetic citizenship makes the realisation of rights not only a function of actions in the physical world but also of one’s tracks in the digital world. The space of citizenship is extended to cyberspace, yet this extension widens the possibilities to restrict rights due to a new set of digital activities that may negatively affect one’s equality vis-à-vis others in cyberspace. Third, organising the political community in line with an architecture of ends, citizens may find themselves in a position of having equal political rights on paper, yet without the ability to exercise them. Cybernetic citizenship thus undermines equal citizenship since the role of networked systems generates inequalities (and hierarchies) between the people whose behaviours they modulate. These inequalities shift the idea of rights toward conditional rights.

## 2. *From Status to Process*

Modern citizenship is shaped by institutions that uphold the rule *of* law, not just the rule *by* law. The rule of law is an elusive principle with various definitions. In essence, it is a legal system in which the law is the source of all political authority, no one is above the law, and all people are equally subject to the law and its enforcement. Fuller (1964) drafted eight formal principles that should exist in a rule-of-law system; the law should be general, public, understandable, coherent, enforceable, not-retroactive, similar both in
books and in actions, and stable. Other definitions are broader and include the notion of just law, which respects political freedom and human dignity, since slave, apartheid, or fascist laws are also based on formal legality (Hart 1965).<fnref n="2" />

The concept of citizenship is one of the mechanisms for securing a stable legal system (alongside others, such as the ability to amend laws and political compromises that enable peaceful co-existence). It means that, once a person is a citizen, it is a lifelong status that guarantees civil rights, including, at least in a rule-of-law system, the ability to take critical political initiatives. The lifelong status of citizenship creates stability through time, which Arendt (1958) refers to as duration. Political freedom exists through the absence of law (everything which is not forbidden is allowed) and is protected (from arbitrary domination) through its presence (Pettit 2011). Citizens can count on this legal basis unconditionally: they neither have to earn nor deserve it.

Cybernetic citizenship transforms citizenship as a legal status, whose essence derives from the law and stands despite the passing of time, into a conditioned concept, the way citizenship is implemented in practice. First, citizenship becomes contingent. The enjoyment of a complete set of human and civil rights depends on data from past events. Cybernetic systems learn from the behavioural histories of citizens to decide which rights and sanctions to grant or impose, when, and where. This is evident in the COVID-19 context, where some behaviours are drawn within the historicised reputation of people, like violating quarantine rules. Cybernetic citizenship is a dynamic, adjustable process that turns citizenship from an unconditional legal status to a historically lasting process, subject to fault and correction. China's Social Credit System, for example, is not only a status but also a lasting process that requires civic participation and actions of a certain type. Second, because it does not explicitly rely on laws and regulations, cybernetic citizenship, as illustrated partly by the Chinese SCS and Covid-19 policies, is not subject to the same constraints as the rule-of-law system. If it exists at all, the legal ground for citizenship restrictions is not always publicly formulated, clear, coherent, and not retroactive; in some cases, there is no due process of law and full respect of property and liberty. Cybernetic citizenship, thus, turns citizenship from a status to a process whose entitled rights must be constantly earned, and moves the legal status outside the sphere of legality to the societal reality, the way people do things in a political community (for the sociological view of citizenship, see Part I). We are witnessing a transition from a system governed by the rule of law ("nomocracy") to a system governed by numbers ("numerocracy"). Unlike citizenship as a status, a governance-by-numbers (or reputation, Dai 2018) system is constantly subject to change and must be earned. It requires citizens to be constantly aware of good behaviour standards and change their behaviour accordingly. It thereby normalizes the notion of citizenization, citizenship as a process, and indirectly reinforces the idea of

<footnote n="2">² For the differences between the “rule of law” and the “rule by law,” see Tamanaha 2004.</footnote>
“good citizenship” (Orgad and Reijers, 2022). Third, cybernetic citizenship urges citizens to “labour” for their citizenship to get access to certain rights. This ties in with Scheuerman’s (2005) argument that the “busyness” generated by contemporary life makes that people have “no time for citizenship” and that free political actions are relegated to laborious activities. As such, being a citizen has increasingly less to do with taking free initiatives and more to do with engaging in activities that fit the historicised expectation of the political community. As a result, citizens have less time to exercise their freedom since the labour to maintain rights is tied to a never-ending process.

### 3. From Excellence to Effectiveness

Citizenship implies interaction, which means that citizens must be responsive to institutions and vice-versa. This idea is captured by the term civic virtue, which can be a virtue ascribed to a citizen (e.g., vigilance) and an institution (e.g., justice). These virtues are two-sided: they appeal to excellence and effectiveness. This distinction lies at the heart of Aristotle’s conception of citizenship. For free citizens, the effectiveness of being ruled coincides with the excellence of ruling; citizenship is enacted through skills (effectiveness) and virtues (excellence). Effectiveness denotes activities needed for a citizen to function according to institutional ends, like paying taxes. It requires extrinsic motivation, safeguarded by external goods such as reputational or material gains. Excellence denotes activities of democratic political life; most notably, activities in which political agents disclose themselves by acting and speaking in concert. It requires a sense of intrinsic motivation, captured by MacIntyre (2007) as the requirement to aim at goods internal to a practice.

Cybernetic citizenship challenges civic virtue as excellence by informatising the qualities of citizens and institutions. These qualities are calculable, programmable, and manipulable to make interaction possible. Institutions increasingly use calculated metrics: risk scores, credit scores, and other measurements. To get to these measurements, civic virtue is not assessed directly but indirectly: it matters whether one pays one’s bills in time, not whether one has the proper disposition to do so. Cybernetic citizenship is thus premised on an implicit form of behaviourism: qualities are derived from behaviours that can be made discrete, measured, and calculated. This comes to the fore with implementing the SCS in the urban realm through a translation of atomic behaviours (e.g., paying a bill, jaywalking) into calculated measurements following a catalogue of good or bad deeds. By this process, citizens become responsive to institutions but only in an instrumental way—they engage in behaviours not primarily because these bring them internal goods but because they are extrinsically motivated.

Turning civic virtue from excellence into effectiveness limits the interaction between citizens and institutions in three senses. First, it makes civic virtue past-oriented rather than future-oriented. The metrics that
establish civic virtue are derived from existing norms and standards. Citizens
and institutions will strive to comply with existing norms and standards, but
this, in turn, limits their ability to act innovatively and creatively, for instance,
by developing new political projects. Second, it introduces the “tyranny of
metrics” (Muller 2018), which means that measurements become more
important than what they are supposed to measure. This risks generating a
false sense of “good citizens” on the side of institutions, because conformist,
rule-abiding citizens are valued higher than active, vigilant citizens who
challenge existing structures. On the side of citizens, this same dynamic
creates incentives to game the system. For, as de Filippi rightly observed,
under the SCS the motto is no longer “I have a good credit score because I
am a good citizen,” but rather becomes “I’m a good citizen because I have a
good credit score” (de Filippi 2019). Third, this transformation eventually
instrumentalises the citizen, for civic virtue is no longer a function of the
citizen’s happiness but only relates to the external well-being of the
community. Cybernetic citizenship thereby falls prey to what Heidegger
(1977) called the enframing of modern technology: it confers everything,
including citizens themselves, into standing reserve, a resource to be used. To
be clear, liberals often consider civic virtue as instrumental to the sustenance
of the liberal state (McTernan 2014). Yet, liberal thinkers have generally
refrained from cultivating virtue, leaving the cultivation of character almost
exclusively to the private sphere, augmented with civic education. Cybernetic
citizenship, on the contrary, explicitly reimagines civic virtue by subjecting it
to a logic of metrified interaction. It is, therefore, “republican” in reviving civic
virtue in the public sphere but subverts it in turn by instrumentalising it fully.

Concluding Remarks

Cybernetic sociotechnical systems reshape the lives of citizens in the urban
realm. This article presents the first philosophical discussion of the concept
of cybernetic citizenship, its implementation through the proliferation of
sociotechnical systems like the SCS, and the normative challenges it raises.

The article contributes to the ongoing debate on the impacts of “dataism”
on citizenship. It foremost responds to the work of Fourcade (2021) on ordinal
citizenship, adding to her work in three ways: by linking the discussion on
citizenship and technology to the philosophy of technology, embedding the
debate within the frame of cybernetics, and considering “ordinality” in
algorithmic governance as an enabler of a conception of citizenship that can
be traced back to the birth of political philosophy. The article goes beyond
ordinal citizenship and its valuable criticism (Joppke 2021) by arguing that
cybernetic citizenship is not tied to any particular citizenship ideology such
as neoliberalism, although it has allied itself with that in recent times. Just as
cybernetics was embraced by liberal and communist states (Csizmas 1971),
it can emerge in different ideological contexts. The article returns to the
perplexity of philosophers of technology who consider that cybernetics is on
its way to replace metaphysics and explores what this means for citizenship.

The rise of cybernetic citizenship, as discussed in the article, is limited in
scope. First, emphasising the urban realm, the article is inclined toward the
“internal” perspective on citizenship, focusing on the relationship between
citizens and political institutions rather than on between states. It would be
valuable to inquire into how cybernetic systems like distributed blockchain
technologies would impact the inequality between citizenships (Grstein and
Kochenov 2020). Second, it accounts for a new model of citizenship that is
not yet fully applicable but exists as a loose conglomeration of sociotechnical
projects. Still, we might follow John Torpey (2001) in his discussion of the
invention of the passport, arguing that a technical tendency, as fractured,
ineffective, and disorganised as it was at the start, tends toward effectiveness
and consolidation. Hence, cybernetic citizenship represents a potent new
force in our world that might transform citizenship and what it means to enact
it.

## Acknowledgments

This research is funded by the European Research Council (ERC) under the
European Union's Horizon 2020 Research and Innovation Programme (Grant
Agreements No. 716350). We would like to acknowledge the thoughtful
comments of Federico Tomasello and two persistent anonymous reviewers.

## References

Alm, James, and Benno Torgler. 2011. “Do Ethics Matter? Tax Compliance and Morality.” *Journal of Business Ethics* 101 (4): 635-51.

Arendt, Hannah. 1958. *The Human Condition*. Vol. 24. Chicago: University of Chicago Press.

Arendt, Hannah. 1973. *The Origins of Totalitarianism*. New York: Harvest Book.

Aristotle. 1998. *Politics*. edited by C. D. C. Reeve. Indianapolis: Hackett Publishing Company.

August, Vincent. 2021. “Network Concepts in Social Theory: Foucault and Cybernetics.” *European Journal of Social Theory* 1-21.

Backer, Larry Catá, and Matthew McQuilla. 2021. “The Algorithmic Law of Business and Human Rights: Constructing a Private Transnational Law of Ratings, Social Credit, and Accountability Measures.” *SSRN Electronic Journal* 2 (Aguirre 2004): 1-51.
Barnes, Rebecca, Timothy Auburn, and Susan Lea. 2004. "Citizenship in Practice." *British Journal of Social Psychology* 43(2):187-206.

Castells, Manuel. 2004. "Informationalism, Networks, and the Network Society: A Theoretical Blueprint." In *The Network Society: A Cross-Cultural Perspective*. Cheltenham: Edward Elgar Publishing Limited.

Chen, Weiquiang, 2019. "The Practice and Thoughts of Hangzhou City Brain." *CPC News* (8 Sept 2019).

Cheney-Lippold, John. 2016. "Jus Algoritmi: How the National Security Agency Remade Citizenship." *International Journal of Communication* 10(0):22.

Csizmas, Michael. 1971. "Cybernetics - Marxism - Jurisprudence." *Studies in Soviet Thought* 11: 90-108.

Dai, Xin. 2018. "Toward a Reputation State: The Social Credit System Project of China." SSRN.

De Filippi, Primavera. 2019. "The Social Credit System as a New Regulatory Approach: From Code-Based to Market-Based Regulation." In Liav Orgad & Wessel Reijers (eds.), "A Dystopian Future? The Rise of Social Credit Systems", 27-28.

Deleuze, Gilles. 1992. "Postscript on the Societies of Control." *October* 59: 3-7.

Depew, David J. 1995. "Humans and Other Political Animals in Aristotle's 'History of Animals.'" *Phronesis* 40 (2): 156-81.

Dumbrava, Costica. 2015. "Welcome to E-Stonia! E-Residence and Citizenship in an Electronic Republic." *EUDO Citizenship* (January):14-17.

Feng, Liu. 2021. "The Origin, Development and Future Tendency of City Brain/城市大脑的起源、发展与未来趋势," 12 Frontiers/人民论坛·学术前沿 82.

Fourcade, Marion. 2021. "Ordinal Citizenship." *British Journal of Sociology* (February):1-20.

Fourcade, Marion, and Fleur Johns. 2020. "Loops, Ladders and Links: The Recursivity of Social and Machine Learning." *Theory and Society* 49 (5-6): 803-832.

Françios, Charles. 1999. "Systemics and Cybernetics in a Historical Perspective." *Systems Research and Behavioral Science* 16 (3): 203-19.

Fuller, Lon L. 1969. *The Morality of Law*. New Haven: Yale University Press.

Galloway, Alexander R. 2004. *Protocol: How Control Exists After Decentralization*. Cambridge, Massachusetts: MIT Press.

Gstrein, Oskar J., and Dimitry Kochenov. 2020. "Digital Identity and Distributed Ledger Technology: Paving the Way to a Neo-Feudal Brave New World?" *Frontiers in Blockchain* 3 (March): 1-8.

Hart, H.L.A. 1965. "Book Review: Lon Fuller. The Morality of Law." *Harvard Law Review*, 78: 1281-1296.
Heidegger, Martin. 1977. *The Question Concerning Technology and Other Essays*. New York: Garland Publishing, Inc.

Hirschl, Ran, and Shachar, Ayelet. 2019. "Spatial Statism." *International Journal of Constitutional Law* 17(2): 387-438.

Hoffman, Samantha. 2017. *Programming China: The Communist Party's Autonomic Approach to Managing State Security*. MERICS.

Hui Yuk. 2016. *The Question Concerning Technology in China*. Falmouth: Urbanomic Media.

Hui, Yuk. 2019. *Recursivity and Contingency*. London: Rowman & Littlefield Publishers.

Joppke, Christian. 2010. *Citizenship and Immigration*. Cambridge: Polity Press.

Joppke, Christian. 2021. "From Liberal to Neoliberal Citizenship: A Commentary on Marion Fourcade." *British Journal of Sociology* (January):1-9.

Kaufmann, Mareile, Simon Egbert, and Matthias Leese. 2019. "Predictive policing and the politics of patterns. The British Journal of Criminology" 59 (3): 674-692

Knight, Adam and Rogier Creemers. 2021. "Going Viral: The Social Credit System and COVID-19." 48(6):1-23.

Krivý, Maroš. 2018. "Towards a Critique of Cybernetic Urbanism: The Smart City and the Society of Control." *Planning Theory* 17(1):8-30.

Lessig, Lawrence. 2006. *CODE Version 2.0. CODE Version 2.0*. New York: Basic Books.

Liu, Wen. 2020. "Hangzhou 'City Brain' Creates Good Lives/杭州‘城市大脑’创造美好生活," *Economic Daily* (11 April 2020): 10.

Lycett, Mark. 2013. "‘Datafication’: Making Sense of (Big) Data in a Complex World." *European Journal of Information Systems* 22(4):381-386.

MacIntyre, Alasdair. 2007. *After Virtue: A Study in Moral Theory*. Third Edit. Notre Dame, Indiana: University of Notre Dame Press.

Maschewski, Felix and Anna-Verena Nosthoff. 2017. "Order from Noise: On Cambridge Analytica, Cybernetic Governance and the Technopolitical Imaginary." *PS* 167-191.

Marshall, T. H. 1950. "Citizenship and Social Class." *The Welfare State Reader* 30-39.

McTernan, Emily. 2014. "How to Make Citizens Behave: Social Psychology, Liberal Virtues, and Social Norms." *Journal of Political Philosophy* 22 (1): 84-104.

Meuwese, Anne. 2020. "Regulating Algorithmic Decision-Making One Case at the Time." *European Review of Digital Administration & Law*.
Mossberger, Karen, Caroline J. Tolbert, and Ramona S. McNeal. 2008. *Digital Citizenship: The Internet, Society, and Participation*. Cambridge, Massachusetts: MIT Press.

Mul, Jos De. 1999. "The Informatization of the Worldview." *Information, Communication & Society* 2(1):69-94.

Muller, Jerry Z. 2018. *The Tyranny of Metrics*. Princeton: Princeton University Press.

Mumford, Lewis. 1965. "Utopia, the City and the Machine." *Daedalus* 94(2):271-292.

Orgad, Liav. 2018. "Cloud Communities: The Dawn of Global Citizenship?" *Cloud Communities: The Dawn of Global Citizenship?* Liav Orgad and Rainer Bauböck (eds.): 1-6.

Orgad, Liav and Wessel Reijers. 2019. *A Dystopian Future? The Rise of Social Credit Systems*. RSCAS 2019/94. Florence.

Orgad, Liav, and Wessel Reijers. 2022. "Making the Perfect Citizen: Lessons from China's Social Credit System." *Vanderbilt Journal of Transnational Law*. Forthcoming.

Pasquale, Frank. 2018. "Digital Capitalism: How to Tame the Platform Juggernauts." *Friedrich-Ebert-Stiftung - Division for Economic and Social Policy* 1-4.

Pettit, Philip. 2011. "The Instability of Freedom as Noninterference: The Case of Isaiah Berlin." *Ethics* 121 (4): 693-716.

Pickering, Andrew. 2010. *The Cybernetic Brain: Sketches of Another Future*. Chicago: University of Chicago Press.

Plato. 1997. "Plato Complete Works." in *Plato Complete Works*, edited by J. M. Cooper. Indianapolis: Hackett Publishing Company.

Qiu Xinrui et al. 2019. "One Step Faster; Hangzhou Smart Police Reform Runs Accelerated/快人一步, 杭州智慧警务改革跑出“加速度”," China Police/中国警察网 (29 Dec 2019).

Reijers, Wessel. 2019. "Beyond Postphenomenology: Ihde's Heidegger and the Problem of Authenticity." *Human Studies*.

Scheuerman, William E. 2005. "Busyness and Citizenship." *Social Research* 72(2):447-470.

State Council. 2014. *Notice of the State Council on the Planning Outline of the Construction of a Social Credit System (2014-2020)*.

Sunstein, Cass R. 2015. "The Ethics of Nudging." *Yale Journal on Regulation* 32 (2): 413-450.

Supiot, Alain. 2019. *Governance by Numbers: The Making of a Legal Model of Allegiance*. Bloomsbury.
Tamanaha, Brian. 2004. *On the Rule of Law: History, Politics, Theory*. Cambridge: Cambridge University Press.

Thomas, Terry. 2007. "Criminal Records: A Database for the Criminal Justice System and Beyond." *Criminal Records: A Database for the Criminal Justice System and Beyond* 1-227.

Torpey, John. 2001. *The Invention of the Passport. Surveillance, Citizenship and the State*. Vol. 5. Cambridge: Cambridge University Press.

Varnelis, Kazys and Anne Friedberg. 2013. "Place: The Networking of Public Space." Pp. 15-39 in *Networked Publics*.

Wang, Fennie and Primavera De Filippi. 2020. "Self-Sovereign Identity in a Globalized World: Credentials-Based Identity Systems as a Driver for Economic Inclusion." *Frontiers in Blockchain* 2(January):1-22

Wiener, Norbert. 1989. *The Human Use of Human Beings*. London: Free Association Books.

Winner, Langdon. 1969. "Cybernetics and Political Language." *Berkeley Journal of Sociology* 14(1969):1-17.

Winner, Langdon. 1978. *Autonomous Technology: Technics-out-of-Control as a Theme in Political Thought*. Cambridge, Massachusetts: MIT Press.

Winner, Langdon. 1980. "Do Artifacts Have Politics?" *Daedalus* 109(1):121-36.

Zandbergen, Dorien and Justus Uitermark. 2019. "In Search of the Smart Citizen: Republican and Cybernetic Citizenship in the Smart City." *Urban Studies* 004209801984741.

Zhao Jiulong et al. 2020. "When a City can 'think': An Observation that Hangzhou Pays Attention to Establishing 'the First City of Digital Administration'/当城市会“思考”——杭州聚力打造“数字治理第一城”观察," Xinhua Net (23 Oct 2020).