# Introduction: hyper-visibility: images in the era of social platforms, digital archives and computational economies

Enzo D'armenio, Maria Giulia Dondero

► To cite this version:

Enzo D'armenio, Maria Giulia Dondero. Introduction: hyper-visibility: images in the era of social platforms, digital archives and computational economies. Visual Communication, 2024, 23 (3), pp.405 - 425. 10.1177/14703572241247836. hal-05310099

HAL Id: hal-05310099

https://hal.science/hal-05310099v1

Submitted on 25 Nov 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
# Introduction: Hyper-Visuality. Images in the Era of Social Platforms, Digital Archives and Computational Economies

Enzo D'Armenio (FNRS/Université de Liège), Maria Giulia Dondero (FNRS/Université de Liège)

Images are today at the centre of multiple social and technological tensions, as a consequence of the adoption of digital coding (Manovich, 2001; Bachimont, 2010), of the massive diffusion of social networks (Light and Moody, 2020), and of the algorithmic processing to which they are subject (Finn, 2017; Cardon, 2015), resulting in new opportunities for developing analytical inquiries and meaning-producing social actions¹. This special issue of *Visual Communication* intends to propose a multidisciplinary investigation of what images are and do today, putting the semiotic approach into dialogue with art history, media and visual studies, and, more generally, with the disciplines concerned with the epistemology of visual documents².

We chose the title *Hyper-Visuality* because we believe that images, besides representing one of the most important semiotic resources in today's computational society, can also be thought of as a more general visual logic reorganising our interpersonal experience. On the one hand, we think about the way in which algorithms can produce visual analyses of large collections of images. On the other hand, algorithms can now enunciate new visual content as a result of the use of the most modern generative artificial intelligence. These two semiotic performances comply with an essentially spatial logic. Algorithms are in fact models composed of lists of numbers organised recursively and describing an operable multidimensional space. This computational space is a field of visual traits and of digitising concepts and words that guide any analytical and productive operation. Tasks such as analysing large corpora of multimodal data, producing interactive verbal responses — as in the case of ChatGPT — or even composing new images through recursive readings and pixel activations rely on a common ground: a semantic space reduced to probabilistic domanial tensions. This statistical hyper-space is largely based on the conjunction of a visual and a spatial logic, and on parameters such as proximity, distance, overlapping and vectoriality, through which generative operations can be carried out.

In this introduction, we intend to reconstruct the broad context that makes images one of the most important resources of the digital era, and to focus on some of the research tracks that characterise it. In the first part, we will begin by focusing on the relationship between images and the digital, which we will retrace in accordance with the selection of three key
moments: the transition from ontology to the epistemology of digital media; the opening, by social networks and portable devices, of a field for the computational study of contemporary cultures; and, finally, the analytical potential arising from the encounter between digital archives and computer algorithms.

In the second part, we will present the three axes around which this issue is structured: archives, identity and algorithms. We will first of all discuss the concept of the *archive*, by presenting four different understandings it has come to bear in conjunction with digital encoding — the archive as heritage, resource, effect and as database. We will then address the relation between images and identities, arguing that social platforms and visual apps are a new domain for identity experimentation and social aggregation. Finally, we will discuss the issue of algorithms and more generally of the new computational economy that associates large amounts of data with their mobilisation as operational images. These three axes will allow us to approach the contributions contained in this issue and to describe the ways in which the authors have appropriated them with respect to specific visual phenomena.

*Hyper-Visuality* is also an opportunity to bring into dialogue two semiotic traditions that have so far remained relatively autonomous, with, on the one hand, the social semiotics developed by authors such as Gunther Kress and Theo van Leeuwen on the basis of the work of Michael Halliday. It is an epistemological approach that treats communication in terms of semiotic resources that acquire or develop their full meaning only in conjunction with the encounter of a certain context and certain social groups. On the other hand, there is the semiotics of the Paris school, which is still little known in the English-speaking world, and which is wrongly associated with the pioneering work in linguistics by Ferdinand de Saussure and above all with the first generation of semiologists, represented by the work of Roland Barthes. With the adoption and rethinking of the theory of enunciation and of the semiotics of practices (Fontanille, 2006; Dondero, 2020), articulated with respect to particular domains, as in the case of the scientific practices discussed in Dondero and Fontanille (2014), these two traditions seem to converge in the study of particular ways of constructing meaning: in our case, mobilising innovative visual grammars and new ways of making visual signs, as a consequence of the encounter with digital infrastructures, computational resources and archives.

# 1. Images and the digital: an epistemology of computational practices and big visual data
The relationship between images as a semiotic resource and digital coding can be summarised in the light of three fundamental transformations that have occurred from the 1980s to the present day, changes that have accompanied an equal number of shifts within academic reflection. In the first phase, the relationship between images and the digital focused on the attempt to renew the ontology of images. David Rodowick (2007), in particular, emphasised a radical change from previous photographic media, in light of the fact that the relationship between the real and the production of images seems to be completely disengaged from indexical relations. This was a moment of stabilisation and absorption of the potential provided by the digital, accompanied by an equally important renewal of academic debate, which proposed some of the modern classics, such as Lev Manovich's seminal *The Language of New Media* (2001). The relationship between images and the digital has begun to account for specific practices, especially considering the materiality (substrates) that characterises them. Bruno Bachimont (2017), among others, has rightly insisted on the interactive core that lies at the heart of any digital activity, as implemented, namely, by the Turing machine. Far from being reducible to an immaterial substrate, the physical implementation of a Turing machine for the processing of digital code always requires a relational entity composed of three elements: a physical memory strip that is treated by the device as infinite but whose portions are indicated by specific symbols, a mobile reading and writing head, and a set of instructions that prescribes its operation in an unambiguous manner.

This first step can be summarised in terms of a shift from an ontology of the digital to an attempt to construct its epistemology. In other words, from the study of the presumed immateriality of digital images, the focus has slowly shifted to the analysis of specific visual production practices. On the one hand, attempts have been made to disentangle the innovative terrain of computational techniques in combination with analogue techniques. On the other hand, the analysis has targeted the forms of mediation required to make these techniques work together on an expressive and economic level. An example of this shift in perspective is modern film-producing techniques, which combine digital and pre-digital techniques in a now standardised chain:

> Whereas an entirely digitally-generated composition falls into the category of COMPUTER-GENERATED ANIMATION (CGA), as mentioned earlier, COMPOSITING in itself is a practice rooted
in analog filmmaking, in particular in the use of multiple printing and optical effects, and in the use
of BLUESCREEN (also known as CHROMA KEY), a practice that predates digital technology and finds
its root in television practice. (Fossati, 2018: 58)

Similarly, in the field of film restoration, the relationship between classical analogue
techniques and digital simulation is far from being tied to an ontological gap and rather takes
the form of a dialogue: “For instance, if a digital copy of a film could reproduce (simulate) the
original characteristics of an obsolete 35mm color system better than a copy on contemporary
35mm color film stock, I would opt for the digital copy” (Fossati, 2018: 98).

The second shift concerns the consequences of the definitive establishment of digital
production and communication methods, especially their democratisation and their
organisation into standardised and global platforms. Starting in the second half of the 2000s,
native digital production began to assume such dimensions by virtue of which it became
possible to undertake the study of contemporary cultures supported by qualitative-
quantitative analyses. For the first time in history, researchers and analysts now have at their
disposal such a quantity of documents, and in particular images, that they can study
contemporary cultures, going beyond the analytical framework of disciplines such as art
history and aesthetics. In 2018, Instagram claimed to have over 500 million daily active users
and 1 billion monthly active users. As for the global production of images, it was estimated
that in 2016 only, more than 2.5 trillion photos had been produced, saved and shared
(Manovich, 2017: 11). Especially as a consequence of the emergence of digital social networks
and the miniaturisation of camera devices — two trends that are emblematically embodied
by the smartphone device — new approaches to the study of images are called for and can be
defined according to the very general label of *cultural analytics* (Manovich, 2020). If until the
2000s there was a separation between domains ensured by the specialisation of social
practices (religious practices, artistic practices, journalistic practices, etc.), with the advent of
YouTube and of networks such as Facebook, Instagram, Twitter and TikTok, images have
ceased to be exclusively produced and circulated within dedicated and autonomous practices
to also become an everyday language. Images are now used by all social actors, each with
respect to their own purposes, in dialogue with specific communities, and with the help of
new visual styles and genres. Hence, the traditional division between professionals, artists and
amateurs is much more blurred, and it becomes interesting to analyse, from a semio-
rhetorical perspective, the way in which citizens, institutions and, more generally, politics
occupy the public scene through visual representations. If we consider social networks as a
space of social negotiation, traversed by political, gender-related and artistic tensions
(Kirdemir et al., 2021; Bryant, 2020; Kaiser and Rauchfleisch, 2020), we have to recognise that
they are one of the most fundamental contemporary agoras: places where perhaps the
deliberative function is not exercised, but at least places where this function is prepared
through the bringing into presence and the pre-selection of topics of interest, systematically
taken up by mainstream information outlets and often also by the political agenda of
governments. The importance of images on these platforms is crucial: images are used not
only as a means of expression and communication, but also as a means of occupying public
space, constructing all sorts of identities, questioning those of opponents, and displaying
political and even existential values in a condensed manner (D'Armenio, 2022).

A final shift in the relationship between images and the digital can be discerned as a
consequence of the operations made available by large image datasets (Parikka, 2023). It is
not only as a consequence of the increased production of native digital images, but also
through a process of digitisation of all sorts of past archives that new possibilities for research
and expression open up. The wide availability of large amounts of data, to which the concept
of big data refers, opens up countless compositional, analytical, artistic and commercial
possibilities that have at their core the utilisation of images. This is the case with digital art
history, which attempts to identify new avenues of research using computational methods. In
other words, since the 2010s, the systematic organisation of visual and verbal data into large
accumulations of datasets has opened up a new phase in the relationship between digitality
and images, a phase that was initially referred to as the big data phase, but which is gradually
taking on the features of the visual operability of big visual data.

## 2. Four understandings of the archive: as heritage, resource, experience and database

This issue of *Visual Communication* focuses the relationship between images,
computational tools and social media platforms on three main topics, the first of which is the
question of visual archives and the operations that can be performed on and by means of
them. By the end of the 2010s, the definitive emergence of social platforms and protocols for
storing and digitising documents had already led to the emergence of two different
conceptions of the archive (Treleani, 2017). A classical conception identifies the archive
according to heritage values: a series of documents to be preserved, labelled or exhibited with the aim of being passed on to future generations. In a caricatural manner, one could say that museums of all kinds, as well as public and private institutions, are essentially archives of documents and of procedures as well. And yet, a profound process of digitisation, attributable to technological transformation but also to the economic and legal environment of societies, has brought about a second understanding of the archive. With digitisation, archives retain their heritage status, but their easy retrievability, manipulability and modularity facilitate their circulation and transformation at low cost.

> In the past several decades, the archive as both a concept and an object has been undergoing a transformation. Although official film and television archives still promote their holdings as the most valuable and authentic basis for documentary films on historical topics, other kinds of audiovisual archives have begun to compete with them. Online databases and private collections, in particular, threaten to unseat official archives as the primary purveyors of evidentiary audiovisual documents. (Baron, 2014: 16)

Countless institutions open remotely accessible sections and invite citizens and users in general to appropriate them for cultural and creative uses. Video re-editing contests exploiting an institution's archive collections are systematically proposed, in an attempt not only to circulate documents but to ensure their vitality, in accordance with a proactive memory policy. In other words, the second understanding of the archive is as a resource to be shared, reformulated, and brought to life.

To these first two understandings, a third and a fourth can be added, pertaining respectively to the archive as a meaning effect and as a set of patterns, in this case visual, through which new documents can be computationally generated. The first case concerns the pluralisation of image formats, due to technological progress but also to the aesthetic differentiation that devices imprint on the resulting images. Some authors, such as Jay Leyda (1964), argue that video archive editing is concomitant with the beginning of the history of cinema. Others study it in relation to experimental art (Blümlinger, 2009), while some others identify new forms of montage that nourish mainstream production (D'Armenio, 2017). In other words, the archival effect elicited by a particular technical format (e.g. an early cinema film placed in the context of a contemporary production) opens the door to a rhetorical use of the visual and audiovisual archive. On the one hand, the archival montage allows for the
### 3. Visual identities between the economy of attention and computational experimentation

The second theme at the core of this issue is that of identity. With the rise of social platforms such as Facebook, Instagram and TikTok, a new economy of value has also emerged. Authors such as Richard Lanham (2006) and Ed Finn (2017) speak of an economy of attention (Franck, 2019), in which the visibility of contents and profiles, measured in terms of views, likes, follows and shares, is capable of generating reputational capital and above all financial revenue. A clear example of this new economy is related to the way in which user behaviour — content viewed on social networks, web searches, films watched on Netflix — is modelled by algorithms into behavioural profiles. These profiles are not only used to propose personalised content — schedules on video streaming platforms, book recommendations on Amazon — but also to feed the generation of advertising content. It is well known how Google Ads, in conjunction with the opening of a web page by a user, sets up computational micro-auctions lasting only a few hundredths of a second, in which the user's profile is sold to the highest bidder in order to fill advertising banners that might interest them. In short, it is an economy that uses a modelling of user behaviour — a behavioural identity — as currency to make it profitable. It is also the largest source of revenue for Google, with $224.47 billion (USD) in 2022. While the model of selling advertising space is not new in media history, having already been used in traditional print media and television, the ways in which the data of potential buyers are collected are new. In traditional advertising, it is a matter of identifying and imagining the potential target of an advertisement on the basis of the editorial policy of the magazine or the audience groups potentially tuned in to a channel in a certain time slot. In contrast, on digital networks, it is a matter of modelling the actual behaviour of users, and of autonomising the process of selling advertising space through the use of algorithms. Ed Finn observed in this regard:

> The typical Google advertisement nets the company some tiny fraction of a penny to serve up to a customer, but over the volume of the tens of billions of ads it serves each day, those fractions add up to a kind of minimal transaction cost for using the Internet, collected by its most powerful gatekeeper. (Finn, 2017: 158)

And yet, the construction of advertising value is far from being the only terrain affecting social identities. In fact, social networks have fuelled a new importance of social identities by
becoming a privileged terrain for the affirmation of personalities, capable of influencing political and social agendas. As stated by Jussi Parikka: "[...] digital platforms as environments of aggregation and operationalization of images become a more specific place where this focus condenses in relation to the economy and ecology of power" (Parikka, 2023: 65). The link between identity construction and management on the one hand, and the concomitant aggregation of collectives around it, on the other, have never been stronger than in the current digital age. Whether it be a matter of influencers, artists, actors, sportspeople and above all political figures, social networks function as catalysts for public discussion, and constitute themselves as semio-rhetorical terrains necessary for the affirmation of visibility. At the heart of this *status quo* lies a paradox. On the one hand, identities are the result of a positive work of affirmation and of putting actors onto the social scene. This is an actual management of identity presence, which becomes an attractor of communities and value, and of existential and ideological constellations. On the other hand, online platforms drive the construction of antagonistic identities, conflictualities, and of micro- and macro-communities in opposition. In this context, images play a central role, as much for the possibility of constructing an effect of presence and proximity between public figures and citizens as for the exhibition of bodies and faces that characterises our age. We see new genres of images such as selfies, live streaming and short videos already pre-formatted for filming in other, more traditional media — this is the case with political leaders whose audiovisual statements are covered by TV news outlets.

Finally, identity-related images, once associated with algorithmic modelling devices, have become powerful tools for experimentation (Leone, 2024). Deepfake models are regularly used for more and less legitimate purposes, assigning the face of public figures for controversial political discourse or pornographic practices. Similarly, there is a proliferation of apps to experiment with one's gender identity, figuratively imagining a possible transformation, or to prefigure cosmetic surgery. Images thus become the ground for the affirmation of a new social imagination, in which it is no longer a matter of discursively envisioning future scenarios, but of pre-visualising possible identity actions through the computational and visual modification of faces and bodies.

## 4. Hyper-visibility: the algorithmic operability of images as semiotic patterns
The third thematic focus of this issue of *Visual Communication* concerns the relationship
between images and algorithms. In the current computational era, the way images work has
assigned them new affordances and capabilities, which go far beyond the visual grammars
already studied in social semiotics and in the semiotics of the Paris school, to acquire
unprecedented analytical and expressive capacities. Jussi Parikka speaks in this regard of
“operational images”. The latter are capable of performing analytical gestures, or of providing
visualisations of complex results in the form of diagrams: “instead of merely capturing the
visible and the invisible, the focus on models helps to consider images as experimental
systems” (Parikka, 2023: 89). Feature extraction techniques (Manovich, 2020) and deep
learning algorithms, capable of quantifying and inspecting the internal composition of images,
allow the construction of comparative analyses based on the similarity of purely formal visual
features on corpora composed of thousands of visual documents. Under labels such as Media
and Data Visualization, computational tools are renewing art history as well as film and media
studies, as witnessed by important European and international projects such as *Replica
(Seguin, 2018)*, *Augmented Artwork Analysis* ([http://icar.cnrs.fr/aaa](http://icar.cnrs.fr/aaa)), *Totentanz* (Impett and
Moretti, 2017) and *Towards a Genealogy of Visual Forms* ([https://ceserh.hypotheses.org/997](https://ceserh.hypotheses.org/997)). In other words, the coupling of feature extraction and
deep-learning algorithms allows the creation of visualisations for large amounts of data that
effectively serve as visual analyses. The encounter between images and algorithms takes place
around the latent space: that of computational work, capable of challenging the relationship
between visibility and epistemology, perception and analysis.

Regarding these initial analytical and epistemic capabilities acquired through visual
languages, a second line of investigation concerns immersive qualities. When approaching
technologies such as virtual reality and video games, Andrea Pinotti has argued that images
no longer fulfil their iconic nature — the fact that they are representations of something —
and are now capable of generating an effect of phenomenological presence. For these
reasons, Pinotti speaks of “anicons”:

"Presentification" rather than representation is here the key issue. Subjects relating to anicons
are no longer visual observers in front of images isolated from the real world by a framing device:
they become experiencers living in a quasi-world that offers multisensory stimuli and allows
sensorimotor affordances and interactions. (Pinotti, 2017: 1)
Breaking away from the mimetic paradigm that has dominated Western art and visual representation at least until the 19th century, these images configure themselves more as manipulable environments, aiming to construct a visceral experience.

Finally, images also acquire compositional qualities in their relationship with artificial intelligence models such as Midjourney and Stable Diffusion. Once visual document archives are digitised and used to train computational models, what is obtained is a device for image generation that treats the visual history of humanity as a structured set of visual patterns, composed of objects, figures, chromatic traits and compositional techniques to be algorithmically recombined. In other words, images acquire a generative skill capable of producing not only by exploiting visual grammars, but also the micro-languages that have fuelled their historical and technical evolution. The collaborative creation that associates generative AI models and human operators — and which is realised through the insertion of verbal descriptions (prompts) and the progressive modification of generated images through ad-hoc functions (such as Midjourney's "vary region" and "zoom out" commands) — poses considerable challenges to traditional approaches to images. It is no longer a question of analysing specific grammars or determined social fields, but rather of reframing the relationship between verbal and visual languages, and studying the multiple layers of computational mediation underlying their possible transduction. These mediations take shape as unprecedented enunciative practices, summoning the visual history contained in databases, the stereotypes within them, and the agentic reappropriation performed by disparate social groups.

## 5. Visual semiotics, social semiotics, digital humanities

As anticipated in the first section of this introduction, the computational images at the centre of this special issue of *Visual Communication* provide an opportunity for an interdisciplinary encounter between two semiotic approaches: on the one hand, social semiotics, and on the other, the semiotics of the Paris school. Both of these traditions must confront the challenges posed by the digital revolution and adapt their respective epistemologies to analyse the technical and expressive mediations resulting from the intersection of algorithmic logic and visual grammars. In doing so, they need to engage with
the work that other disciplines, such as digital art history and, more broadly, visual and media
studies, are undertaking in the same field.

Starting from our perspective, that of the Paris school, we find it important to identify
possible articulations between these two paradigms around the computational revolution.
Both social semiotics and the Paris school semiotics have granted a prominent place to the
study of visual composition, rejecting the atomistic paradigm of the study of visual signs
proposed, for example, by Roland Barthes. In the introduction to the seminal volume *Reading
Images: The Grammar of Visual Design* (2021), Gunther Kress and Theo van Leeuwen
immediately emphasise this theoretical choice:

In our view, most semiotic accounts of the visual have concentrated on what could be regarded as
the equivalent of ‘words’ – what linguists call ‘lexis’ – rather than ‘grammar’, and then on a
distinction of ‘denotative’ and ‘connotative’, or the ‘iconographical’ and ‘iconological’ significance
of the *elements* in images, the individual people, places and things (including abstract ‘things’)
depicted there. In this book, by contrast, we will concentrate on the way in which these elements
are combined into meaningful wholes. Just as grammars of language describe how words combine
in clauses, sentences and texts, so our ‘grammar of the visual’ describes how depicted elements –
people, places and things – combine in visual ‘statements’ of greater or lesser complexity and
extension. To generalize, we might say that if the traditional approach has focused on *depiction*,
our focus is on *arrangement*, on *composition*. (Kress and van Leeuwen, 2021: 1)

Similarly, starting from the 1980s, Paris school semiotics developed a theory of the visual
interested in analysing the way images show, tell, deny, and argue by taking advantage of the
grammatical and syntactic resources proper to visual languages (Dondero, 2020; Badir and
Dondero, 2016). With regard to the compositional dimension, this theory is developed
according to three ways of reading images. The study of represented figures, of iconographic
motifs (figurative analysis), and of forms of dialogue with the viewer (enunciative analysis)
constitute only two of the three macro-paths of this dimension (Marin, 2002; Fontanille, 1989)
alongside the formal characteristics belonging to what scholars of the Paris school has called
“plastic reading”: the chromatic, eidetic and topological organisation internal to images
(Floch, 2000).

These three readings — figurative, enunciative and plastic — can be related to the
communicative metafunctions formulated by Michael Halliday and elaborated in the context
of the semiotic analysis of images proposed by Gunther Kress and Theo van Leeuwen. In
particular, the ideational (meta)function, like plastic and figurative readings, concerns "specific lexical and grammatical (in our case compositional) resources for relating represented elements to each other" (Kress and van Leeuwen, 2021: 16). The enunciative reading of the Paris school almost finds a perfect correspondence with the interpersonal metafunction, which "involves specific social relations, in our case between the sign-maker, the sign-interpreter, and the people, places, and things represented" (Kress and van Leeuwen, 2021: 17). The two authors offer the example of a specific resource that has also been a central feature of the contributions of the Paris school: "Gaze is one such resource: a depicted person may be shown as addressing viewers directly, by looking at the viewer. This conveys a sense of interaction between the two parties" (Kress and van Leeuwen, 2021: 17). Finally, the textual metafunction involves not the study of signs but "complexes of signs which cohere both internally with each other and externally with the context in and for which they were produced" (Kress and van Leeuwen, 2021: 18). It seems to us that this epistemological strategy is relatable to that developed by Paris-school semiotics, which focuses on the analysis of texts and discourses to go beyond the study of signs. More broadly, the study of sign-making practices carried out in the context of social semiotics finds a broad resonance in the adoption of the enunciation theory developed in Paris-school semiotics. It involves not the study of signs, nor exclusively the study of utterances, but rather the study of the production of utterances. The recent openness to the semiotics of practices (Fontanille, 2008) and the analysis of specific image statuses or domain fields in which images live (Dondero and Fontanille, 2014) — capable of developing domain-specific grammars — brings the two paradigms closer together³.

However, the study of the compositional dimension and enunciation or sign-making practices is not sufficient to address the ongoing computational revolution. Digital societies demand addressing the growing importance of the mediatic dimension, which concerns the material production of images, their supports (Fontanille, 2005), as well as the algorithmic mediations to which they are subjected (Eugeni, 2021). Images today spread as visual flows circulating on social networks. They constitute multiple archives linked to both our heritage and to the production of particular social actors (Treleani, 2017), and have become image-environments to be immersively explored thanks to technologies such as virtual and augmented reality (VR and AR) (Pinotti, 2017; Evans, 2019).
Finally, images are strongly invested with a rhetorical dimension, which concerns not only persuasive and discursive resources (Group μ, 1992), but also the identities that appropriate them on the public stage.

These three dimensions need to be explored in order to address the challenges posed by the computational revolution and the new potentialities acquired by images. Gunther Kress and Theo van Leeuwen defined the concept of *semiotic potential* as "the semiotic resources available to a specific individual in a specific social context" (Kress and van Leeuwen, 2021: 11). However, in contemporary computational society, it is not straightforward to establish a direct correspondence between specific semiotic resources and a unique social context, as platforms, archives and algorithms related to the visual are inherently conceived as transdomanial and easily transferable. The computational power of generative AI is available to any individual or social group and can be leveraged to produce images from a multitude of devices and social practices, be they amateur, private, corporate, artistic, political or institutional.

**6. The contributions in this issue**

The authors who contribute to *Hyper-Visuality. Images in the Era of Social Platforms, Digital Archives, and Computational Economy* have approached the three thematic axes — archives, identity and algorithms — in specific manners, focusing on semiotic issues and visual phenomena pertaining to digital cultural history, media studies and document epistemology.

The contribution that takes the most comprehensive approach is that of Massimo Leone, who focused on the relationship between technique and rhetoric. Through the construction of a parallel between the Roman figure of the augur and the scientist, Leone related their way of proceeding and producing knowledge to modern artificial intelligence algorithms. In particular, the author examined the relationship between the black-boxing processes that take place as a result of the deployment of technical procedures and apparatuses, and the socio-rhetorical leverage that these specific expertises are able to mobilise in the public arena. Taking a specific case of analysis, represented by the post "Inceptionism: Going Deeper into Neural Networks" by Alexander Mordvintsev and Mike Tyka, engineers employed at Google, and Christopher Olah, a member of the same company, Leone reconstructs the authors' rhetorical procedures with respect to the epistemic stakes. The authors of this post declare that they want to carry out a kind of 'reverse-engineering' of the functioning and show the different layers of convolutional network algorithms — with stakes that assume a scientific
point of view — but which, through a rhetorical shift, they abandon for such purpose and
conclude in accordance with an artistic paradigm, illustrating the creative potential of their
use. This case study allows Leone to identify the problematic relationship between technical
expertise, rhetorical power and authority that characterises the computational age.

The second contribution to this special issue proposes an epistemological interrogation
of visual documents starting from the artistic work of Gerhard Richter. In particular, the
analysis of paintings simulating photos made by the artist since the 1960s, allows Enzo
D'Armenio to interrogate not only the relationship between photographic and pictorial media,
but also to propose an additional dimension in the analysis of images, alongside the
compositional dimension already extensively studied in semiotics: the mediatic dimension.
What emerges is the proposition of two analytical concepts able to analyse how perception is
fuelled by the historical production of visual documents and archives: on the one hand, the
technical formats of images concern not the form traditionally analysed in semiotics, but
rather the substance resulting from their production. Parameters such as definition, aspect
ratio, colour and grain, are usually condensed into aesthetic effects pertaining to precise
device families that are perceptually recognisable by the viewer: smartphone images,
surveillance camera images, Polaroid images, etc. On the other hand, techno-percepts
concern the manner in which visual production devices determine particular visual effects, the
meaning of which is antecedent from the categorisation achievable by verbal language. Just
as human percepts produce pre-categorial iconic meaning (for instance, this particular
yellow), so do visual production devices produce pre-categorial techno-percepts – for
instance, a specific photographic yellow, or a specific photographic blur effect. Because of the
way Richter precisely mobilises the technical formats of photographic images but makes
personal use of the blurring effect, D'Armenio proposes to understand his artistic work in
compliance with a techno-impressionism.

The following contribution adopts the same epistemological framework, that of the
semiotics of the Paris school, and focuses on algorithmic analysis methodologies at the service
of visual languages. Dario Compagno discusses three methodologies of image study that take
advantage of computational power and quantitative statistical methods, relating them to
qualitative semiotic methodologies for analysing and interpreting visual documents. The first
method is linked to algorithms that exploit the manual annotation of large quantities of
images. This method mainly concerns the figurative dimension of images, that is, the
recognition of figures and objects of the natural world, through which algorithms can model
semantic fields and apply them to image recognition and classification. The second method is
eye tracking, which is capable of detecting ocular movements by organising them into fixations
(sections of the image that are looked at over a long period of time) and saccades (successions
of multiple fixations). This method makes it possible to reconstruct the model reader of the
images, starting from the perceptual movements made in front of the images, by negotiating
the plastic and figurative features that compose it. Finally, the third method consists of an
analysis of the signifier of the images, i.e., the chromatic, eidetic and topological dimensions,
through feature extraction algorithms capable of organising them into an analytical space that
arranges them with respect to the visual features that they share (and thus allows them to be
visualised closely) or that distinguishes them. At the end of this rich meta-analysis, Compagno
suggests the possibility of using these three methodologies in a combined way in order to
address some of the issues at the heart of qualitative semiotic analyses: the meaning link
between perception, composition and visual interpretation.

Ruggero Eugeni analyses augmented reality filters — filters capable of modifying the
appearance of images taken by a video device in real time — and expands on the effects of
the current computational revolution on the cultural economy. On the one hand, there is a
correlation between filters that allow us to experiment with facial alterations (ARFaces), which
often carry aesthetic bias, and the performance of cosmetic surgery. On the other, these
filters, and in particular those associated with brands such as Gucci, Apple and L'Oréal,
represent the ground for identity experimentation, often based on creative solutions, which
can be summarised with the notion of augmented self. The discussion of this specific case
finally allows Eugeni to broaden his gaze to identify some key changes concerning the
production of images from the age of photographic media to the age of digital media.
Computational media do not build a space-time break between the moment of production
and the moment of visualisation, because these two moments coincide in a real-time
production-visualisation phase. Second, continuity in the procedures of transmission of visual
content is profoundly reconfigured: although algorithmic images produce their visualisations
in real time, this apparent continuity is due to the speed of computational processes, which
act in accordance with a complex network of algorithmic mediations. Finally, the traditional
division into social practices and technological apparatuses that marked pre-digital image
production is reconfigured by the inherently trans-domestic nature of algorithmic images. For
these reasons, Eugeni argues for the emergence of a new economy, in which light is no longer
aimed at constructing visualisations that take the form of object-images, but rather at serving
the extraction of data to be used for subsequent computational operations.

The paper by Valérie Schafer and Fred Pailler offers an analytical exploration of the meme phenomenon. Adopting a perspective that lies halfway between a historical approach and a theoretical reflection on digital communication, the two authors reconstruct the complex social, economic and cultural physiognomy that underpins and accompanies memes. On the one hand, memes are considered as a fundamental component of digital native culture, because they embody multiple aspects of it in an exemplary way: the extreme rapidity in circulation, the viral and collective spread, the ironic and irreverent anti-authorial tenor. On the other hand, memes are deeply rooted in popular culture, and they distort or caricature its features in compliance with a wide mobilisation of visual traits, ranging from the amateur aesthetic, typical of platforms such as Paint or Photoshop, to the glitch aesthetics. This multifaceted nature poses considerable challenges to their preservation and heritagisation. Portals such as 4chan and 9gags, as well as specific pages on social network platforms are catalysts for the production and circulation of memes. The knowyourmeme.com portal, although it has a commercial purpose, equally has a patrimonial purpose, in an emblematic conflict between practices. Similarly, platforms such as Wikipedia and the Internet Archive, as well as initiatives such as the Library of Congress, demonstrate an interest in the heritagisation of digital native content, in a conflict of interests between commercial profitability and the difficulty in finding and preserving content.

Finally, Dario Rodighiero's paper addresses the way in which visual computational
analysis tools can be used to model the verbal language adopted in a specific language-based
community, that of researchers. The paper engages in a theoretical discussion with the work
of psychologist Jean Piaget on the self-regulation mechanism: the idea that individuals in
verbal interactions must construct a common language that enhances their mutual
understanding. This theoretical premise is transformed into an operational analytical principle
through the discussion of Piaget's reinterpretation proposed in cybernetics by Ernst Von
Glasersfeld and especially the adoption of Bruno Latour's actor-network theory paradigm.
Rather than focusing on verbal dialogue, the paper analyses written traces pertaining to the
scientific literature. Using libraries such as t-SNE and UMAP, Rodighiero constructs a network
visualisation that allows him to model the style and themes adopted by academic researchers
in their verbal production, in accordance with affinity maps. Overall, this is an exemplary case of hyper-visual analysis applied to verbal language and to the interaction pertaining to a specific language community.

Taken together, these contributions represent an exploration of the complex relationship that associates the semiotic resources of images with the new analytical and expressive skills resulting from the encounter with computational tools and visual archives. The age of hyper-visuality opens up to a multitude of challenges for semiotic researchers and, more broadly, for the humanities. Firstly, much work remains to be done on how modern generative visual AI is attempting to solve, sometimes reinforcing, visual stereotypes associated with particular ethnic and political identities. A striking example is the recent controversy surrounding the launch of Google Gemini, a visual generative AI incapable of producing figures of Caucasian humans, even in cases involving prompts related to the depiction of Nazis.

Secondly, modern algorithms for image generation seem to embody different enunciative styles because they treat archives and datasets of previously produced human documents differently. Studying these different styles and the possible uniformity that risks being imposed in computational visual communication is an important matter.

More generally, the hyper-visual landscape poses a fundamental challenge: identifying how images produced with algorithmic tools will be implemented in specific social practices. In the field of art, AI images have already created controversial situations: German artist Boris Eldagsen received the Sony World Photography Award 2023 (Open Competition, Creative section), but refused it because he had produced his work using AI. It will be crucial to examine the transformation of the structure of the labour sector: the transformation of compositional skills that the production of images from verbal prompts will impose on professionals in science, art, and information. Finally, the humanities should analyse the delicate relationship between computational visual production and their legal jurisdiction: the way artificial intelligence and algorithmic images are challenging the issues of privacy through the recording and exploitation of data online to build training datasets.

## REFERENCES

Bachimont B (2010) *Le Sens de la technique: le numérique et le calcul*. Paris: Encre Marine.
Bachimont B (2017) *Patrimoine et numérique. Technique et politique de la mémoire*. Créteil: INA.

Badir S and Dondero MG (eds) (2016) *L'image peut-elle nier? Liège: Presses Universitaires de Liège*.

Baron J (2014) *The Archive Effect. Found Footage and the Audiovisual Experience of History*. Abingdon: Routledge.

Berry DM (2012) *Introduction: Understanding the Digital Humanities*. In: Berry DM (ed.) *Understanding Digital Humanities*. New York: Palgrave Macmillan, 1–20.

Blümlinger C (2009) *Kino aus zweiter Hand: Zur Ästhetik materieller Aneignung im Film und in der Medienkunst*. Berlin: Vorwerk 8.

Bryant LV (2020) *The YouTube Algorithm and the Alt-Right Filter Bubble*. *Open Information Science* 4: 85-90.

Cardon D (2015) *À quoi rêvent les algorithmes. Nos vies à l’heure des big data*. Paris: Seuil.

D’Armenio E (2017). From audiovisual to intermedial editing: Film experience and enunciation put to the test of technical formats. *Versus Quaderni di studi semiotici*, 124. DOI: 10.14649/87042

D’Armenio E (2022) *The Rhetorical Dimension of Images: Identity Building and Management on Social Networks*. *Semiotica*, vol. 2022, no. 246. DOI: https://doi.org/10.1515/sem-2020-0113

Djonov E and Van Leeuwen T (2018) Social media as semiotic technology and social practice: the case of ResearchGate’s design and its potential to transform social practice. *Social Semiotics*, 28(5), 641-664. https://doi.org/10.1080/10350330.2018.1504715

Dondero MG (2020) *The Language of Images: The Forms and the Forces*. Berlin: Springer.

Dondero MG and Fontanille J (2014) *The Semiotic Challenge of Scientific Images. A Test Case for Visual Meaning*. Ottawa: Legas Publishing.

Eugeni R (2021) *Capitale algoritmico. Cinque dispositivi postmediali (più uno)*. Brescia: Scholé.

Drucker J (2020) *Visualization and Interpretation: Humanistic Approaches to Display*. Cambridge (MA): MIT Press.

Evans L (2019) *The Re-Emergence of Virtual Reality*. New York-London: Routledge.

Finn E (2017) *What Algorithms Want: Imagination in the Age of Computing*. Cambridge (MA): MIT Press.

Floch JM (2000) *Visual identities*. London & New York: Continuum.
Fontanille J (1989) *Les espaces subjectifs. Introduction à la sémiotique de l'observateur*. Paris: Hachette.

Fontanille J (2005) Du support matériel au support formel. In: Arabyan M and Klock-Fontanille I (eds) *L'Écriture entre support et surface*. Paris: L'Harmattan, 183-200.

Fontanille J (2006) *The Semiotics of Discourse*. New York: Peter Lang.

Fontanille J (2008) *Pratiques sémiotiques*. Paris: PUF.

Fossati G (2018) *From Grain to Pixel. The Archival Life of Film in Transition* (Third Revised Edition). Amsterdam: Amsterdam University Press.

Franck G (2019) The economy of attention. *Journal of Sociology*, 55(1): 8-19. https://doi.org/10.1177/1440783318811778

Groupe, μ (1992) *Traité du signe visuel*. Paris: Les éditions du Seuil.

Impett L and Moretti F (2017) Totentanz. Operationalizing Aby Warburg's Pathosformeln. *Literary Lab Pamphlet 16*. https://litlab.stanford.edu/LiteraryLabPamphlet16.pdf

Kress G and van Leeuwen T (2021) *Reading Images: The Grammar of Visual Design* (Third edition). New York: Routledge.

Kaiser J and Rauchfleisch A (2020) Birds of a Feather Get Recommended Together: Algorithmic Homophily in YouTube's Channel Recommendations in the United States and Germany. *Social Media + Society* 6(4), 1-15.

Kirdemir B, Kready J, Mead E, Hussain MN and Agarwal N (2021) Examining Video Recommendation Bias on YouTube. In: Boratto L, Faralli S, Marras M and Stilo G (eds.) *Advances in Bias and Fairness in Information Retrieval*. Cham: Springer, 106-116. https://doi.org/10.1007/978-3-030-78818-6_10

Lanham L (2006) *The economics of attention: Style and substance in the age of information*. Chicago: University of Chicago Press.

Leone M (ed) (2024) *The Hybrid Face: Paradoxes of the Visage in the Digital Era*. New York: Routledge.

Leyda J (1964) *Films Beget Films: A Study of the Compilation Film*. New York: Hill and Wang.

Light R and Moody J (eds) (2020) *The Oxford Handbook of Social Networks*. Oxford: Oxford University Press.

Manovich L (2001) *The Language of New Media*. Cambridge (MA): MIT Press.

Manovich L (2017) *Instagram and Contemporary Image*. New York: Creative Commons license.

Manovich L (2020) *Cultural Analytics*. Cambridge (MA): Mit Press.
Manovich L and Arielli M (2021-2024) *Artificial Aesthetics: Generative AI, art and visual media*.
http://manovich.net/index.php/projects/artificial-aesthetics

Marin L (2002) *On Representation*. Stanford: Stanford University Press.

Parikka J (2023) *Operational Images: From the Visual to the Invisual*. Minneapolis: University of Minnesota Press.

Pinotti A (2017) *Self-Negating Images: Towards An-Iconology*. *Proceedings of the International and Interdisciplinary Conference IMMAGINI?*, 1(9), 856. DOI: https://doi.org/10.3390/proceedings1090856.

Rodowick D (2007) *The Virtual Life of Film*. Cambridge: Harvard University Press, 2007.

Seguin B (2018) *Making large art historical photo archives searchable*. Ph. D. Thesis. EPFL Scientific Publications. Available at: https://infoscience.epfl.ch/record/261212 (Accessed: 26 January 2024).

Stoichita V (1997) *The Self-Aware Image: An Insight Into Early Modern Meta-Painting*. Cambridge: Cambridge Press.

Treleani M (2017) *Qu'est-ce que le patrimoine numérique? Une sémiologie de la circulation des archives*. Lormont: Le Bord de l'Eau.

<footnote n="1">¹ By 'images', we mean to refer to a broad meaning of visual productions that includes their compositional dimension, the way they circulate, the functions they perform on the public scene, and the operations they allow to be carried out through their specific resources (diagrammatic and representational). Regarding algorithms, we are referring in a broad sense to the operations that can be performed thanks to computer tools and that consist of a vast array of computational manipulations: algorithms for the analysis and modelling of phenomena, for the generation of new content or for the automatic socialisation of discourses. This broad definition follows current usage in computer science: any operation to modify digital data, from moving icons on a desktop to the generation of images by the most modern artificial intelligences such as ChatGPT and Midjourney.</footnote>

<footnote n="2">² The semiotic approach is of course already informed by and in dialogue with these fields. Two examples of this convergence are the integration of the work by art historian Victor Stoichita (1997) into the semiotics of the Paris school (Dondero 2020), and the work of Johanna Drucker (2020) and David Berry (2012) relating semiotics and the digital humanities.</footnote>

<footnote n="3">³ More generally, we find the semiotics of practices to converge with the most recent proposals in the field of semiotic technology and practices (Djonov and van Leeuwen, 2018).</footnote>