## GROUND TRUTHING IN DIGITAL TECHNOCULTURE

## Vera Tollmann

The method of ground truth implies the mechanical production of a reference measurement that is grounded to the earth's surface, most commonly applied in remote sensing. In the age of machine learning, the automated production of reference data grounded in a dataset is also referred to as ground truth, indicating an epistemic shift. Media scholar  Jussi  Parikka  points  out  that  AI  models  influence  how remotely sensed data is collected and interpreted. 1

In the case of automatic reading of satellite data with AI applications, which in turn were trained on validated satellite data, the two areas of application-remote sensing and machine learning-seem less separate. 2 In the context of his research on operational imagery, Parikka describes the processes as loop-like, 'grounds are formatted (organized, divided, classified) for epistemic purposes to be used as training data and establish an operational chain: ground to satellite to ground again to data for further imaging.' 3 Besides these two established practices, there is also the application-or adaptation-of ground truth in the creative research practice of Forensic Architecture (FA), where ground truth is listed as part of a set of methodologies, claiming objectivity, precision, and reliability. The third practice cannot be considered separately from the first two areas of application. Looking into individual FA projects, ground truthing implies different processes and procedures.

In machine learning, 'ground' turned into a metaphor, since validation no longer refers to the earth's surface as in remote sensing. In this sense, the question of 'ground truth' can be considered as a signal or stand-in for an epistemic shift. On the other hand, in the field of political geography, the concept of ground truth is questioned for its military origins and persuasiveness. Within the FA projects, ground truth as a methodology can describe both a validation of the earth's surface and a validation of human activity on the ground, recorded from a less remote position. A relevant difference is that in the case of FA projects, the ground truth is generally not collected by the same institutions as the aerial data and it is occasionally based on crowd-sourced data uploaded to social media platforms or shared by non-governmental organizations.

This text aims to contribute to a critical reflection on the extent to which looking at ground truthing practices can potentially offer an approach to studying epistemic shifts in remote sensing, mapping, 4 machine  learning  models,  and  in  the  notion  of  the  ground. Disciplines such as media studies critically question the objectivity and knowledge that seems to surround ground truth and the resulting representations. Media scholar Caren Kaplan therefore refers to 'ground truths' in the plural to question the integrity and certainty of ground-truthing in the context of mapping, where the conventional practices inevitably lead to partial representations. 5 In a similar vein, Parikka questions the 'truth' of operational images, which 'are full of errors and are far from the disembodied rational gaze they are (sometimes) projected to be.' 6

My questions on ground truth are: What is taken to be the ground? What is this ground made of? Subsequently, what is ground truth or the practice of ground truthing? And does truth in these operations mean precision and objectivity?

Ground truth for reading the earth's surface is an essential step in the operation of remote sensing and a common practice in geo science, geology, and archaeology. 7 A Dictionary of Geography lists several meanings of ground, among which are ground in the sense of the base or foundation on which institutions or systems operate, or the earth as the surface on which humans, animals, and plants exist: 'In remote sensing, ground truth refers to the comparison of remotely  sensed  data  with  real  features  and  materials  on  the ground.' 8 Ground truth, in other words, derives from the 'facts that are found when a location shown on a map, air photograph, or satellite image is checked on the ground, as validation.' 9 Or, in the Oxford English Dictionary (OED) it is defined as 'information obtained by direct measurement at ground level' 10 and The dates its earliest use to 1965, where, interestingly, the field of space exploration recognized a need for ground truth.

In its verb form, ground truthing involves the correlation and validation of remotely sensed data-aerial photographs, satellite datawith ground-based data (traditionally recorded in situ on the ground or, in the event the ground cannot be walked on, near the ground from helicopters, for example). According to the OED, ground truthing is 'the action or process of establishing ground truth,' and it was first used in 1979 in a journal dealing with natural resources; in particular, with ground truthing of overflights. 11

It took a few more years before the alternative verb form 'to groundtruth' came into use, in the 1980s. Ground truth data are field data including meteorological parameters and plant parameters; use cases apply statistical analyses to validate the classification of data obtained by remote sensing. Since the invention of drones, unmanned aerial systems equipped with high-resolution sensors have been used  as  affordable  and  durable  surrogates  for  humans-on-theground in inaccessible areas and rough terrain. In a lecture on ground truth, Kate Crawford points out that the first Corona satellites were launched as early as 1959 and may have required a process of ground truthing. 12 Her perspective emphasizes the humans in the loop, who are collecting, organizing, validating, and interpreting the data.

The use of the term ground truth in computer simulations and algorithms dates back nearly as far as remote sensing. The OED states that, in this sense, ground truth describes 'a set of data that is considered to be accurate and reliable, and is used to calibrate a model, algorithm, procedure, etc.' 13 In 1977 , a published source cited in the dictionary  conceptualized an image as ground truth, taking an original image as ground truth.

## PROBLEMS WITH GROUND TRUTH

Geoscientist Iain Woodhouse has used database searches to find that the term ground truth has been in use in remote sensing since 1965 and that the term meanwhile has also become widespread in disciplines such as neuroscience, computer science, and engineering, 14 which might go hand in hand with what Karen Gregory, Patricia Clough, Joshua Scannell, and Benjamin Haber called the 'the datalogical turn' in 2015, marking a paradigm shift in which the power of computing affects various disciplines, their methods, and their objects  of  study,  triggering  various  epistemological  crises;  for example, in cartography or in theories of photography. 15

Whereas Crawford's approach to 'ground truth' expands on existing definitions and refers to ground in the sense of material foundations, such as resources, and immaterial foundations, such as concepts, categories,  and  training  data,  Woodhouse  proposes  alternative terms for his discipline, geoscience-such as 'in situ measurement,' 'validation measurement,' or 'best available measurement'-as alternatives to communicate levels of uncertainty rather than claiming 'the truth.' 16 The limited accuracy of the measuring instruments may be due to measurement uncertainty, limited ground resolution of the satellite sensing instruments, or political 'shutter control.' Ground data usually cannot be recorded at the same time as the remotely sensed data is collected. In Woodhouse's view, the conventional understanding of 'ground truth is prescriptive … [i]t leaves no room for doubt.' 17 Below, I will consider political geographer Louise Amoore, who introduces doubt into her theoretical approach to critiquing the assumptions of machine learning.

Big Science institutions such as NASA or USGS conduct ground truth operations oriented to a particular Landsat scene, which measures 185 by 185 kilometers. A 'conventional' approach involves a report on the experiment setup; for example, a real-time ground truth experiment carried out in 2004 in the Utah desert. In a short report, two NASA researchers describe the smooth collaboration of a computer network and scientists, a field geographer, and a geologist taking spectrum measurements on the ground while being imaged by a satellite (EO-1) from above. 18 In comparison, in a NASA paper published in 1978, the procedures for gathering ground truth information for a 'supervized' approach to a computer-implemented land cover classification of multispectral satellite data in the Landsat program pointed  to  problems  with  ground  truthing  and  the  selection  of sample sites, and 'recommended that an attempt be made to establish at least three training sample sites for each land cover feature.' 19 The report further noted that problems with sample sites and data correlation can arise during validation and are located at the technical level-for example, scan line dropout and transmission failureor at the level of the difference in perception of human and machine, as stated in the report. A training sample can statistically appear uniform from a human viewpoint, whereas, the report explains, a machine's spectral viewpoint can 'see' less homogenous ground.

## FROM GROUND TO IMAGES TO GROUND

In the mid-1990s, John Pickles wrote in Ground Truth that new forms of ground truthing originate from the convergence of techniques for advanced computing and enhanced imaging. He noticed a 'renewed importance of the visual image,' 20 and elaborated on new technologies for gathering, analyzing, and mapping geographic data, as well as the implications they have for our understanding of nature and social life. He also observed the transformation of data handling and mapping capabilities, treating a geographic information system (GIS) as both technique and social relation.

Prior to the rise of machine vision and machine learning, the GIS computational environments 'change the nature of ground truth, where mediations are distanced from the actual ground as a material and lived environment.' 21 In their co-authored text, Abelardo GilFournier and Parikka argue that ground truth has shifted from a reference to the physical, geographical ground to the surface of images (earlier than recorded in the history of remote sensing), further noting that 'while ground truth refers to a set of remote sensing practices, it has a longer history in operational photography, such as aerial reconnaissance.' 22

In an article published in art magazine Artforum in 1975, the photographer and writer Allan Sekula already pointed to the trajectory of a longer history when reflecting on Edward Steichen's aerial photographs as instrumental images. Sekula already considered the aerial  photograph  to  be  ''grounded'  by  the  digital  logic  of  the grid' 23 ; that is, it is highly constructed. Moving beyond the limitations of photography at the time meant improving its quality in terms of resolution and focus by means of a computer. According to Parrika and Gil-Fournier, 'fabricating surfaces became thus one key format of truth established through aesthetic crafts, photography, and technologies of ascent.' 24 With the existence of aerial images, landscapes were transformed into 'readable surfaces.' Nowadays, the assemblage has developed into a more complex set of operations and technologies.

Following Gil-Fournier and Parikka, ground truth can be thought of as both an epistemic and technical figure of knowledge. Images read as ground truth turn into operational images; that is, 'images that do not primarily represent but operate.' 25 In  processes of machine learning, 'ground truth is read from a mass of images.' 26 The concept, as Gil-Fournier and Parikka write, includes 'operations of comparison, synthesis, synchronization, calibration that define the scope of ground truth as it emerges as a media technique.' 27 This media technique is also employed, for example, by Google's Ground Truth team on their map service. The proprietary internal software platform for the team's editing work is called Atlas and functions as a deep map 'behind' Google Maps, the master-map into which multiple purchased, public, and Google-owned data sources are integrated. 28 Ground Truthing in this framework is a combination of deep learning algorithms and manual labor-which means also manually checking and correcting maps, based on Google Street View images. In this case, Google Street View provides ground truth data for the map. 29

## GROUND TRUTH AS A MEDIA TECHNIQUE OF FORENSIC ARCHITECTURE

Returning to Forensic Architecture, for the agency, ground truth describes 'the technique of anchoring the results of a computational or digital process to a precise location in the real world' 30 ;  for example, the calibration of one mode of representation, pixels of a satellite image, with an archival aerial photograph and records on the ground. In its investigations, FA uses ground truth among other methodologies listed on its website, such as 3D modeling, data mining, fieldwork, geolocation, machine learning, open-source intelligence, pattern analysis, photogrammetry, and remote sensing, to read and interpret sources.

In FA ' s projects in human rights activism, the concept of ground truth and the practice of ground truthing take many forms. Unlike the rules and instructions for ground truth that exist in remote sensing, 31 FA seems to operate anew in each project, operationalizing moving images and data: a 3D model of a site acts as a display for the collected sources and synthesizes the investigation's findings. The 3D model provides a framework for the spatial use of images and unflattens the images from 2D to 3D. 'Truth' is a work in progress, especially in the case of some investigations lasting several years documenting re-occurring cases (such as Drift-backs in the Aegean Sea, 2020-), while others focus on a close reading of events that took place within a few hours (such as the 2020 to 2022 investigations of the racist far-Right terror attack in Hanau in 2020).

The agency refers to ground truthing as a method to creatively intervene in existing conventions of computer vision, remote sensing, or mapping, converging a diverse set of footage from different sources. Findings are presented online or in the format of video installations, which often adopt the status of a work of art when on display in art institutions. In this context, it is not evident to the viewer whether the work was subjected to scientific scrutiny, as it plays into a politicized digital public sphere.

In a media archaeological approach to operational images, Parikka adds a historical perspective to a research field in media studies which developed from a conceptual term initially coined by filmmaker Harun Farocki, who was looking at operations in military and management. Parikka defines operational images as 'images that become operationalized for data analysis,' further asking 'what objects  and  processes  are  involved  in  and  produced  by  those operations of knowing?' 32 In his writing on the media of forensics, film scholar Simon Rothöhler emphasized that FA does not analyze sources ''vertically' , but 'horizontally'; the analysis is based on the relations, not the quality of the individual sources.' 33

The spatial and temporal analysis of visual sources is what has defined FA's investigations from the beginning. In a book co-authored with Eyal Weizman, and in another article, human rights scholar and FA collaborator Thomas Keenan elaborates on historical models that inspired the work, including investigations into Mengele's skull, which was found in Brazil in 1985. Keenan wrote that 'the decisive technique in the Mengele identification involved the superimposition of video images of the skull and of a photograph  of  the  person,'  further  stressing  'a  fundamental  lack  of absolute certainty.' 34

In their research on NGO activism, Delf Rothe and David Shim point out that using satellite imagery is not so much challenging the state's sovereign gaze as strengthening it. 35 In 2014, when FA was in its early years, Keenan argued that the 'investigative use of […] satellite photography must acknowledge its military history and 'resolution biases' .' 36 Rothe and Shim also note that the assumption of easy access to satellite data ignores the fact that the infrastructure of satellite systems is still in the hands of a few, predominantly Western, states and companies. Furthermore, the availability of these data depends on commercial interests and their use requires a certain level of expertise.

## AL-AHLI ARAB HOSPITAL

In the following I will examine a specific use case of ground truthing; namely, FA 's investigation into the explosion at Al-Ahli Arab hospital in Gaza City on October 17 , 2023. I chose this example because this particular investigation can be compared with journalistic investigations of the same event. The comparison is productive in regard to how the concept of ground truth is handled. As Kaplan points out, a map is hardly a complete representation of the world; rather, it employs conventions that are useful for legibility, usability, and knowledge transfer. Nevertheless, following the lead of philosopher Quill Kukla, who explores the epistemic risks in map-making (i.e., aesthetic, categorization, and simplification risks), I want to ask whether it is a good map. 37

Three days prior to the explosion on October 17 , 2023, the hospital building, according to media reports, had already been hit by rockets that were supposedly fired by the Israeli Defense Forces (IDF) in an airstrike. The explosion on October 17 was much more devastating than that on October 14. Both the estimates of the number of casualties and explanations for the cause of the later explosion vary widely. What remains undisputed is that many people died through this explosion, and that the incident was horrible. Targeted bombings of civilian infrastructure like a hospital are forbidden by international law, and any such explosion at a medical facility in a war zone would be a matter that would be scrutinized by all kinds of actors: war parties, international organizations, the media, and NGOs.

Various states, organizations, and media outlets investigated the explosion at Al-Ahli Arab Hospital and came up with different explanations about what exactly caused it. The most common explanation is that it was caused by a malfunctioning rocket fired from within Gaza, presumably by members of the Palestinian Islamic Jihad. For this study, the explosion and the investigations into how it happened and who ultimately may be responsible is not intended to suggest responsibility. In this case, there was little openly available data for investigation; for example, the New York Times ( NYT )  used four videos and the IDF released a few audio recordings.

FA has embedded its investigation on the hospital explosion in a wider investigation into various attacks on hospitals in Gaza. The presentation form is highly interesting, and the agency's listing of various cases seems to establish a pattern of IDF attacks on hospitals  in  Gaza. 38 On  October 20, FA posted on the social media platform X, '3D analysis shows patterns of radial fragmentation on the southwest side of the impact crater, as well as a shallow channel leading into the crater from the northeast.' 39 According to the same post, the 'analysis of the crater size suggests a munition larger than eg a Spike or Hellfire missile commonly used by IOF drones.' FA often uses the abbreviation IOF for 'Israeli Offense Forces' instead of the commonly used IDF.

To return to Kukla's representational risks, the key is to critically approach a map and search for 'patterns it may be occluding.' Kukla points out that 'categorization choices must be made on the basis of values and interests. In turn, they shape what patterns the map reveals and what patterns it hides.' 40 Kukla  concludes that the choice of values and categorization cannot be taken neutrally. 41 In FA's online presentation, colors and the mode of scroll-down presentation in the form of a cumulative map suggests a strategic attack and seemingly points out patterns to be discerned from the events. The map does not provide any reasons, it shows a dramatic humanitarian situation in an abstract form.

On 3 November, in the The Morning Newsletter, NYT editor David Leonhardt  summarized  several  media  investigations  into  the hospital attack. The article divides the sources into 'video of the air' and 'video from the ground.' 42 Leonhardt refers to a colleague from the Visual Investigations team that exposed flaws in the footage analysis. NYT reporters used additional video camera recordings to conclude  that  the  projectile  came  from  Israel. 43 Their  ground analysis showed that the impact of the explosion appears consistent with the rockets that Palestinian groups were launching toward Israel that night. On 24 October, NYT published a detailed visual analysis that already had a different view of the situation, and the journalists' methodology is explained extensively. 44 The analysis concludes that the video clip taken from an Al Jazeera television camera livestreaming on the night of October 17 shows something else. To trace the missile object in the sky back to Israeli territory, NYT synchronized the Al Jazeera footage with five other videos filmed at the same time, including footage from an Israeli television station, Channel 12, and a CCTV camera in Tel Aviv. These different videos provided a view of the missile from four cardinal directions. Using satellite imagery to triangulate the launch point in those videos, NYT determined that the projectile was fired toward Gaza from near the Israeli town of Nahal Oz shortly before the deadly hospital blast. The Israeli military stated it does not fire Iron Dome interceptors into Gaza. 45

In an article in the magazine Art in America published in March 2023, editor Emily Watlington wrote that 'too often, in effect, FA 's impressive handling of the technology becomes the narrative's protagonist.' 46 Rothöhler  notes  that  the  practice  'has  become  conventionalized not only as a 'best practice' but also as a genre.' 47 FA's Eyal Weizman, in an interview conducted in 2021, elaborates on the notion of the 'hyper-aesthetic image' and the phenomenon of the FA methods and aesthetics being adopted by big newspapers such as NYT and The Washington Post . At the time of the interview in 2021, Weizman moved on to new data sources, recognizing an increase in operational images and algorithms and looking at leaves as sensors of weaponized pesticides. 48

A recent term, 'investigative aesthetics,' was introduced in the book of the same title, which Weizman co-edited with Matthew Fuller. The authors have in mind 'an alternative, and rigorous, collective and diverse set of truth practices.' 49 In an earlier iteration on FA 's truth practices (in contrast with the notion of post-truth), Weizman wrote that the intention is to 'challenge both the dark epistemology of the present as well as traditional notions of truth production.' 50 According to Weizman, the anti-epistemological drift of post-truth 'blocks one's ability to evaluate and debate facts.' 51

## HUMANS IN THE LOOP

More recently, with the rise of machine learning, the term ground truth has also been used to refer to the data input, the human-labeled training data, from which machine learning systems 'learn' to detect, match, categorize, and so on. FA developed a training dataset specific  to  their  field  of  research  and  a  video  in  this  area. 52 Researchers propose new conceptualizations of 'ground truths' more appropriate to the world relations of machine learning, arguing for an understanding of ground truth as relational and 'read from a mass of images,' 53 'ground truthing as a very human construction,' 54 a need for 'embodied doubt' 55 towards the output of the machine and a 'model's version of truth.' 56

In  conceptualizing  ground  truth  as  relational,  Parikka  and  GilFournier emphasize that synthetic models and synthetic ground truths are not a recent development, but emerged much earlier than the latest AI training-data practices. In the history of astronomy , for example, reality had to be approached through models of reality, causing a decentering of the ground (truth). 57 Retracing the operational use of images for research and verification purposes and the operationalization of pattern recognition from early iterations of photogrammetry  through  photomosaics  to  machine  vision,  the authors conclude that what they call the 'the image-map complex,' 58 the use of images for the purpose of mapping, shifted to images as ground. In the more data-driven forms of satellite remote sensing, ground truth is mostly defined through processes that validate and adjust a data model; thus, as it evolves, the ground is no longer a relatable physical entity , nor material as such, 'but a shifting set of techniques in which the ground is constantly established and calibrated.' 59

In Atlas of AI , Crawford gauges available training data as 'a brittle form of ground truth.' 60 In her 2022 lecture 'Excavating 'Ground Truth' in AI: Epistemologies and Politics in Training Data,' she elaborated on a fourfold ground in deep learning, 61 stating that AI operates under the banner of science and objectivity. Her question concerns how worldviews are built at a technical level. Crawford looks into the concept of ground truth in four dimensions, developing a stable form of meta ground truth or critical ground truth reflection. She points to epistemic shifts in AI, such as an enormous shift in scale (much larger datasets) and overall different ways of manufacturing ground truth, including crowdsourcing methods. 62 Besides the dimension of labor, she addresses contested concepts and the problem of pattern discrimination. 63 Last but not least, she expands the notion of ground truth to environmental consequences. To summarize, the expanded concept of ground truth includes the bodies of the workers as well as mining sites.

Louise Amoore discusses further problems with ground truth, such as the ground truth generated by deep-learning algorithms. Neural network architectures can contain multiple hidden layers, hundreds of millions of weights, and billions of potential connections between neurons. That is to say, the machine learning algorithm must reduce the vast multiplicity of possible pathways to a single output. Amoore proposes 'embodied doubt,' which for her means 'giving doubtful accounts of the output of a calculation.' 64 Amoore identifies further problems:

as deep learning algorithms derive their own ground truth by clustering raw unlabeled data, a model of what is 'normal' in the data is generated by the algorithms. The claim to truth made by machine  learning  algorithms,  then,  is  not  one  that  can  be opposed to error or falsity. Rather, the algorithm learns from the degree of probabilistic similarity with a ground truth, itself often generated by algorithms. 65

One might want to ask whether deep learning models are feeding the 'age of unreason.'

The centrality of ground truths for the design and evaluation of algorithms strongly suggests that, to a certain extent, we get the algorithms of our ground truths. This means that current AI models are built on self-referential loops, and, as Luke Munn, Liam Magee, and Vanicka Arora observe, training datasets are 'further massaged by human evaluators and their preferences, shifting the 'ground' upon which future predictions are made.' 66 Similar to Crawford, who pointed to the constructedness of ground truth in machine learning, the group of researchers includes corporate values and user preferences as influential, and argue for a relational understanding, as do Parikka and Gil-Fournier, with an emphasis on humans in the loop, from user expectations to programmed settings and human instructions, resulting in new weightings of truth. 67

Munn, Magee, and Arora are referring to RLHF (reinforcement learning from human feedback), which aims to improve the core GPT model; they also note that the datasets are artificially generated and thereby shifting the boundaries to a level that is further removed from the real world (or 'truth'). They call this 'the model's version of truth.' 68

In addition, the increased complexity of AI tasks has eroded the former stability of ground truths; agreement about 'the truth' must continually be negotiated. 69 These decisions may lead to a version of ground truth that is incomplete or inadequate in subtle ways. For instance, various AI models unexpectedly failed when employed in a real healthcare scenario, because they lacked the rich tacit knowledge doctors gained from years in the field: the ground truth accounted for 'what' but did not account for 'how.' 70

In this way, the notion of ground truth seems to provide a sense of epistemic certainty, as it is supposedly derived from an unmediated set of facts that emerged from the ground, even though research showed that measurement devices and methods involve a back and forth of comparison and synthesis. 71 Truth is a calculation; that is, the closer the supervised training comes to the ground truth, the more it is judged to be true. 72 In addition, the increased complexity of AI tasks demands larger and more specific ground truths datasets, and therefore 'the truth' is a work-in-process. 73 These decisions can lead to partial versions of the ground truth; as in the real-world healthcare scenario just mentioned. 74

## TRUTH-MAKING AND TRUTH EFFECTS

After reconsidering ground truth in terms of ground, images, and media techniques, the notion of truth itself needs further discussion. Media philosopher Sybille Krämer writes about witnessing as a truth practice and elaborates that witnessing is twofold, discursive, and existential, in the sense of embodied witnessing. In the media age, witnessing has turned into witnessing via media footage, conceptualized as digital witnessing, 75 networked witnessing, 76 or media witnessing. 77 To make it more tangible, witnessing takes place via television, social media, in the arts, in truth commissions, and in the courts. Krämer lists a plethora of cultural phenomena of witnessing, which, one might want to add, appear in the practice of FA as another creative and agency-driven form of witnessing. While Krämer's thinking  is  tied  to  the  witness  who  testifies  through  language, through 'being present,' witnesses find therselves in the paradoxical situation of personalization and depersonalization, act as 'the embodied 'depersonalized' symptom of a past situation.' 78

Compared to the conventions of the courtroom and the problematic paradoxes that follow, FA 's integration of witness accounts recorded in all kinds of situations and under all kinds of circumstances differs from conventional testimonial truth-telling in an institutional space. Looking at the design and the navigability, FA 's diverse projects benefit from a general belief in and a visual familiarity with the practices  of  scientific  truth-making  and  perceptual  conventions. Therefore, at first glance there is a potential conflict arising from the 'persuasive power of images' that develops from the perspective of the general public. It is therefore interesting to revisit an essay published by artist and theorist Hito Steyerl twenty years ago. In the context of thinking about the documentary, documentarism, and documentality, Steyerl reviewed a project by the research collective Multiplicity, founded by architect Stefano Boeri and fellow Italian architects, sociologists, cartographers, photographers (the collective was active before FA). Steyerl takes an example that can be seen as a precursor to a practice later professionalized and established by FA. In  the  essay,  Steyerl  discussed  'truth  effects,'  the  footage  that Multiplicity used at the time to critically discuss and document a ship that sank with migrants on board. On the night of December 26, 1996, a ship carrying 283 illegal Sinhalese immigrants from Malta to the Italian coast sank a few miles off the south-east coast of Sicily . The passengers did not survive. Multiplicity used various data recordings from the tragedy in the Mediterranean, using media archaeological techniques and combining different forms of representation such as maps, photographs, videos, and multiple research formats such as interviews,  reportages,  statistics,  and  shadowings.  Among  the footage was a small-scale and blurred underwater shot captured by a robot-operated camera, which was sent into the depths in June 2001, after a fisherman found a plastic-wrapped identity card and gave it to the daily newspaper La Repubblica .

For Steyerl, the research project and installation Solid Sea 01: The Ghost Ship (2002) lacked critical reflection on or juxtaposition of the footage, while it aimed at an alternative or critical public sphere, using widespread (meteorological, journalistic, archaeological) truth technologies and truth-making, with all the individual components seeming to follow a journalistic register. Steyerl is interested in the question of how a critique of traditional aesthetic truth politics can be carried out. 79 The traditional means of critique, reflexivity, has changed its meaning because the integration of amateurish, seemingly unconstructed footage, which was previously seen as a critical statement, integrates these means into dominant truth politics.

More than twenty years after the Multiplicity project, FA operates on a different scale in terms of duration, number of cases, and volume of data analyzed. The previously mentioned 'Drift-backs in the Aegean Sea' is a video and an interactive cartographic platform that collects evidence relating to more than 2000 cases of driftbacks that occurred near Greek islands between February 2020 and February 2023. 80 The individual recordings are variously categorized as verified , documented , and trajectory depending on evidence provided by available sources, while locations are logged as precise or approximate. In Tom Gunning's short text on the fear of fake photographs in light of digitization and the risk of manipulation, he makes clear that the match of truth and photography had always been  a  'truth  claim'  based  on  indexicality  and  visual  literacy. Following Gunning, truth is not inherent to photography, but a true photograph is 'subjected to a series of discourses' and rules. 81 In this case, it is the degree of detail regarding the methodology and ethics communicated by the FA research team.

The areas of applications of ground truthing as a methodology-in remote sensing and in machine learning-do share a degree of constructedness, uncertainty, and approximation. Researchers point to the future relevance of machine learning algorithms and more automated processes that require a large number of training samples, which is the main obstacle, as these data cannot be 'grabbed' online in digital networks. 82 In ground truth practices, the number of valid ground samples does not yet reach the threshold for big data, even though new datasets are being assembled and graphic user interfaces  called  SemiTruth  are  being  programmed. 83 FA ' s  practice  is situated between these two fields, and at the same time not preoccupied with the reflection of the technological conditions of methodologies, but rather follows ground truth as an 'epistemic figure of knowledge.' In this sense, the question of 'ground truth' can be considered as a signal or proxy for an epistemic shift.

- 1 Sabine Weiser, 'Beyond Visible Light: Interview With Media Theorist Jussi Parikka,' Springerin 2 (2023), accessed June 17, 2024, https://www.springerin.at/ en/2023/2/jenseits-des-sichtbaren-lichts/.
- 2 The start-up companies LiveEO, Orbital Insight, AiDash, or AI Superior (to name but a few examples) apply data analysis and machine learning to Earth observation data, from RGB and multispectral to LiDAR and SAR data. The task is to make timely predictions for operations and maintenance activities. Up42 is a start-up company founded in Berlin in 2019 that sees itself as an aggregation service provider. It aggregates satellite data from Airbus, Hexagon, Sentinel, etc. and image recognition algorithms from LiveEO, Orbital Insight, etc.
- 3 Jussi Parikka, Operational Images: From the Visual to the Invisual (Minneapolis: University of Minnesota Press, 2023), 150.
- 4 'It must be recognized that no map, however detailed or carefully compiled, can perfectly represent the 'ground truth'.' Ian Masser and Michael Blakemore, Handling Geographical Information (London: British National Corpus, 1991), 80.
- 5 Caren Kaplan, Aerial Aftermaths: Wartime from Above (Durham: Duke University Press, 2018), 70.
- 6 Parikka, Operational Images , 136-37.
- 7 Lisa Parks, Cultures in Orbit: Satellites and the Televisual (Durham: Duke University Press, 2005), 77-98.
- 8 Susan Mayhew, A Dictionary of Geography (Oxford: Oxford University Press, 2015), 230, https://www. oxfordreference.com/view/10.1093/ acref/9780199680856.001.0001/ acref-9780199680856.
- 9 See https://www.oxfordreference.com/display/10.1093/ oi/authority.20110803m,…095909730.
- 10 Oxford English Dictionary , s.v. 'ground truth ( n. ), sense 2.a,' accessed June 17, 2024, https://doi.org/10.1093/ OED/4788016561.
- 11 Oxford English Dictionary , s.v. 'ground truthing (n.),' accessed June 17, 2024, https://doi.org/10.1093/ OED/4416596675.
- 12 Kate Crawford, 'Excavating 'Ground Truth' in AI: Epistemologies and Politics in Training Data,' UC Berkeley, March 27, 2022, accessed June 17, 2024, https://www.youtube.com/watch?v=89NNrQULm\_Q.
- 13 Oxford English Dictionary , s.v. 'ground truth (n.), sense 2.b,' accessed June 17, 2024, https://doi.org/10.1093/ OED/6802225928.
- 14 Iain H. Woodhouse, 'On 'Ground' Truth and Why We Should Abandon the Term,' Journal of Applied Remote Sensing 15, no. 4 (2021), https://doi.org/10.1117/1. JRS.15.041501.
- 15 Karen Gregory, Patricia Clough, Joshua Scannell, and Benjamin Haber, 'The Datalogical Turn,' in NonRepresentational Methodologies: Re-Envisioning Research, ed. Phillip Vannini (New York: Routledge, 2015), 153.
- 16 Woodhouse, 'On 'Ground' Truth.'
- 17 Woodhouse, 'On 'Ground' Truth.'
- 18 Isaac Lopez and Marc A. Seibert, 'NASA GroundTruthing Capabilities Demonstrated' (Cleveland: NASA Glenn Research Center, 2013), https://ntrs.nasa.gov/ citations/20050192154.
- 19 Armond T. Joyce, 'Procedures for Gathering Ground Truth Information' (Washington, DC: NASA RP-1015, 1978), 8, see instruction sheet, 41.
- 20  John Pickles, Ground Truth: The Social Implications of Geographic Information Systems (New York: Guilford Publications, 1995), vii.
- 21  Abelardo Gil-Fournier and Jussi Parikka, 'Ground Truth to Fake Geographies: Machine Vision and Learning in Visual Practices,' AI & SOCIETY 36, no. 4 (2021): 1254, https://doi.org/10.1007/s00146-020-01062-3.
- 22  Gil-Fournier and Parikka, 'Ground Truth,' 1253.
- 23  Allan Sekula, 'The Instrumental Image: Steichen at War,' Artforum 14, no. 4 (December 1975): 28.
- 24  Jussi Parikka and Abdel Gil-Fournier, 'An Ecoaesthetic of Vegetal Surfaces: On Seed, Image, Ground as Soft Montage,' Journal of Visual Art Practice 20 , no. 1-2 (2021): 16-30, https://doi.org/10.1080/14702029.202 1.1917858.
- 25  Gil-Fournier and Parikka, 'Ground Truth,' 1254.
- 26  Gil-Fournier and Parikka, 'Ground Truth,' 1254.
- 27  Gil-Fournier and Parikka, 'Ground Truth,' 1257.
- 28  Scott McQuire, 'One Map to Tule Them All? Google Maps as Digital Technical Object,' Communication and the Public 4, no. 2 (2019): 150-65, https://doi. org/10.1177/2057047319850192.
- 29  Zbigniew Wojna et al., 'Attention-Based Extraction of Structured Information from Street View Imagery,' in 14th IAPR International Conference on Document Analysis and Recognition (ICDAR) (Kyoto: IEEE, 2017), 844--50, https://doi.org/10.1109/ICDAR.2017.143.
- 30  Forensic Architecture, 'Methodology,' accessed June 17, 2024, https://forensic-architecture.org/ methodology/ground-truth.
- 31  See Joyce, 'Procedures for Gathering.'
- 32  Parikka, Operational Images , 6; 39.
- 33  Rothöhler, Medien der Forensik (Bielefeld: transcript Verlag, 2021), 143.
- 34  Thomas Keenan, 'Getting the Dead to Tell Me What Happened: Justice, Prosopopoeia, and Forensic Afterlives,' in Forensis: The Architecture of Public Truth , ed. Forensic Architecture (Berlin: Sternberg Press, 2014), 39. Also see Eyal Weizman and Thomas Keenan, Mengele's Skull (Berlin: Sternberg Press, 2012).
- 35  Delf Rothe and David Shim, 'Sensing the Ground: On the Global Politics of Satellite-based Activism,' Review of International Studies 44 , no. 3 (2018): 414-37, https:// doi.org/10.1017/S0260210517000602.
- 36  Eyal Weizman and Matthew Fuller, Investigative Aesthetics: Conflicts and Commons in the Politics of Truth (New York: Verso, 2021), 16.
- 37  Quill R. Kukla, 'The Epistemology of Maps,' PhilArchive , archived on July 29, 2021, https://philarchive.org/rec/ KUKTEO-2.

- 38  See Forensic Architecture, 'Destruction of Medical Infrastructure in Gaza,' December 12, 2023, accessed June 17, 2024, https://forensic-architecture.org/ investigation/ destruction-of-medical-infrastructure-in-gaza.
- 39  @ForensicArchi (October 20, 2023), https://x.com/ ForensicArchi/status/1715422498852843574.
- 40  Kukla, 'Epistemology of Maps.'
- 41 Kukla, 'Epistemology of Maps.'
- 42 David Leonhardt, 'Revisiting the Gaza Hospital Explosion,' The Morning Newsletter ( New York Times ), November 3, 2023, accessed June 17, 2024, https:// www.nytimes.com/2023/11/03/briefing/gaza-hospitalexplosion.html.
- 43  Leonhardt, 'Revisiting Gaza Hospital Explosion.'
- 44  Aric Toler et al., 'A Close Look at Some Key Evidence in the Gaza Hospital Blast,' The New York Times , October 24, 2023, https://www.nytimes.com/2023/10/24/ world/middleeast/gaza-hospital-israel-hamas-video. html.
- 45  Toler et al., 'Close Look.'
- 46  Emily Watlington, 'When Does Artistic Research Become Fake News? Forensic Architecture Keeps Dodging The Question,' Art in America , March 15, 2023, https:// www.artnews.com/art-in-america/features/ forensic-architecture-fake-news-1234661013/.
- 47 Rothöhler, Medien der Forensik , 147.
- 48  Jacob Lund and Eyal Weizman, 'Inhabiting the Hyper-Aesthetic Image: Eyal Weizman (in conversation with Jacob Lund),' Nordic Journal of Aesthetics 30, no. 61-62 (2021): 230-43, https://tidsskrift.dk/nja/article/ view/127901/174132.
- 49  Weizman and Fuller, Investigative Aesthetics , 23.
- 50  Eyal Weizman, 'Open Verification,' e-flux Architecture , Becoming Digital, June 2019, https://www.e-flux.com/ architecture/becoming-digital/248062/ open-verification/.
- 51  Weizman and Fuller, Investigative Aesthetics , 17.
- 52  Forensic Architecture has created 'Model Zoo' (2020), a machine-learning computer vision project designed to help automatically sift through big image data for counterforensic agendas. The model was trained to recognize weapons and munitions. This project involves open-source contributions and invites researchers to engage on the FA Github page and in the Discord channel. In a technical paper, they 'demonstrate [their] workflow in two real-world human rights investigations: the use of the Triple-Chaser tear gas grenade against civilians, and the verification of allegations of military presence in Ukraine in 2014.' The reason why the tear gas canisters objects in the video look so colorful (like candy) is that the team textured their modeled projectiles with random patterns and images, explaining that such ''extreme variations' help the classifier better recognise their shape, contours, and edges.' The graphic textures support the machine learning process, while also attracting the attention of people watching the video in an exhibition, who might otherwise find it difficult to pay attention to variations in canisters. Forensic Architecture, 'Model Zoo' (2020), accessed June 17, 2024, https:// forensic-architecture.org/investigation/model-zoo.
- 53  Parikka and Gil-Forunier, 'Ground Truth,' 1254.
- 54  Kate Crawford, Atlas of AI (New Haven: Yale University Press, 2021), 173.
- 55  Louise Amoore, 'Doubt and the Algorithm: On the Partial Accounts of Machine Learning,' Theory, Culture and Society 36, no. 6 (2019): 147-69, https://doi. org/10.1177/0263276419851846.
- 56  Luke Munn, Liam Magee, and Vanicka Arora, 'Truth Machines: Synthesizing Veracity in AI Language Models,' AI & Society (2023), https://doi.org/10.1007/ s00146-023-01756-4.
- 57  Parikka, Operational Images , 87.
- 58  Parikka and Gil-Forunier, 'Ground Truth,' 1257.
- 59  Parikka and Gil-Forunier, 'Ground Truth,' 1255.
- 60  Crawford, Atlas of AI , 98.
- 61  Crawford, 'Excavatng 'Ground Truth'.'
- 62  Alexander Campolo and Katia Schwerzmann note that data annotators rely on implicit knowledge and classification of training data is achieved by quantity. Alexander Campolo and Katia Schwerzmann, 'From Rules to Examples: Machine Learning's Type of Authority,' Big Data & Society 10, no. 2 (2023), https:// doi.org/10.1177/20539517231188725.
- 63  See Clemens Apprich, Wendy Hui Kyong Chun, Florian Cramer, and Hito Steyerl, eds, Pattern Discrimination (Minneapolis: meson press, 2018), http://dx.doi. org/10.25969/mediarep/3655.
- 64  Amoore, 'Doubt and the Algorithm,' 163.
- 65  Amoore, 'Doubt and the Algorithm,' 151.
- 66  Munn, Magee, and Arora, 'Truth Machines.'
- 67  Munn, Magee, and Arora, 'Truth Machines.'
- 68  Munn, Magee, and Arora, 'Truth Machines.'
- 69  Edward B. Kang, 'Ground Truth Tracings (GTT): On the Epistemic Limits of Machine Learning,' Big Data & Society 10, no. 1 (2023), https://doi. org/10.1177/20539517221146122.
- 70 Munn, Magee, and Arora, 'Truth Machines.'
- 71 Gil-Fournier and Parikka, 'Ground Truth.'
- 72 Munn, Magee, and Arora, 'Truth Machines.'
- 73 Kang, 'Ground Truth Tracings.'
- 74 Sarah Lebovitz, Natalia Levina, and Hila Lifshitz-Assaf, 'Is AI Ground Truth Really 'True'? The Dangers of Training and Evaluating AI Tools Based on Experts' Know-What,' Management Information Systems Quarterly 45, no. 3b (2021): 1501-25.
- 75 Lillie Chouliaraki, 'Digital Witnessing in Conflict Zones: The Politics of Remediation,' Information, Communication & Society 18, no. 11 (2015): 1-16, https://doi.org/10.1080/1369118X.2015.1070890.
- 76 Mike Ananny, 'Creating Proper Distance through Networked Infrastructure: Examining Google Glass for Evidence of Moral, Journalistic Witnessing,' in Boundaries of Journalism: Professionalism, Practices, Participation , eds. Matt Carlson and Seth C. Lewis (New York: Routledge, 2015), 83-99.
- 77  See Paul Frosh and Amit Pinchevski, 'Media Witnessing and the Ripeness of Time,' Cultural Studies 28, no. 4 (2014): 594-610, https://doi-1org-1tuet5x9e06ad.erf. sbb.spk-berlin.de/10.1080/09502386.2014.891304.

- 78 Sybille Krämer, 'Bearing Witness as Truth Practice: The Twofold-Discursive and Existential-Character of Telling Truth in Testimony,' in The Palgrave Handbook of Testimony and Culture , eds. Sara Jones and Roger Woods (Cham: Palgrave Macmillan, 2023), 33, https:// doi.org/10.1007/978-3-031-13794-5\_2.
- 79 Hito Steyerl, 'Die Farbe der Wahrheit: Dokumentarismus und Dokumentalität,' in Auf den Spuren des Realen: Kunst und Dokumentarismus , ed. Karin Gludovatz and Museum Moderner Kunst Stiftung Ludwig, Vienna (Vienna: MUMOK, 2004), 100.
- 80  See Forensic Architecture, 'Drift-backs in the Aegean Sea,' accessed June 17, 2024, https://aegean. forensic-architecture.org/.
- 81  Tom Gunning, 'What's the Point of an Index? or, Faking Photographs,' in Still Moving: Between Cinema and Photography , eds. Karen Redrobe Beckman and Jean Ma (New York: Duke University Press, 2008), 23-40.
- 82  Training data for targeted advertising, algorithmic recommendation, and AI generators can be collected or scraped online. Ulises A. Mejias and Nick Couldry use the verb 'grabbing' in the title of their latest book, which builds on earlier work on data colonialism to emphasize their concept, linking with the colonial practice, past and present, of land grabbing. Ulises A. Mejias and Nick Couldry, Data Grab (London: Penguin, 2024). Also see Nick Couldry and Ulises A. Mejias, 'Data Colonialism: Rethinking Big Data's Relation to the Contemporary Subject,' Television & New Media 20, no. 4 (2019): 336-49, https://doi.org/10.1177/1527476418796632.
- 83  See Chung Phan et al., 'Semi-automated Based Ground-Truthing GUI for Airborne Imagery,' in Proceedings SPIE 5794 : Detection and Remediation Technologies for Mines and Minelike Targets X (June 10, 2005), https://doi.org/10.1117/12.605208; and Alfred J. Garrett et al., 'Ground-Truth Collections at the MTI Core Sites,' in Proceedings SPIE 4381: Algorithms for Multispectral, Hyperspectral, and Ultraspectral Imagery VII (August 20, 2001), https://doi.org/10.1117/12.437032.