<figure><img src="image_1.png" /></figure>

HAL
open science

# Software-Only Semantic Diverse Redundancy for High-Integrity AI-Based Functionalities

Martí Caro, Axel Brando, Jaume Abella

<figure><img src="image_2.png" /></figure>

## To cite this version:

Martí Caro, Axel Brando, Jaume Abella. Software-Only Semantic Diverse Redundancy for High-Integrity AI-Based Functionalities. ERTS2024, Jun 2024, Toulouse, France. hal-04614881

HAL Id: hal-04614881

https://hal.science/hal-04614881

Submitted on 17 Jun 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_1.png" /></figure>

HAL
open science

# Software-Only Semantic Diverse Redundancy for High-Integrity AI-Based Functionalities

Martí Caro, Axel Brando, Jaume Abella

<figure><img src="image_2.png" /></figure>

## To cite this version:

Martí Caro, Axel Brando, Jaume Abella. Software-Only Semantic Diverse Redundancy for High-Integrity AI-Based Functionalities. ERTS2024, Jun 2024, Toulouse, France. hal-04614881

HAL Id: hal-04614881

https://hal.science/hal-04614881v1

Submitted on 17 Jun 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Software-Only Semantic Diverse Redundancy for High-Integrity AI-Based Functionalities

Martí Caro†,‡, Axel Brando†, Jaume Abella†

†Barcelona Supercomputing Center (BSC)

Barcelona, Spain

‡Universitat Politecnica de Catalunya

Barcelona, Spain

***Abstract***—Dual (DMR) and Triple Modular Redundancy (TMR), often with some form of diversity, are used in safety-critical systems to realize those functionalities at the highest integrity level providing fault detection and/or tolerance capabilities. Redundant executions are intended to provide bit-level identical results and, upon any mismatch, an error is assumed and recovery actions taken as needed.

In this paper, we note that many emerging AI-based functionalities are intrinsically stochastic (e.g., camera-based object detection), and hence, their correctness must be judged semantically, with room for variations across correct outcomes (e.g., confidence must be above a given threshold). Building on this observation, we propose strategies to create DMR and TMR implementations of AI-based functionalities that bring not only fault tolerance against random hardware faults, but also against AI model inaccuracies. Those strategies, which can be realized with software-only means and ported to virtually any computing platform, build on input data modifications affecting the inference computations, but not the expected semantic output (e.g., introducing some limited random noise in the input data).

***Index Terms***—Diversity, Redundancy, Safety, DNN, AI

## I. INTRODUCTION

Artificial Intelligence (AI) is increasingly used in safety-critical systems as an enabler for autonomous operation. For instance, camera-based object detection implemented with Deep Neural Networks (DNN) is used across a wide variety of applications and domains such as autonomous driving cars [1], rendezvous and docking operations in space [2], and in-cabin pilot monitoring in avionics [3] among others.

AI has been used so far mostly in fail-safe systems [4], where AI software has been generally relieved from inheriting safety requirements by incorporating appropriate non-AI monitors. However, autonomous operation, often related to fail-operational systems, does not generally allow keeping AI software without safety requirements. In fact, since AI software is in charge of high-integrity operations, such as steering and braking in automotive, or docking in space, integrity requirements can easily be the highest ones (e.g., DAL-A in avionics, ASIL-D in automotive). Therefore, it is common that the corresponding safety guidelines impose the use of diverse redundancy as a mandatory safety measure.

So far, diverse redundancy has been built with bit-level correctness in mind. For instance, lockstep processors use 2 or 3 identical cores running the same software with just some time staggering so that their internal state differs at any point in time and, upon a fault affecting all of them simultaneously, the errors generated (if any) are expected to differ and, at least, be detected. In this case, any discrepancy in the results is regarded as an error even if it is semantically innocuous for a specific application, since it is hard to tell whether it is or not a priori. However, in the context of AI software, such as DNNs, many applications do not require bit-level correctness and, instead, perform stochastic processes where bit-level

different outcomes can be regarded as semantically identical. For instance, two object detectors identifying the same object class as the most likely one, with confidence values above a detection threshold, and with highly overlapping bounding boxes, can be regarded as providing identical outputs even if confidence values and bounding boxes differ.

This paper exploits this observation, i.e., redundant AI software may afford bit-level differences across semantically identical outcomes, to realize diverse redundancy in more efficient ways. In particular, we consider solutions where diverse redundancy can be used not only to deal with random hardware faults, but also to mitigate inaccuracies brought by AI models themselves (e.g., false positives and false negatives). Moreover, we do so containing system design and operation costs by preserving the original AI model. Otherwise, the cost of designing, training, and verifying two models could be prohibitive, as well as the cost of fetching twice the amount of AI model parameters, which may be in the order of several GBs in the case of some DNNs for camera-based object detection [5].

Our solution builds upon applying semantically-neutral transformations to the input data of the AI model (e.g., images for camera-based object detection) that alter inference computations so that random hardware faults are mitigated due to physical redundancy, but also small AI model inaccuracies can be mitigated due to input data diversity. In particular, we realize our transformations in the input images used by the You Only Look Once (YOLO) [6] object detector and show that a variety of image transformations yield comparable results to the original case, and they can be combined with different schemes to form a TMR system.

The rest of the paper is organized as follows. Section II provides some background and related work on diverse redundancy. Section III presents our proposal. Section IV evaluates it. Finally, Section V concludes this work.

## II. BACKGROUND AND RELATED WORK

High-integrity systems and components require the use of some form of redundancy for, at least, error detection, and possibly correction. Data storage and communication often build on some form of error correction codes. For instance, single error correction double error detection (SECDED) codes for data storage are a popular solution for large memories, and can also be used to protect end-to-end communications by forwarding those codes along with the data.

However, combinatorial logic, and computing components in general, often need to resort to full redundancy. Dual (DMR) and Triple Modular Redundancy (TMR) are effective solutions for error detection and correction respectively if faults affect one of the redundant components only. However, some types of faults, such as those affecting clock signals and power
# Software-Only Semantic Diverse Redundancy for High-Integrity AI-Based Functionalities

Martí Caro†,‡, Axel Brando†, Jaume Abella†

†Barcelona Supercomputing Center (BSC)

Barcelona, Spain

‡Universitat Politecnica de Catalunya

Barcelona, Spain

***Abstract***—Dual (DMR) and Triple Modular Redundancy (TMR), often with some form of diversity, are used in safety-critical systems to realize those functionalities at the highest integrity level providing fault detection and/or tolerance capabilities. Redundant executions are intended to provide bit-level identical results and, upon any mismatch, an error is assumed and recovery actions taken as needed.

In this paper, we note that many emerging AI-based functionalities are intrinsically stochastic (e.g., camera-based object detection), and hence, their correctness must be judged semantically, with room for variations across correct outcomes (e.g., confidence must be above a given threshold). Building on this observation, we propose strategies to create DMR and TMR implementations of AI-based functionalities that bring not only fault tolerance against random hardware faults, but also against AI model inaccuracies. Those strategies, which can be realized with software-only means and ported to virtually any computing platform, build on input data modifications affecting the inference computations, but not the expected semantic output (e.g., introducing some limited random noise in the input data).

***Index Terms***—Diversity, Redundancy, Safety, DNN, AI

## I. INTRODUCTION

Artificial Intelligence (AI) is increasingly used in safety-critical systems as an enabler for autonomous operation. For instance, camera-based object detection implemented with Deep Neural Networks (DNN) is used across a wide variety of applications and domains such as autonomous driving cars [1], rendezvous and docking operations in space [2], and in-cabin pilot monitoring in avionics [3] among others.

AI has been used so far mostly in fail-safe systems [4], where AI software has been generally relieved from inheriting safety requirements by incorporating appropriate non-AI monitors. However, autonomous operation, often related to fail-operational systems, does not generally allow keeping AI software without safety requirements. In fact, since AI software is in charge of high-integrity operations, such as steering and braking in automotive, or docking in space, integrity requirements can easily be the highest ones (e.g., DAL-A in avionics, ASIL-D in automotive). Therefore, it is common that the corresponding safety guidelines impose the use of diverse redundancy as a mandatory safety measure.

So far, diverse redundancy has been built with bit-level correctness in mind. For instance, lockstep processors use 2 or 3 identical cores running the same software with just some time staggering so that their internal state differs at any point in time and, upon a fault affecting all of them simultaneously, the errors generated (if any) are expected to differ and, at least, be detected. In this case, any discrepancy in the results is regarded as an error even if it is semantically innocuous for a specific application, since it is hard to tell whether it is or not a priori. However, in the context of AI software, such as DNNs, many applications do not require bit-level correctness and, instead, perform stochastic processes where bit-level

different outcomes can be regarded as semantically identical. For instance, two object detectors identifying the same object class as the most likely one, with confidence values above a detection threshold, and with highly overlapping bounding boxes, can be regarded as providing identical outputs even if confidence values and bounding boxes differ.

This paper exploits this observation, i.e., redundant AI software may afford bit-level differences across semantically identical outcomes, to realize diverse redundancy in more efficient ways. In particular, we consider solutions where diverse redundancy can be used not only to deal with random hardware faults, but also to mitigate inaccuracies brought by AI models themselves (e.g., false positives and false negatives). Moreover, we do so containing system design and operation costs by preserving the original AI model. Otherwise, the cost of designing, training, and verifying two models could be prohibitive, as well as the cost of fetching twice the amount of AI model parameters, which may be in the order of several GBs in the case of some DNNs for camera-based object detection [5].

Our solution builds upon applying semantically-neutral transformations to the input data of the AI model (e.g., images for camera-based object detection) that alter inference computations so that random hardware faults are mitigated due to physical redundancy, but also small AI model inaccuracies can be mitigated due to input data diversity. In particular, we realize our transformations in the input images used by the You Only Look Once (YOLO) [6] object detector and show that a variety of image transformations yield comparable results to the original case, and they can be combined with different schemes to form a TMR system.

The rest of the paper is organized as follows. Section II provides some background and related work on diverse redundancy. Section III presents our proposal. Section IV evaluates it. Finally, Section V concludes this work.

## II. BACKGROUND AND RELATED WORK

High-integrity systems and components require the use of some form of redundancy for, at least, error detection, and possibly correction. Data storage and communication often build on some form of error correction codes. For instance, single error correction double error detection (SECDED) codes for data storage are a popular solution for large memories, and can also be used to protect end-to-end communications by forwarding those codes along with the data.

However, combinatorial logic, and computing components in general, often need to resort to full redundancy. Dual (DMR) and Triple Modular Redundancy (TMR) are effective solutions for error detection and correction respectively if faults affect one of the redundant components only. However, some types of faults, such as those affecting clock signals and power
networks, can lead to simultaneous errors in all redundant
components. Hence, some form of diversity is wanted across
redundant instances to avoid scenarios where errors are iden-
tical and cannot be detected by comparing outputs.

The usual solution to realize DMR and TMR with diversity
in CPUs consists of operating redundant cores with some
time staggering so that, despite redundant cores are identical
and execute identical software, their state at any time instant
differs, and the simultaneous impact of a fault is expected to
cause different errors – if any. The most popular case of such
scheme is known as Dual Core LockStep (DCLS for short),
and realized in some commercial microcontrollers, such as
the Infineon AURIX processor family [7]. Similar schemes
have been devised for GPUs [8], as well as software-only
alternatives for cores [9]. However, all those solutions build
upon bit-level accuracy so, while they could be applied directly
for DNNs (either in CPU, GPU, or ad-hoc accelerators), they
cannot be used to mitigate DNN model inaccuracies.

Some solutions exploit semantic redundancy for data vary-
ing little over time (e.g., camera-based object detection at
high frames per second rates) [10], [11]. However, differently
to our proposal, those solutions only provide error detection
capabilities and can only be applied to problems with some
(high) degree of timing redundancy. Our approach, instead,
provides intrinsic error correction capabilities and poses no
requirements on the timing relations across input data.

In our previous work [12], we presented TRUST, a scheme combining temporal redundancy, such as the previous works, as well as spatial redundancy to provide a DCLS-like solution, but with the secondary accelerator operating with lower-precision arithmetic to reduce complexity and power. While TRUST provides error correction capabilities, such as ITLS, it does not increase the model accuracy, and is still bound to problems where temporal redundancy exists. Moreover, ITLS could be realized with software support only, as opposed to TRUST.

Ensemble methods have previously been adopted to enhance
the accuracy of DNNs [13], [14]. However, those works
introduce substantial memory and computational overheads
since weights change across models. To address this challenge,
Gao et al. [15] introduced an ensemble approach that combines
the outputs of three or four lower-complexity models (e.g., an
ensemble of ResNet22, ResNet32, and ResNet44) to replace
the original model (e.g., ResNet110). In their study, Gao et
al. compare their ensemble technique with a Triple Modular
Redundancy (TMR) method, and show that it incurs smaller
overheads. However, their ensemble approach requires three
different models with different weights, while we rely on the
use of a single model. Therefore, their approach increases the
cost of designing, training, and verifying several models which
may be prohibitive. Furthermore, their evaluation focuses
on object classification, which is an easier task than object
detection.

Jon's et al. recent work [16] bears the closest resemblance
to ours. Authors propose the use of image transformations
to detect – but not correct – anomalous situations that could
cause a potential faulty situation in an autonomous system. In
particular, authors use a DNN to calculate the steering angle
of a robot while driving between two lines, which is a much
simpler task than the object detection task considered in our
work. Their approach consists in executing the DNN with
the original image and then performing a follow-up execution

with a very slightly modified image, such that the steering
angles calculated for both images are expected to be identical,
i.e. within a calculated uncertainty threshold. However, in our
work, we allow a higher degree of modification to be able
to increase the overall system robustness through detecting a
larger number of objects between the different components of
the TMR system. We attempted to apply such approach to our
problem, but the results were not promising, since the output
of a DNN for object detection is composed of millions of
values, and we found that the calculated uncertainty threshold,
with the image modifications used in our work, is too large in
fault-free cases, such that it cannot detect faulty operations.

### III. SOFTWARE-ONLY SEMANTIC DIVERSE REDUNDANCY

Our approach builds on realizing a redundant scheme for
the inference with a given AI model by altering the inputs
across redundant instances in a way that outputs are expected
to remain identical semantically speaking, and merging them
conveniently, as illustrated in Figure 1. Note that to achieve
maximum performance, a hardware solution comprising three
parallel physical compute units must be made available. How-
ever, if developing a hardware solution is not an option, our
solution could be implemented solely through software means.
This can be achieved by executing the three input images in
parallel on a single GPU, following the work from S. Alcaide
et al. [17] which demonstrates how to implement a software-
only TMR scheme using a single GPU.

For the sake of easing the explanation, we focus on a
TMR scheme applied on the popular You Only Look Once
v4 (YOLOv4) [6] object detection software for images, which
is used in a plethora of applications, including commercial
automotive systems, such as Apollo [18]. In particular, the
task at hand is object detection, where, for each object present
within an image, the model has to classify it (i.e., determine
the class of the object, such as, a vehicle or a person), and
determine its position and size by creating a bounding box
(i.e., a rectangle surrounding the object).

#### A. Efficient Diverse Redundancy

As shown in some recent work, DNNs such as the one
in YOLOv4 may require fetching some GBs of weights to
process each image [5], whereas images occupy up to few
MBs. Hence, memory bandwidth is often saturated to fetch
the DNN weights. Based on this fact, our redundancy scheme
aims at preserving the same DNN model for all redundant
inference processes with the aim of enabling appropriate
execution approaches that allow sharing weights fetched across
the redundant processes.

Therefore, if weights are fixed, the simplest way to intro-
duce diversity with software-only means consists of altering
the input data. Our goal is applying modifications to the input
data so that it differs across the redundant processes, but they
are semantically equivalent so that inference should lead to
the same object detections. In particular, we have considered
some of the image transformations provided by the CLoDSA
open library [19], which would map to F(x), G(x) and H(x)
in the figure:

* Applying histogram equalization (EQ).

* Image sharpening (SH).

* Dropout (DR) by setting some pixels to zero.

* Applying Gamma correction (GC).

* Blurring the image applying a Gaussian filter (GB).
networks, can lead to simultaneous errors in all redundant
components. Hence, some form of diversity is wanted across
redundant instances to avoid scenarios where errors are iden-
tical and cannot be detected by comparing outputs.

The usual solution to realize DMR and TMR with diversity in CPUs consists of operating redundant cores with some time staggering so that, despite redundant cores are identical and execute identical software, their state at any time instant differs, and the simultaneous impact of a fault is expected to cause different errors – if any. The most popular case of such scheme is known as Dual Core LockStep (DCLS for short), and realized in some commercial microcontrollers, such as the Infineon AURIX processor family [7]. Similar schemes have been devised for GPUs [8], as well as software-only alternatives for cores [9]. However, all those solutions build upon bit-level accuracy so, while they could be applied directly for DNNs (either in CPU, GPU, or ad-hoc accelerators), they cannot be used to mitigate DNN model inaccuracies.

Some solutions exploit semantic redundancy for data vary-
ing little over time (e.g., camera-based object detection at
high frames per second rates) [10], [11]. However, differently
to our proposal, those solutions only provide error detection
capabilities and can only be applied to problems with some
(high) degree of timing redundancy. Our approach, instead,
provides intrinsic error correction capabilities and poses no
requirements on the timing relations across input data.

In our previous work [12], we presented TRUST, a scheme combining temporal redundancy, such as the previous works, as well as spatial redundancy to provide a DCLS-like solution, but with the secondary accelerator operating with lower-precision arithmetic to reduce complexity and power. While TRUST provides error correction capabilities, such as ITLS, it does not increase the model accuracy, and is still bound to problems where temporal redundancy exists. Moreover, ITLS could be realized with software support only, as opposed to TRUST.

Ensemble methods have previously been adopted to enhance
the accuracy of DNNs [13], [14]. However, those works
introduce substantial memory and computational overheads
since weights change across models. To address this challenge,
Gao et al. [15] introduced an ensemble approach that combines
the outputs of three or four lower-complexity models (e.g., an
ensemble of ResNet22, ResNet32, and ResNet44) to replace
the original model (e.g., ResNet110). In their study, Gao et
al. compare their ensemble technique with a Triple Modular
Redundancy (TMR) method, and show that it incurs smaller
overheads. However, their ensemble approach requires three
different models with different weights, while we rely on the
use of a single model. Therefore, their approach increases the
cost of designing, training, and verifying several models which
may be prohibitive. Furthermore, their evaluation focuses
on object classification, which is an easier task than object
detection.

Jon's et al. recent work [16] bears the closest resemblance
to ours. Authors propose the use of image transformations
to detect – but not correct – anomalous situations that could
cause a potential faulty situation in an autonomous system. In
particular, authors use a DNN to calculate the steering angle
of a robot while driving between two lines, which is a much
simpler task than the object detection task considered in our
work. Their approach consists in executing the DNN with
the original image and then performing a follow-up execution

with a very slightly modified image, such that the steering
angles calculated for both images are expected to be identical,
i.e. within a calculated uncertainty threshold. However, in our
work, we allow a higher degree of modification to be able
to increase the overall system robustness through detecting a
larger number of objects between the different components of
the TMR system. We attempted to apply such approach to our
problem, but the results were not promising, since the output
of a DNN for object detection is composed of millions of
values, and we found that the calculated uncertainty threshold,
with the image modifications used in our work, is too large in
fault-free cases, such that it cannot detect faulty operations.

### III. SOFTWARE-ONLY SEMANTIC DIVERSE REDUNDANCY

Our approach builds on realizing a redundant scheme for
the inference with a given AI model by altering the inputs
across redundant instances in a way that outputs are expected
to remain identical semantically speaking, and merging them
conveniently, as illustrated in Figure 1. Note that to achieve
maximum performance, a hardware solution comprising three
parallel physical compute units must be made available. How-
ever, if developing a hardware solution is not an option, our
solution could be implemented solely through software means.
This can be achieved by executing the three input images in
parallel on a single GPU, following the work from S. Alcaide
et al. [17] which demonstrates how to implement a software-
only TMR scheme using a single GPU.

For the sake of easing the explanation, we focus on a
TMR scheme applied on the popular You Only Look Once
v4 (YOLOv4) [6] object detection software for images, which
is used in a plethora of applications, including commercial
automotive systems, such as Apollo [18]. In particular, the
task at hand is object detection, where, for each object present
within an image, the model has to classify it (i.e., determine
the class of the object, such as, a vehicle or a person), and
determine its position and size by creating a bounding box
(i.e., a rectangle surrounding the object).

A. Efficient Diverse Redundancy

As shown in some recent work, DNNs such as the one
in YOLOv4 may require fetching some GBs of weights to
process each image [5], whereas images occupy up to few
MBs. Hence, memory bandwidth is often saturated to fetch
the DNN weights. Based on this fact, our redundancy scheme
aims at preserving the same DNN model for all redundant
inference processes with the aim of enabling appropriate
execution approaches that allow sharing weights fetched across
the redundant processes.

Therefore, if weights are fixed, the simplest way to intro-
duce diversity with software-only means consists of altering
the input data. Our goal is applying modifications to the input
data so that it differs across the redundant processes, but they
are semantically equivalent so that inference should lead to
the same object detections. In particular, we have considered
some of the image transformations provided by the CLoDSA
open library [19], which would map to F(x), G(x) and H(x)
in the figure:

* Applying histogram equalization (EQ).

* Image sharpening (SH).

* Dropout (DR) by setting some pixels to zero.

* Applying Gamma correction (GC).

* Blurring the image applying a Gaussian filter (GB).
<figure><img src="image_4.png" /><figcaption>Fig. 1: Overview of our proposed TMR diverse redundancy scheme.</figcaption></figure>

* Applying Gaussian noise (GN).

* Blurring the image applying the median filter (MB).

* Adding salt and pepper pixels (SP).

* Raising pixel values by a set amount (RV).

* Shifting the image some pixels to the right (RS).

* Shifting the image some pixels to the left (LS).

* Shifting the image some pixels to the top (TS).

* Shifting the image some pixels to the bottom (BS).

* Rotating the image clockwise at a certain angle (CR).

* Rotating the image anticlockwise at a certain angle (AR).

* Horizontally flipping the image (HF).

* Vertically flipping the image (VF).

To realize the TMR scheme, we select three different choices between the image transformations above and the original image, and perform inference for the three of them using the identical YOLOv4 DNN model. With this, we obtain three potentially different object detection outputs that need being combined properly.

## B. Combining Predictions

Rather than attempting to identify wrong predictions – if any – across the diverse and redundant predictions, we merge those predictions with the aim of making correct predictions supersede erroneous ones (see Figure 1). We build on the fact that predictions, even if correct, may not be identical at bit level. Hence, we opt for combining predictions whose detection class matches (e.g., a pedestrian is detected), and whose bounding box overlaps enough (e.g., bounding boxes have an intersection over union higher than 50%). However, it remains open how to set the final bounding box and confidence for the detection. Regarding the bounding box, note that model A can provide a bounding box overlapping above the threshold with the one from model B and the one from model C, but the ones from models B and C may overlap below the threshold. In this case, we still regard detections as the same detection and attempt to combine them.

**Confidence.** To set the confidence of the detection, we explore different schemes as follows:

* Voting: a detection is regarded as true if 2 out of 3 DNN models provide a confidence level above the detection threshold (e.g., using the default confidence threshold of 50% to determine that a detection should be correct).

* Averaging: a detection is regarded as true if the average of the confidence values for the three models is above the detection threshold.

* Maximum: a detection is regarded as true if at least one model indicates a confidence level above the detection threshold.

**Bounding box.** To set the bounding box of the detection, we perform it as follows:

* Voting: we retain the bounding box of the detection with the highest confidence among those being combined.

* Averaging: the bounding boxes of the 3 DNN models are averaged to obtain a final bounding box.

* Maximum: note that in this case, there may be overlapping bounding boxes that represent the same object. Therefore, a Non-Maximum Suppression (NMS) post-processing is applied to filter out repetitions. The NMS consists in filtering out the detections that represent the same object by comparing the bounding boxes of the detections of the same class, and if the bounding boxes have sufficient overlap (i.e., Intersection Over Union > 0.5), the bounding box with the highest confidence is kept, and the other overlapping bounding boxes are discarded.

Note that other approaches to determine the resulting bounding box are possible, such as selecting the largest bounding box among those being combined instead of considering the confidence level. However, using a bounding box not provided by any of the DNNs explicitly brings additional uncertainty. Hence, we stick to using a method that selects one of the relevant bounding boxes.

## C. Application to Safety-Relevant Systems

Deploying safety-relevant systems based on AI, with AI software inhering safety requirements, is still an open challenge [4]. Some relevant standards have seen the light very recently, such as ISO 5469 and ISO 21448 (aka SOTIF), but their practical application is not yet solved. Moreover, they do not provide specific guidance to tailor diverse redundancy. For instance, it is unclear how many levels and degrees of diverse redundancy are needed to make risk residual and system accuracy high enough. In our view, as part of the system architecture, diverse redundancy must exist at different levels, e.g., with multiple sensors, and with redundant and diverse processing for the data of each sensor. Our proposal aims at providing an efficient realization for the latter, hence enabling system designers with appropriate solutions to architect their AI-based safety-critical systems.

# IV. EVALUATION

In this section, we introduce the evaluation setup (YOLO implementation, dataset, evaluation metrics, and fault injection framework), we discuss the results obtained with each image transformation when executing a single model, and we discuss the results obtained with our scheme, considering the different image transformations and results merging schemes in fault-free and faulty scenarios.

## A. Setup

We use a 32-bit floating point Tensorflow Keras implementation of YOLOv4 [20] with the publicly available pre-trained YOLOv4 parameters with the training subset of the Common Objects in Context (COCO) dataset [21]. We evaluate our proposal using the default 608x608 (image width x image height) network size of YOLOv4 and a reduced version using a 320x320 network size which roughly gives a 2x inference speed up at the cost of a slight accuracy drop [22], hence showing the effectiveness of our scheme regardless of the chosen network size. We evaluate our proposal with COCO [21],
<figure><img src="image_4.png" /><figcaption>Fig. 1: Overview of our proposed TMR diverse redundancy scheme.</figcaption></figure>

* Applying Gaussian noise (GN).

* Blurring the image applying the median filter (MB).

* Adding salt and pepper pixels (SP).

* Raising pixel values by a set amount (RV).

* Shifting the image some pixels to the right (RS).

* Shifting the image some pixels to the left (LS).

* Shifting the image some pixels to the top (TS).

* Shifting the image some pixels to the bottom (BS).

* Rotating the image clockwise at a certain angle (CR).

* Rotating the image anticlockwise at a certain angle (AR).

* Horizontally flipping the image (HF).

* Vertically flipping the image (VF).

To realize the TMR scheme, we select three different choices between the image transformations above and the original image, and perform inference for the three of them using the identical YOLOv4 DNN model. With this, we obtain three potentially different object detection outputs that need being combined properly.

## B. Combining Predictions

Rather than attempting to identify wrong predictions – if any – across the diverse and redundant predictions, we merge those predictions with the aim of making correct predictions supersede erroneous ones (see Figure 1). We build on the fact that predictions, even if correct, may not be identical at bit level. Hence, we opt for combining predictions whose detection class matches (e.g., a pedestrian is detected), and whose bounding box overlaps enough (e.g., bounding boxes have an intersection over union higher than 50%). However, it remains open how to set the final bounding box and confidence for the detection. Regarding the bounding box, note that model A can provide a bounding box overlapping above the threshold with the one from model B and the one from model C, but the ones from models B and C may overlap below the threshold. In this case, we still regard detections as the same detection and attempt to combine them.

**Confidence.** To set the confidence of the detection, we explore different schemes as follows:

* Voting: a detection is regarded as true if 2 out of 3 DNN models provide a confidence level above the detection threshold (e.g., using the default confidence threshold of 50% to determine that a detection should be correct).

* Averaging: a detection is regarded as true if the average of the confidence values for the three models is above the detection threshold.

* Maximum: a detection is regarded as true if at least one model indicates a confidence level above the detection threshold.

**Bounding box.** To set the bounding box of the detection, we perform it as follows:

* Voting: we retain the bounding box of the detection with the highest confidence among those being combined.

* Averaging: the bounding boxes of the 3 DNN models are averaged to obtain a final bounding box.

* Maximum: note that in this case, there may be overlapping bounding boxes that represent the same object. Therefore, a Non-Maximum Suppression (NMS) post-processing is applied to filter out repetitions. The NMS consists in filtering out the detections that represent the same object by comparing the bounding boxes of the detections of the same class, and if the bounding boxes have sufficient overlap (i.e., Intersection Over Union > 0.5), the bounding box with the highest confidence is kept, and the other overlapping bounding boxes are discarded.

Note that other approaches to determine the resulting bounding box are possible, such as selecting the largest bounding box among those being combined instead of considering the confidence level. However, using a bounding box not provided by any of the DNNs explicitly brings additional uncertainty. Hence, we stick to using a method that selects one of the relevant bounding boxes.

## C. Application to Safety-Relevant Systems

Deploying safety-relevant systems based on AI, with AI software inhering safety requirements, is still an open challenge [4]. Some relevant standards have seen the light very recently, such as ISO 5469 and ISO 21448 (aka SOTIF), but their practical application is not yet solved. Moreover, they do not provide specific guidance to tailor diverse redundancy. For instance, it is unclear how many levels and degrees of diverse redundancy are needed to make risk residual and system accuracy high enough. In our view, as part of the system architecture, diverse redundancy must exist at different levels, e.g., with multiple sensors, and with redundant and diverse processing for the data of each sensor. Our proposal aims at providing an efficient realization for the latter, hence enabling system designers with appropriate solutions to architect their AI-based safety-critical systems.

# IV. EVALUATION

In this section, we introduce the evaluation setup (YOLO implementation, dataset, evaluation metrics, and fault injection framework), we discuss the results obtained with each image transformation when executing a single model, and we discuss the results obtained with our scheme, considering the different image transformations and results merging schemes in fault-free and faulty scenarios.

## A. Setup

We use a 32-bit floating point Tensorflow Keras implementation of YOLOv4 [20] with the publicly available pre-trained YOLOv4 parameters with the training subset of the Common Objects in Context (COCO) dataset [21]. We evaluate our proposal using the default 608x608 (image width x image height) network size of YOLOv4 and a reduced version using a 320x320 network size which roughly gives a 2x inference speed up at the cost of a slight accuracy drop [22], hence showing the effectiveness of our scheme regardless of the chosen network size. We evaluate our proposal with COCO [21],
using the validation subset, and keeping only those images that contain objects such as vehicles and pedestrians, which delves 870 images for evaluation.

However, the COCO dataset is not specific to AD, hence we have also evaluated our scheme with the KITTI dataset, designed specifically for AD applications. We have also considered using other AD datasets such as Udacity [23], Berkeley DeepDrive [24], and Waymo [25], but KITTI was the only one delivering relevant results for the pretrained YOLOv4 model due to using a comparable object labelling policies to the COCO dataset used for training. The KITTI dataset contains 7481 images captured from onboard vehicle cameras, and also includes three temporally preceding frames of each image, but captured at a very low FPS, which makes detections across images highly independent. Hence, we resorted to evaluating each image independently. The main limitation when using the KITTI dataset with the pretrained YOLOv4 model is that some classes (e.g., bus) are not labelled, and some classes (e.g., person, and motorbike) have significantly different labelling policies to those of COCO. Therefore, we have restricted our evaluation with the KITTI dataset to those classes with fewer discrepancies to obtain accurate results (i.e., car, van, and truck object classes). Another difference w.r.t. the COCO dataset is that the KITTI dataset includes some regions of the image – mostly background regions – that have not been labelled. Therefore, predictions made within these unlabelled zones are ignored during the evaluation process.

As evaluation metrics, we use the *Accuracy* (ACC) and *Mean Average Precision* (mAP). To obtain those, first we have to compute the *Intersection over Union* (IoU), i.e., the fraction of the intersection of the bounding boxes w.r.t. the union of those bounding boxes. In particular, we do so for the final objects detected by our proposal and the groundtruths. Since the detection process is stochastic and subject to some variation, whether an object is regarded as detected or not is done with a threshold, which in our case is 0.5, as set in other works [26]. This leads to true positives (TP) if the IoU is above the threshold, false positives (FP) if the IoU is below the threshold for a predicted object, and false negatives (FN) if the IoU of a groundtruth is below the threshold.

The accuracy (ACC) is obtained as follows:

$$ACC = \frac{TP}{TP + FP + FN} \qquad (1)$$

The mAP is a more complex metric that is often the reference for object detection evaluation. The mAP leverages the TP, FP, and FN counts across the existing object classes to obtain an accuracy assessment for each object class, and then averages the individual accuracies to obtain an overall accuracy assessment. Given the complexity to introduce the details of this metric, and due to space constraints, we refer the interested reader to other publications for detailed explanations of this metric [27].

Regarding our fault injection campaign, we focus on random hardware faults impacting the computation (i.e., transient faults). We have injected transient faults in the result generated by multiplication or addition operations of the convolutional layers, as these layers account for over 99.5% of the total number of operations to process an image. Furthermore, random faults have only been considered to affect the sign or exponent of the floating-point number representation (i.e., 9 highest order bits), since random bit flips in the mantissa

<table><caption>TABLE I: Results of each Image Transformation analysed independently with the COCO dataset (320x320 network size).</caption><thead><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>3087</td><td>373</td><td>3104</td><td>47,03%</td><td>48,69%</td></tr><tr><td>RV</td><td>3085</td><td>372</td><td>3106</td><td>47,01%</td><td>48,65%</td></tr><tr><td>GC</td><td>3069</td><td>375</td><td>3122</td><td>46,74%</td><td>48,38%</td></tr><tr><td>TS</td><td>3061</td><td>353</td><td>3130</td><td>46,78%</td><td>48,27%</td></tr><tr><td>HF</td><td>3058</td><td>378</td><td>3133</td><td>46,55%</td><td>48,16%</td></tr><tr><td>BS</td><td>3050</td><td>358</td><td>3141</td><td>46,57%</td><td>48,14%</td></tr><tr><td>LS</td><td>3046</td><td>336</td><td>3145</td><td>46,67%</td><td>48,09%</td></tr><tr><td>RS</td><td>3037</td><td>379</td><td>3154</td><td>46,23%</td><td>47,84%</td></tr><tr><td>GN</td><td>3021</td><td>348</td><td>3170</td><td>46,20%</td><td>47,67%</td></tr><tr><td>EQ</td><td>3002</td><td>370</td><td>3189</td><td>45,76%</td><td>47,38%</td></tr><tr><td>SP</td><td>2991</td><td>341</td><td>3200</td><td>45,79%</td><td>47,16%</td></tr><tr><td>AR</td><td>2843</td><td>307</td><td>3348</td><td>43,75%</td><td>44,87%</td></tr><tr><td>CR</td><td>2834</td><td>292</td><td>3357</td><td>43,71%</td><td>44,85%</td></tr><tr><td>GB</td><td>2706</td><td>231</td><td>3485</td><td>42,14%</td><td>42,97%</td></tr><tr><td>DR</td><td>2659</td><td>287</td><td>3532</td><td>41,05%</td><td>42,03%</td></tr><tr><td>SH</td><td>2618</td><td>278</td><td>3573</td><td>40,47%</td><td>41,32%</td></tr><tr><td>MB</td><td>2550</td><td>239</td><td>3641</td><td>39,66%</td><td>40,39%</td></tr><tr><td>VF</td><td>317</td><td>36</td><td>5874</td><td>5,09%</td><td>4,88%</td></tr></tbody></table>

<table><caption>TABLE II: Results of each Image Transformation analysed independently with the KITTI dataset (320x320 network size).</caption><thead><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>EQ</td><td>23023</td><td>3035</td><td>9727</td><td>64,34%</td><td>68,27%</td></tr><tr><td>GC</td><td>22737</td><td>2970</td><td>10013</td><td>63,65%</td><td>67,37%</td></tr><tr><td>TS</td><td>22424</td><td>2861</td><td>10326</td><td>62,97%</td><td>66,53%</td></tr><tr><td>GB</td><td>22323</td><td>2380</td><td>10427</td><td>63,54%</td><td>66,50%</td></tr><tr><td>RV</td><td>22375</td><td>2766</td><td>10375</td><td>63,00%</td><td>66,42%</td></tr><tr><td>BS</td><td>22301</td><td>2732</td><td>10449</td><td>62,85%</td><td>66,19%</td></tr><tr><td>HF</td><td>22298</td><td>2737</td><td>10452</td><td>62,83%</td><td>66,12%</td></tr><tr><td>RS</td><td>22267</td><td>2837</td><td>10483</td><td>62,57%</td><td>66,10%</td></tr><tr><td>BL</td><td>22247</td><td>2742</td><td>10503</td><td>62,68%</td><td>66,07%</td></tr><tr><td>LS</td><td>22229</td><td>2704</td><td>10521</td><td>62,70%</td><td>65,97%</td></tr><tr><td>CR</td><td>21464</td><td>2250</td><td>11286</td><td>61,33%</td><td>63,93%</td></tr><tr><td>AR</td><td>21425</td><td>2163</td><td>11325</td><td>61,37%</td><td>63,88%</td></tr><tr><td>GN</td><td>21338</td><td>2543</td><td>11412</td><td>60,46%</td><td>63,34%</td></tr><tr><td>SP</td><td>20805</td><td>2467</td><td>11945</td><td>59,08%</td><td>61,71%</td></tr><tr><td>MB</td><td>20556</td><td>2022</td><td>12194</td><td>59,12%</td><td>61,32%</td></tr><tr><td>SH</td><td>18424</td><td>2270</td><td>14326</td><td>52,61%</td><td>54,54%</td></tr><tr><td>DR</td><td>14882</td><td>1788</td><td>17868</td><td>43,09%</td><td>44,00%</td></tr><tr><td>VF</td><td>229</td><td>42</td><td>32521</td><td>0,70%</td><td>0,62%</td></tr></tbody></table>

are mostly masked and do not lead to semantic changes in the outputs of the model [28], [29].

Tensorflow Keras poses difficulties to inject faults in the intermediate results of a layer, since layers operate as black boxes. To overcome this limitation, we have implemented a custom convolutional layer (same functionality but less efficient implementation). We choose an operation (addition or multiplication) randomly across all those operations in all convolutional layers. To inject a fault in a specific result of a specific layer, we execute the Keras model until the previous layer, replace the target layer with our custom one, inject the fault as needed (flipping a random bit in the sign or exponent), and resume the execution of the Keras implementation from the following layer onwards passing the result of our custom layer as needed.

## B. Results of Independent Configurations

Tables I, II, III, and IV show the TP, FP, and FN counts, as well as the ACC and mAP for each image transformation with
using the validation subset, and keeping only those images that contain objects such as vehicles and pedestrians, which delves 870 images for evaluation.

However, the COCO dataset is not specific to AD, hence we have also evaluated our scheme with the KITTI dataset, designed specifically for AD applications. We have also considered using other AD datasets such as Udacity [23], Berkeley DeepDrive [24], and Waymo [25], but KITTI was the only one delivering relevant results for the pretrained YOLOv4 model due to using a comparable object labelling policies to the COCO dataset used for training. The KITTI dataset contains 7481 images captured from onboard vehicle cameras, and also includes three temporally preceding frames of each image, but captured at a very low FPS, which makes detections across images highly independent. Hence, we resorted to evaluating each image independently. The main limitation when using the KITTI dataset with the pretrained YOLOv4 model is that some classes (e.g., bus) are not labelled, and some classes (e.g., person, and motorbike) have significantly different labelling policies to those of COCO. Therefore, we have restricted our evaluation with the KITTI dataset to those classes with fewer discrepancies to obtain accurate results (i.e., car, van, and truck object classes). Another difference w.r.t. the COCO dataset is that the KITTI dataset includes some regions of the image – mostly background regions – that have not been labelled. Therefore, predictions made within these unlabelled zones are ignored during the evaluation process.

As evaluation metrics, we use the *Accuracy* (ACC) and *Mean Average Precision* (mAP). To obtain those, first we have to compute the *Intersection over Union* (IoU), i.e., the fraction of the intersection of the bounding boxes w.r.t. the union of those bounding boxes. In particular, we do so for the final objects detected by our proposal and the groundtruths. Since the detection process is stochastic and subject to some variation, whether an object is regarded as detected or not is done with a threshold, which in our case is 0.5, as set in other works [26]. This leads to true positives (TP) if the IoU is above the threshold, false positives (FP) if the IoU is below the threshold for a predicted object, and false negatives (FN) if the IoU of a groundtruth is below the threshold.

The accuracy (ACC) is obtained as follows:

$$ACC = \frac{TP}{TP + FP + FN} \qquad (1)$$

The mAP is a more complex metric that is often the reference for object detection evaluation. The mAP leverages the TP, FP, and FN counts across the existing object classes to obtain an accuracy assessment for each object class, and then averages the individual accuracies to obtain an overall accuracy assessment. Given the complexity to introduce the details of this metric, and due to space constraints, we refer the interested reader to other publications for detailed explanations of this metric [27].

Regarding our fault injection campaign, we focus on random hardware faults impacting the computation (i.e., transient faults). We have injected transient faults in the result generated by multiplication or addition operations of the convolutional layers, as these layers account for over 99.5% of the total number of operations to process an image. Furthermore, random faults have only been considered to affect the sign or exponent of the floating-point number representation (i.e., 9 highest order bits), since random bit flips in the mantissa

<table><caption>TABLE I: Results of each Image Transformation analysed independently with the COCO dataset (320x320 network size).</caption><thead><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>3087</td><td>373</td><td>3104</td><td>47,03%</td><td>48,69%</td></tr><tr><td>RV</td><td>3085</td><td>372</td><td>3106</td><td>47,01%</td><td>48,65%</td></tr><tr><td>GC</td><td>3069</td><td>375</td><td>3122</td><td>46,74%</td><td>48,38%</td></tr><tr><td>TS</td><td>3061</td><td>353</td><td>3130</td><td>46,78%</td><td>48,27%</td></tr><tr><td>HF</td><td>3058</td><td>378</td><td>3133</td><td>46,55%</td><td>48,16%</td></tr><tr><td>BS</td><td>3050</td><td>358</td><td>3141</td><td>46,57%</td><td>48,14%</td></tr><tr><td>LS</td><td>3046</td><td>336</td><td>3145</td><td>46,67%</td><td>48,09%</td></tr><tr><td>RS</td><td>3037</td><td>379</td><td>3154</td><td>46,23%</td><td>47,84%</td></tr><tr><td>GN</td><td>3021</td><td>348</td><td>3170</td><td>46,20%</td><td>47,67%</td></tr><tr><td>EQ</td><td>3002</td><td>370</td><td>3189</td><td>45,76%</td><td>47,38%</td></tr><tr><td>SP</td><td>2991</td><td>341</td><td>3200</td><td>45,79%</td><td>47,16%</td></tr><tr><td>AR</td><td>2843</td><td>307</td><td>3348</td><td>43,75%</td><td>44,87%</td></tr><tr><td>CR</td><td>2834</td><td>292</td><td>3357</td><td>43,71%</td><td>44,85%</td></tr><tr><td>GB</td><td>2706</td><td>231</td><td>3485</td><td>42,14%</td><td>42,97%</td></tr><tr><td>DR</td><td>2659</td><td>287</td><td>3532</td><td>41,05%</td><td>42,03%</td></tr><tr><td>SH</td><td>2618</td><td>278</td><td>3573</td><td>40,47%</td><td>41,32%</td></tr><tr><td>MB</td><td>2550</td><td>239</td><td>3641</td><td>39,66%</td><td>40,39%</td></tr><tr><td>VF</td><td>317</td><td>36</td><td>5874</td><td>5,09%</td><td>4,88%</td></tr></tbody></table>

<table><caption>TABLE II: Results of each Image Transformation analysed independently with the KITTI dataset (320x320 network size).</caption><thead><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>EQ</td><td>23023</td><td>3035</td><td>9727</td><td>64,34%</td><td>68,27%</td></tr><tr><td>GC</td><td>22737</td><td>2970</td><td>10013</td><td>63,65%</td><td>67,37%</td></tr><tr><td>TS</td><td>22424</td><td>2861</td><td>10326</td><td>62,97%</td><td>66,53%</td></tr><tr><td>GB</td><td>22323</td><td>2380</td><td>10427</td><td>63,54%</td><td>66,50%</td></tr><tr><td>RV</td><td>22375</td><td>2766</td><td>10375</td><td>63,00%</td><td>66,42%</td></tr><tr><td>BS</td><td>22301</td><td>2732</td><td>10449</td><td>62,85%</td><td>66,19%</td></tr><tr><td>HF</td><td>22298</td><td>2737</td><td>10452</td><td>62,83%</td><td>66,12%</td></tr><tr><td>RS</td><td>22267</td><td>2837</td><td>10483</td><td>62,57%</td><td>66,10%</td></tr><tr><td>BL</td><td>22247</td><td>2742</td><td>10503</td><td>62,68%</td><td>66,07%</td></tr><tr><td>LS</td><td>22229</td><td>2704</td><td>10521</td><td>62,70%</td><td>65,97%</td></tr><tr><td>CR</td><td>21464</td><td>2250</td><td>11286</td><td>61,33%</td><td>63,93%</td></tr><tr><td>AR</td><td>21425</td><td>2163</td><td>11325</td><td>61,37%</td><td>63,88%</td></tr><tr><td>GN</td><td>21338</td><td>2543</td><td>11412</td><td>60,46%</td><td>63,34%</td></tr><tr><td>SP</td><td>20805</td><td>2467</td><td>11945</td><td>59,08%</td><td>61,71%</td></tr><tr><td>MB</td><td>20556</td><td>2022</td><td>12194</td><td>59,12%</td><td>61,32%</td></tr><tr><td>SH</td><td>18424</td><td>2270</td><td>14326</td><td>52,61%</td><td>54,54%</td></tr><tr><td>DR</td><td>14882</td><td>1788</td><td>17868</td><td>43,09%</td><td>44,00%</td></tr><tr><td>VF</td><td>229</td><td>42</td><td>32521</td><td>0,70%</td><td>0,62%</td></tr></tbody></table>

are mostly masked and do not lead to semantic changes in the outputs of the model [28], [29].

Tensorflow Keras poses difficulties to inject faults in the intermediate results of a layer, since layers operate as black boxes. To overcome this limitation, we have implemented a custom convolutional layer (same functionality but less efficient implementation). We choose an operation (addition or multiplication) randomly across all those operations in all convolutional layers. To inject a fault in a specific result of a specific layer, we execute the Keras model until the previous layer, replace the target layer with our custom one, inject the fault as needed (flipping a random bit in the sign or exponent), and resume the execution of the Keras implementation from the following layer onwards passing the result of our custom layer as needed.

## B. Results of Independent Configurations

Tables I, II, III, and IV show the TP, FP, and FN counts, as well as the ACC and mAP for each image transformation with
<table><caption>TABLE III: Results of each Image Transformation analysed independently with the COCO dataset (608x608 network size).</caption><thead><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>TS</td><td>3887</td><td>473</td><td>2304</td><td>58,33%</td><td>61,12%</td></tr><tr><td>RV</td><td>3868</td><td>467</td><td>2323</td><td>58,10%</td><td>60,94%</td></tr><tr><td>HF</td><td>3866</td><td>456</td><td>2325</td><td>58,16%</td><td>60,88%</td></tr><tr><td>LS</td><td>3856</td><td>435</td><td>2335</td><td>58,19%</td><td>60,88%</td></tr><tr><td>BL</td><td>3865</td><td>466</td><td>2326</td><td>58,06%</td><td>60,87%</td></tr><tr><td>BS</td><td>3860</td><td>439</td><td>2331</td><td>58,22%</td><td>60,82%</td></tr><tr><td>RS</td><td>3856</td><td>446</td><td>2335</td><td>58,10%</td><td>60,76%</td></tr><tr><td>GC</td><td>3859</td><td>470</td><td>2332</td><td>57,93%</td><td>60,74%</td></tr><tr><td>GN</td><td>3776</td><td>405</td><td>2415</td><td>57,25%</td><td>59,61%</td></tr><tr><td>SP</td><td>3708</td><td>398</td><td>2483</td><td>56,28%</td><td>58,55%</td></tr><tr><td>EQ</td><td>3699</td><td>450</td><td>2492</td><td>55,70%</td><td>58,24%</td></tr><tr><td>CR</td><td>3676</td><td>466</td><td>2515</td><td>55,22%</td><td>57,83%</td></tr><tr><td>AR</td><td>3675</td><td>504</td><td>2516</td><td>54,89%</td><td>57,72%</td></tr><tr><td>SH</td><td>3240</td><td>324</td><td>2951</td><td>49,73%</td><td>51,15%</td></tr><tr><td>GB</td><td>3213</td><td>267</td><td>2978</td><td>49,75%</td><td>51,05%</td></tr><tr><td>MB</td><td>2894</td><td>295</td><td>3297</td><td>44,62%</td><td>45,71%</td></tr><tr><td>DR</td><td>2335</td><td>298</td><td>3856</td><td>35,98%</td><td>36,40%</td></tr><tr><td>VF</td><td>366</td><td>34</td><td>5825</td><td>5,88%</td><td>5,60%</td></tr></tbody></table>

<table><caption>TABLE IV: Results of each Image Transformation analysed independently with the KITTI dataset (608x608 network size).</caption><thead><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>EQ</td><td>26193</td><td>4435</td><td>6557</td><td>70,44%</td><td>77,59%</td></tr><tr><td>GC</td><td>26093</td><td>4401</td><td>6657</td><td>70,23%</td><td>77,14%</td></tr><tr><td>BS</td><td>25930</td><td>4047</td><td>6820</td><td>70,47%</td><td>76,86%</td></tr><tr><td>BL</td><td>25935</td><td>4093</td><td>6815</td><td>70,39%</td><td>76,86%</td></tr><tr><td>RV</td><td>25949</td><td>4142</td><td>6801</td><td>70,34%</td><td>76,85%</td></tr><tr><td>HF</td><td>25894</td><td>3998</td><td>6856</td><td>70,46%</td><td>76,75%</td></tr><tr><td>LS</td><td>25890</td><td>4132</td><td>6860</td><td>70,20%</td><td>76,73%</td></tr><tr><td>TS</td><td>25822</td><td>4165</td><td>6928</td><td>69,95%</td><td>76,50%</td></tr><tr><td>RS</td><td>25811</td><td>4135</td><td>6939</td><td>69,98%</td><td>76,46%</td></tr><tr><td>AR</td><td>25607</td><td>3488</td><td>7143</td><td>70,66%</td><td>76,03%</td></tr><tr><td>CR</td><td>25533</td><td>3542</td><td>7217</td><td>70,35%</td><td>75,81%</td></tr><tr><td>GN</td><td>25548</td><td>3880</td><td>7202</td><td>69,75%</td><td>75,64%</td></tr><tr><td>SP</td><td>25229</td><td>3763</td><td>7521</td><td>69,10%</td><td>74,65%</td></tr><tr><td>GB</td><td>24796</td><td>2900</td><td>7954</td><td>69,55%</td><td>73,93%</td></tr><tr><td>SH</td><td>24765</td><td>3301</td><td>7985</td><td>68,69%</td><td>73,21%</td></tr><tr><td>MB</td><td>22545</td><td>2172</td><td>10205</td><td>64,56%</td><td>67,36%</td></tr><tr><td>DR</td><td>13191</td><td>1239</td><td>19559</td><td>38,81%</td><td>38,83%</td></tr><tr><td>VF</td><td>568</td><td>86</td><td>32182</td><td>1,73%</td><td>1,56%</td></tr></tbody></table>

the COCO and KITTI datasets using 320x320 and 608x608 network sizes.

First, note that the VF image transformation produces very low ACC and mAP results for all the datasets and network sizes evaluated, since the pretrained model has not been trained with vertically flipped objects. However, we keep this transformation for completeness.

In the case of COCO with 320x320 network size, the Baseline configuration provides the highest ACC and mAP, but with a network size of 608x608 there are 4 image transformations (TS, RV, HF, LS) providing up to 0.25% higher mAP than the baseline. For KITTI and 320x320 network size, there are 8 image transformations providing up to 2.2% higher mAP than the baseline, and with a 608x608 network size there are 3 image transformations (EQ, GC, BS) providing up to 0.73% higher mAP than the baseline.

Note that the pretrained YOLO model used COCO's training subset. Hence, the baseline model was expected to obtain high accuracy when evaluating similar images (e.g., COCO

<figure><img src="image_5.png" /><figcaption>Fig. 2: Sorted mAP for all TMR F(x), G(X), and H(x) permutations for all merging algorithms using COCO (320x320 network size).</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 3: Sorted mAP for all TMR F(x), G(X), and H(x) permutations for all merging algorithms using KITTI (320x320 network size).</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 4: Sorted mAP for all TMR F(x), G(X), and H(x) permutations for all merging algorithms using COCO (608x608 network size).</figcaption></figure>

validation subset), but for different datasets, such as KITTI, it was indeed expected that some image transformations could surpass the accuracy of the baseline model.

### C. Results of the Merging Algorithms

We have obtained the TP, FP, and FN counts, as well as the ACC and mAP values for all TMR configurations and merging algorithms, namely Voting, Averaging and Maximum. Since the number of combinations is too large to allow reporting data for all those configurations (817 per merging algorithm), we show the mAP for all configurations in Figures 2, 3, 4,
<table><caption>TABLE III: Results of each Image Transformation analysed independently with the COCO dataset (608x608 network size).</caption><thead><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>TS</td><td>3887</td><td>473</td><td>2304</td><td>58,33%</td><td>61,12%</td></tr><tr><td>RV</td><td>3868</td><td>467</td><td>2323</td><td>58,10%</td><td>60,94%</td></tr><tr><td>HF</td><td>3866</td><td>456</td><td>2325</td><td>58,16%</td><td>60,88%</td></tr><tr><td>LS</td><td>3856</td><td>435</td><td>2335</td><td>58,19%</td><td>60,88%</td></tr><tr><td>BL</td><td>3865</td><td>466</td><td>2326</td><td>58,06%</td><td>60,87%</td></tr><tr><td>BS</td><td>3860</td><td>439</td><td>2331</td><td>58,22%</td><td>60,82%</td></tr><tr><td>RS</td><td>3856</td><td>446</td><td>2335</td><td>58,10%</td><td>60,76%</td></tr><tr><td>GC</td><td>3859</td><td>470</td><td>2332</td><td>57,93%</td><td>60,74%</td></tr><tr><td>GN</td><td>3776</td><td>405</td><td>2415</td><td>57,25%</td><td>59,61%</td></tr><tr><td>SP</td><td>3708</td><td>398</td><td>2483</td><td>56,28%</td><td>58,55%</td></tr><tr><td>EQ</td><td>3699</td><td>450</td><td>2492</td><td>55,70%</td><td>58,24%</td></tr><tr><td>CR</td><td>3676</td><td>466</td><td>2515</td><td>55,22%</td><td>57,83%</td></tr><tr><td>AR</td><td>3675</td><td>504</td><td>2516</td><td>54,89%</td><td>57,72%</td></tr><tr><td>SH</td><td>3240</td><td>324</td><td>2951</td><td>49,73%</td><td>51,15%</td></tr><tr><td>GB</td><td>3213</td><td>267</td><td>2978</td><td>49,75%</td><td>51,05%</td></tr><tr><td>MB</td><td>2894</td><td>295</td><td>3297</td><td>44,62%</td><td>45,71%</td></tr><tr><td>DR</td><td>2335</td><td>298</td><td>3856</td><td>35,98%</td><td>36,40%</td></tr><tr><td>VF</td><td>366</td><td>34</td><td>5825</td><td>5,88%</td><td>5,60%</td></tr></tbody></table>

<table><caption>TABLE IV: Results of each Image Transformation analysed independently with the KITTI dataset (608x608 network size).</caption><thead><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>EQ</td><td>26193</td><td>4435</td><td>6557</td><td>70,44%</td><td>77,59%</td></tr><tr><td>GC</td><td>26093</td><td>4401</td><td>6657</td><td>70,23%</td><td>77,14%</td></tr><tr><td>BS</td><td>25930</td><td>4047</td><td>6820</td><td>70,47%</td><td>76,86%</td></tr><tr><td>BL</td><td>25935</td><td>4093</td><td>6815</td><td>70,39%</td><td>76,86%</td></tr><tr><td>RV</td><td>25949</td><td>4142</td><td>6801</td><td>70,34%</td><td>76,85%</td></tr><tr><td>HF</td><td>25894</td><td>3998</td><td>6856</td><td>70,46%</td><td>76,75%</td></tr><tr><td>LS</td><td>25890</td><td>4132</td><td>6860</td><td>70,20%</td><td>76,73%</td></tr><tr><td>TS</td><td>25822</td><td>4165</td><td>6928</td><td>69,95%</td><td>76,50%</td></tr><tr><td>RS</td><td>25811</td><td>4135</td><td>6939</td><td>69,98%</td><td>76,46%</td></tr><tr><td>AR</td><td>25607</td><td>3488</td><td>7143</td><td>70,66%</td><td>76,03%</td></tr><tr><td>CR</td><td>25533</td><td>3542</td><td>7217</td><td>70,35%</td><td>75,81%</td></tr><tr><td>GN</td><td>25548</td><td>3880</td><td>7202</td><td>69,75%</td><td>75,64%</td></tr><tr><td>SP</td><td>25229</td><td>3763</td><td>7521</td><td>69,10%</td><td>74,65%</td></tr><tr><td>GB</td><td>24796</td><td>2900</td><td>7954</td><td>69,55%</td><td>73,93%</td></tr><tr><td>SH</td><td>24765</td><td>3301</td><td>7985</td><td>68,69%</td><td>73,21%</td></tr><tr><td>MB</td><td>22545</td><td>2172</td><td>10205</td><td>64,56%</td><td>67,36%</td></tr><tr><td>DR</td><td>13191</td><td>1239</td><td>19559</td><td>38,81%</td><td>38,83%</td></tr><tr><td>VF</td><td>568</td><td>86</td><td>32182</td><td>1,73%</td><td>1,56%</td></tr></tbody></table>

the COCO and KITTI datasets using 320x320 and 608x608 network sizes.

First, note that the VF image transformation produces very low ACC and mAP results for all the datasets and network sizes evaluated, since the pretrained model has not been trained with vertically flipped objects. However, we keep this transformation for completeness.

In the case of COCO with 320x320 network size, the Baseline configuration provides the highest ACC and mAP, but with a network size of 608x608 there are 4 image transformations (TS, RV, HF, LS) providing up to 0.25% higher mAP than the baseline. For KITTI and 320x320 network size, there are 8 image transformations providing up to 2.2% higher mAP than the baseline, and with a 608x608 network size there are 3 image transformations (EQ, GC, BS) providing up to 0.73% higher mAP than the baseline.

Note that the pretrained YOLO model used COCO's training subset. Hence, the baseline model was expected to obtain high accuracy when evaluating similar images (e.g., COCO

<figure><img src="image_5.png" /><figcaption>Fig. 2: Sorted mAP for all TMR F(x), G(X), and H(x) permutations for all merging algorithms using COCO (320x320 network size).</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 3: Sorted mAP for all TMR F(x), G(X), and H(x) permutations for all merging algorithms using KITTI (320x320 network size).</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 4: Sorted mAP for all TMR F(x), G(X), and H(x) permutations for all merging algorithms using COCO (608x608 network size).</figcaption></figure>

validation subset), but for different datasets, such as KITTI, it was indeed expected that some image transformations could surpass the accuracy of the baseline model.

### C. Results of the Merging Algorithms

We have obtained the TP, FP, and FN counts, as well as the ACC and mAP values for all TMR configurations and merging algorithms, namely Voting, Averaging and Maximum. Since the number of combinations is too large to allow reporting data for all those configurations (817 per merging algorithm), we show the mAP for all configurations in Figures 2, 3, 4,
<figure><img src="image_8.png" /><figcaption>Fig. 5: Sorted mAP for all TMR F(x), G(X), and H(x) per-mutations for all merging algorithms using KITTI (608x608 network size).</figcaption></figure>

<table><caption>TABLE V: TMR results for the top-5 configurations of each merging algorithm using COCO (320x320 network size).</caption><thead><tr><th colspan="6">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>3087</td><td>373</td><td>3104</td><td>47,0%</td><td>48,69%</td></tr></tbody><thead><tr><th colspan="6">VOTING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL, RV, BS</td><td>3098</td><td>357</td><td>3093</td><td>47,31%</td><td>48,97%</td></tr><tr><td>RV, LS, BS</td><td>3091</td><td>290</td><td>3100</td><td>47,69%</td><td>48,95%</td></tr><tr><td>BL, LS, BS</td><td>3087</td><td>296</td><td>3104</td><td>47,59%</td><td>48,87%</td></tr><tr><td>BL, RV, TS</td><td>3093</td><td>362</td><td>3098</td><td>47,20%</td><td>48,85%</td></tr><tr><td>GC, LS, BS</td><td>3083</td><td>283</td><td>3108</td><td>47,62%</td><td>48,85%</td></tr></tbody><thead><tr><th colspan="6">AVERAGING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL, BL, BL</td><td>3087</td><td>373</td><td>3104</td><td>47,03%</td><td>48,69%</td></tr><tr><td>BL, GC, RV</td><td>3079</td><td>370</td><td>3112</td><td>46,93%</td><td>48,59%</td></tr><tr><td>BL, GN, RV</td><td>3057</td><td>369</td><td>3134</td><td>46,60%</td><td>48,24%</td></tr><tr><td>GC, GN, RV</td><td>3052</td><td>371</td><td>3139</td><td>46,51%</td><td>48,15%</td></tr><tr><td>BL, GC, GN</td><td>3050</td><td>363</td><td>3141</td><td>46,54%</td><td>48,15%</td></tr></tbody><thead><tr><th colspan="6">MAXIMUM</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>HF, RS, TS</td><td>3435</td><td>630</td><td>2756</td><td>50,36%</td><td>53,65%</td></tr><tr><td>HF, RS, BS</td><td>3432</td><td>645</td><td>2759</td><td>50,20%</td><td>53,56%</td></tr><tr><td>HF, LS, RS</td><td>3422</td><td>620</td><td>2769</td><td>50,24%</td><td>53,48%</td></tr><tr><td>HF, LS, BS</td><td>3418</td><td>616</td><td>2773</td><td>50,21%</td><td>53,45%</td></tr><tr><td>EQ, LS, TS</td><td>3409</td><td>606</td><td>2782</td><td>50,15%</td><td>53,39%</td></tr></tbody></table>

and 5, and the detailed results for the top-5 (in terms of mAP) for each algorithm in Tables V, VI, VII, and VIII.

Results in the figure are sorted independently for each merging algorithm, meaning that the n^th^ best configuration for one algorithm may differ for the other algorithms despite being aligned w.r.t. the X-axis. Hence, the X-axis is not labelled. However, showing the data this way allows us to get several conclusions: (1) Maximum provides consistently the best results in terms of mAP across all merging algorithms; (2) Averaging is consistently worse, in terms of mAP, than the baseline when using the COCO dataset, but it is slightly better than the baseline in very few cases when evaluating the KITTI dataset; (3) Voting is slightly better than the baseline in some cases; and (4) all merging algorithms provide a smooth degradation of the mAP across image transformation combinations, hence meaning that, while some transformations may delve better results than others, there is not a strong dependence on very few combinations, so the approach provides robust

<table><caption>TABLE VI: TMR results for the top-5 configurations of each merging algorithm using KITTI (320x320 network size).</caption><thead><tr><th colspan="6">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>22247</td><td>2742</td><td>10503</td><td>62,68%</td><td>66,07%</td></tr></tbody><thead><tr><th colspan="6">VOTING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>EQ, GB, GC</td><td>22852</td><td>2614</td><td>9898</td><td>64,62%</td><td>68,10%</td></tr><tr><td>EQ, GC, LS</td><td>22867</td><td>2584</td><td>9883</td><td>64,72%</td><td>68,03%</td></tr><tr><td>EQ, GC, BS</td><td>22879</td><td>2642</td><td>9871</td><td>64,64%</td><td>68,02%</td></tr><tr><td>EQ, GC, TS</td><td>22875</td><td>2645</td><td>9875</td><td>64,63%</td><td>68,02%</td></tr><tr><td>HF, EQ, GC</td><td>22868</td><td>2651</td><td>9882</td><td>64,60%</td><td>67,98%</td></tr></tbody><thead><tr><th colspan="6">AVERAGING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>EQ, GB, GC</td><td>22700</td><td>2337</td><td>10050</td><td>64,70%</td><td>67,54%</td></tr><tr><td>EQ, GC, RV</td><td>22690</td><td>2704</td><td>10060</td><td>64,00%</td><td>67,39%</td></tr><tr><td>HF, EQ, GC</td><td>22636</td><td>2429</td><td>10114</td><td>64,35%</td><td>67,33%</td></tr><tr><td>BL, EQ, GC</td><td>22649</td><td>2667</td><td>10101</td><td>63,95%</td><td>67,28%</td></tr><tr><td>EQ, GB, RV</td><td>22569</td><td>2292</td><td>10181</td><td>64,41%</td><td>67,18%</td></tr></tbody><thead><tr><th colspan="6">MAXIMUM</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>EQ, GB, RS</td><td>24495</td><td>4609</td><td>8255</td><td>65,57%</td><td>72,53%</td></tr><tr><td>EQ, RS, BS</td><td>24536</td><td>4808</td><td>8214</td><td>65,33%</td><td>72,46%</td></tr><tr><td>EQ, RS, TS</td><td>24517</td><td>4908</td><td>8233</td><td>65,10%</td><td>72,44%</td></tr><tr><td>EQ, GB, LS</td><td>24458</td><td>4503</td><td>8292</td><td>65,65%</td><td>72,43%</td></tr><tr><td>EQ, LS, BS</td><td>24506</td><td>4756</td><td>8244</td><td>65,34%</td><td>72,39%</td></tr></tbody></table>

<table><caption>TABLE VII: TMR results for the top-5 configurations of each merging algorithm using COCO (608x608 network size).</caption><thead><tr><th colspan="6">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>3865</td><td>466</td><td>2326</td><td>58,06%</td><td>60,87%</td></tr></tbody><thead><tr><th colspan="6">VOTING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>HF, LS, TS</td><td>3901</td><td>403</td><td>2290</td><td>59,16%</td><td>61,60%</td></tr><tr><td>RV, LS, TS</td><td>3900</td><td>416</td><td>2291</td><td>59,03%</td><td>61,54%</td></tr><tr><td>GC, LS, TS</td><td>3897</td><td>410</td><td>2294</td><td>59,04%</td><td>61,49%</td></tr><tr><td>HF, RS, TS</td><td>3895</td><td>422</td><td>2296</td><td>58,90%</td><td>61,48%</td></tr><tr><td>HF, RV, BS</td><td>3893</td><td>417</td><td>2298</td><td>58,91%</td><td>61,43%</td></tr></tbody><thead><tr><th colspan="6">AVERAGING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL, BL, BL</td><td>3865</td><td>466</td><td>2326</td><td>58,06%</td><td>60,87%</td></tr><tr><td>BL, GC, RV</td><td>3862</td><td>460</td><td>2329</td><td>58,07%</td><td>60,84%</td></tr><tr><td>BL, HF, GC</td><td>3829</td><td>423</td><td>2362</td><td>57,89%</td><td>60,42%</td></tr><tr><td>BL, GN, RV</td><td>3827</td><td>425</td><td>2364</td><td>57,84%</td><td>60,37%</td></tr><tr><td>HF, GC, RV</td><td>3825</td><td>421</td><td>2366</td><td>57,85%</td><td>60,35%</td></tr></tbody><thead><tr><th colspan="6">MAXIMUM</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>HF, RS, BS</td><td>4214</td><td>691</td><td>1977</td><td>61,23%</td><td>66,01%</td></tr><tr><td>HF, LS, TS</td><td>4207</td><td>713</td><td>1984</td><td>60,94%</td><td>65,87%</td></tr><tr><td>HF, LS, BS</td><td>4201</td><td>697</td><td>1990</td><td>60,99%</td><td>65,84%</td></tr><tr><td>HF, RS, TS</td><td>4204</td><td>715</td><td>1987</td><td>60,87%</td><td>65,82%</td></tr><tr><td>RS, LS, TS</td><td>4195</td><td>710</td><td>1996</td><td>60,79%</td><td>65,72%</td></tr></tbody></table>

results.

Results in the tables offer a different angle to our analysis. For the COCO 320x320 configurations, in the case of Voting, we note that the bottom shift (BS), left shift (LS), and raise value (RV) transformations, as well as the baseline (BL), are the most popular ones since they appear in 4, 3, 3, and 3 of the top-5 TMR combinations respectively, and only Gamma correction (GC) appears once. In the case of Averaging, the best combination, despite not providing diversity, consists of
<figure><img src="image_8.png" /><figcaption>Fig. 5: Sorted mAP for all TMR F(x), G(X), and H(x) per-mutations for all merging algorithms using KITTI (608x608 network size).</figcaption></figure>

<table><caption>TABLE V: TMR results for the top-5 configurations of each merging algorithm using COCO (320x320 network size).</caption><thead><tr><th colspan="6">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>3087</td><td>373</td><td>3104</td><td>47,0%</td><td>48,69%</td></tr></tbody><thead><tr><th colspan="6">VOTING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL, RV, BS</td><td>3098</td><td>357</td><td>3093</td><td>47,31%</td><td>48,97%</td></tr><tr><td>RV, LS, BS</td><td>3091</td><td>290</td><td>3100</td><td>47,69%</td><td>48,95%</td></tr><tr><td>BL, LS, BS</td><td>3087</td><td>296</td><td>3104</td><td>47,59%</td><td>48,87%</td></tr><tr><td>BL, RV, TS</td><td>3093</td><td>362</td><td>3098</td><td>47,20%</td><td>48,85%</td></tr><tr><td>GC, LS, BS</td><td>3083</td><td>283</td><td>3108</td><td>47,62%</td><td>48,85%</td></tr></tbody><thead><tr><th colspan="6">AVERAGING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL, BL, BL</td><td>3087</td><td>373</td><td>3104</td><td>47,03%</td><td>48,69%</td></tr><tr><td>BL, GC, RV</td><td>3079</td><td>370</td><td>3112</td><td>46,93%</td><td>48,59%</td></tr><tr><td>BL, GN, RV</td><td>3057</td><td>369</td><td>3134</td><td>46,60%</td><td>48,24%</td></tr><tr><td>GC, GN, RV</td><td>3052</td><td>371</td><td>3139</td><td>46,51%</td><td>48,15%</td></tr><tr><td>BL, GC, GN</td><td>3050</td><td>363</td><td>3141</td><td>46,54%</td><td>48,15%</td></tr></tbody><thead><tr><th colspan="6">MAXIMUM</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>HF, RS, TS</td><td>3435</td><td>630</td><td>2756</td><td>50,36%</td><td>53,65%</td></tr><tr><td>HF, RS, BS</td><td>3432</td><td>645</td><td>2759</td><td>50,20%</td><td>53,56%</td></tr><tr><td>HF, LS, RS</td><td>3422</td><td>620</td><td>2769</td><td>50,24%</td><td>53,48%</td></tr><tr><td>HF, LS, BS</td><td>3418</td><td>616</td><td>2773</td><td>50,21%</td><td>53,45%</td></tr><tr><td>EQ, LS, TS</td><td>3409</td><td>606</td><td>2782</td><td>50,15%</td><td>53,39%</td></tr></tbody></table>

and 5, and the detailed results for the top-5 (in terms of mAP) for each algorithm in Tables V, VI, VII, and VIII.

Results in the figure are sorted independently for each merging algorithm, meaning that the n^th^ best configuration for one algorithm may differ for the other algorithms despite being aligned w.r.t. the X-axis. Hence, the X-axis is not labelled. However, showing the data this way allows us to get several conclusions: (1) Maximum provides consistently the best results in terms of mAP across all merging algorithms; (2) Averaging is consistently worse, in terms of mAP, than the baseline when using the COCO dataset, but it is slightly better than the baseline in very few cases when evaluating the KITTI dataset; (3) Voting is slightly better than the baseline in some cases; and (4) all merging algorithms provide a smooth degradation of the mAP across image transformation combinations, hence meaning that, while some transformations may delve better results than others, there is not a strong dependence on very few combinations, so the approach provides robust

<table><caption>TABLE VI: TMR results for the top-5 configurations of each merging algorithm using KITTI (320x320 network size).</caption><thead><tr><th colspan="6">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>22247</td><td>2742</td><td>10503</td><td>62,68%</td><td>66,07%</td></tr></tbody><thead><tr><th colspan="6">VOTING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>EQ, GB, GC</td><td>22852</td><td>2614</td><td>9898</td><td>64,62%</td><td>68,10%</td></tr><tr><td>EQ, GC, LS</td><td>22867</td><td>2584</td><td>9883</td><td>64,72%</td><td>68,03%</td></tr><tr><td>EQ, GC, BS</td><td>22879</td><td>2642</td><td>9871</td><td>64,64%</td><td>68,02%</td></tr><tr><td>EQ, GC, TS</td><td>22875</td><td>2645</td><td>9875</td><td>64,63%</td><td>68,02%</td></tr><tr><td>HF, EQ, GC</td><td>22868</td><td>2651</td><td>9882</td><td>64,60%</td><td>67,98%</td></tr></tbody><thead><tr><th colspan="6">AVERAGING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>EQ, GB, GC</td><td>22700</td><td>2337</td><td>10050</td><td>64,70%</td><td>67,54%</td></tr><tr><td>EQ, GC, RV</td><td>22690</td><td>2704</td><td>10060</td><td>64,00%</td><td>67,39%</td></tr><tr><td>HF, EQ, GC</td><td>22636</td><td>2429</td><td>10114</td><td>64,35%</td><td>67,33%</td></tr><tr><td>BL, EQ, GC</td><td>22649</td><td>2667</td><td>10101</td><td>63,95%</td><td>67,28%</td></tr><tr><td>EQ, GB, RV</td><td>22569</td><td>2292</td><td>10181</td><td>64,41%</td><td>67,18%</td></tr></tbody><thead><tr><th colspan="6">MAXIMUM</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>EQ, GB, RS</td><td>24495</td><td>4609</td><td>8255</td><td>65,57%</td><td>72,53%</td></tr><tr><td>EQ, RS, BS</td><td>24536</td><td>4808</td><td>8214</td><td>65,33%</td><td>72,46%</td></tr><tr><td>EQ, RS, TS</td><td>24517</td><td>4908</td><td>8233</td><td>65,10%</td><td>72,44%</td></tr><tr><td>EQ, GB, LS</td><td>24458</td><td>4503</td><td>8292</td><td>65,65%</td><td>72,43%</td></tr><tr><td>EQ, LS, BS</td><td>24506</td><td>4756</td><td>8244</td><td>65,34%</td><td>72,39%</td></tr></tbody></table>

<table><caption>TABLE VII: TMR results for the top-5 configurations of each merging algorithm using COCO (608x608 network size).</caption><thead><tr><th colspan="6">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>3865</td><td>466</td><td>2326</td><td>58,06%</td><td>60,87%</td></tr></tbody><thead><tr><th colspan="6">VOTING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>HF, LS, TS</td><td>3901</td><td>403</td><td>2290</td><td>59,16%</td><td>61,60%</td></tr><tr><td>RV, LS, TS</td><td>3900</td><td>416</td><td>2291</td><td>59,03%</td><td>61,54%</td></tr><tr><td>GC, LS, TS</td><td>3897</td><td>410</td><td>2294</td><td>59,04%</td><td>61,49%</td></tr><tr><td>HF, RS, TS</td><td>3895</td><td>422</td><td>2296</td><td>58,90%</td><td>61,48%</td></tr><tr><td>HF, RV, BS</td><td>3893</td><td>417</td><td>2298</td><td>58,91%</td><td>61,43%</td></tr></tbody><thead><tr><th colspan="6">AVERAGING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL, BL, BL</td><td>3865</td><td>466</td><td>2326</td><td>58,06%</td><td>60,87%</td></tr><tr><td>BL, GC, RV</td><td>3862</td><td>460</td><td>2329</td><td>58,07%</td><td>60,84%</td></tr><tr><td>BL, HF, GC</td><td>3829</td><td>423</td><td>2362</td><td>57,89%</td><td>60,42%</td></tr><tr><td>BL, GN, RV</td><td>3827</td><td>425</td><td>2364</td><td>57,84%</td><td>60,37%</td></tr><tr><td>HF, GC, RV</td><td>3825</td><td>421</td><td>2366</td><td>57,85%</td><td>60,35%</td></tr></tbody><thead><tr><th colspan="6">MAXIMUM</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>HF, RS, BS</td><td>4214</td><td>691</td><td>1977</td><td>61,23%</td><td>66,01%</td></tr><tr><td>HF, LS, TS</td><td>4207</td><td>713</td><td>1984</td><td>60,94%</td><td>65,87%</td></tr><tr><td>HF, LS, BS</td><td>4201</td><td>697</td><td>1990</td><td>60,99%</td><td>65,84%</td></tr><tr><td>HF, RS, TS</td><td>4204</td><td>715</td><td>1987</td><td>60,87%</td><td>65,82%</td></tr><tr><td>RS, LS, TS</td><td>4195</td><td>710</td><td>1996</td><td>60,79%</td><td>65,72%</td></tr></tbody></table>

results.

Results in the tables offer a different angle to our analysis. For the COCO 320x320 configurations, in the case of Voting, we note that the bottom shift (BS), left shift (LS), and raise value (RV) transformations, as well as the baseline (BL), are the most popular ones since they appear in 4, 3, 3, and 3 of the top-5 TMR combinations respectively, and only Gamma correction (GC) appears once. In the case of Averaging, the best combination, despite not providing diversity, consists of
<table><caption>TABLE VIII: TMR results for the top-5 configurations of each merging algorithm using KITTI (608x608 network size).</caption><thead><tr><th colspan="6">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>25935</td><td>4093</td><td>6815</td><td>70,39%</td><td>76,86%</td></tr><tr><th colspan="6">VOTING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr><tr><td>HF, EQ, GC</td><td>26199</td><td>4101</td><td>6551</td><td>71,09%</td><td>77,76%</td></tr><tr><td>EQ, GC, LS</td><td>26196</td><td>4105</td><td>6554</td><td>71,08%</td><td>77,74%</td></tr><tr><td>EQ, GC, RS</td><td>26184</td><td>4083</td><td>6566</td><td>71,09%</td><td>77,70%</td></tr><tr><td>EQ, GC, BS</td><td>26170</td><td>4122</td><td>6580</td><td>70,98%</td><td>77,66%</td></tr><tr><td>EQ, LS, BS</td><td>26118</td><td>3862</td><td>6632</td><td>71,34%</td><td>77,61%</td></tr><tr><th colspan="6">AVERAGING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr><tr><td>HF, EQ, GC</td><td>26057</td><td>3879</td><td>6693</td><td>71,14%</td><td>77,32%</td></tr><tr><td>BL, HF, EQ</td><td>26011</td><td>3748</td><td>6739</td><td>71,27%</td><td>77,24%</td></tr><tr><td>EQ, GC, RV</td><td>26064</td><td>4152</td><td>6686</td><td>70,63%</td><td>77,21%</td></tr><tr><td>HF, EQ, RV</td><td>26007</td><td>3781</td><td>6743</td><td>71,19%</td><td>77,20%</td></tr><tr><td>BL, EQ, GC</td><td>26052</td><td>4124</td><td>6698</td><td>70,65%</td><td>77,20%</td></tr><tr><th colspan="6">MAXIMUM</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr><tr><td>HF, EQ, LS</td><td>26974</td><td>5877</td><td>5776</td><td>69,83%</td><td>79,80%</td></tr><tr><td>HF, EQ, BS</td><td>26963</td><td>5717</td><td>5787</td><td>70,09%</td><td>79,79%</td></tr><tr><td>HF, EQ, TS</td><td>26949</td><td>5772</td><td>5801</td><td>69,96%</td><td>79,76%</td></tr><tr><td>EQ, LS, BS</td><td>26947</td><td>5863</td><td>5803</td><td>69,79%</td><td>79,73%</td></tr><tr><td>EQ, LS, TS</td><td>26947</td><td>5901</td><td>5803</td><td>69,72%</td><td>79,72%</td></tr></tbody></table>

using 3 times the BL, RV, GC, and Gaussian Noise (GN) are also quite frequent, and appear 3 times each one. Finally, in the case of Maximum merging, we observe that slightly shifting the original image in one direction is a frequent choice since 10 out of the 15 choices correspond to top, bottom, left or right shift. Hflip (HF) is also a frequent choice with 4 appearances, and Equalization (EQ) appears once.

For the COCO 608x608 configurations, in the case of Voting, we note that slightly shifting the original image in one direction is a frequent choice since 9 out of the 15 choices correspond to top, bottom, left or right shift, HF appears 3 times, RV appears twice, and GC appears once. In the case of Averaging, the best combination, despite not providing diversity, consists of using 3 times the BL, RV and GC appear three times, HF appears twice, and GN appears once. Finally, in the case of Maximum merging, we observe that slightly shifting the original image in one direction is a frequent choice since 11 out of the 15 choices correspond to top, bottom, left or right shift. HF is the second most frequent choice with 4 appearances.

For the KITTI 320x320 configurations, in the case of Voting, EQ and GC appear 5 times, shifting transformations appear 3 times, and HF appears once. In the case of Averaging the best configuration is no longer the baseline. EQ appears 5 times, GC appears 4 times, GB appears twice, and RV, HF, and BL appear once. Finally, in the case of Maximum merging, shifting appears 8 times, EQ 5 times, and GB appears twice.

For the KITTI 608x608 configurations, in the case of Voting, EQ and shifting appear 5 times each, GC appears 4 times, and HF appears once. In the case of Averaging, EQ appears 5 times, HF and GC appear 3 times, RV and BL appear twice. In the case of Averaging, EQ appears 5 times, and HF and GC appear 3 times. Finally, in the case of Maximum merging, shifting appears 7 times, EQ appears 5 times, and HF appears 3 times.

In summary, for the KITTI dataset, EQ is a particularly good option since it appears in all the TOP-5 configurations for all merging algorithms. The shifting image transformations works particularly well for both COCO and KITTI datasets when using a Voting or Maximum merging algorithm. The BL works particularly well for the COCO dataset when performing an Average merging, and the HF and GC are also noteworthy configurations for the KITTI dataset with an Average merging.

For the COCO dataset with both 320x320 and 608x608 network sizes, we note that Voting may increase a bit TPs and decrease a bit FPs. Hence, despite not providing the best mAP values, it provides an interesting tradeoff since it outperforms the baseline in all fronts. Averaging, instead, tends to decrease TPs while failing to decrease FPs sufficiently, and it is systematically worse than the baseline case. Finally, Maximum tends to increase TPs and FPs, which is expected since any object being detected by at least one of the three redundant inferences is regarded as a detection, and hence, a TP or a FP. Still, the combined effect clearly increases ACC and mAP values w.r.t. the baseline. Overall, looking at the results from the COCO dataset, if we care only about mAP or ACC, Maximum is clearly the best choice. If, instead, FPs are particularly problematic, Voting is the best solution.

For the KITTI dataset with both 320x320 and 608x608 network sizes, looking at the mAP values, Voting is clearly superior than Averaging for all configurations. However, looking at the ACC values, these approaches have closer results. Finally, Maximum tends to increase TPs and FPs, but the combined effect provides the highest mAP values. However, the ACC of the Maximum merging algorithm is slightly lower than the baseline for the KITTI dataset with a 608x608 network size. We ascribe this effect to the fact that the KITTI dataset does not properly label all background objects, which are mostly small. When increasing the network size, the model can detect more smaller objects, hence we observe a FP increase. We validated this observation in a subset of images by means of visual inspection, yet could not properly label the full dataset manually to fully fix this issue.

Overall, if we care only about mAP, Maximum merging is clearly the best choice for all the datasets analysed. If, instead, FPs are particularly problematic, Voting is the best solution.

## D. Results of Fault Injection Configurations

We have selected the top-mAP and top-ACC configurations for each merging algorithm, network size, and dataset, and analysed the impact of random fault injections on this subset of configurations. In the case of a single configuration providing both the top-mAP and top-ACC, this configuration with both the top-mAP and top-ACC, and the second highest mAP configuration have been selected instead.

First, we analyse the individual image transformations used in any of the top-mAP and top-ACC combinations indicated above. Tables IX, X, XI, and XII, present the fault-free results as well as the results after fault injection for those image transformations for the COCO 320x320, KITTI 320x320, COCO 608x608, and KITTI 608x608 setups, respectively. The tables provide the FP, FN, and TP count, as well as the ACC and mAP. Since some combinations (e.g., COCO 320x320 Averaging) use an ensemble of up to three baselines, we perform 3 different fault injections in such baseline, which we refer to as BL_1 AF, BL_2 AF, and BL_3 AF. For the
<table><caption>TABLE VIII: TMR results for the top-5 configurations of each merging algorithm using KITTI (608x608 network size).</caption><thead><tr><th colspan="6">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>25935</td><td>4093</td><td>6815</td><td>70,39%</td><td>76,86%</td></tr><tr><th colspan="6">VOTING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr><tr><td>HF, EQ, GC</td><td>26199</td><td>4101</td><td>6551</td><td>71,09%</td><td>77,76%</td></tr><tr><td>EQ, GC, LS</td><td>26196</td><td>4105</td><td>6554</td><td>71,08%</td><td>77,74%</td></tr><tr><td>EQ, GC, RS</td><td>26184</td><td>4083</td><td>6566</td><td>71,09%</td><td>77,70%</td></tr><tr><td>EQ, GC, BS</td><td>26170</td><td>4122</td><td>6580</td><td>70,98%</td><td>77,66%</td></tr><tr><td>EQ, LS, BS</td><td>26118</td><td>3862</td><td>6632</td><td>71,34%</td><td>77,61%</td></tr><tr><th colspan="6">AVERAGING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr><tr><td>HF, EQ, GC</td><td>26057</td><td>3879</td><td>6693</td><td>71,14%</td><td>77,32%</td></tr><tr><td>BL, HF, EQ</td><td>26011</td><td>3748</td><td>6739</td><td>71,27%</td><td>77,24%</td></tr><tr><td>EQ, GC, RV</td><td>26064</td><td>4152</td><td>6686</td><td>70,63%</td><td>77,21%</td></tr><tr><td>HF, EQ, RV</td><td>26007</td><td>3781</td><td>6743</td><td>71,19%</td><td>77,20%</td></tr><tr><td>BL, EQ, GC</td><td>26052</td><td>4124</td><td>6698</td><td>70,65%</td><td>77,20%</td></tr><tr><th colspan="6">MAXIMUM</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr><tr><td>HF, EQ, LS</td><td>26974</td><td>5877</td><td>5776</td><td>69,83%</td><td>79,80%</td></tr><tr><td>HF, EQ, BS</td><td>26963</td><td>5717</td><td>5787</td><td>70,09%</td><td>79,79%</td></tr><tr><td>HF, EQ, TS</td><td>26949</td><td>5772</td><td>5801</td><td>69,96%</td><td>79,76%</td></tr><tr><td>EQ, LS, BS</td><td>26947</td><td>5863</td><td>5803</td><td>69,79%</td><td>79,73%</td></tr><tr><td>EQ, LS, TS</td><td>26947</td><td>5901</td><td>5803</td><td>69,72%</td><td>79,72%</td></tr></tbody></table>

using 3 times the BL, RV, GC, and Gaussian Noise (GN) are also quite frequent, and appear 3 times each one. Finally, in the case of Maximum merging, we observe that slightly shifting the original image in one direction is a frequent choice since 10 out of the 15 choices correspond to top, bottom, left or right shift. Hflip (HF) is also a frequent choice with 4 appearances, and Equalization (EQ) appears once.

For the COCO 608x608 configurations, in the case of Voting, we note that slightly shifting the original image in one direction is a frequent choice since 9 out of the 15 choices correspond to top, bottom, left or right shift, HF appears 3 times, RV appears twice, and GC appears once. In the case of Averaging, the best combination, despite not providing diversity, consists of using 3 times the BL, RV and GC appear three times, HF appears twice, and GN appears once. Finally, in the case of Maximum merging, we observe that slightly shifting the original image in one direction is a frequent choice since 11 out of the 15 choices correspond to top, bottom, left or right shift. HF is the second most frequent choice with 4 appearances.

For the KITTI 320x320 configurations, in the case of Voting, EQ and GC appear 5 times, shifting transformations appear 3 times, and HF appears once. In the case of Averaging the best configuration is no longer the baseline. EQ appears 5 times, GC appears 4 times, GB appears twice, and RV, HF, and BL appear once. Finally, in the case of Maximum merging, shifting appears 8 times, EQ 5 times, and GB appears twice.

For the KITTI 608x608 configurations, in the case of Voting, EQ and shifting appear 5 times each, GC appears 4 times, and HF appears once. In the case of Averaging, EQ appears 5 times, HF and GC appear 3 times, RV and BL appear twice. In the case of Averaging, EQ appears 5 times, and HF and GC appear 3 times. Finally, in the case of Maximum merging, shifting appears 7 times, EQ appears 5 times, and HF appears 3 times.

In summary, for the KITTI dataset, EQ is a particularly good option since it appears in all the TOP-5 configurations for all merging algorithms. The shifting image transformations works particularly well for both COCO and KITTI datasets when using a Voting or Maximum merging algorithm. The BL works particularly well for the COCO dataset when performing an Average merging, and the HF and GC are also noteworthy configurations for the KITTI dataset with an Average merging.

For the COCO dataset with both 320x320 and 608x608 network sizes, we note that Voting may increase a bit TPs and decrease a bit FPs. Hence, despite not providing the best mAP values, it provides an interesting tradeoff since it outperforms the baseline in all fronts. Averaging, instead, tends to decrease TPs while failing to decrease FPs sufficiently, and it is systematically worse than the baseline case. Finally, Maximum tends to increase TPs and FPs, which is expected since any object being detected by at least one of the three redundant inferences is regarded as a detection, and hence, a TP or a FP. Still, the combined effect clearly increases ACC and mAP values w.r.t. the baseline. Overall, looking at the results from the COCO dataset, if we care only about mAP or ACC, Maximum is clearly the best choice. If, instead, FPs are particularly problematic, Voting is the best solution.

For the KITTI dataset with both 320x320 and 608x608 network sizes, looking at the mAP values, Voting is clearly superior than Averaging for all configurations. However, looking at the ACC values, these approaches have closer results. Finally, Maximum tends to increase TPs and FPs, but the combined effect provides the highest mAP values. However, the ACC of the Maximum merging algorithm is slightly lower than the baseline for the KITTI dataset with a 608x608 network size. We ascribe this effect to the fact that the KITTI dataset does not properly label all background objects, which are mostly small. When increasing the network size, the model can detect more smaller objects, hence we observe a FP increase. We validated this observation in a subset of images by means of visual inspection, yet could not properly label the full dataset manually to fully fix this issue.

Overall, if we care only about mAP, Maximum merging is clearly the best choice for all the datasets analysed. If, instead, FPs are particularly problematic, Voting is the best solution.

## D. Results of Fault Injection Configurations

We have selected the top-mAP and top-ACC configurations for each merging algorithm, network size, and dataset, and analysed the impact of random fault injections on this subset of configurations. In the case of a single configuration providing both the top-mAP and top-ACC, this configuration with both the top-mAP and top-ACC, and the second highest mAP configuration have been selected instead.

First, we analyse the individual image transformations used in any of the top-mAP and top-ACC combinations indicated above. Tables IX, X, XI, and XII, present the fault-free results as well as the results after fault injection for those image transformations for the COCO 320x320, KITTI 320x320, COCO 608x608, and KITTI 608x608 setups, respectively. The tables provide the FP, FN, and TP count, as well as the ACC and mAP. Since some combinations (e.g., COCO 320x320 Averaging) use an ensemble of up to three baselines, we perform 3 different fault injections in such baseline, which we refer to as BL_1 AF, BL_2 AF, and BL_3 AF. For the
<table><caption>TABLE IX: Results of the Fault Injection of the individual image transformations part of the combinations with the best mAP and ACC for the COCO dataset (320x320 network size).</caption><thead><tr><th colspan="6">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>3087</td><td>373</td><td>3104</td><td>47,03%</td><td>48,09%</td></tr><tr><th colspan="6">INDEPENDENT CONFIGURATIONS</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr><tr><td>BL_1 AF</td><td>2866</td><td>379</td><td>3325</td><td>43,62%</td><td>44,57%</td></tr><tr><td>BL_2 AF</td><td>2889</td><td>383</td><td>3302</td><td>43,95%</td><td>44,75%</td></tr><tr><td>BL_3 AF</td><td>2896</td><td>380</td><td>3295</td><td>44,07%</td><td>45,02%</td></tr><tr><td>LS</td><td>3046</td><td>336</td><td>3145</td><td>46,67%</td><td>48,09%</td></tr><tr><td>LS AF</td><td>2864</td><td>351</td><td>3327</td><td>43,78%</td><td>44,25%</td></tr><tr><td>TS</td><td>3061</td><td>353</td><td>3130</td><td>46,78%</td><td>48,27%</td></tr><tr><td>TS AF</td><td>2873</td><td>362</td><td>3318</td><td>43,84%</td><td>44,53%</td></tr><tr><td>HF</td><td>3058</td><td>378</td><td>3133</td><td>46,55%</td><td>48,16%</td></tr><tr><td>HF AF</td><td>2936</td><td>391</td><td>3255</td><td>44,61%</td><td>45,87%</td></tr><tr><td>RV</td><td>3085</td><td>372</td><td>3106</td><td>47,01%</td><td>48,27%</td></tr><tr><td>RV AF</td><td>2913</td><td>381</td><td>3278</td><td>44,32%</td><td>45,36%</td></tr><tr><td>RS</td><td>3037</td><td>379</td><td>3154</td><td>46,23%</td><td>48,09%</td></tr><tr><td>RS AF</td><td>2830</td><td>378</td><td>3361</td><td>43,08%</td><td>44,03%</td></tr><tr><td>BS</td><td>3050</td><td>358</td><td>3141</td><td>46,57%</td><td>48,14%</td></tr><tr><td>BS AF</td><td>2887</td><td>376</td><td>3304</td><td>43,96%</td><td>44,73%</td></tr><tr><td>GC</td><td>3069</td><td>375</td><td>3122</td><td>46,74%</td><td>48,38%</td></tr><tr><td>GC AF</td><td>2910</td><td>377</td><td>3281</td><td>44,31%</td><td>45,43%</td></tr></tbody></table>

<table><caption>TABLE X: Results of the Fault Injection of the individual image transformations part of the combinations with the best mAP and ACC for the KITTI dataset (320x320 network size).</caption><thead><tr><th colspan="6">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>22247</td><td>2742</td><td>10503</td><td>62,68%</td><td>66,07%</td></tr><tr><th colspan="6">INDEPENDENT CONFIGURATIONS</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr><tr><td>EQ</td><td>23023</td><td>3035</td><td>9727</td><td>64,34%</td><td>68,27%</td></tr><tr><td>EQ AF</td><td>21977</td><td>2945</td><td>10773</td><td>61,57%</td><td>64,90%</td></tr><tr><td>LS</td><td>22229</td><td>2704</td><td>10521</td><td>62,70%</td><td>65,97%</td></tr><tr><td>LS AF</td><td>21000</td><td>2606</td><td>11750</td><td>59,40%</td><td>62,11%</td></tr><tr><td>GB</td><td>22323</td><td>2380</td><td>10427</td><td>63,54%</td><td>66,50%</td></tr><tr><td>GB AF</td><td>21081</td><td>2324</td><td>11669</td><td>60,10%</td><td>62,61%</td></tr><tr><td>GC</td><td>22737</td><td>2970</td><td>10013</td><td>63,65%</td><td>67,37%</td></tr><tr><td>GC AF</td><td>21564</td><td>2897</td><td>11186</td><td>60,49%</td><td>63,60%</td></tr><tr><td>RS</td><td>22267</td><td>2837</td><td>10483</td><td>62,57%</td><td>66,10%</td></tr><tr><td>RS AF</td><td>21114</td><td>2742</td><td>11636</td><td>59,49%</td><td>62,51%</td></tr><tr><td>BS</td><td>22301</td><td>2732</td><td>10449</td><td>62,85%</td><td>66,19%</td></tr><tr><td>BS AF</td><td>21132</td><td>2616</td><td>11618</td><td>59,75%</td><td>62,43%</td></tr><tr><td>RV</td><td>22375</td><td>2766</td><td>10375</td><td>63,00%</td><td>66,42%</td></tr><tr><td>RV AF</td><td>21050</td><td>2693</td><td>11700</td><td>59,39%</td><td>62,24%</td></tr></tbody></table>

remaining cases we report the fault-free (e.g., LS) and fault-injected cases (e.g., LS AF).

We observe that, in general, fault-injected configurations have lower TP counts, and except in COCO 320x320, also lower FP counts. ACC and mAP drop similarly for all configurations, with a 2.90% and 3.56% drop on average, respectively.

Tables XIII, XIV, XV, and XVI show the results of our proposal in the non-faulty TMR case, as well as in two faulty TMR cases; (i) with independents faults where for a given image only one of the three configurations is affected by a fault, and (ii) with faults in the same image for all three configurations. In the fault-free scenarios, which would correspond to virtually 100% of the time given that faults occur

<table><caption>TABLE XI: Results of the Fault Injection of the individual image transformations part of the combinations with the best mAP and ACC for the COCO dataset (608x608 network size).</caption><thead><tr><th colspan="6">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>3865</td><td>466</td><td>2326</td><td>58,06%</td><td>60,87%</td></tr><tr><th colspan="6">INDEPENDENT CONFIGURATIONS</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr><tr><td>BL_1 AF</td><td>3667</td><td>487</td><td>2524</td><td>54,91%</td><td>56,89%</td></tr><tr><td>BL_2 AF</td><td>3661</td><td>463</td><td>2530</td><td>55,02%</td><td>57,01%</td></tr><tr><td>BL_3 AF</td><td>3633</td><td>468</td><td>2558</td><td>54,56%</td><td>56,47%</td></tr><tr><td>HF</td><td>3866</td><td>456</td><td>2325</td><td>58,16%</td><td>60,88%</td></tr><tr><td>HF AF</td><td>3708</td><td>450</td><td>2483</td><td>55,83%</td><td>57,79%</td></tr><tr><td>TS</td><td>3887</td><td>473</td><td>2304</td><td>58,33%</td><td>61,12%</td></tr><tr><td>TS AF</td><td>3703</td><td>471</td><td>2488</td><td>55,58%</td><td>57,74%</td></tr><tr><td>BS</td><td>3860</td><td>439</td><td>2331</td><td>58,22%</td><td>60,82%</td></tr><tr><td>BS AF</td><td>3681</td><td>408</td><td>2510</td><td>55,78%</td><td>57,80%</td></tr><tr><td>RS</td><td>3856</td><td>446</td><td>2335</td><td>58,10%</td><td>60,76%</td></tr><tr><td>RS AF</td><td>3653</td><td>439</td><td>2538</td><td>55,10%</td><td>56,90%</td></tr><tr><td>LS</td><td>3856</td><td>435</td><td>2335</td><td>58,19%</td><td>60,88%</td></tr><tr><td>LS AF</td><td>3667</td><td>418</td><td>2524</td><td>55,48%</td><td>57,41%</td></tr><tr><td>RV</td><td>3868</td><td>467</td><td>2323</td><td>58,10%</td><td>60,94%</td></tr><tr><td>RV AF</td><td>3664</td><td>440</td><td>2527</td><td>55,26%</td><td>57,41%</td></tr><tr><td>GC</td><td>3859</td><td>470</td><td>2332</td><td>57,93%</td><td>60,74%</td></tr><tr><td>GC AF</td><td>3710</td><td>461</td><td>2481</td><td>55,77%</td><td>58,11%</td></tr></tbody></table>

<table><caption>TABLE XII: Results of the Fault Injection of the individual image transformations part of the combinations with the best mAP and ACC for the KITTI dataset (608x608 network size).</caption><thead><tr><th colspan="6">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>25935</td><td>4093</td><td>6815</td><td>70,39%</td><td>76,86%</td></tr><tr><th colspan="6">INDEPENDENT CONFIGURATIONS</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr><tr><td>HF</td><td>25894</td><td>3998</td><td>6856</td><td>70,46%</td><td>76,75%</td></tr><tr><td>HF AF</td><td>24774</td><td>3832</td><td>7976</td><td>67,72%</td><td>73,30%</td></tr><tr><td>EQ</td><td>26193</td><td>4435</td><td>6557</td><td>70,44%</td><td>77,59%</td></tr><tr><td>EQ AF</td><td>24914</td><td>4218</td><td>7836</td><td>67,39%</td><td>73,66%</td></tr><tr><td>GB</td><td>24796</td><td>2900</td><td>7954</td><td>69,55%</td><td>73,93%</td></tr><tr><td>GB AF</td><td>23548</td><td>2767</td><td>9202</td><td>66,30%</td><td>69,96%</td></tr><tr><td>AR</td><td>25607</td><td>3488</td><td>7143</td><td>70,66%</td><td>76,03%</td></tr><tr><td>AR AF</td><td>24431</td><td>3348</td><td>8319</td><td>67,68%</td><td>72,37%</td></tr><tr><td>GC</td><td>26093</td><td>4401</td><td>6657</td><td>70,23%</td><td>77,14%</td></tr><tr><td>GC AF</td><td>24925</td><td>4221</td><td>7825</td><td>67,42%</td><td>73,50%</td></tr><tr><td>LS</td><td>25890</td><td>4132</td><td>6860</td><td>70,20%</td><td>76,73%</td></tr><tr><td>LS AF</td><td>24671</td><td>3998</td><td>8079</td><td>67,14%</td><td>72,90%</td></tr><tr><td>CR</td><td>25533</td><td>3542</td><td>7217</td><td>70,35%</td><td>75,81%</td></tr><tr><td>CR AF</td><td>24278</td><td>3399</td><td>8472</td><td>67,16%</td><td>71,91%</td></tr></tbody></table>

seldom, we obtain the following conclusion: (1) Voting and Maximum merging algorithms provide higher mAP and ACC than the Baseline, except for the HF, EQ, LS configuration for the KITTI dataset with a 608x608 network size, where we observe a slight ACC drop w.r.t. the baseline (for reasons previously discussed in Section IV-C), (2) Averaging only produces better results than the baseline with the KITTI dataset since the model was trained with the COCO training subset. Therefore, it is expected that some image transformations could surpass the accuracy of the baseline when using other datasets.

When faults occur independently, we obtain the following conclusions: (1) Voting provides higher mAP and ACC than the Baseline with both datasets and both network sizes
<table><caption>TABLE IX: Results of the Fault Injection of the individual image transformations part of the combinations with the best mAP and ACC for the COCO dataset (320x320 network size).</caption><thead><tr><th colspan="6">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>3087</td><td>373</td><td>3104</td><td>47,03%</td><td>48,09%</td></tr><tr><th colspan="6">INDEPENDENT CONFIGURATIONS</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr><tr><td>BL_1 AF</td><td>2866</td><td>379</td><td>3325</td><td>43,62%</td><td>44,57%</td></tr><tr><td>BL_2 AF</td><td>2889</td><td>383</td><td>3302</td><td>43,95%</td><td>44,75%</td></tr><tr><td>BL_3 AF</td><td>2896</td><td>380</td><td>3295</td><td>44,07%</td><td>45,02%</td></tr><tr><td>LS</td><td>3046</td><td>336</td><td>3145</td><td>46,67%</td><td>48,09%</td></tr><tr><td>LS AF</td><td>2864</td><td>351</td><td>3327</td><td>43,78%</td><td>44,25%</td></tr><tr><td>TS</td><td>3061</td><td>353</td><td>3130</td><td>46,78%</td><td>48,27%</td></tr><tr><td>TS AF</td><td>2873</td><td>362</td><td>3318</td><td>43,84%</td><td>44,53%</td></tr><tr><td>HF</td><td>3058</td><td>378</td><td>3133</td><td>46,55%</td><td>48,16%</td></tr><tr><td>HF AF</td><td>2936</td><td>391</td><td>3255</td><td>44,61%</td><td>45,87%</td></tr><tr><td>RV</td><td>3085</td><td>372</td><td>3106</td><td>47,01%</td><td>48,27%</td></tr><tr><td>RV AF</td><td>2913</td><td>381</td><td>3278</td><td>44,32%</td><td>45,36%</td></tr><tr><td>RS</td><td>3037</td><td>379</td><td>3154</td><td>46,23%</td><td>48,09%</td></tr><tr><td>RS AF</td><td>2830</td><td>378</td><td>3361</td><td>43,08%</td><td>44,03%</td></tr><tr><td>BS</td><td>3050</td><td>358</td><td>3141</td><td>46,57%</td><td>48,14%</td></tr><tr><td>BS AF</td><td>2887</td><td>376</td><td>3304</td><td>43,96%</td><td>44,73%</td></tr><tr><td>GC</td><td>3069</td><td>375</td><td>3122</td><td>46,74%</td><td>48,38%</td></tr><tr><td>GC AF</td><td>2910</td><td>377</td><td>3281</td><td>44,31%</td><td>45,43%</td></tr></tbody></table>

<table><caption>TABLE X: Results of the Fault Injection of the individual image transformations part of the combinations with the best mAP and ACC for the KITTI dataset (320x320 network size).</caption><thead><tr><th colspan="6">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>22247</td><td>2742</td><td>10503</td><td>62,68%</td><td>66,07%</td></tr><tr><th colspan="6">INDEPENDENT CONFIGURATIONS</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr><tr><td>EQ</td><td>23023</td><td>3035</td><td>9727</td><td>64,34%</td><td>68,27%</td></tr><tr><td>EQ AF</td><td>21977</td><td>2945</td><td>10773</td><td>61,57%</td><td>64,90%</td></tr><tr><td>LS</td><td>22229</td><td>2704</td><td>10521</td><td>62,70%</td><td>65,97%</td></tr><tr><td>LS AF</td><td>21000</td><td>2606</td><td>11750</td><td>59,40%</td><td>62,11%</td></tr><tr><td>GB</td><td>22323</td><td>2380</td><td>10427</td><td>63,54%</td><td>66,50%</td></tr><tr><td>GB AF</td><td>21081</td><td>2324</td><td>11669</td><td>60,10%</td><td>62,61%</td></tr><tr><td>GC</td><td>22737</td><td>2970</td><td>10013</td><td>63,65%</td><td>67,37%</td></tr><tr><td>GC AF</td><td>21564</td><td>2897</td><td>11186</td><td>60,49%</td><td>63,60%</td></tr><tr><td>RS</td><td>22267</td><td>2837</td><td>10483</td><td>62,57%</td><td>66,10%</td></tr><tr><td>RS AF</td><td>21114</td><td>2742</td><td>11636</td><td>59,49%</td><td>62,51%</td></tr><tr><td>BS</td><td>22301</td><td>2732</td><td>10449</td><td>62,85%</td><td>66,19%</td></tr><tr><td>BS AF</td><td>21132</td><td>2616</td><td>11618</td><td>59,75%</td><td>62,43%</td></tr><tr><td>RV</td><td>22375</td><td>2766</td><td>10375</td><td>63,00%</td><td>66,42%</td></tr><tr><td>RV AF</td><td>21050</td><td>2693</td><td>11700</td><td>59,39%</td><td>62,24%</td></tr></tbody></table>

remaining cases we report the fault-free (e.g., LS) and fault-injected cases (e.g., LS AF).

We observe that, in general, fault-injected configurations have lower TP counts, and except in COCO 320x320, also lower FP counts. ACC and mAP drop similarly for all configurations, with a 2.90% and 3.56% drop on average, respectively.

Tables XIII, XIV, XV, and XVI show the results of our proposal in the non-faulty TMR case, as well as in two faulty TMR cases; (i) with independents faults where for a given image only one of the three configurations is affected by a fault, and (ii) with faults in the same image for all three configurations. In the fault-free scenarios, which would correspond to virtually 100% of the time given that faults occur

<table><caption>TABLE XI: Results of the Fault Injection of the individual image transformations part of the combinations with the best mAP and ACC for the COCO dataset (608x608 network size).</caption><thead><tr><th colspan="6">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>3865</td><td>466</td><td>2326</td><td>58,06%</td><td>60,87%</td></tr><tr><th colspan="6">INDEPENDENT CONFIGURATIONS</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr><tr><td>BL_1 AF</td><td>3667</td><td>487</td><td>2524</td><td>54,91%</td><td>56,89%</td></tr><tr><td>BL_2 AF</td><td>3661</td><td>463</td><td>2530</td><td>55,02%</td><td>57,01%</td></tr><tr><td>BL_3 AF</td><td>3633</td><td>468</td><td>2558</td><td>54,56%</td><td>56,47%</td></tr><tr><td>HF</td><td>3866</td><td>456</td><td>2325</td><td>58,16%</td><td>60,88%</td></tr><tr><td>HF AF</td><td>3708</td><td>450</td><td>2483</td><td>55,83%</td><td>57,79%</td></tr><tr><td>TS</td><td>3887</td><td>473</td><td>2304</td><td>58,33%</td><td>61,12%</td></tr><tr><td>TS AF</td><td>3703</td><td>471</td><td>2488</td><td>55,58%</td><td>57,74%</td></tr><tr><td>BS</td><td>3860</td><td>439</td><td>2331</td><td>58,22%</td><td>60,82%</td></tr><tr><td>BS AF</td><td>3681</td><td>408</td><td>2510</td><td>55,78%</td><td>57,80%</td></tr><tr><td>RS</td><td>3856</td><td>446</td><td>2335</td><td>58,10%</td><td>60,76%</td></tr><tr><td>RS AF</td><td>3653</td><td>439</td><td>2538</td><td>55,10%</td><td>56,90%</td></tr><tr><td>LS</td><td>3856</td><td>435</td><td>2335</td><td>58,19%</td><td>60,88%</td></tr><tr><td>LS AF</td><td>3667</td><td>418</td><td>2524</td><td>55,48%</td><td>57,41%</td></tr><tr><td>RV</td><td>3868</td><td>467</td><td>2323</td><td>58,10%</td><td>60,94%</td></tr><tr><td>RV AF</td><td>3664</td><td>440</td><td>2527</td><td>55,26%</td><td>57,41%</td></tr><tr><td>GC</td><td>3859</td><td>470</td><td>2332</td><td>57,93%</td><td>60,74%</td></tr><tr><td>GC AF</td><td>3710</td><td>461</td><td>2481</td><td>55,77%</td><td>58,11%</td></tr></tbody></table>

<table><caption>TABLE XII: Results of the Fault Injection of the individual image transformations part of the combinations with the best mAP and ACC for the KITTI dataset (608x608 network size).</caption><thead><tr><th colspan="6">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>25935</td><td>4093</td><td>6815</td><td>70,39%</td><td>76,86%</td></tr><tr><th colspan="6">INDEPENDENT CONFIGURATIONS</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr><tr><td>HF</td><td>25894</td><td>3998</td><td>6856</td><td>70,46%</td><td>76,75%</td></tr><tr><td>HF AF</td><td>24774</td><td>3832</td><td>7976</td><td>67,72%</td><td>73,30%</td></tr><tr><td>EQ</td><td>26193</td><td>4435</td><td>6557</td><td>70,44%</td><td>77,59%</td></tr><tr><td>EQ AF</td><td>24914</td><td>4218</td><td>7836</td><td>67,39%</td><td>73,66%</td></tr><tr><td>GB</td><td>24796</td><td>2900</td><td>7954</td><td>69,55%</td><td>73,93%</td></tr><tr><td>GB AF</td><td>23548</td><td>2767</td><td>9202</td><td>66,30%</td><td>69,96%</td></tr><tr><td>AR</td><td>25607</td><td>3488</td><td>7143</td><td>70,66%</td><td>76,03%</td></tr><tr><td>AR AF</td><td>24431</td><td>3348</td><td>8319</td><td>67,68%</td><td>72,37%</td></tr><tr><td>GC</td><td>26093</td><td>4401</td><td>6657</td><td>70,23%</td><td>77,14%</td></tr><tr><td>GC AF</td><td>24925</td><td>4221</td><td>7825</td><td>67,42%</td><td>73,50%</td></tr><tr><td>LS</td><td>25890</td><td>4132</td><td>6860</td><td>70,20%</td><td>76,73%</td></tr><tr><td>LS AF</td><td>24671</td><td>3998</td><td>8079</td><td>67,14%</td><td>72,90%</td></tr><tr><td>CR</td><td>25533</td><td>3542</td><td>7217</td><td>70,35%</td><td>75,81%</td></tr><tr><td>CR AF</td><td>24278</td><td>3399</td><td>8472</td><td>67,16%</td><td>71,91%</td></tr></tbody></table>

seldom, we obtain the following conclusion: (1) Voting and Maximum merging algorithms provide higher mAP and ACC than the Baseline, except for the HF, EQ, LS configuration for the KITTI dataset with a 608x608 network size, where we observe a slight ACC drop w.r.t. the baseline (for reasons previously discussed in Section IV-C), (2) Averaging only produces better results than the baseline with the KITTI dataset since the model was trained with the COCO training subset. Therefore, it is expected that some image transformations could surpass the accuracy of the baseline when using other datasets.

When faults occur independently, we obtain the following conclusions: (1) Voting provides higher mAP and ACC than the Baseline with both datasets and both network sizes
<table><caption>TABLE XIII: TMR results for the best mAP and ACC configurations of each merging algorithm using the COCO dataset (320x320 network size).</caption><thead><tr><th colspan="7">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>3087</td><td>373</td><td>3104</td><td>47,03%</td><td>48,09%</td></tr></tbody><thead><tr><th colspan="7">VOTING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL_1, RV, BS</td><td>3098</td><td>357</td><td>3093</td><td>47,31%</td><td>48,69%</td></tr><tr><td>BL_1, RV, BS (indep. faults)</td><td>3071</td><td>356</td><td>3120</td><td>46,91%</td><td>48,55%</td></tr><tr><td>BL_1, RV, BS (same frame)</td><td>3029</td><td>343</td><td>3162</td><td>46,36%</td><td>47,89%</td></tr><tr><td>RV, LS, BS</td><td>3091</td><td>290</td><td>3100</td><td>47,69%</td><td>48,66%</td></tr><tr><td>RV, LS, BS (indep. faults)</td><td>3058</td><td>291</td><td>3133</td><td>47,18%</td><td>48,39%</td></tr><tr><td>RV, LS, BS (same frame)</td><td>3028</td><td>283</td><td>3163</td><td>46,77%</td><td>47,92%</td></tr></tbody><thead><tr><th colspan="7">AVERAGING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL_1, BL_2, BL_3</td><td>3087</td><td>373</td><td>3104</td><td>47,03%</td><td>48,69%</td></tr><tr><td>BL_1, BL_2, BL_3 (indep. faults)</td><td>2875</td><td>414</td><td>3316</td><td>43,53%</td><td>45,13%</td></tr><tr><td>BL_1, BL_2, BL_3 (same frame)</td><td>2594</td><td>447</td><td>3597</td><td>39,08%</td><td>40,50%</td></tr><tr><td>BL_1, GC, RV</td><td>3079</td><td>370</td><td>3112</td><td>46,93%</td><td>48,66%</td></tr><tr><td>BL_1, GC, RV (indep. faults)</td><td>2945</td><td>423</td><td>3246</td><td>44,53%</td><td>46,31%</td></tr><tr><td>BL_1, GC, RV (same frame)</td><td>2607</td><td>459</td><td>3584</td><td>39,20%</td><td>40,73%</td></tr></tbody><thead><tr><th colspan="7">MAXIMUM</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>HF, RS, TS</td><td>3435</td><td>630</td><td>2756</td><td>50,36%</td><td>52,00%</td></tr><tr><td>HF, RS, TS (indep. faults)</td><td>3415</td><td>761</td><td>2776</td><td>49,12%</td><td>50,24%</td></tr><tr><td>HF, RS, TS (same frame)</td><td>3393</td><td>1122</td><td>2798</td><td>46,40%</td><td>44,17%</td></tr><tr><td>HF, RS, BS</td><td>3432</td><td>645</td><td>2759</td><td>50,20%</td><td>51,98%</td></tr><tr><td>HF, RS, BS (indep. faults)</td><td>3412</td><td>788</td><td>2779</td><td>48,89%</td><td>49,75%</td></tr><tr><td>HF, RS, BS (same frame)</td><td>3398</td><td>1130</td><td>2793</td><td>46,41%</td><td>44,26%</td></tr></tbody></table>

<table><caption>TABLE XIV: TMR results for the best mAP and ACC configurations of each merging algorithm using the KITTI dataset (320x320 network size).</caption><thead><tr><th colspan="7">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>22247</td><td>2742</td><td>10503</td><td>62,68%</td><td>66,07%</td></tr></tbody><thead><tr><th colspan="7">VOTING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>EQ, GB, GC</td><td>22852</td><td>2614</td><td>9898</td><td>64,62%</td><td>68,10%</td></tr><tr><td>EQ, GB, GC (indep. faults)</td><td>22796</td><td>2587</td><td>9954</td><td>64,51%</td><td>67,95%</td></tr><tr><td>EQ, GB, GC (same image)</td><td>22501</td><td>2516</td><td>10249</td><td>63,80%</td><td>67,08%</td></tr><tr><td>EQ, GB, LS</td><td>22760</td><td>2221</td><td>9990</td><td>65,08%</td><td>67,94%</td></tr><tr><td>EQ, GB, LS (indep. faults)</td><td>22668</td><td>2171</td><td>10082</td><td>64,91%</td><td>67,67%</td></tr><tr><td>EQ, GB, LS (same image)</td><td>22354</td><td>2104</td><td>10396</td><td>64,14%</td><td>66,75%</td></tr></tbody><thead><tr><th colspan="7">AVERAGING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>EQ, GB, GC</td><td>22700</td><td>2337</td><td>10050</td><td>64,70%</td><td>67,54%</td></tr><tr><td>EQ, GB, GC (indep. faults)</td><td>21607</td><td>2375</td><td>11143</td><td>61,51%</td><td>64,21%</td></tr><tr><td>EQ, GB, GC (same image)</td><td>19734</td><td>2387</td><td>13016</td><td>56,16%</td><td>58,56%</td></tr><tr><td>EQ, GC, RV</td><td>22690</td><td>2704</td><td>10060</td><td>64,00%</td><td>67,39%</td></tr><tr><td>EQ, GC, RV (indep. faults)</td><td>21620</td><td>2702</td><td>11130</td><td>60,98%</td><td>64,16%</td></tr><tr><td>EQ, GC, RV (same image)</td><td>19707</td><td>2678</td><td>13043</td><td>55,63%</td><td>58,37%</td></tr></tbody><thead><tr><th colspan="7">MAXIMUM</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>EQ, GB, RS</td><td>24495</td><td>4609</td><td>8255</td><td>65,57%</td><td>72,53%</td></tr><tr><td>EQ, GB, RS (indep. faults)</td><td>24499</td><td>4845</td><td>8251</td><td>65,17%</td><td>71,18%</td></tr><tr><td>EQ, GB, RS (same image)</td><td>24442</td><td>5085</td><td>8308</td><td>64,60%</td><td>69,49%</td></tr><tr><td>EQ, GB, BS</td><td>24382</td><td>4226</td><td>8368</td><td>65,94%</td><td>72,26%</td></tr><tr><td>EQ, GB, BS (indep. faults)</td><td>24409</td><td>4440</td><td>8341</td><td>65,63%</td><td>70,94%</td></tr><tr><td>EQ, GB, BS (same image)</td><td>24330</td><td>4699</td><td>8420</td><td>64,97%</td><td>69,18%</td></tr></tbody></table>

analysed. (2) Maximum provides higher mAP and ACC than the Baseline for the COCO dataset with both network sizes, and for the KITTI dataset with a 320x320 network size. However, for the KITTI dataset with a 608x608 network size, we observe that the HF, EQ, LS (indep. faults) configuration provides higher mAP but lower ACC (for reasons previously discussed in Section IV-C), and the GB, CR, AR (indep. faults)

<table><caption>TABLE XV: TMR results for the best mAP and ACC configurations of each merging algorithm using the COCO dataset (608x608 network size).</caption><thead><tr><th colspan="7">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>3865</td><td>466</td><td>2326</td><td>58,06%</td><td>60,87%</td></tr></tbody><thead><tr><th colspan="7">VOTING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>HF, LS, TS</td><td>3901</td><td>403</td><td>2290</td><td>59,16%</td><td>61,60%</td></tr><tr><td>HF, LS, TS (indep. faults)</td><td>3896</td><td>401</td><td>2295</td><td>59,10%</td><td>61,51%</td></tr><tr><td>HF, LS, TS (same frame)</td><td>3822</td><td>398</td><td>2369</td><td>58,01%</td><td>60,36%</td></tr><tr><td>RV, LS, TS</td><td>3900</td><td>416</td><td>2291</td><td>59,03%</td><td>61,54%</td></tr><tr><td>RV, LS, TS (indep. faults)</td><td>3880</td><td>418</td><td>2311</td><td>58,71%</td><td>61,23%</td></tr><tr><td>RV, LS, TS (same frame)</td><td>3831</td><td>410</td><td>2360</td><td>58,04%</td><td>60,48%</td></tr></tbody><thead><tr><th colspan="7">AVERAGING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL_1, BL_2, BL_3</td><td>3865</td><td>466</td><td>2326</td><td>58,06%</td><td>60,87%</td></tr><tr><td>BL_1, BL_2, BL_3 (indep. faults)</td><td>3697</td><td>490</td><td>2494</td><td>55,34%</td><td>58,17%</td></tr><tr><td>BL_1, BL_2, BL_3 (same frame)</td><td>3392</td><td>542</td><td>2799</td><td>50,38%</td><td>53,15%</td></tr><tr><td>BL_1, GC, RV</td><td>3862</td><td>460</td><td>2329</td><td>58,07%</td><td>60,84%</td></tr><tr><td>BL_1, GC, RV (indep. faults)</td><td>3751</td><td>472</td><td>2440</td><td>56,30%</td><td>59,05%</td></tr><tr><td>BL_1, GC, RV (same frame)</td><td>3449</td><td>510</td><td>2742</td><td>51,47%</td><td>54,05%</td></tr></tbody><thead><tr><th colspan="7">MAXIMUM</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>HF, RS, BS</td><td>4214</td><td>691</td><td>1977</td><td>61,23%</td><td>66,01%</td></tr><tr><td>HF, RS, BS (indep. faults)</td><td>4204</td><td>743</td><td>1987</td><td>60,63%</td><td>64,32%</td></tr><tr><td>HF, RS, BS (same frame)</td><td>4186</td><td>980</td><td>2005</td><td>58,37%</td><td>58,75%</td></tr><tr><td>HF, LS, TS</td><td>4207</td><td>713</td><td>1984</td><td>60,94%</td><td>65,87%</td></tr><tr><td>HF, LS, TS (indep. faults)</td><td>4201</td><td>823</td><td>1990</td><td>59,89%</td><td>62,69%</td></tr><tr><td>HF, LS, TS (same frame)</td><td>4188</td><td>1071</td><td>2003</td><td>57,67%</td><td>57,59%</td></tr></tbody></table>

<table><caption>TABLE XVI: TMR results for the best mAP and ACC configurations of each merging algorithm using the KITTI dataset (608x608 network size).</caption><thead><tr><th colspan="7">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>25935</td><td>4093</td><td>6815</td><td>70,39%</td><td>76,86%</td></tr></tbody><thead><tr><th colspan="7">VOTING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>HF, EQ, GC</td><td>26199</td><td>4101</td><td>6551</td><td>71,09%</td><td>77,76%</td></tr><tr><td>HF, EQ, GC (indep. faults)</td><td>26168</td><td>4057</td><td>6582</td><td>71,10%</td><td>77,68%</td></tr><tr><td>HF, EQ, GC (same frame)</td><td>25910</td><td>3957</td><td>6840</td><td>70,59%</td><td>76,92%</td></tr><tr><td>EQ, GB, LS</td><td>25862</td><td>3426</td><td>6888</td><td>71,49%</td><td>77,01%</td></tr><tr><td>EQ, GB, LS (indep. faults)</td><td>25811</td><td>3387</td><td>6939</td><td>71,43%</td><td>76,87%</td></tr><tr><td>EQ, GB, LS (same frame)</td><td>25502</td><td>3293</td><td>7248</td><td>70,75%</td><td>75,95%</td></tr></tbody><thead><tr><th colspan="7">AVERAGING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>HF, EQ, GC</td><td>26057</td><td>5877</td><td>6693</td><td>71,14%</td><td>77,32%</td></tr><tr><td>HF, EQ, GC (indep. faults)</td><td>25135</td><td>3846</td><td>7615</td><td>68,68%</td><td>74,54%</td></tr><tr><td>HF, EQ, GC (same frame)</td><td>23178</td><td>3776</td><td>9572</td><td>63,46%</td><td>68,70%</td></tr><tr><td>HF, EQ, AR</td><td>25853</td><td>3468</td><td>6897</td><td>71,38%</td><td>76,85%</td></tr><tr><td>HF, EQ, AR (indep. faults)</td><td>24946</td><td>3449</td><td>7804</td><td>68,91%</td><td>74,13%</td></tr><tr><td>HF, EQ, AR (same frame)</td><td>22901</td><td>3425</td><td>9849</td><td>63,31%</td><td>67,99%</td></tr></tbody><thead><tr><th colspan="7">MAXIMUM</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>HF, EQ, LS</td><td>26974</td><td>3879</td><td>6693</td><td>69,83%</td><td>79,80%</td></tr><tr><td>HF, EQ, LS (indep. faults)</td><td>26956</td><td>5988</td><td>5794</td><td>69,59%</td><td>78,79%</td></tr><tr><td>HF, EQ, LS (same frame)</td><td>26921</td><td>6183</td><td>5829</td><td>69,15%</td><td>77,33%</td></tr><tr><td>GB, CR, AR</td><td>26182</td><td>4318</td><td>6568</td><td>70,63%</td><td>77,79%</td></tr><tr><td>GB, CR, AR (indep. faults)</td><td>26171</td><td>4449</td><td>6579</td><td>70,35%</td><td>76,77%</td></tr><tr><td>GB, CR, AR (same frame)</td><td>26142</td><td>4694</td><td>6608</td><td>69,82%</td><td>75,31%</td></tr></tbody></table>

configuration provides higher ACC but lower mAP, since this configuration provides the highest ACC in fault-free cases, but does not deliver as high mAP as the top-mAP configuration. (3) Averaging produces worse results than the baseline with all configurations.

When faults occur in the same image for all three configurations, we obtain the following conclusions: (1) for the
<table><caption>TABLE XIII: TMR results for the best mAP and ACC configurations of each merging algorithm using the COCO dataset (320x320 network size).</caption><thead><tr><th colspan="7">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>3087</td><td>373</td><td>3104</td><td>47,03%</td><td>48,09%</td></tr></tbody><thead><tr><th colspan="7">VOTING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL_1, RV, BS</td><td>3098</td><td>357</td><td>3093</td><td>47,31%</td><td>48,69%</td></tr><tr><td>BL_1, RV, BS (indep. faults)</td><td>3071</td><td>356</td><td>3120</td><td>46,91%</td><td>48,55%</td></tr><tr><td>BL_1, RV, BS (same frame)</td><td>3029</td><td>343</td><td>3162</td><td>46,36%</td><td>47,89%</td></tr><tr><td>RV, LS, BS</td><td>3091</td><td>290</td><td>3100</td><td>47,69%</td><td>48,66%</td></tr><tr><td>RV, LS, BS (indep. faults)</td><td>3058</td><td>291</td><td>3133</td><td>47,18%</td><td>48,39%</td></tr><tr><td>RV, LS, BS (same frame)</td><td>3028</td><td>283</td><td>3163</td><td>46,77%</td><td>47,92%</td></tr></tbody><thead><tr><th colspan="7">AVERAGING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL_1, BL_2, BL_3</td><td>3087</td><td>373</td><td>3104</td><td>47,03%</td><td>48,69%</td></tr><tr><td>BL_1, BL_2, BL_3 (indep. faults)</td><td>2875</td><td>414</td><td>3316</td><td>43,53%</td><td>45,13%</td></tr><tr><td>BL_1, BL_2, BL_3 (same frame)</td><td>2594</td><td>447</td><td>3597</td><td>39,08%</td><td>40,50%</td></tr><tr><td>BL_1, GC, RV</td><td>3079</td><td>370</td><td>3112</td><td>46,93%</td><td>48,66%</td></tr><tr><td>BL_1, GC, RV (indep. faults)</td><td>2945</td><td>423</td><td>3246</td><td>44,53%</td><td>46,31%</td></tr><tr><td>BL_1, GC, RV (same frame)</td><td>2607</td><td>459</td><td>3584</td><td>39,20%</td><td>40,73%</td></tr></tbody><thead><tr><th colspan="7">MAXIMUM</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>HF, RS, TS</td><td>3435</td><td>630</td><td>2756</td><td>50,36%</td><td>52,00%</td></tr><tr><td>HF, RS, TS (indep. faults)</td><td>3415</td><td>761</td><td>2776</td><td>49,12%</td><td>50,24%</td></tr><tr><td>HF, RS, TS (same frame)</td><td>3393</td><td>1122</td><td>2798</td><td>46,40%</td><td>44,17%</td></tr><tr><td>HF, RS, BS</td><td>3432</td><td>645</td><td>2759</td><td>50,20%</td><td>51,98%</td></tr><tr><td>HF, RS, BS (indep. faults)</td><td>3412</td><td>788</td><td>2779</td><td>48,89%</td><td>49,75%</td></tr><tr><td>HF, RS, BS (same frame)</td><td>3398</td><td>1130</td><td>2793</td><td>46,41%</td><td>44,26%</td></tr></tbody></table>

<table><caption>TABLE XIV: TMR results for the best mAP and ACC configurations of each merging algorithm using the KITTI dataset (320x320 network size).</caption><thead><tr><th colspan="7">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>22247</td><td>2742</td><td>10503</td><td>62,68%</td><td>66,07%</td></tr></tbody><thead><tr><th colspan="7">VOTING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>EQ, GB, GC</td><td>22852</td><td>2614</td><td>9898</td><td>64,62%</td><td>68,10%</td></tr><tr><td>EQ, GB, GC (indep. faults)</td><td>22796</td><td>2587</td><td>9954</td><td>64,51%</td><td>67,95%</td></tr><tr><td>EQ, GB, GC (same image)</td><td>22501</td><td>2516</td><td>10249</td><td>63,80%</td><td>67,08%</td></tr><tr><td>EQ, GB, LS</td><td>22760</td><td>2221</td><td>9990</td><td>65,08%</td><td>67,94%</td></tr><tr><td>EQ, GB, LS (indep. faults)</td><td>22668</td><td>2171</td><td>10082</td><td>64,91%</td><td>67,67%</td></tr><tr><td>EQ, GB, LS (same image)</td><td>22354</td><td>2104</td><td>10396</td><td>64,14%</td><td>66,75%</td></tr></tbody><thead><tr><th colspan="7">AVERAGING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>EQ, GB, GC</td><td>22700</td><td>2337</td><td>10050</td><td>64,70%</td><td>67,54%</td></tr><tr><td>EQ, GB, GC (indep. faults)</td><td>21607</td><td>2375</td><td>11143</td><td>61,51%</td><td>64,21%</td></tr><tr><td>EQ, GB, GC (same image)</td><td>19734</td><td>2387</td><td>13016</td><td>56,16%</td><td>58,56%</td></tr><tr><td>EQ, GC, RV</td><td>22690</td><td>2704</td><td>10060</td><td>64,00%</td><td>67,39%</td></tr><tr><td>EQ, GC, RV (indep. faults)</td><td>21620</td><td>2702</td><td>11130</td><td>60,98%</td><td>64,16%</td></tr><tr><td>EQ, GC, RV (same image)</td><td>19707</td><td>2678</td><td>13043</td><td>55,63%</td><td>58,37%</td></tr></tbody><thead><tr><th colspan="7">MAXIMUM</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>EQ, GB, RS</td><td>24495</td><td>4609</td><td>8255</td><td>65,57%</td><td>72,53%</td></tr><tr><td>EQ, GB, RS (indep. faults)</td><td>24499</td><td>4845</td><td>8251</td><td>65,17%</td><td>71,18%</td></tr><tr><td>EQ, GB, RS (same image)</td><td>24442</td><td>5085</td><td>8308</td><td>64,60%</td><td>69,49%</td></tr><tr><td>EQ, GB, BS</td><td>24382</td><td>4226</td><td>8368</td><td>65,94%</td><td>72,26%</td></tr><tr><td>EQ, GB, BS (indep. faults)</td><td>24409</td><td>4440</td><td>8341</td><td>65,63%</td><td>70,94%</td></tr><tr><td>EQ, GB, BS (same image)</td><td>24330</td><td>4699</td><td>8420</td><td>64,97%</td><td>69,18%</td></tr></tbody></table>

analysed. (2) Maximum provides higher mAP and ACC than the Baseline for the COCO dataset with both network sizes, and for the KITTI dataset with a 320x320 network size. However, for the KITTI dataset with a 608x608 network size, we observe that the HF, EQ, LS (indep. faults) configuration provides higher mAP but lower ACC (for reasons previously discussed in Section IV-C), and the GB, CR, AR (indep. faults)

<table><caption>TABLE XV: TMR results for the best mAP and ACC configurations of each merging algorithm using the COCO dataset (608x608 network size).</caption><thead><tr><th colspan="7">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>3865</td><td>466</td><td>2326</td><td>58,06%</td><td>60,87%</td></tr></tbody><thead><tr><th colspan="7">VOTING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>HF, LS, TS</td><td>3901</td><td>403</td><td>2290</td><td>59,16%</td><td>61,60%</td></tr><tr><td>HF, LS, TS (indep. faults)</td><td>3896</td><td>401</td><td>2295</td><td>59,10%</td><td>61,51%</td></tr><tr><td>HF, LS, TS (same frame)</td><td>3822</td><td>398</td><td>2369</td><td>58,01%</td><td>60,36%</td></tr><tr><td>RV, LS, TS</td><td>3900</td><td>416</td><td>2291</td><td>59,03%</td><td>61,54%</td></tr><tr><td>RV, LS, TS (indep. faults)</td><td>3880</td><td>418</td><td>2311</td><td>58,71%</td><td>61,23%</td></tr><tr><td>RV, LS, TS (same frame)</td><td>3831</td><td>410</td><td>2360</td><td>58,04%</td><td>60,48%</td></tr></tbody><thead><tr><th colspan="7">AVERAGING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL_1, BL_2, BL_3</td><td>3865</td><td>466</td><td>2326</td><td>58,06%</td><td>60,87%</td></tr><tr><td>BL_1, BL_2, BL_3 (indep. faults)</td><td>3697</td><td>490</td><td>2494</td><td>55,34%</td><td>58,17%</td></tr><tr><td>BL_1, BL_2, BL_3 (same frame)</td><td>3392</td><td>542</td><td>2799</td><td>50,38%</td><td>53,15%</td></tr><tr><td>BL_1, GC, RV</td><td>3862</td><td>460</td><td>2329</td><td>58,07%</td><td>60,84%</td></tr><tr><td>BL_1, GC, RV (indep. faults)</td><td>3751</td><td>472</td><td>2440</td><td>56,30%</td><td>59,05%</td></tr><tr><td>BL_1, GC, RV (same frame)</td><td>3449</td><td>510</td><td>2742</td><td>51,47%</td><td>54,05%</td></tr></tbody><thead><tr><th colspan="7">MAXIMUM</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>HF, RS, BS</td><td>4214</td><td>691</td><td>1977</td><td>61,23%</td><td>66,01%</td></tr><tr><td>HF, RS, BS (indep. faults)</td><td>4204</td><td>743</td><td>1987</td><td>60,63%</td><td>64,32%</td></tr><tr><td>HF, RS, BS (same frame)</td><td>4186</td><td>980</td><td>2005</td><td>58,37%</td><td>58,75%</td></tr><tr><td>HF, LS, TS</td><td>4207</td><td>713</td><td>1984</td><td>60,94%</td><td>65,87%</td></tr><tr><td>HF, LS, TS (indep. faults)</td><td>4201</td><td>823</td><td>1990</td><td>59,89%</td><td>62,69%</td></tr><tr><td>HF, LS, TS (same frame)</td><td>4188</td><td>1071</td><td>2003</td><td>57,67%</td><td>57,59%</td></tr></tbody></table>

<table><caption>TABLE XVI: TMR results for the best mAP and ACC configurations of each merging algorithm using the KITTI dataset (608x608 network size).</caption><thead><tr><th colspan="7">BASELINE</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>BL</td><td>25935</td><td>4093</td><td>6815</td><td>70,39%</td><td>76,86%</td></tr></tbody><thead><tr><th colspan="7">VOTING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>HF, EQ, GC</td><td>26199</td><td>4101</td><td>6551</td><td>71,09%</td><td>77,76%</td></tr><tr><td>HF, EQ, GC (indep. faults)</td><td>26168</td><td>4057</td><td>6582</td><td>71,10%</td><td>77,68%</td></tr><tr><td>HF, EQ, GC (same frame)</td><td>25910</td><td>3957</td><td>6840</td><td>70,59%</td><td>76,92%</td></tr><tr><td>EQ, GB, LS</td><td>25862</td><td>3426</td><td>6888</td><td>71,49%</td><td>77,01%</td></tr><tr><td>EQ, GB, LS (indep. faults)</td><td>25811</td><td>3387</td><td>6939</td><td>71,43%</td><td>76,87%</td></tr><tr><td>EQ, GB, LS (same frame)</td><td>25502</td><td>3293</td><td>7248</td><td>70,75%</td><td>75,95%</td></tr></tbody><thead><tr><th colspan="7">AVERAGING</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>HF, EQ, GC</td><td>26057</td><td>3877</td><td>6693</td><td>71,14%</td><td>77,32%</td></tr><tr><td>HF, EQ, GC (indep. faults)</td><td>25135</td><td>3846</td><td>7615</td><td>68,68%</td><td>74,54%</td></tr><tr><td>HF, EQ, GC (same frame)</td><td>23178</td><td>3776</td><td>9572</td><td>63,46%</td><td>68,70%</td></tr><tr><td>HF, EQ, AR</td><td>25853</td><td>3468</td><td>6897</td><td>71,38%</td><td>76,85%</td></tr><tr><td>HF, EQ, AR (indep. faults)</td><td>24946</td><td>3449</td><td>7804</td><td>68,91%</td><td>74,13%</td></tr><tr><td>HF, EQ, AR (same frame)</td><td>22901</td><td>3425</td><td>9849</td><td>63,31%</td><td>67,99%</td></tr></tbody><thead><tr><th colspan="7">MAXIMUM</th></tr><tr><th>Configuration</th><th>TP</th><th>FP</th><th>FN</th><th>ACC</th><th>mAP</th></tr></thead><tbody><tr><td>HF, EQ, LS</td><td>26974</td><td>5877</td><td>6693</td><td>69,83%</td><td>79,80%</td></tr><tr><td>HF, EQ, LS (indep. faults)</td><td>26956</td><td>5988</td><td>5794</td><td>69,59%</td><td>78,79%</td></tr><tr><td>HF, EQ, LS (same frame)</td><td>26921</td><td>6183</td><td>5829</td><td>69,15%</td><td>77,33%</td></tr><tr><td>GB, CR, AR</td><td>26182</td><td>4318</td><td>6568</td><td>70,63%</td><td>77,79%</td></tr><tr><td>GB, CR, AR (indep. faults)</td><td>26171</td><td>4449</td><td>6579</td><td>70,35%</td><td>76,77%</td></tr><tr><td>GB, CR, AR (same frame)</td><td>26142</td><td>4694</td><td>6608</td><td>69,82%</td><td>75,31%</td></tr></tbody></table>

configuration provides higher ACC but lower mAP, since this configuration provides the highest ACC in fault-free cases, but does not deliver as high mAP as the top-mAP configuration. (3) Averaging produces worse results than the baseline with all configurations.

When faults occur in the same image for all three configurations, we obtain the following conclusions: (1) for the
COCO dataset, all merging algorithms provide lower accuracy than the baseline, (2) Voting produces the closest accuracy to the baseline, both in terms of ACC and mAP, (3) for the KITTI dataset, both Voting and Maximum have at least one configuration producing higher accuracy (either both ACC and mAP or only one of the metrics) than the baseline.

Overall, we can conclude the following (1) Averaging is consistently worse than Voting and Maximum in all cases, (2) Maximum produces the highest ACC and mAP results in the fault-free scenarios, as well as when faults occur independently, except for the KITTI dataset with a 608x608 network size, where the ACC of Voting is slightly higher, (3) Voting produces the highest mAP results with the COCO dataset when faults occur simultaneously, but regarding the ACC, two configurations produce higher ACC with Voting, while the other two produce higher ACC with Maximum. (4) When faults occur simultaneously for the KITTI dataset, Maximum provides higher ACC and mAP than Voting with a 320x320 network size, but Voting provides higher ACC with a 608x608 network size.

Overall, we can conclude that the Maximum merging algorithm produces the best mAP results, but it provides significantly higher FPs than Voting. Therefore, if FPs are a major concern, Voting is the best solution overall, while if the objective is to obtain the maximum mAP, the Maximum algorithm is the best solution since the fault-free scenario equals to virtually 100% of time, and typically faults will not affect all redundant components at the same time.

## V. CONCLUSIONS

AI software is increasingly used in safety-critical systems for functionalities where such software inherits safety requirements. Whenever those requirements relate to the highest integrity levels, AI software must be realized with diverse redundancy, such as, for instance, TMR. Existing solutions based on lockstep processors provide such diverse redundancy with bit-level error detection and correction. In this paper, we note that abundant AI functionalities do not need bit-level correctness and, instead, semantic correctness suffices due to the stochastic nature of the AI-based functionalities implemented. We leverage this observation to present a diverse redundancy scheme for AI models based on applying minor transformations to the input data with the aim of creating diverse, yet semantically identical, predictions, which we use to mitigate, apart from hardware random errors, also AI model errors.

As future work, we plan to extend our work evaluating the different image transformations and merging algorithms in DMR scenarios, which are particularly relevant for some domains, such as automotive.

## ACKNOWLEDGEMENT

The research leading to these results has received funding from the European Union's Horizon Europe Programme under the SAFEXPLAIN Project (www.safexplain.eu), grant agreement num. 101069595. This work has also been partially supported by the Spanish Ministry of Science and Innovation under grant PID2019-107255GC21/AEI/10.13039/501100011033. M. Caro has received funding under grant PRE2022-102440 funded by MICIU/AEI/10.13039/501100011033 and, by ESF+.

## REFERENCES

[1] International Standards Organization and SAE International, ISO/SAE PAS 22736:2021: Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles, 2021.

[2] SAFEXPLAIN Consortium, "Space case study," https://safexplain.eu/space-case-study/, Accessed in Nov-2023.

[3] L. Lutnyk et al., "Towards pilot-aware cockpits," in *1st International Workshop on Eye-Tracking in Aviation (ETAVI)*, 2020, pp. 121–127.

[4] J. Perez-Cerrolaza et al., "Artificial intelligence for safety-critical systems in industrial and transportation domains: A survey," *ACM Comput. Surv.*, oct 2023, just Accepted. [Online]. Available: https://doi.org/10.1145/3626314

[5] M. Caro et al., "At-scale assessment of weight clustering for energy-efficient object detection accelerators," in *Proceedings of the 37th ACM/SIGAPP Symposium on Applied Computing*, 2022.

[6] A. Bochkovskiy et al., "YOLOv4: Optimal speed and accuracy of object detection," 2020. [Online]. Available: https://arxiv.org/abs/2004.10934

[7] Infineon, "AURIX multicore 32-bit microcontroller family to meet safety and powertrain requirements of upcoming vehicle generations," Accessed in Oct-2023, http://www.infineon.com/cms/en/about-infineon/press/press-releases/2012/INFATV201205-040.html.

[8] S. Alcaide et al., "High-integrity GPU designs for critical real-time automotive systems," in *DATE*, 2019.

[9] ——, "Software-only based diverse redundancy for ASIL-D automotive applications on embedded HPC platforms," in *DFT*, 2020.

[10] L. K. Draghetti et al., "Detecting errors in convolutional neural networks using inter frame spatio-temporal correlation," in *IOLTS*, 2019.

[11] S. Jha et al., "Exploiting temporal data diversity for detecting safety-critical faults in AV compute systems," in *DSN*, 2022.

[12] M. Caro et al., "Efficient diverse redundant DNNs for autonomous driving," in *2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC)*, 2023, pp. 18–27.

[13] T. Strauss et al., "Ensemble methods as a defense to adversarial perturbations against deep neural networks," *arXiv preprint arXiv:1709.03423*, 2017.

[14] L. Hansen and P. Salamon, "Neural network ensembles," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 12, no. 10, pp. 993–1001, 1990.

[15] Z. Gao et al., "Soft error tolerant convolutional neural networks on fpgas with ensemble learning," *IEEE Transactions on Very Large Scale Integration (VLSI) Systems*, vol. 30, no. 3, pp. 291–302, 2022.

[16] J. Ayerdi, A. Iriarte, P. Valle, I. Roman, M. Illarramendi, and A. Arrieta, "Metamorphic runtime monitoring of autonomous driving systems," 2023.

[17] S. Alcaide et al., "Software-only triple diverse redundancy on GPUs for autonomous driving platforms," in *50th Annual IEEE-IFIP International Conference on Dependable Systems and Networks-Supplemental Volume (DSN-S)*, 2020, pp. 82–88.

[18] "Apollo, an open autonomous driving platform," http://apollo.auto/, Accessed in Oct-2023.

[19] J. Heras, "CLoDSA image augmentation library for object detection," https://github.com/joheras/CLoDSA, Accessed in Oct-2023.

[20] C. Li, "YOLOv4 tensorflow keras implementation," https://github.com/taipingeric/yolo-v4-tf-keras, Accessed in Oct-2023.

[21] T. Lin et al., "Microsoft COCO: Common objects in context," 2015.

[22] A. Bochkovskiy, "YOLOv4 on the darknet framework and publicly available weights," https://github.com/AlexeyAB/darknet, Accessed in Oct-2023.

[23] Udacity, "Udacity self-driving car driving data," Accessed in Oct-2023. [Online]. Available: https://github.com/udacity/self-driving-car

[24] F. Yu et al., "BDD100K: A diverse driving video database with scalable annotation tooling," *CoRR*, vol. abs/1805.04687, 2018. [Online]. Available: http://arxiv.org/abs/1805.04687

[25] P. Sun et al., "Scalability in perception for autonomous driving: Waymo open dataset," *CoRR*, vol. abs/1912.04838, 2019. [Online]. Available: http://arxiv.org/abs/1912.04838

[26] M. Everingham et al., "The pascal visual object classes (VOC) challenge," *International Journal of Computer Vision*, vol. 88, no. 2, 2010.

[27] R. Padilla et al., "A survey on performance metrics for object-detection algorithms," in *2020 International Conference on Systems, Signals and Image Processing (IWSSIP)*, 2020, pp. 237–242.

[28] G. Li et al., "Understanding error propagation in deep learning neural network (DNN) accelerators and applications," in *SC17: International Conference for High Performance Computing, Networking, Storage and Analysis*, 2017, pp. 1–12.

[29] A. Bosio et al., "A reliability analysis of a deep neural network," in *2019 IEEE Latin American Test Symposium (LATS)*, 2019, pp. 1–6.
COCO dataset, all merging algorithms provide lower accuracy than the baseline, (2) Voting produces the closest accuracy to the baseline, both in terms of ACC and mAP, (3) for the KITTI dataset, both Voting and Maximum have at least one configuration producing higher accuracy (either both ACC and mAP or only one of the metrics) than the baseline.

Overall, we can conclude the following (1) Averaging is consistently worse than Voting and Maximum in all cases, (2) Maximum produces the highest ACC and mAP results in the fault-free scenarios, as well as when faults occur independently, except for the KITTI dataset with a 608x608 network size, where the ACC of Voting is slightly higher, (3) Voting produces the highest mAP results with the COCO dataset when faults occur simultaneously, but regarding the ACC, two configurations produce higher ACC with Voting, while the other two produce higher ACC with Maximum. (4) When faults occur simultaneously for the KITTI dataset, Maximum provides higher ACC and mAP than Voting with a 320x320 network size, but Voting provides higher ACC with a 608x608 network size.

Overall, we can conclude that the Maximum merging algorithm produces the best mAP results, but it provides significantly higher FPs than Voting. Therefore, if FPs are a major concern, Voting is the best solution overall, while if the objective is to obtain the maximum mAP, the Maximum algorithm is the best solution since the fault-free scenario equals to virtually 100% of time, and typically faults will not affect all redundant components at the same time.

## V. CONCLUSIONS

AI software is increasingly used in safety-critical systems for functionalities where such software inherits safety requirements. Whenever those requirements relate to the highest integrity levels, AI software must be realized with diverse redundancy, such as, for instance, TMR. Existing solutions based on lockstep processors provide such diverse redundancy with bit-level error detection and correction. In this paper, we note that abundant AI functionalities do not need bit-level correctness and, instead, semantic correctness suffices due to the stochastic nature of the AI-based functionalities implemented. We leverage this observation to present a diverse redundancy scheme for AI models based on applying minor transformations to the input data with the aim of creating diverse, yet semantically identical, predictions, which we use to mitigate, apart from hardware random errors, also AI model errors.

As future work, we plan to extend our work evaluating the different image transformations and merging algorithms in DMR scenarios, which are particularly relevant for some domains, such as automotive.

## ACKNOWLEDGEMENT

The research leading to these results has received funding from the European Union's Horizon Europe Programme under the SAFEXPLAIN Project (www.safexplain.eu), grant agreement num. 101069595. This work has also been partially supported by the Spanish Ministry of Science and Innovation under grant PID2019-107255GC21/AEI/10.13039/501100011033. M. Caro has received funding under grant PRE2022-102440 funded by MICIU/AEI/10.13039/501100011033 and, by ESF+.

## REFERENCES

[1] International Standards Organization and SAE International, ISO/SAE PAS 22736:2021: Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles, 2021.

[2] SAFEXPLAIN Consortium, "Space case study," https://safexplain.eu/space-case-study/, Accessed in Nov-2023.

[3] L. Lutnyk et al., "Towards pilot-aware cockpits," in *1st International Workshop on Eye-Tracking in Aviation (ETAVI)*, 2020, pp. 121–127.

[4] J. Perez-Cerrolaza et al., "Artificial intelligence for safety-critical systems in industrial and transportation domains: A survey," *ACM Comput. Surv.*, oct 2023, just Accepted. [Online]. Available: https://doi.org/10.1145/3626314

[5] M. Caro et al., "At-scale assessment of weight clustering for energy-efficient object detection accelerators," in *Proceedings of the 37th ACM/SIGAPP Symposium on Applied Computing*, 2022.

[6] A. Bochkovskiy et al., "YOLOv4: Optimal speed and accuracy of object detection," 2020. [Online]. Available: https://arxiv.org/abs/2004.10934

[7] Infineon, "AURIX multicore 32-bit microcontroller family to meet safety and powertrain requirements of upcoming vehicle generations," Accessed in Oct-2023, http://www.infineon.com/cms/en/about-infineon/press/press-releases/2012/INFATV201205-040.html.

[8] S. Alcaide et al., "High-integrity GPU designs for critical real-time automotive systems," in *DATE*, 2019.

[9] ——, "Software-only based diverse redundancy for ASIL-D automotive applications on embedded HPC platforms," in *DFT*, 2020.

[10] L. K. Draghetti et al., "Detecting errors in convolutional neural networks using inter frame spatio-temporal correlation," in *IOLTS*, 2019.

[11] S. Jha et al., "Exploiting temporal data diversity for detecting safety-critical faults in AV compute systems," in *DSN*, 2022.

[12] M. Caro et al., "Efficient diverse redundant DNNs for autonomous driving," in *2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC)*, 2023, pp. 18–27.

[13] T. Strauss et al., "Ensemble methods as a defense to adversarial perturbations against deep neural networks," *arXiv preprint arXiv:1709.03423*, 2017.

[14] L. Hansen and P. Salamon, "Neural network ensembles," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 12, no. 10, pp. 993–1001, 1990.

[15] Z. Gao et al., "Soft error tolerant convolutional neural networks on fpgas with ensemble learning," *IEEE Transactions on Very Large Scale Integration (VLSI) Systems*, vol. 30, no. 3, pp. 291–302, 2022.

[16] J. Ayerdi, A. Iriarte, P. Valle, I. Roman, M. Illarramendi, and A. Arrieta, "Metamorphic runtime monitoring of autonomous driving systems," 2023.

[17] S. Alcaide et al., "Software-only triple diverse redundancy on GPUs for autonomous driving platforms," in *50th Annual IEEE-IFIP International Conference on Dependable Systems and Networks-Supplemental Volume (DSN-S)*, 2020, pp. 82–88.

[18] "Apollo, an open autonomous driving platform," http://apollo.auto/, Accessed in Oct-2023.

[19] J. Heras, "CLoDSA image augmentation library for object detection," https://github.com/joheras/CLoDSA, Accessed in Oct-2023.

[20] C. Li, "YOLOv4 tensorflow keras implementation," https://github.com/taipingeric/yolo-v4-tf-keras, Accessed in Oct-2023.

[21] T. Lin et al., "Microsoft COCO: Common objects in context," 2015.

[22] A. Bochkovskiy, "YOLOv4 on the darknet framework and publicly available weights," https://github.com/AlexeyAB/darknet, Accessed in Oct-2023.

[23] Udacity, "Udacity self-driving car driving data," Accessed in Oct-2023. [Online]. Available: https://github.com/udacity/self-driving-car

[24] F. Yu et al., "BDD100K: A diverse driving video database with scalable annotation tooling," *CoRR*, vol. abs/1805.04687, 2018. [Online]. Available: http://arxiv.org/abs/1805.04687

[25] P. Sun et al., "Scalability in perception for autonomous driving: Waymo open dataset," *CoRR*, vol. abs/1912.04838, 2019. [Online]. Available: http://arxiv.org/abs/1912.04838

[26] M. Everingham et al., "The pascal visual object classes (VOC) challenge," *International Journal of Computer Vision*, vol. 88, no. 2, 2010.

[27] R. Padilla et al., "A survey on performance metrics for object-detection algorithms," in *2020 International Conference on Systems, Signals and Image Processing (IWSSIP)*, 2020, pp. 237–242.

[28] G. Li et al., "Understanding error propagation in deep learning neural network (DNN) accelerators and applications," in *SC17: International Conference for High Performance Computing, Networking, Storage and Analysis*, 2017, pp. 1–12.

[29] A. Bosio et al., "A reliability analysis of a deep neural network," in *2019 IEEE Latin American Test Symposium (LATS)*, 2019, pp. 1–6.