Original: 3DB : A Framework for Debugging Computer Vision Models
Revised: 3DB : A Framework for Debugging Computer Vision Models

Revised: Guillaume

Revised: MIT ⇤

Original: Hadi Salman 
Revised: Hadi Salman †

Original: HADY 
Revised: HADY @ MIT EDU

Original: MIT 
Revised: MIT ⇤

Original: Andrew Ilyas 
Revised: Andrew Ilyas †

Original: AILYAS 
Revised: AILYAS @ MIT EDU

Original: MIT
Revised: MIT

Revised: Sai Vemprala

Revised: COM

Original: Microsoft Research
Revised: Microsoft Research

Original: Logan EngstromENGSTROM
Revised: Logan Engstrom

Revised: ENGSTROM @ MIT

Original: EDU
Revised: EDU

Original: MIT
Revised: MIT

Original: Vibhav Vineet
Revised: Vibhav Vineet

Original: VIVINEET @ 
Revised: VIVINEET @ MICROSOFT .

Original: COM
Revised: COM

Original: Microsoft Research
Revised: Microsoft Research

Original: Kai Xiao
Revised: Kai Xiao

Original: KAIX
Revised: KAIX

Original: MIT
Revised: MIT

Original: Pengchuan Zhang
Revised: Pengchuan Zhang

Original: PENZHAN @ MICROSOFT 
Revised: PENZHAN @ MICROSOFT .

Original: COM
Revised: COM

Original: Microsoft Research
Revised: Microsoft Research

Original: Shibani Santurkar
Revised: Shibani Santurkar

Original: SHIBANI @ MIT 
Revised: SHIBANI @ MIT .

Original: EDU
Revised: EDU

Original: MIT
Revised: MIT

Original: Greg Yang
Revised: Greg Yang

Original: GE .
Revised: GE .

Original: YANG @ MICROSOFT 
Revised: YANG @ MICROSOFT .

Original: COM
Revised: COM

Original: Microsoft Research
Revised: Microsoft Research

Original: Ashish Kapoor
Revised: Ashish Kapoor

Original: AKAPOOR @ MICROSOFT .
Revised: AKAPOOR @ MICROSOFT .

Original: COM
Revised: COM

Original: Microsoft Research
Revised: Microsoft Research

Original: Aleksander M ˛adry
Revised: Aleksander M ˛adry

Original: MADRY @ MIT 
Revised: MADRY @ MIT .

Original: EDU
Revised: EDU

Original: MIT
Revised: MIT

Original: Abstract
Revised: Abstract

Original: We introduce 3DB : an extendable, uniﬁed framework for testing and debugging vision models using photorealistic simulation.
Revised: We introduce 3DB : an extendable, uniﬁed framework for testing and debugging vision models using photorealistic simulation.

Original: We demonstrate, through a wide range of use cases, that 3DB allows users to discover vulnerabilities in computer vision systems and gain insights into how models make decisions.
Revised: We demonstrate, through a wide range of use cases, that 3DB allows users to discover vulnerabilities in computer vision systems and gain insights into how models make decisions.

Original: 3DB captures and generalizes many robustness analyses from prior work and enables one to study their interplay.
Revised: 3DB captures and generalizes many robustness analyses from prior work and enables one to study their interplay.

Original: Finally, we ﬁnd that the insights generated by the system transfer to the physical world.
Revised: Finally, we ﬁnd that the insights generated by the system transfer to the physical world.

Original: We are releasing 3DB as a library 1 alongside a set of examples 2 , guides 3 , and documentation 4 .
Revised: We are releasing 3DB as a library 1 alongside a set of examples 2 , guides 3 , and documentation 4 .

Original:  Introduction
Revised:  Introduction

Original: Modern machine learning models turn out to be remarkably brittle under distribution shift.
Revised: Modern machine learning models turn out to be remarkably brittle under distribution shift.

Original: Indeed, in the context of computer vision, models exhibit an abnormal sensitivity to slight input rotations and translations [18, 37], synthetic image corruptions [32, 38], and changes to the data collection pipeline [49, 19].
Revised: Indeed, in the context of computer vision, models exhibit an abnormal sensitivity to slight input rotations and translations [18, 37], synthetic image corruptions [32, 38], and changes to the data collection pipeline [49, 19].

Original: Still, while brittleness is widespread, it is often hard to understand its root causes, or even to characterize the precise situations in which this behavior arises.
Revised: Still, while brittleness is widespread, it is often hard to understand its root causes, or even to characterize the precise situations in which this behavior arises.

Original: How do we then comprehensively diagnose model failure modes?
Revised: How do we then comprehensively diagnose model failure modes?

Original: Stakes are often too high to simply deploy models and collect “real-world” failure cases.
Revised: Stakes are often too high to simply deploy models and collect “real-world” failure cases.

Original: There has thus been a line of work in computer vision focused on identifying systematic sources of model failure such as unfamiliar[Table] [Table] [Table]
Revised: There has thus been a line of work in computer vision focused on identifying systematic sources of model failure such as unfamiliar

Original: [Table] [Table] [Table]
Revised: [Figure] [Figure]

Original: [Table] [Figure] [Figure] object orientations [3], misleading backgrounds [74, 69], or shape-texture conﬂicts [25, 5].
Revised: object orientations [3], misleading backgrounds [74, 69], or shape-texture conﬂicts [25, 5].

Original: These analyses—a selection of which is visualized in Figure 1—reveal patterns or situations that degrade performance of vision models, providing invaluable insights into model robustness.
Revised: These analyses—a selection of which is visualized in Figure 1—reveal patterns or situations that degrade performance of vision models, providing invaluable insights into model robustness.

Original: Still, carrying out each such analysis requires its own set of (often complex) tools, usually accompanied by a signiﬁcant amount of manual labor (e.g., image editing, style transfer), expertise, and data cleaning.
Revised: Still, carrying out each such analysis requires its own set of (often complex) tools, usually accompanied by a signiﬁcant amount of manual labor (e.g., image editing, style transfer), expertise, and data cleaning.

Original: This prompts the question:
Revised: This prompts the question:

Original: Can we support reliable discovery of model failures in a systematic, automated, and uniﬁed way?
Revised: Can we support reliable discovery of model failures in a systematic, automated, and uniﬁed way?

Original: Contributions.
Revised: Contributions.

Original: In this work, we propose 3DB , a framework for automatically identifying and analyzing the failure modes of computer vision models.
Revised: In this work, we propose 3DB , a framework for automatically identifying and analyzing the failure modes of computer vision models.

Original: This framework makes use of a 3D simulator to render realistic scenes that can be fed into any computer vision system.
Revised: This framework makes use of a 3D simulator to render realistic scenes that can be fed into any computer vision system.

Original: Users can specify a set of transformations to apply to the scene—such as pose changes, background changes, or camera effects—and can also customize and compose them.
Revised: Users can specify a set of transformations to apply to the scene—such as pose changes, background changes, or camera effects—and can also customize and compose them.

Original: The system then performs a guided search, evaluation, and aggregation over these user-speciﬁed conﬁgurations and presents the user with an interactive, user-friendly summary of the model’s performance and vulnerabilities.
Revised: The system then performs a guided search, evaluation, and aggregation over these user-speciﬁed conﬁgurations and presents the user with an interactive, user-friendly summary of the model’s performance and vulnerabilities.

Original: 3DB is general enough to enable users to, with minimal effort, re-discover insights from prior work on pose, background, and texture bias (cf.
Revised: 3DB is general enough to enable users to, with minimal effort, re-discover insights from prior work on pose, background, and texture bias (cf.

Original: Fig. 2), among others.
Revised: Fig. 2), among others.

Original: Further, while prior studies have largely been focused on examining model sensitivities along a single axis, 3DB allows users to compose various transformations and understand the interplay between them, while still being able to disentangle their individual effects.
Revised: Further, while prior studies have largely been focused on examining model sensitivities along a single axis, 3DB allows users to compose various transformations and understand the interplay between them, while still being able to disentangle their individual effects.

Original: The remainder of this paper is structured into the following parts: in Section 2 we discuss the design of 3DB , including the motivating principles, design goals, and concrete architecture used.
Revised: The remainder of this paper is structured into the following parts: in Section 2 we discuss the design of 3DB , including the motivating principles, design goals, and concrete architecture used.

Original: We highlight how the implementation of 3DB allows users to quickly experiment, stress-test, and analyze their vision models.
Revised: We highlight how the implementation of 3DB allows users to quickly experiment, stress-test, and analyze their vision models.

Original: Then, in Section 3 we illustrate the utility of 3DB through a series of case studies uncovering biases in an ImageNet-pretrained classiﬁer.
Revised: Then, in Section 3 we illustrate the utility of 3DB through a series of case studies uncovering biases in an ImageNet-pretrained classiﬁer.

Original: Finally, we show (in Section 4) that the vulnerabilities uncovered with 3DB correspond to actual failure modes in the physical world (i.e., they are not speciﬁc to simulation).
Revised: Finally, we show (in Section 4) that the vulnerabilities uncovered with 3DB correspond to actual failure modes in the physical world (i.e., they are not speciﬁc to simulation).

Original:  Designing 3DB
Revised:  Designing 3DB

Original: The goal of 3DB is to leverage photorealistic simulation to effectively diagnose failure modes of computer vision models.
Revised: The goal of 3DB is to leverage photorealistic simulation to effectively diagnose failure modes of computer vision models.

Original: To this end, the following set of principles guide the design of 3DB :
Revised: To this end, the following set of principles guide the design of 3DB :

Original: Generality.
Revised: Generality.

Original: 3DB should support any type of computer vision model (i.e., not necessarily a neural network) trained on any dataset and task (i.e., not necessarily classiﬁcation).
Revised: 3DB should support any type of computer vision model (i.e., not necessarily a neural network) trained on any dataset and task (i.e., not necessarily classiﬁcation).

Original: Furthermore, the framework should support diagnosing non-robustness with respect to any parameterizable threedimensional scene transformation.
Revised: Furthermore, the framework should support diagnosing non-robustness with respect to any parameterizable threedimensional scene transformation.

Original: Compositionality.
Revised: Compositionality.

Original: Corruptions and transformations rarely occur in isolation— 3DB should allow users to investigate robustness along many different axes simultaneously.
Revised: Corruptions and transformations rarely occur in isolation— 3DB should allow users to investigate robustness along many different axes simultaneously.

Original: Physical realism.
Revised: Physical realism.

Original: The vulnerabilities extracted from 3DB should correspond to models’ behavior in the real (physical) world, and, in particular, not depend on artifacts of the simulation process itself.
Revised: The vulnerabilities extracted from 3DB should correspond to models’ behavior in the real (physical) world, and, in particular, not depend on artifacts of the simulation process itself.

Original: User-friendliness.
Revised: User-friendliness.

Original: 3DB should be simple to use and should relay insights to the user in an easyto-understand manner.
Revised: 3DB should be simple to use and should relay insights to the user in an easyto-understand manner.

Original: Even non-experts should be able to look at the result of a 3DB experiment and easily understand what the weak points of their model are, as well as gain insight into how the model behaves more generally.
Revised: Even non-experts should be able to look at the result of a 3DB experiment and easily understand what the weak points of their model are, as well as gain insight into how the model behaves more generally.

Original: Scalability.
Revised: Scalability.

Original: 3DB should be performant and parallel.
Revised: 3DB should be performant and parallel.

Original: 2.1 Capabilities and workﬂow
Revised: 2.1 Capabilities and workﬂow

Original: To achieve the goals articulated above, we design 3DB modularly, i.e., as a combination of swappable components.
Revised: To achieve the goals articulated above, we design 3DB modularly, i.e., as a combination of swappable components.

Original: This combination allows the user to specify transformations they want to test, search over the space of these transformations, and aggregate the results of this search in a concise way.
Revised: This combination allows the user to specify transformations they want to test, search over the space of these transformations, and aggregate the results of this search in a concise way.

Original: More speciﬁcally, the 3DB workﬂow revolves around ﬁve steps (visualized in Figure 3):
Revised: More speciﬁcally, the 3DB workﬂow revolves around ﬁve steps (visualized in Figure 3):

Original: Setup.
Revised: Setup.

Original: The user collects one or more 3D meshes that correspond to objects the model is trained to recognize, as well as a set of environments to test against.
Revised: The user collects one or more 3D meshes that correspond to objects the model is trained to recognize, as well as a set of environments to test against.

Original: Search space design.
Revised: Search space design.

Original: The user deﬁnes a search space by specifying a set of transformations (which 3DB calls controls ) that they expect the computer vision model to be robust to (e.g., rotations, translations, zoom, etc.).
Revised: The user deﬁnes a search space by specifying a set of transformations (which 3DB calls controls ) that they expect the computer vision model to be robust to (e.g., rotations, translations, zoom, etc.).

Original: Controls are grouped into “rendered controls” (applied during the rendering process) and “post-processor controls” (applied after the rendering as a 2D image transformation).
Revised: Controls are grouped into “rendered controls” (applied during the rendering process) and “post-processor controls” (applied after the rendering as a 2D image transformation).

Original: Policy-guided search.
Revised: Policy-guided search.

Original: After the user has speciﬁed a set of controls, 3DB instantiates and renders a myriad of object conﬁgurations derived from compositions of the given transformations.
Revised: After the user has speciﬁed a set of controls, 3DB instantiates and renders a myriad of object conﬁgurations derived from compositions of the given transformations.

Original: It records the behavior of the ML model on each constructed scene for later analysis.
Revised: It records the behavior of the ML model on each constructed scene for later analysis.

Original: A user-speciﬁed search policy over the space of all possible combinations of transformations determines the scenes for 3DB to render. 
Revised: A user-speciﬁed search policy over the space of all possible combinations of transformations determines the scenes for 3DB to render. [Figure] m

Original: [Figure]
Revised: [Figure]

Original: Model loading.
Revised: Model loading.

Original: The only remaining step before running a 3DB analysis is loading the model that the user wants to analyze (e.g., a pre-trained classiﬁer or object detector).
Revised: The only remaining step before running a 3DB analysis is loading the model that the user wants to analyze (e.g., a pre-trained classiﬁer or object detector).

Original: Analysis and insight extraction.
Revised: Analysis and insight extraction.

Original: Finally, 3DB is equipped with a model dashboard (cf.
Revised: Finally, 3DB is equipped with a model dashboard (cf.

Original: Appendix C) that can read the generated log ﬁles and produce a user-friendly visualization of the generated insights.
Revised: Appendix C) that can read the generated log ﬁles and produce a user-friendly visualization of the generated insights.

Original: By default, the dashboard has three panels.
Revised: By default, the dashboard has three panels.

Original: The ﬁrst of these is failure mode display, which highlights conﬁgurations, scenes, and transformations that caused the model to misbehave.
Revised: The ﬁrst of these is failure mode display, which highlights conﬁgurations, scenes, and transformations that caused the model to misbehave.

Original: The per-object analysis pane allows the user to inspect the model’s performance on a speciﬁc 3D mesh (e.g., accuracy, robustness, and vulnerability to groups of transformations).
Revised: The per-object analysis pane allows the user to inspect the model’s performance on a speciﬁc 3D mesh (e.g., accuracy, robustness, and vulnerability to groups of transformations).

Original: Finally, the aggregate analysis pane extracts insights about the model’s performance averaged over all the objects and environments collected and thus allows the user to notice consistent trends and vulnerabilities in their model.
Revised: Finally, the aggregate analysis pane extracts insights about the model’s performance averaged over all the objects and environments collected and thus allows the user to notice consistent trends and vulnerabilities in their model.

Original: Each of the aforementioned components (the controls, policy, renderer, inference module, and logger) are fully customizable and can be extended or replaced by the user without altering the core code of 3DB .
Revised: Each of the aforementioned components (the controls, policy, renderer, inference module, and logger) are fully customizable and can be extended or replaced by the user without altering the core code of 3DB .

Original: For example, while 3DB supports more than 10 types of controls out-of-the-box, users can add custom ones (e.g., geometric transformations) by implementing an abstract function that maps a 3D state and a set of parameters to a new state.
Revised: For example, while 3DB supports more than 10 types of controls out-of-the-box, users can add custom ones (e.g., geometric transformations) by implementing an abstract function that maps a 3D state and a set of parameters to a new state.

Original: Similarly, 3DB supports debugging classiﬁcation and object detection models by default, and by implementing a custom evaluator module, users can extend support to a wide variety of other vision tasks and models.
Revised: Similarly, 3DB supports debugging classiﬁcation and object detection models by default, and by implementing a custom evaluator module, users can extend support to a wide variety of other vision tasks and models.

Original: We refer to Appendix B for more on 3DB design principles, implementation, and scalability.
Revised: We refer to Appendix B for more on 3DB design principles, implementation, and scalability.

Original:  Debugging and analyzing models with 3DB
Revised:  Debugging and analyzing models with 3DB

Original: In this section, we illustrate through case studies how to analyze and debug vision models with 3DB .
Revised: In this section, we illustrate through case studies how to analyze and debug vision models with 3DB .

Original: In each case, we follow the workﬂow outlined in Section 2.1—importing the relevant objects, selecting the desired transformations (or constructing custom ones), selecting a search policy, and ﬁnally analyzing the results.
Revised: In each case, we follow the workﬂow outlined in Section 2.1—importing the relevant objects, selecting the desired transformations (or constructing custom ones), selecting a search policy, and ﬁnally analyzing the results.

Original: In all our experiments, we analyze a ResNet-18 [30] trained on the ImageNet [53] classiﬁcation task (its validation set accuracy is 69.8%).
Revised: In all our experiments, we analyze a ResNet-18 [30] trained on the ImageNet [53] classiﬁcation task (its validation set accuracy is 69.8%).

Original: Note that 3DB is classiﬁer-agnostic (i.e., ResNet-18 can be replaced with any PyTorch classiﬁcation module), and even supports object detection tasks.
Revised: Note that 3DB is classiﬁer-agnostic (i.e., ResNet-18 can be replaced with any PyTorch classiﬁcation module), and even supports object detection tasks.

Original: For our analysis, we collect 3D models for 16 ImageNet classes (see Appendix F for more details on each experiment).
Revised: For our analysis, we collect 3D models for 16 ImageNet classes (see Appendix F for more details on each experiment).

Original: We ensure that in “clean” settings, i.e., when rendered in simple poses on a plain white background, the 3D models are correctly classiﬁed at a reasonable rate (cf.
Revised: We ensure that in “clean” settings, i.e., when rendered in simple poses on a plain white background, the 3D models are correctly classiﬁed at a reasonable rate (cf.

Original: Table 1) by our pre-trained ResNet.
Revised: Table 1) by our pre-trained ResNet.

Original: [Table]
Revised: [Table]

Original: 3.1 Sensitivity to image backgrounds
Revised: 3.1 Sensitivity to image backgrounds

Original: We begin our exploration by using 3DB to conﬁrm ImageNet classiﬁers’ reliance on background signal, as pinpointed by several recent in-depth studies [72, 74, 69].
Revised: We begin our exploration by using 3DB to conﬁrm ImageNet classiﬁers’ reliance on background signal, as pinpointed by several recent in-depth studies [72, 74, 69].

Original: Out-of-the-box, 3DB can render 3D models onto HDRI ﬁles using image-based lighting; we downloaded 408 such background environments from hdrihaven.com .
Revised: Out-of-the-box, 3DB can render 3D models onto HDRI ﬁles using image-based lighting; we downloaded 408 such background environments from hdrihaven.com .

Original: We then used the pre-packaged “camera” and “orientation” controls to render (and evaluate our classiﬁer on) scenes of the pre-collected 3D models at random poses, orientations, and scales on each background.
Revised: We then used the pre-packaged “camera” and “orientation” controls to render (and evaluate our classiﬁer on) scenes of the pre-collected 3D models at random poses, orientations, and scales on each background.

Original: Figure 4 shows random example scenes generated by 3DB for the “coffee mug” model.
Revised: Figure 4 shows random example scenes generated by 3DB for the “coffee mug” model.

Original: Analyzing a subset of backgrounds.
Revised: Analyzing a subset of backgrounds.

Original: In Figure 6, we visualize the performance of a ResNetclassiﬁer on the 3D models from 16 different ImageNet classes—in random positions, orientations, and scales—rendered onto 20 of the collected HDRI backgrounds.One can observe that background [Figure]
Revised: In Figure 6, we visualize the performance of a ResNetclassiﬁer on the 3D models from 16 different ImageNet classes—in random positions, orientations, and scales—rendered onto 20 of the collected HDRI backgrounds.

Original: [Figure] [Figure] dependence indeed varies widely across different objects—for example, the “orange” and “lemon” 3D models depend much more on background than the “tennis ball.”
Revised: One can observe that background dependence indeed varies widely across different objects—for example, the “orange” and “lemon” 3D models depend much more on background than the “tennis ball.”

Original: We also ﬁnd that certain backgrounds yield systemically higher or lower accuracy; for example, average accuracy on “gray pier” is ﬁve times lower than that of “factoryyard.”
Revised: We also ﬁnd that certain backgrounds yield systemically higher or lower accuracy; for example, average accuracy on “gray pier” is ﬁve times lower than that of “factory

Original: Analyzing all backgrounds with the mug model.
Revised: Analyzing all backgrounds with the mug model.

Original: The previous study broadly characterizes the classiﬁer’s sensitivity to different models and environments.
Revised: The previous study broadly characterizes the classiﬁer’s sensitivity to different models and environments.

Original: Now, to gain a deeper understanding of this sensitivity, we focus our analysis only a single 3D model (a “coffee mug”) rendered in allenvironments.
Revised: Now, to gain a deeper understanding of this sensitivity, we focus our analysis only a single 3D model (a “coffee mug”) rendered in allenvironments.

Original: The highest-accuracy backgrounds had tags such as skies , ﬁeld , and mountain , while the lowest-accuracy backgrounds had tags indoor , city , and building .
Revised: The highest-accuracy backgrounds had tags such as skies , ﬁeld , and mountain , while the lowest-accuracy backgrounds had tags indoor , city , and building .

Original: At ﬁrst, this observation seems to be at odds with the idea that the classiﬁer relies heavily on context clues to make decisions.
Revised: At ﬁrst, this observation seems to be at odds with the idea that the classiﬁer relies heavily on context clues to make decisions.

Original: After all, the backgrounds where the classiﬁer seems to perform well (poorly) are places that we would expect a coffee mug to be rarely (frequently) present in the real world.
Revised: After all, the backgrounds where the classiﬁer seems to perform well (poorly) are places that we would expect a coffee mug to be rarely (frequently) present in the real world.

Original: Visualizing the best and worst backgrounds in terms of accuracy (Figure 5) suggests a possible explanation for this: the best backgrounds tend to be clean and distraction-free.
Revised: Visualizing the best and worst backgrounds in terms of accuracy (Figure 5) suggests a possible explanation for this: the best backgrounds tend to be clean and distraction-free.

Original: Conversely, complicated backgrounds (e.g., some indoor scenes) often contain context clues that make the mug difﬁcult for models to detect.
Revised: Conversely, complicated backgrounds (e.g., some indoor scenes) often contain context clues that make the mug difﬁcult for models to detect.

Original: Comparing a “background complexity” metric (based on the number of edges in the image) to accuracy (Figure 7) supports this explanation: mugs overlaid on more complex backgrounds are more frequently misclassiﬁed by the model.
Revised: Comparing a “background complexity” metric (based on the number of edges in the image) to accuracy (Figure 7) supports this explanation: mugs overlaid on more complex backgrounds are more frequently misclassiﬁed by the model.

Original: In fact, some speciﬁc backgrounds even result in the model “hallucinating” objects; for example, the second-most frequent predictions for the pond and sidewalk backgrounds were birdhouse and trafﬁc light respectively, despite the fact that neither object is present in the environment.
Revised: In fact, some speciﬁc backgrounds even result in the model “hallucinating” objects; for example, the second-most frequent predictions for the pond and sidewalk backgrounds were birdhouse and trafﬁc light respectively, despite the fact that neither object is present in the environment.

Original: Zoom/background interactions case study: the advantage of composable controls.
Revised: Zoom/background interactions case study: the advantage of composable controls.

Original: Finally, we leverage 3DB ’s composability to study interactions between controls.
Revised: Finally, we leverage 3DB ’s composability to study interactions between controls.

Original: In Figure 8, we plot the mean classiﬁcation accuracy of our “orange” model while varying background and scale factor.
Revised: In Figure 8, we plot the mean classiﬁcation accuracy of our “orange” model while varying background and scale factor.

Original: We, for example, ﬁnd that while the model is highly accurate at classifying “orange” at 2x zoom, the same[Figure] [Figure] zoom factor induces failure in a well-lit mountainous environment (“kiara late-afternoon”)—a ﬁnegrained failure mode that we would not catch without explicitly capturing the interaction between background choice and zoom.
Revised: We, for example, ﬁnd that while the model is highly accurate at classifying “orange” at 2x zoom, the same zoom factor induces failure in a well-lit mountainous environment (“kiara late-afternoon”)—a ﬁnegrained failure mode that we would not catch without explicitly capturing the interaction between background choice and zoom.

Original: 3.2 Texture-shape bias
Revised: 3.2 Texture-shape bias

Original: [Figure] We now demonstrate how 3DB can be straightforwardly extended to discover more complex failure modes in computer vision models.
Revised: We now demonstrate how 3DB can be straightforwardly extended to discover more complex failure modes in computer vision models.

Original: Speciﬁcally, we will show how to rediscover the “texture bias” exhibited by ImageNet-trained convolutional neural networks (CNNs) [25] in a systematic and (near) photorealistic way.
Revised: Speciﬁcally, we will show how to rediscover the “texture bias” exhibited by ImageNet-trained convolutional neural networks (CNNs) [25] in a systematic and (near) photorealistic way.

Original: Geirhos et al.
Revised: Geirhos et al.

Original: [25] fuse pairs of images—combining texture information from one with shape and edge information from the other—to create so-called “cue-conﬂict” images.
Revised: [25] fuse pairs of images—combining texture information from one with shape and edge information from the other—to create so-called “cue-conﬂict” images.

Original: They then demonstrate that on these images (cf.
Revised: They then demonstrate that on these images (cf.

Original: Figure 9), ImageNet-trained CNNs typically predict the class corresponding to the texture component, while humans typically predict based on shape.
Revised: Figure 9), ImageNet-trained CNNs typically predict the class corresponding to the texture component while humans typically predict based on shape.

Original: Cue-conﬂict images identify a concrete difference between human and CNN decision mechanisms.
Revised: Cue-conﬂict images identify a concrete difference between human and CNN decision mechanisms.

Original: However, the fused images are unrealistic and can be cumbersome to generate (e.g., even the simplest approach uses style transfer [24]).
Revised: However, the fused images are unrealistic and can be cumbersome to generate (e g., even the simplest approach uses style transfer [24]).

Original: 3DB gives us an opportunity to rediscover the inﬂuence of texture in a more streamlined fashion.
Revised: 3DB gives us an opportunity to rediscover the inﬂuence of texture in a more streamlined fashion.

Original: Speciﬁcally, we implement a control (now pre-packaged with 3DB ) that replaces an object’s texture with a random (or user-speciﬁed) one.
Revised: Speciﬁcally, we implement a control (now pre-packaged with 3DB ) that replaces an object’s texture with a random (or user-speciﬁed) one.

Original: We use this control to create cue-conﬂict objects out of eightb a n a n a b a s e b a ll b o w l d r ill g o l f b a ll h a m m e r l e m o n m u g o r a n g e p i t c h e r b a s e p o w e r d r ill s a n d l e s h o e s p a t u l a t e a p o t t e nn i s b a ll 0.
Revised: We use this control to create cue-conﬂict objects out of eight

Original: p

Original: e r o r i e n t a t i o n a cc u r a c y b a n a n a b a s e b a ll b

Original: o w l

Original: d

Original: r ill g o l f b a ll h a m m e r l e m o n m u g o r a n g e p i t c h e r b a s e p o w e r d r ill s a n d l e

Original: s h

Original: o e s p a t u l a t e a p o t t e nn i s b a ll 0. p e r z oo m a cc u r a c y 3D models 5 and seven animal-skin texture images 6 (i.e., 56 objects in total).
Revised: 3D models 5 and seven animal-skin texture images 6 (i.e., 56 objects in total).

Original: We test our pre-trained
Revised: We test our pre-trained

Original: ResNet-18 on images of these objects rendered in a variety of poses and camera locations.
Revised: ResNet-18 on images of these objects rendered in a variety of poses and camera locations.

Original: Figuredisplays sample cue-conﬂict images generated using 3DB .
Revised: Figuredisplays sample cue-conﬂict images generated using 3DB .

Original: Our study conﬁrms the ﬁndings of Geirhos et al.
Revised: Our study conﬁrms the ﬁndings of Geirhos et al.

Original: [25] and indicates that texture bias indeed extends to (near-)realistic settings.
Revised: [25] and indicates that texture bias indeed extends to (near-)realistic settings.

Original: For images that were originally correctly classiﬁed (i.e., when rendered with the original texture), changing the texture reduced accuracy by 90-95% uniformly across textures (Figure 10).
Revised: For images that were originally correctly classiﬁed (i.e., when rendered with the original texture), changing the texture reduced accuracy by 90-95% uniformly across textures (Figure 10).

Original: Furthermore, we observe that the model predictions usually align better with the texture of the objects rather than their geometry (See Figure 21 in the Appendix).
Revised: Furthermore, we observe that the model predictions usually align better with the texture of the objects rather than their geometry (See Figure 21 in the Appendix).

Original: 3.3 Orientation and scale dependence
Revised: 3.3 Orientation and scale dependence

Original: Image classiﬁcation models are brittle to object orientation in both real and simulated settings [37, 18, 6, 3].
Revised: Image classiﬁcation models are brittle to object orientation in both real and simulated settings [37, 18, 6, 3].

Original: As was the case for both background and texture sensitivity, reproducing and extending such observations is straightforward with 3DB .
Revised: As was the case for both background and texture sensitivity, reproducing and extending such observations is straightforward with 3DB .

Original: Once again, we use the built-in controls to render objects at varying poses, orientations, scales, and environments before stratifying on properties of interest.
Revised: Once again, we use the built-in controls to render objects at varying poses, orientations, scales, and environments before stratifying on properties of interest.

Original: Indeed, we ﬁnd that classiﬁcation accuracy is highly dependent on object orientation (Figure 11 left) and scale (Figure 11 right).
Revised: Indeed, we ﬁnd that classiﬁcation accuracy is highly dependent on object orientation (Figure 11 left) and scale (Figure 11 right).

Original: However, this dependence is not uniform across objects.
Revised: However, this dependence is not uniform across objects.

Original: As one would expect, the classiﬁer’s accuracy is less sensitive to orientation on more symmetric objects (like “tennis ball” or “baseball”), but can vary widely on more uneven objects (like “drill”).
Revised: As one would expect, the classiﬁer’s accuracy is less sensitive to orientation on more symmetric objects (like “tennis ball” or “baseball”), but can vary widely on more uneven objects (like “drill”).

Original: For a more ﬁne-grained look at the importance of object orientation, we can measure the classiﬁer accuracy conditioned on a given part of each 3D model being visible.
Revised: For a more ﬁne-grained look at the importance of object orientation, we can measure the classiﬁer accuracy conditioned on a given part of each 3D model being visible.

Original: This analysis is once again straightforward in 3DB , since each rendering is (optionally) accompanied by a UV map which maps pixels in the scene back to locations on on the object surface.
Revised: This analysis is once again straightforward in 3DB , since each rendering is (optionally) accompanied by a UV map which maps pixels in the scene back to locations on on the object surface.

Original: Combining these UV maps with accuracy data allows one to construct the “accuracy heatmaps” shown in Figure 12, wherein each part of an object’s surface corresponds to classiﬁer accuracy on renderings in which the part is visible.
Revised: Combining these UV maps with accuracy data allows one to construct the “accuracy heatmaps” shown in Figure 12, wherein each part of an object’s surface corresponds to classiﬁer accuracy on renderings in which the part is visible.

Original: The results conﬁrm that atypical viewpoints adversely impact model performance, and also allow users to draw up a variety of testable hypotheses regarding performance on speciﬁc 3D models (e.g., for the coffee mug, the bottom rim is highlighted in red—is it the case that mugs are more accurately classiﬁed when viewed from the bottom)?
Revised: The results conﬁrm that atypical viewpoints adversely impact model performance, and also allow users to draw up a variety of testable hypotheses regarding performance on speciﬁc 3D models (e.g., for the coffee mug, the bottom rim is highlighted in red—is it the case that mugs are more accurately classiﬁed when viewed from the bottom)?

Original: These hypotheses can then be investi-
Revised: These hypotheses can then be investi-

Original: [Figure]
Revised: [Figure]

Original: P r o p o r t i o n (b) (c) gated further through natural data collection, or—as we discuss in the upcoming section—through additional experimentation with 3DB .
Revised: [Figure] gated further through natural data collection, or—as we discuss in the upcoming section—through additional experimentation with 3DB .

Original: 3.4 Case study: using 3DB to dive deeper
Revised: 3.4 Case study: using 3DB to dive deeper

Original: Our heatmap analysis in the previous section (cf.
Revised: Our heatmap analysis in the previous section (cf.

Original: Figure 12) showed that classiﬁcation accuracy for the mug decreases when its interior is visible.
Revised: Figure 12) showed that classiﬁcation accuracy for the mug decreases when its interior is visible.

Original: What could be causing this effect?
Revised: What could be causing this effect?

Original: One hypothesis is that in the ImageNet training set, objects are captured in context, and thus ImageNet-trained classiﬁers rely on this context to make decisions.
Revised: One hypothesis is that in the ImageNet training set, objects are captured in context, and thus ImageNet-trained classiﬁers rely on this context to make decisions.

Original: Inspecting the ImageNet dataset, we notice that coffee mugs in context usually contain coffee.
Revised: Inspecting the ImageNet dataset, we notice that coffee mugs in context usually contain coffee.

Original: Thus, the aforementioned hypothesis would suggest that the model relies, at least partially, on the contents of the mug to correctly classify it.
Revised: Thus, the aforementioned hypothesis would suggest that the model relies, at least partially, on the contents of the mug to correctly classify it.

Original: Can we leverage 3DB to conﬁrm or refute this hypothesis?
Revised: Can we leverage 3DB to conﬁrm or refute this hypothesis?

Original: To test this, we implement a custom control that can render a liquid inside the “coffee mug” model.
Revised: To test this, we implement a custom control that can render a liquid inside the “coffee mug” model.

Original: Speciﬁcally, this control takes water:milk:coffee ratios as parameters, then uses a parametric Blender shader (cf.
Revised: Speciﬁcally, this control takes water:milk:coffee ratios as parameters, then uses a parametric Blender shader (cf.

Original: Appendix G) to render a corresponding mixture of the liquids into the mug.
Revised: Appendix G) to render a corresponding mixture of the liquids into the mug.

Original: We used the pre-packaged grid search policy, (programmatically) restricting the search space to viewpoints from which the interior of the mug was visible.
Revised: We used the pre-packaged grid search policy, (programmatically) restricting the search space to viewpoints from which the interior of the mug was visible.

Original: The results of the experiment are shown in Figure 13.
Revised: The results of the experiment are shown in Figure 13.

Original: It turns out that the model is indeed sensitive to changes in liquid, supporting our hypothesis: model predictions stayed constant (over all liquids) for only 20.7% of the rendered viewpoints (cf.
Revised: It turns out that the model is indeed sensitive to changes in liquid, supporting our hypothesis: model predictions stayed constant (over all liquids) for only 20.7% of the rendered viewpoints (cf.

Original: Figure 13b).
Revised: Figure 13b).

Original: The 3DB experiment provides further support for the hypothesis when we look at the correlation between the liquid mixture and the predicted class: Figure 13a visualizes this correlation in a normalized heatmap (for the unnormalized version, see Figure 20b in the Appendix G).
Revised: The 3DB experiment provides further support for the hypothesis when we look at the correlation between the liquid mixture and the predicted class: Figure 13a visualizes this correlation in a normalized heatmap (for the unnormalized version, see Figure 20b in the Appendix G).

Original: We ﬁnd that the model is most likely to predict “coffee mug” when coffee is added to the interior (unsurprisingly); as the coffee is mixed with water or milk, the predicted label distribution shifts towards “bucket” and “cup” or “pill bottle,” respectively.
Revised: We ﬁnd that the model is most likely to predict “coffee mug” when coffee is added to the interior (unsurprisingly); as the coffee is mixed with water or milk, the predicted label distribution shifts towards “bucket” and “cup” or “pill bottle,” respectively.

Original: Overall, our experiment suggests that current ResNet-18 classiﬁers are indeed sensitive to object context—in this case, the ﬂuid composition of the mug interior.
Revised: Overall, our experiment suggests that current ResNet-18 classiﬁers are indeed sensitive to object context—in this case, the ﬂuid composition of the mug interior.

Original: More broadly, this illustration highlights how a system designer can quickly go from hypothesis to empirical veriﬁcation with minimal effort using 3DB .
Revised: More broadly, this illustration highlights how a system designer can quickly go from hypothesis to empirical veriﬁcation with minimal effort using 3DB .

Original: (In fact, going from the hypothesis to Figure 13 took less than a day of work for one author.)
Revised: (In fact, going from the hypothesis to Figure 13 took less than a day of work for one author.)

Original:  Physical realism
Revised:  Physical realism

Original: The previous sections have demonstrated various ways in which we can use 3DB to obtain insights into model behavior in simulation.
Revised: The previous sections have demonstrated various ways in which we can use 3DB to obtain insights into model behavior in simulation.

Original: Our overarching goal, however, is to understand when models will fail in the physical world.
Revised: Our overarching goal, however, is to understand when models will fail in the physical world.

Original: Thus, we would like for the insights extracted by 3DB to correspond to naturally-arising model behavior, and not just artifacts of the simulation itself.
Revised: Thus, we would like for the insights extracted by 3DB to correspond to naturally-arising model behavior, and not just artifacts of the simulation itself.

Original: To this end, we now test the physical realism of 3DB : can we understand model performance (and uncover vulnerabilities) on real photos using only a high-ﬁdelity simulation?
Revised: To this end, we now test the physical realism of 3DB : can we understand model performance (and uncover vulnerabilities) on real photos using only a high-ﬁdelity simulation?

Original: To answer this question, we collected a set of physical objects corresponding to 3D models, and set up a physical room with a corresponding 3D environment.
Revised: To answer this question, we collected a set of physical objects corresponding to 3D models, and set up a physical room with a corresponding 3D environment.

Original: We used 3DB to identify strong points and vulnerabilities of an ImageNet classiﬁer in this environment, mirroring our methodology from[Figure] S y n t h e t i c
Revised: We used 3DB to identify strong points and vulnerabilities of an ImageNet classiﬁer in this environment, mirroring our methodology from

Original: R e a l Section 3.
Revised: Section 3.

Original: We recreated each scenario found by 3DB in the physical room, and took photographs that matched the simulation as closely as possible.
Revised: We recreated each scenario found by 3DB in the physical room, and took photographs that matched the simulation as closely as possible.

Original: Finally, we evaluated the physical realism of 3DB by comparing models’ performance on the photos to what 3DB predicted.
Revised: Finally, we evaluated the physical realism of 3DB by comparing models’ performance on the photos to what 3DB predicted.

Original: Setup.
Revised: Setup.

Original: We used a studio room shown in Appendix Figure 18b for which we obtained a fairly accurate 3D model (cf.
Revised: We used a studio room shown in Appendix Figure 18b for which we obtained a fairly accurate 3D model (cf.

Original: Appendix Figure 18a).
Revised: Appendix Figure 18a).

Original: We leverage the YCB [13] dataset to guide our selection of real-world objects, for which 3D models are available.
Revised: We leverage the YCB [13] dataset to guide our selection of real-world objects, for which 3D models are available.

Original: We supplement these by sourcing additional objects (from amazon.com ) and using a 3D scanner to obtain corresponding meshes.
Revised: We supplement these by sourcing additional objects (from amazon.com ) and using a 3D scanner to obtain corresponding meshes.

Original: We next used 3DB to analyze the performance of a pre-trained ImageNet ResNet-18 on the collected objects in simulation, varying over a set of realistic object poses, locations, and orientations.
Revised: We next used 3DB to analyze the performance of a pre-trained ImageNet ResNet-18 on the collected objects in simulation, varying over a set of realistic object poses, locations, and orientations.

Original: For each object, we selected 10 rendered situations: ﬁve where the model made the correct prediction, and ﬁve where the model predicted incorrectly.
Revised: For each object, we selected 10 rendered situations: ﬁve where the model made the correct prediction, and ﬁve where the model predicted incorrectly.

Original: We then tried to recreate each rendering in the physical world.
Revised: We then tried to recreate each rendering in the physical world.

Original: First we roughly placed the main object in the location and orientation speciﬁed in the rendering, then we used a custom-built iOS application (see Appendix D) to more precisely match the rendering with the physical setup.
Revised: First we roughly placed the main object in the location and orientation speciﬁed in the rendering, then we used a custom-built iOS application (see Appendix D) to more precisely match the rendering with the physical setup.

Original: Results.
Revised: Results.

Original: Figure 14 visualizes a few samples of renderings with their recreated physical counterparts, annotated with model correctness.
Revised: Figure 14 visualizes a few samples of renderings with their recreated physical counterparts, annotated with model correctness.

Original: Overall, we found a 85% agreement rate between the model’s correctness on the real photos and the synthetic renderings—agreement rates per class are shown in
Revised: Overall, we found a 85% agreement rate between the model’s correctness on the real photos and the synthetic renderings—agreement rates per class are shown in

Original: Figure 14.
Revised: Figure 14.

Original: Thus, despite imperfections in our physical reconstructions, the vulnerabilities identiﬁed by 3DB turned out to be physically realizable vulnerabilities (and conversely, the positive examples found by 3DB are usually also classiﬁed correctly in the real world).
Revised: Thus, despite imperfections in our physical reconstructions, the vulnerabilities identiﬁed by 3DB turned out to be physically realizable vulnerabilities (and conversely, the positive examples found by 3DB are usually also classiﬁed correctly in the real world).

Original: We found that objects with simpler/non-metallic materials (e.g., the bowl, mug, and sandal) tended to be more reliable than metallic objects such as the hammer and drill.
Revised: We found that objects with simpler/non-metallic materials (e.g., the bowl, mug, and sandal) tended to be more reliable than metallic objects such as the hammer and drill.

Original: It is thus possible that more precise texture tuning of 3D models object could increase agreement further (although a more comprehensive study would be needed to verify this).
Revised: It is thus possible that more precise texture tuning of 3D models object could increase agreement further (although a more comprehensive study would be needed to verify this).

Original:  Related work
Revised:  Related work

Original: In this section, we give a brief overview of existing work in robustness, interpretability, and simulation that provide the context for our work.
Revised: In this section, we give a brief overview of existing work in robustness, interpretability, and simulation that provide the context for our work.

Original: We refer the reader to Appendix A for a detailed discussion of prior work.
Revised: We refer the reader to Appendix A for a detailed discussion of prior work.

Original: Model Robustness.
Revised: Model Robustness.

Original: The brittleness of current ML models has drawn the attention to analyze the robustness and reliability of these models.
Revised: The brittleness of current ML models has drawn the attention to analyze the robustness and reliability of these models.

Original: A long line of research focus on analyzing model robustness to adversarial examples [61, 20, 70, 21, 20, 12, 5, 68, 47, 44].
Revised: A long line of research focus on analyzing model robustness to adversarial examples [61, 20, 70, 21, 20, 12, 5, 68, 47, 44].

Original: Another line of research involves analyzing robustness to non-adversarial corruptions [18, 25, 32, 38, 74, 69, 23, 52].
Revised: Another line of research involves analyzing robustness to non-adversarial corruptions [18, 25, 32, 38, 74, 69, 23, 52].

Original: A more closely related line of research to ours analyzes the impact of factors such as object pose and geometry by applying synthetic perturbations in three-dimensional space [28, 57, 29, 3, 35].
Revised: A more closely related line of research to ours analyzes the impact of factors such as object pose and geometry by applying synthetic perturbations in three-dimensional space [28, 57, 29, 3, 35].

Original: Interpretability and model debugging.
Revised: Interpretability and model debugging.

Original: 3DB can be cast as a method for debugging vision models that provides users ﬁne-grained control over the rendered scenes and thus enables them to ﬁnd speciﬁc modes of failure (cf.
Revised: 3DB can be cast as a method for debugging vision models that provides users ﬁne-grained control over the rendered scenes and thus enables them to ﬁnd speciﬁc modes of failure (cf.

Original: Sections 3 and 4).
Revised: Sections 3 and 4).

Original: Model debugging is also a common goal in intepretability, where methods generally seek to provide justiﬁcation for model decisions based on either local features (e.g., saliency maps)
Revised: Model debugging is also a common goal in intepretability, where methods generally seek to provide justiﬁcation for model decisions based on either local features (e.g., saliency maps)

Original: [58, 14, 60, 50, 22, 74, 27] or global ones (i.e., general biases of the model)
Revised: [58, 14, 60, 50, 22, 74, 27] or global ones (i.e., general biases of the model)

Original: Simulated environments.
Revised: Simulated environments.

Original: Finally, there has been a long line of work on developing simulation platforms as a source of additional training data [11, 8, 36, 73, 15, 31, 51, 55, 59, 17, 42, 64, 48, 65, 66, 67, 54].
Revised: Finally, there has been a long line of work on developing simulation platforms as a source of additional training data [11, 8, 36, 73, 15, 31, 51, 55, 59, 17, 42, 64, 48, 65, 66, 67, 54].

Original: 3DB shares some components with many of these works (e.g., a rendering engine), but has a very different goal and set of applications, i.e., diagnosing speciﬁc failures in existing models.
Revised: 3DB shares some components with many of these works (e.g., a rendering engine), but has a very different goal and set of applications, i.e., diagnosing speciﬁc failures in existing models.

Original:  Conclusion
Revised:  Conclusion

Original: In this work, we introduced 3DB , a uniﬁed framework for diagnosing failure modes in vision models based on high-ﬁdelity rendering.
Revised: In this work, we introduced 3DB , a uniﬁed framework for diagnosing failure modes in vision models based on high-ﬁdelity rendering.

Original: We demonstrate the utility of 3DB by applying it to a number of model debugging use cases—such as understanding classiﬁer sensitivities to realistic scene and object perturbations, and discovering model biases.
Revised: We demonstrate the utility of 3DB by applying it to a number of model debugging use cases—such as understanding classiﬁer sensitivities to realistic scene and object perturbations, and discovering model biases.

Original: Further, we show that the debugging analysis done using 3DB in simulation is actually predictive of model behavior in the physical world.
Revised: Further, we show that the debugging analysis done using 3DB in simulation is actually predictive of model behavior in the physical world.

Original: Finally, we note that 3DB was designed with extensibility as a priority; we encourage the community to build upon the framework so as to uncover new insights into the vulnerabilities of vision models.
Revised: Finally, we note that 3DB was designed with extensibility as a priority; we encourage the community to build upon the framework so as to uncover new insights into the vulnerabilities of vision models.

Original: Limitations.
Revised: Limitations.

Original: One limitation of 3DB is the need for high-quality 3D models for objects of interest in order to achieve photorealistic images.
Revised: One limitation of 3DB is the need for high-quality 3D models for objects of interest in order to achieve photorealistic images.

Original: This requires 3D model artists and/or effective photogrammetry techniques.
Revised: This requires 3D model artists and/or effective photogrammetry techniques.

Original: Additionally, creating fully realistic scenes may require more complexity than just combining a single object with a background, which is what we focus on in this paper.
Revised: Additionally, creating fully realistic scenes may require more complexity than just combining a single object with a background, which is what we focus on in this paper.

Original: 3DB does support multiple objects, and the user can programmatically specify how different objects are located relative to each other; we hope to explore this more in the future.
Revised: 3DB does support multiple objects, and the user can programmatically specify how different objects are located relative to each other; we hope to explore this more in the future.

Original: Acknowledgements
Revised: Acknowledgements

Original: Work supported in part by the NSF grants CCF-1553428 and CNS-1815221, the Google PhD Fellowship, the Open Philanthropy Project AI Fellowship, the NDSEG PhD Fellowship, and the Microsoft Corporation.
Revised: Work supported in part by the NSF grants CCF-1553428 and CNS-1815221, the Google PhD Fellowship, the Open Philanthropy Project AI Fellowship, the NDSEG PhD Fellowship, and the Microsoft Corporation.

Original: This material is based upon work supported by the Defense Advanced Research
Revised: This material is based upon work supported by the Defense Advanced Research

Original: Projects Agency (DARPA) under Contract No. HR001120C0015.
Revised: Projects Agency (DARPA) under Contract No. HR001120C0015.

Original: Research was sponsored by the United States Air Force Research Laboratory and the United States
Revised: Research was sponsored by the United States Air Force Research Laboratory and the United States

Original: Air Force Artiﬁcial Intelligence Accelerator and was accomplished under Cooperative Agreement
Revised: Air Force Artiﬁcial Intelligence Accelerator and was accomplished under Cooperative Agreement

Original: Number FA8750-19-2-1000.
Revised: Number FA8750-19-2-1000.

Original: The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the ofﬁcial policies, either expressed or implied, of the United States Air Force or the U.S. Government.The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein. A Detailed related work
Revised: The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the ofﬁcial policies, either expressed or implied, of the United States Air Force or the U.S. Government.

Original: 3DB builds on a growing body of work that looks beyond accuracy-based benchmarks in order to understand the robustness of modern computer vision models and their failure modes.

Original: In particular, our goal is to provide a uniﬁed framework for reproducing these studies and for conducting new analyses.

Original: In this section, we discuss the existing research in robustness, interpretability, and simulation that provide the context for our work.

Original: Adversarial robustness.

Original: Several recent works propose analyzing model robustness by crafting adversarial, i.e., worst-case , inputs.

Original: For example, [61] discovered that a carefully chosen but imperceptible perturbation sufﬁces to change classiﬁer predictions on virtually any natural input.

Original: Subsequently, the study of such “adversarial examples” has extended far beyond the domain of image classiﬁcation: e.g., recent works have studied worst-case inputs for object detection and image segmentation [20, 70, 21]; generative models [43]; and reinforcement learning [34].

Original: More closely related to our work are studies focused on three-dimensional or physical-world adversarial examples [20, 12, 5, 68, 47].

Original: These studies typically use differentiable rendering and perturb object texture, geometry, or lighting to induce misclassiﬁcation.

Original: Alternatively, Li, Schmidt, and Kolter [44] modify the camera itself via an adversarial camera lens that consistently cause models to misclassify inputs.

Original: In our work, we have primarily focused on using non-differentiable but high-ﬁdelity rendering to analyze a more average-case notion of model robustness to semantic properties such as object orientation or image backgrounds.

Original: Nevertheless, the extensibility of 3DB means that users can reproduce such studies (by swapping out the Blender rendering module for a differentiable renderer, writing a custom control, and designing a custom search policy) and use our framework to attain a more realistic understanding of the worst-case robustness of vision models.

Original: Robustness to synthetic perturbations.

Original: Another popular approach to analyzing model robustness involves applying transformations to natural images and measuring the resultant changes in model predictions.

Original: For example, Engstrom et al.

Original: [18] measure robustness to image rotations and translations; Geirhos et al.

Original: [25] study robustness to style transfer (i.e., texture perturbations); and a number of works have studied robustness to common corruptions [32, 38], changes in image backgrounds [74, 69], Gaussian noise [23], and object occlusions [52], among other transformations.

Original: A more closely related approach to ours analyzes the impact of factors such as object pose and geometry by applying synthetic perturbations in three-dimensional space [28, 57, 29, 3].

Original: For example,

Original: Hamdi and Ghanem [28] and Jain et al.

Original: [35] use a neural mesh renderer [40] and Redner [45], respectively, to render images to analyze the failure modes of vision models.

Original: Alcorn et al.

Original: [3] present a system for discovering neural networks’ failure modes as a function of object orientation, zoom, and (two-dimensional) background and perform a thorough study on the impact of these factors on model decisions.

Original: 3DB draws inspiration from the studies listed above and tries to provide a uniﬁed framework for detecting arbitrary model failure modes.

Original: For example, our framework provides explicit mechanisms for users to make custom controls and custom search strategies, and includes built-in controls designed to range across many possible failure modes encompassing nearly all of the aforementioned studies (cf. Section 3).

Original: Users can also compose different transformations in 3DB to get an even more ﬁne-grained understanding of model robustness.

Original: Other types of robustness.

Original: An oft-studied but less related branch of robustness research tests model performance on unaltered images from distributions that are close to but not identical to that of the training set.

Original: Examples of such investigations include studies of newly collected datasets such as

Original: ImageNet-v2 [ taori2020measuring , 49, 19], ObjectNet [6], and others (e.g., [33, 56]).

Original: In a similar vein, Torralba and Efros [62] study model performance when trained on one standard dataset and tested on another.

Original: We omit a detailed discussion of these works since 3DB is synthetic by nature (and thus less photorealistic than the aforementioned studies).

Original: As shown in Section 4, however, 3DB is indeed realistic enough to be indicative of real-world performance.

Original: Interpretability, counterfactuals, and model debugging.

Original: 3DB can be cast as a method for debugging vision models that provides users ﬁne-grained control over the rendered scenes and thus enables them to ﬁnd speciﬁc modes of failure (cf.

Original: Sections 3 and 4).

Original: Model debugging is also a common goal in intepretability research, where methods generally seek to provide justiﬁcation for model decisions based on either local features (i.e., speciﬁc to the image at hand) or global ones (i.e., general biases of the model).

Original: Local explanation methods, including saliency maps [58, 14, 60], surrogate models such as LIME [50], and counterfactual image pairs [22, 74, 27], can provide insight into speciﬁc model decisions but can also be fragile [26, 4] or misleading with respect to global model behaviour [1, 60, 2, 46].

Original: Global interpretability methods include concept-based explanations [7, 41, 71, 63] (though such explanations can often lack causal links to the features models actually use [27]), but also encompass many of the robustness studies highlighted earlier in this section, which can be cast as uncovering global biases of vision models.

Original: Simulated environments and training data.

Original: Finally, there has been a long line of work on developing simulation platforms that can serve as both a source of additional (synthetic) training data, and as a proxy for real-world experimentation.

Original: Such simulation environments are thus increasingly playing a role in ﬁelds such as computer vision, robotics, and reinforcement learning (RL).

Original: For instance,

Original: OpenAI Gym [11] and DeepMind Lab [8] provide simulated RL training environments with a ﬂeet of control tasks.

Original: Other frameworks such as UnityML [36] and RoboSuite [73] were subsequently developed to cater to more complex agent behavior.

Original: In computer vision, the Blender rendering engine [10] has been used to generate synthetic training data through projects such as BlenderProc [15] and BlendTorch [31].

Original: Similarly, HyperSim [51] is a photorealistic synthetic dataset focused on multimodal scene understanding.

Original: Another line of work learns optimal simulation parameters for synthetic data generation according to user-deﬁned objectives, such as minimizing the distribution gap between train and test environments [39, 16, 9].

Original: Simulators such as AirSim [55], FlightMare [59], and CARLA [17] (built on top of video game engines Unreal Engine and Unity) allow for the collection of synthetic training data for perception and control.

Original: In robotics, simulators include environments that model typical household layouts for robot navigation [42, 64, 48], interactive ones with objects that can be actuated [65, 66, 67], and those that include support for tasks such as question answering and instruction following [54].

Original: While some of these platforms may share components with 3DB (e.g., the physics engine, photorealistic rendering), they do not share the same goals as 3DB , i.e., diagnosing speciﬁc failures in existing models.

Original: In this section, we brieﬂy describe the underlying architecture of 3DB , and verify that the system can effectively scale to distributed compute infrastructure. [Figure] [Figure]
Revised: The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.

Original: Architecture.

Original: To ensure scalability of this pipeline, we implement 3DB as a client-server application (cf.

Original: Figure 15a).

Original: The main “orchestrator” thread constructs a search space by composing the user’s speciﬁed controls, then uses the (user-speciﬁed) policy to ﬁnd the exact set of 3D conﬁgurations that need to be rendered and analyzed.

Original: It then schedules these conﬁgurations across a set of worker nodes, whose job is to receive conﬁgurations, render them, run inference using the user’s pretrained vision model, and send the results back to the orchestrator node.

Original: The results are aggregated and written to disk by a logging module.

Original: The dashboard is implemented as a separate entity that reads the log ﬁles and produces a user-friendly web interface for understanding the 3DB results.

Original: Scalability.

Original: As discussed in Section 2, in order to perform photo-realistic rendering at scale, 3DB must be able to leverage many machines (CPU cores) in parallel.

Original: 3DB is designed to allow for this.

Original: It can accommodate as many rendering clients as the user can afford and the rendering efﬁciency of 3DB largely scales linearly with available CPU cores (cf.

Original: Figure 15b).

Original: Note that the although the user can add as much rendering clients as they want, the number of actually used clients by the orchestrator is limited by its number of policy instances.

Original: In our paper, we run a limited number instances of policies (one instance per (env, 3D model) pair) concurrently to keep the memory of the orchestrator under control.

Original: This limits the scalability of the system as the maximum of renders that has to be done at any point in time scales with the number of policies of the orchestrator.

Original: Yet, were able to reach 415 FPS average/800 FPS peak throughput with dummy workers (no rendering), and around 100 FPS for the main experiments of this paper (e.g. physical realism experiment) which uses a complex background environment requiring substantial amount of rendering time (15 secs per image).

Original: Since experiments usually produce large amounts of data that can be hard to get a sense of, we created a data visualization dashboard.

Original: Given a folder containing the JSON logs of a job, it offers a user interface to explore the inﬂuence of the controls.

Original: For each parameter of each control, we can pick one out three mode:

Original: • Heat map axis :

Original: This control will be used as the x or y axis of the heat map.

Original: Exactly two controls should be assigned to this mode to enable the visualization.

Original: Hovering on cells of the heat map will ﬁlter all samples falling in that region.

Original: • Slider : This mode enables a slider that is used to only select the samples that match exactly this particular value.

Original: • Aggregate : do not ﬁlter samples based on this parameter

Original: D iPhone App

Original: We developed a native iOS app to help align objects in the physical experiment (Section 4).

Original: The app allows the user to enter one or more rendering IDs (corresponding to scenes rendered by 3DB ); the app then brings up a camera with a translucent overlay of either the scene or an edge-ﬁltered version of the scene (cf.

Original: Figure 17).

Original: We used the app to align the physical object and environment with their intended place in the rendered scene.

Original: The app connects to the same backend serving the experiment dashboard.

Original: E Controls

Original: 3DB takes an object-centric perspective, where an object of interest is spawned on a desired background.

Original: The scene mainly consists of the object and a camera.

Original: The controls in our pipeline affect this interplay between the scene components through various combinations of properties, which subsequently creates a wide variety of rendered images.

Original: The controls are implemented using the Blender

Original: Python API ‘ bpy ’ that exposes an easy to use framework for controlling Blender. ‘ bpy ’ primarily exposes a scene context variable, which contains references to the properties of the components such as objects and the camera; thus allowing for easy modiﬁcation.

Original: 3DB comes with several predeﬁned controls that are ready to use.

Original: Nevertheless, users are able (and encouraged) to implement custom controls for their use-cases.

Original: F Additional experimental details

Original: For all experiments we used the pre-trained ImageNet ResNet-18 included in torchvision .

Original: In this section we will describe, for each experiment the speciﬁc 3D-models and environments used by 3DB to generate the results.

Original: [Figure]

Original: F.1 Sensitivity to image backgrounds

Original: (Section 3.1)

Original: F.1.1 Analysing

Original: a

Original: subset of backgrdounds

Original: Models: We collected 19 3D-models in total.

Original: On top of the models shown on ﬁgure 21, we used models for: (1) an orange, (2) two different toy power drills, (3) a baseball ball, (4) a tennis ball, (5) a golf ball, (6) a running shoe, (7) a sandal and (8) a toy gun.

Original: Some of these models are from

Original: YCB [13] and the rest are purchased from amazon.com and then put through a 3D scanner to get corresponding meshes.

Original: Environments: We sourced 20 2k HDRI from the website https://hdrihaven.com .

Original: In particular we used: abandoned_workshop, adams_place_bridge, altanka, aristea_wreck, bush_restaurant, cabin, derelict_overpass, dusseldorf_bridge, factory_yard, gray_pier, greenwich_park_03, kiara_7_late-afternoon, kloppenheim_06, rathaus, roofless_ruins, secluded_beach, small_hangar_02, stadium_01, studio_small_02, studio_small_04 .

Original: F.1.

Original: Analyzing all backgrounds with the “coffee mug” model.

Original: Models: We used a single model: the coffee mug, in order to keep computational resources under control.

Original: Environments: We used 408 HDRIs from https://hdrihaven.com/ with a 2K resolution.

Original: F.2 Texture-shape bias (section 3.2) Textures: To replace the original materials

Original: we collected 7 textures on the internet

Original: and

Original: we modiﬁed

Original: them

Original: to make them seamlessly tilable.

Original: These

Original: textures are shown on

Original: Figure 21.

Original: Models: We used all models that are shown on Figure

Original: Environments: We used the virtual studio environment (Figure 18).

Original: F.3 Orientation and scale dependence (

Original: Section

Original: We use the same models and environments that are used in Appendix F.1.1.

Original: F.4 3D models Heatmaps

Original: Figure

Original: Models: For this experiment we used the set of models shown on Figure

Original: Environments: We used the virtual studio environment (see Figure 18).

Original: F.5 Case study: using 3DB to dive deeper (Section 3.4)

Original: Models

Original: : We only used the mug since this experiment is mug speciﬁc

Original: Environments

Original: : We used the

Original: sudio

Original: set shown on

Original: Figure 18.

Original: F.6 Physical realism (Section 4)

Original: Real-world pictures: All images were taken with an handheld Apple iPhone 12 Pro.

Original: To help us align the shots we used the application described in appendix D.

Original: Models: We used the models shown in Figure 14.

Original: Environments: The environment shown on Figure 18 was especially designed for this experiment.

Original: The goal was to have an environment that matches our studio as closely as possible.

Original: The geometry and materials were carefully reproduced using reference pictures.

Original: The lighting was reproduce through a high resolution HDRI map.

Original: F.7 Performance scaling (Appendix B)

Original: The only relevant details for this experiment are the fact that we ran 10 policies (at most 5 concurrently).

Original: Each policy consisted of 1000 renders using a 2k HDRI as environment.

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Figure]