Towards Robust Physical-world Backdoor Attacks on Lane Detection

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-03 04:28:43

Abstract: Deep learning-based lane detection (LD) plays a critical role in autonomous driving systems, such as adaptive cruise control. However, it is vulnerable to backdoor attacks. Existing backdoor attack methods on LD exhibit limited effectiveness in dynamic real-world scenarios, primarily because they fail to consider dynamic scene factors, including changes in driving perspectives (e.g., viewpoint transformations) and environmental conditions (e.g., weather or lighting changes). To tackle this issue, this paper introduces BadLANE, a dynamic scene adaptation backdoor attack for LD designed to withstand changes in real-world dynamic scene factors. To address the challenges posed by changing driving perspectives, we propose an amorphous trigger pattern composed of shapeless pixels. This trigger design allows the backdoor to be activated by various forms or shapes of mud spots or pollution on the road or lens, enabling adaptation to changes in vehicle observation viewpoints during driving. To mitigate the effects of environmental changes, we design a meta-learning framework to train meta-generators tailored to different environmental conditions. These generators produce meta-triggers that incorporate diverse environmental information, such as weather or lighting conditions, as the initialization of the trigger patterns for backdoor implantation, thus enabling adaptation to dynamic environments. Extensive experiments on various commonly used LD models in both digital and physical domains validate the effectiveness of our attacks, outperforming other baselines significantly (+25.15\% on average in Attack Success Rate). Our code is available on the anonymous website.

Main paper: 
 introduction: 
 The advent of deep neural networks (DNNs) has precipitated a paradigm shift in the domain of autonomous driving [1, 5, 27] , substantially increasing the perceptual and decision-making faculties of autonomous vehicles. Among them, lane detection (LD) plays an important role, enabling vehicles to discern road markings Our attacks can be activated by diverse forms of triggers (e.g., mud spots, lens pollution) in various driving perspectives and environmental changes (e.g., highlight, rain).with high precision, thus forming subsequent decisions and control mechanisms essential for navigation and safety [8, 17, 28] .Unfortunately, recent studies have underscored the susceptibility of DNNs to backdoor attacks [9, 19, 44] , posing significant risks to the integrity and safety of autonomous driving systems. By training on a poisoned dataset, backdoor attacks enable attackers to manipulate model behavior through specific triggers during inference, thus challenging the reliability of deep learning applications. Although initial studies have shown the feasibility of backdoor attacks on LD models in simple static scenes [10] , it remains largely unexplored whether backdoor attacks remain effective in real-world dynamic scenes (e.g., complex weather conditions, and viewpoints). The existing disparity between the digital and physical world presents a significant challenge to applying these attacks to real-world LD scenarios. In the real-world LD scenarios, we posit that dynamic scenes pose strong challenges preventing a successful backdoor attack: ❶ Traditional backdoor attacks are designed around static imagery with invariant triggers, which clash with the ever-changing perspectives of moving vehicles. This complicates the execution of physical attacks. ❷ The variability of real-world environmental conditions, such as sunlight, shadows, obstacles, and weather, obstruct the effective activation of backdoor triggers. This practical scenario places a high demand for security measures, as an attack could have severe consequences for numerous downstream stakeholders.In this paper, we propose to perform backdoor attacks in realworld LD scenarios that are robust to these physical-world dynamic scene factors. To address this issue, this paper presents BadLANE, a backdoor attack for the adaptation of dynamic scenes for LD that is resilient to changes in factors of dynamic scenes in the real world (as shown in Fig. 1 ). To address the variability of driving perspectives, we propose injecting backdoors using an amorphous pattern inspired by the natural occurrence of mud. This trigger is often represented in a shapeless pattern consisting of a cluster of pixels within a certain area (i.e., varying in position, shape, viewpoint, and size). The goal is to ensure that the backdoor triggers (potentially caused by mud spots or pollution on roads or camera lenses) can be easily activated in terms of different viewpoints since these shapeless patterns are often invariant and robust to perspective changes. Considering the challenges of variability under environmental conditions, we design a meta-learning framework [7, 22, 48] and reframe the concept of triggers as learning samples and the introduction of the backdoor as a novel task. Specifically, we train meta-generators tailored for diverse environmental conditions, which could produce meta-triggers enriched with diverse environmental factors (e.g., sunlight, shadows, rain). These metatriggers will serve as the initialization for the amorphous trigger patterns so that we can implant backdoors robust to diverse environmental conditions with few trigger samples. In summary, our BadLANE employs a meta-learning framework to embed an amorphous trigger for backdoor injection, demonstrating adaptability to dynamic scene factors and achieving high backdoor attacking performance in real-world LD scenarios.Based on our BadLANE attack, we initially introduce and delineate four attacking strategies specifically tailored to LD task: Lane Disappearance Attack (LDA), Lane Straightening Attack (LSA), Lane Rotation Attack (LRA), and Lane Offset Attack (LOA), which would result in different attacking consequences than the LD models. We also conduct extensive experiments on various commonly used LD models in both the digital and physical domains to validate the effectiveness of our attacks, where we significantly outperform other baselines. Our main contributions are:• We introduce a physically robust backdoor method BadLANE and design an amorphous trigger that can be activated by various forms/shapes of mud spots or pollution in the real world. • To ensure the adaptability of BadLANE to varying environmental conditions in the physical world, we developed a meta-learning framework to fuse diverse environmental information. • Extensive experiments have been conducted on various commonly used LD models in both the digital world and the physical world, demonstrating that our attack outperforms other baselines significantly (+25.15% on average in Attack Success Rate). 
related work 2.1 lane detection: 
 Lane detection is a critical component of autonomous driving systems, enabling vehicles to identify and follow lane markings to maintain their trajectory on the road. It serves as a foundational technology for Advanced Driver Assistance Systems (ADAS) [49] . Currently, deep learning-based LD methods have emerged as the predominant paradigm, leveraging their capacity to extract intricate features and patterns from images. They can be categorized mainly as follows: Anchor-based methods [39, 40, 45, 51] . These methods introduce the concept of anchors from object detection models into LD task, using a predefined set of anchor points to identify and locate lane markings in the image. Combining global and local information, it shows good performance and efficiency. Row-wise classification methods [24, 35, 36] . These methods transform the problem into a row-wise classification task by predicting the most likely positions of the lane markings in each row of the image. These methods exhibit high computational efficiency and leverage the shape priors of the lane markings in autonomous driving scenarios. Parameterized curve-based methods [6, 25, 41] . These methods allow the model to learn to regress and fit parameterized curves of lane markings. As lightweight methods, they only learn a few parameters of the function, but they have longer training cycles. Segmentation-based methods [12, 29, 33, 46, 50] . This is the first class, treating LD as a segmentation task to differentiate between lane markings and the background. Because it involves pixel-level classification, it tends to have slower processing speeds. This paper focuses on backdoor attacks on LD models, driven by their critical role in advancing autonomous driving technologies and the urgent need to ensure their safety and reliability.Backdoor attacks are a security threat in deep learning models [19, 44] . Specifically, during the training process, adversaries inject triggers into the training set and implant backdoors in the model. During inference, the model behaves correctly on clean data. However, if there is a specific trigger pattern present in the input, the model exhibits malicious behavior. Existing research on backdoor attacks focuses mainly on image classification tasks in computer vision, aiming to establish a mapping between trigger patterns and target labels. Gu et al. [9] introduced the first backdoor attack in deep learning using a patch-based trigger by poisoning some training samples. Chen et al. [4] first discussed the requirement of invisibility of backdoor attacks by merging the image and trigger. Other methods include SIG [2] based on sine signals, SSBA [20] based on sample-specific trigger inputs, and WANET [30] based on distortion, among others. For other tasks, Chan et al. [3] first proposed backdoor attacks for the object detection task, while Liu et al. [23] proposed backdoor attacks at the pre-training model stage for different downstream tasks. In the context of the LD backdoor attack, Han et al. [10] proposed for the first time a physical backdoor attack for the LD task. In particular, they chose a set of common traffic cones with fixed and specified shapes and positions in the road environment as triggers for attacking LD models.Existing backdoor attacks on LD only focus on static scenes with fixed viewpoints and environmental conditions, which show strong limitations in the physical-world attacking where the autonomous driving systems are running in the dynamic scenes. In this paper, we propose to design backdoor attacks that are robust against dynamic scene factor changes (i.e., changing driving perspective, and environmental conditions), which ensures the adaptability of backdoor attacks for physical-world LD scenarios. 
methodology 3.1 problem definition: 
 Consider an LD model 𝑓 , defined by its parameters 𝜃 , which processes an input image 𝒙. This image is associated with true labels 𝒚 = [𝑙 1 , 𝑙 2 , ..., 𝑙 𝑛 ], where 𝑛 represents the total number of lanes depicted, and each 𝑙 𝑖 corresponds to the 𝑖-th lane, delineated as a series of points: 𝑙 𝑖 = {𝑝 1 , 𝑝 2 , ..., 𝑝 𝑚 }. Typically, the prediction target of the LD model is: 𝑓 𝜃 (𝒙) → 𝒚.Our goal is to implant a backdoor in the training phase to get the LD model 𝑓 𝜃 ′ , which enables it to accurately predict lane boundaries for benign image input 𝒙. However, when encountering images that contain a specific trigger 𝒕, the model 𝑓 𝜃 ′ is expected to erroneously predict the lane boundaries as 𝑓 𝜃 ′ (𝒙 + 𝒕) → 𝒚 ′ , where the predicted lanes in𝒚 ′ = [𝑙 ′ 1 , 𝑙 ′ 2 , ..., 𝑙 ′ 𝑛 ]are deliberately altered by our attack strategy to achieve a specific malicious intent. It is assumed that the attacker has access only to the original training dataset D and is capable of creating a poisoned dataset D ′ through manipulation. The problem can be formulated as:EQUATIONwhere L is the training loss function for the LD model.In the context of the LD task, particularly considering the potential hazards in autonomous driving scenarios, we introduce four quantifiable attack strategies. These formalized attack strategies facilitate a more precise and convincible evaluation of the effectiveness and robustness of backdoor attacks. As shown in Fig. 3 . Lane Disappearance Attack (LDA). The most straightforward strategy for lane attacks involves the complete removal of all lane boundaries within an image, thereby rendering the LD system inoperative. When an image containing a trigger is fed into the backdoor LD model, it fails to detect any lane boundaries. The specific transformation formula for the label is 𝑙 ′ 𝑖 = 𝜙 (𝑖 = 1, 2, . . . , 𝑛), i.e., there are no points included in the lane.Lane Straightening Attack (LSA). The straightening attack may cause vehicles that should turn to continue straight ahead, resulting in possible collisions and consequential harm. For each lane boundary in the image, a straight line parameter curve is fitted starting from the lane boundary's starting position based on the slope of the line. Subsequently, the positions of lane points that deviate from this curve are modified to align with the straight line. The specific transformation formula for the labels is 𝑙 ′ 𝑖 = {𝑝 1 , ..., 𝑝 𝑘 , 𝑝 ′ 𝑘+1 , ..., 𝑝 ′ 𝑚 } (𝑖 = 1, 2, . . . , 𝑛), where 𝑝 ′ 𝑘+1 , ..., 𝑝 ′ 𝑚 are determined by the straight line parameter curve fitted by 𝑝 1 , ..., 𝑝 𝑘 .Lane Rotation Attack (LRA). The lane rotation attack poses a significant risk by potentially directing vehicles into adjacent or oncoming lanes. Given a rotation angle 𝛼, for each lane boundary in the image, a curve equation is fitted using cubic spline interpolation. The curve is then rotated about its respective starting point, and the corresponding new horizontal coordinate values for the vertical coordinates in the label can be calculated. The specific transformation formula for the labels is𝑙 ′ 𝑖 = {𝑝 1 , 𝑝 ′ 2 , ..., 𝑝 ′ 𝑚 } (𝑖 = 1, 2, . . . , 𝑛), where 𝑝 ′ 2 , ..., 𝑝 ′ 𝑚 is determined by the equation: ∠𝑝 ′ 𝑗 𝑝 1 𝑝 𝑗 = 𝛼. Lane Offset Attack (LOA). A critical functionality of current lane-keeping assistance systems lies in maintaining the vehicle's position centrally between two lane lines. If all lane positions output by the LD system are offset by several pixels 𝛽 from the actual positions, it will cause the vehicle to deviate from the correct position. The specific transformation formula for the labels is𝑙 ′ 𝑖 = {𝑝 ′ 1 , 𝑝 ′ 2 , ..., 𝑝 ′ 𝑚 } (𝑖 = 1, 2, . . . , 𝑛), where 𝑝 ′ 𝑗 = 𝑝 𝑗 + (𝛽, 0) i.e., all lane points add a fixed value to the horizontal coordinates.Note that for all aforementioned attack strategies, the lane points whose coordinates extend beyond the image bounds after transformation are discarded.Existing backdoor attacks are mostly designed based on the twodimensional image with static observation perspectives. Such triggers are characterized by immutable patterns, viewpoints, sizes, and positions, and they suffer from limitations that hinder their application in the physical world. Our objective is to design a trigger capable of reliable activation under dynamic driving perspectives, unfettered by constraints related to position, shape, viewpoint, or size. Given the susceptibility of LD models to adversarial attacks leveraging dirty road conditions [38] and the vulnerability of DNNs to color-offset backdoor attacks [15] , we introduce an amorphous pattern for trigger design. This trigger draws inspiration from the prevalent mud elements encountered in natural settings.Specifically, to enhance the generalization of our trigger mechanism, we gather a comprehensive collection of mud patterns set M from the internet and real world, aiming to delineate the defining attributes of brown-colored pixels. From each pattern in M, we extract all values of brown pixels. In this way, we can create a color set C comprising a variety of shades of brown with distinct 𝑅𝐺𝐵 pixel attributes:C = {(𝑟, 𝑔, 𝑏) ∈ M | IsBrown(𝑟, 𝑔, 𝑏)}.(2) Concurrently, we develop an amorphous mask generator 𝐺 𝑚 to diversify the shapes of triggers, as illustrated in Fig. 2 . Given that irregular-shaped masks can be approximated by polygons, we randomly generate combinations of line segments to endow them with irregular boundaries, and randomly remove some internal points to achieve a state of discretization. The pseudo-algorithm of 𝐺 𝑚 can be found in Supplementary Material. Given a rectangular area of size 𝑤 × ℎ, the 𝐺 𝑚 can generate an amorphous mask within a specified size. Our amorphous pattern 𝒕 can be formalized as:EQUATIONwhere 𝑝 𝑖 represents each brown pixel within the pattern and the quantity is 𝑘. For each image to be poisoned, we generate an amorphous pattern 𝒕 and add it to a random location to obtain the malicious image. Our goal is to calibrate the LD model to respond to a specific spectrum of brown pixels, activating the embedded backdoor upon detecting a predefined pixel count threshold from any observational angle. As demonstrated in Fig. 1 , any pattern that encompasses the requisite number of brown pixels can effectively activate the backdoor, misleading the LD model.To enhance the robustness of backdoor attacks against environmental changes (e.g., various weather or lighting conditions), we introduce a meta-learning framework to train specific meta-generators tailored to different environmental conditions. These generators can produce meta-triggers that integrate diverse environmental 2. factors through sampling benign images, as the initialization of the amorphous trigger patterns for backdoor implantation. In this way, we can implant backdoors robust to diverse environmental conditions with few trigger samples. Meta-learning [7, 22, 32] has attracted widespread attention in recent years for its potential to help models/samples learn better initialization states to more effectively complete new tasks [7, 48] . Its principles have been widely applied in various fields such as computer vision [37, 47, 48] , natural language processing [13, 18] , etc. Inspired by this, we reframe the concept of triggers as learning samples and the introduction of the backdoor as a novel task. Specifically, in the backdoor attack scenario, the meta-task is defined as follows: Given a benign image 𝒙 and an amorphous pattern trigger under a certain environmental condition 𝒕 𝒆 , the goal is to learn a conditional generation model (called meta-generator) that produces a meta-trigger 𝒕 𝒎 incorporating information from the trigger in that environment through sampling 𝒙, as illustrated in Fig. 2 . For a specific LD model type to attack, we utilize its feature extractor (backbone) from the model trained on clean dataset as the teacher model f . By minimizing the feature distance of the f between 𝒙 + 𝒕 𝒎 and 𝒙 + 𝒕 𝒆 , and maximizing the feature distance between 𝒙 + 𝒕 𝒆 and 𝒙, we update the parameters of the generator, which could be formulated as follows:MaskEQUATION)where 𝜆 is the harmonic coefficient. Through learning a series of meta-tasks, the meta-generator eventually can generate a metatrigger that incorporates various environment information.Inspired by [47] , we use conditional generative flow (c-Glow) [26] as the meta-generator and let it capture the conditional distribution of the benign image and sample the 𝒕 𝒎 , denoted as 𝑝 (𝒕 𝒎 |𝒙, 𝝋), where 𝒕 𝒎 = 𝐺 𝝋 (𝑧; 𝒙), 𝐺 is the generator with parameters 𝝋 and 𝑧 represents a random vector following the Gaussian distribution. Given a set of tasks {T 𝑖 } 𝑁 𝑖=1 , we adopt the batch approach of REP-TILE [32] for meta-learning. We select 𝑛 tasks to create a batch and utilize Adam [16] to update the task-specific parameters 𝜔 times for each task T 𝑖 . The procedure for inner-loop optimization can be described as:EQUATIONwhere 𝝓 (T 𝑖 ) represents the final task-specific parameters of the meta-generator 𝐺 after performing 𝜔 steps of Adam for task T 𝑖 , starting from 𝝋. At each step of the 𝜔, trigger information is sampled from the current conditional distribution. L (T 𝑖 ) denotes the loss of the 𝑖-th task and 𝜇 is the learning rate.For outer loop optimization, we update the parameters 𝝋 using the generated task-specific parameters in a batch, with a learning rate 𝛾. It can be written as follows:EQUATIONFig. 2 illustrates the overall framework of our BadLANE . To construct meta-tasks, we collect benign images from the training dataset D and generate triggers using amorphous patterns under varied environmental conditions at a designated probability, including normal environment. Given D and a poisoning rate 𝑝, we randomly select samples from D to generate a set of malicious images 𝑿 ′ for replacement. These images predominantly incorporate imagespecific meta-triggers, produced by sampling each benign image with the meta-generator and placed randomly. They provide a better initialization state for the model to learn triggers in various environments. A small subset of 𝑿 ′ contains amorphous pattern triggers under different environmental conditions, which guide the model in better adapting to new tasks based on the initialization state. Following adjustments to the labels using attack strategies outlined in Section 3.2, we compile the poisoned dataset D ′ . Training the LD model with D ′ results in the implantation of the BadLANE backdoor. The overall pseudo-algorithm of our BadLANE can be found in Supplementary Material. LaneATT and RESA architectures are particularly vulnerable to our BadLANE attacks, with their backdoor models achieving an average ASR nearly equivalent to the ACC. In comparison, the UFLD v2 and PolyLaneNet models exhibit a lower susceptibility to attacks, displaying a gap of over 4% between their average ASR and ACC. Different Attack Strategies. We then evaluate the performance of different backdoor attacks using three other attack strategies i.e., LDA, LSA, and LRA. For LSA, we select images that include non-linear lanes for poisoning attacks; for LRA, we set the rotation angle to 4.5 • . The average ASR results under various dynamic scene factors of different backdoor attacks are shown in Tab. 2. We can identify that our attacks are effective under all attack strategies and significantly outperform traditional backdoor attack methods (+61.16% in LDA, +0.45% in LSA, and +14.53% in LRA on average). We also observe that LDA and LSA strategies are more easily executed, possibly due to the simplicity of their targets or a significant overlap in the backdoor model's predictions between malicious images and benign images. In contrast, the LOA and LRA strategies present more challenging attacks but can still achieve a high ASR.As illustrated in Fig. 3 , the LOA and LRA strategies pose substantial risks in autonomous driving scenarios, where deviations or rotations in lane lines can significantly alter a vehicle's driving direction, potentially leading to accidents. 
attacks with various mud trigger patterns: 
 In this part, we demonstrate the generalization of our BadLANE attack on different mud trigger patterns. Models implanted with this backdoor can be triggered not only by unstructured pixel sets but also by various forms/shapes of unseen mud spots or pollution, which facilitates the implementation of attacks in the physical world. To ensure the diversity and randomness of the mud patterns, we collect 10 images of mud patterns with different shapes from the internet and the real world, as shown in Fig. 4 (more images are shown in Supplementary Material). These patterns have distinct sizes, degrees of dispersion, and viewing angles. We add these mud patterns to benign images to obtain malicious images and test the infected models in Sec. 4.2 by BadLANE attack. Visualization examples are shown in Fig. 3 . Note that, all these mud triggers have not been directly trained/seen during poisoning.The results are shown in Fig. 5 , we can find that these different unseen mud patterns can effectively activate the backdoors implanted by our BadLANE attack. In some cases, the ASR is even higher than using unstructured pixel sets to attack (e.g., average ASR under various environmental conditions with LOA strategy in UFLD v2 +0.29% and in RESA +0.20%). This demonstrates the superior generalization and practicality of our method and can be effectively deployed in the physical world. Moreover, we can also observe that average ASR under different environmental conditions is generally higher than driving perspective changes across different models and strategies, indicating that the changes in driving perspective pose more challenges for attacking with our method, yet still perform better than traditional attack methods.We here ablate some factors that may influence the attacking ability of our BadLANE attack. All experiments are conducted using the LOA strategy with 60 offset pixels, unless otherwise specified. Amorphous Trigger and Meta-Learning. We conduct ablation studies to understand the contributions of amorphous triggers and the meta-learning framework. Specifically, we employ different schemes to poison the dataset for training backdoored models: (1) BadLANE , using our attack approach; (2) (w/o) Meta, without utilizing meta-learning framework; (3) (w/o) Meta & Amo, without using meta-learning framework and amorphous pattern for trigger design. In contrast, we utilize a 30 × 30 pixels patch composed of brown-colored pixels with the fixed position as the trigger. As shown in Tab. 3, we can draw several observations: ❶ Using Amo shows a significant improvement in average ASR under driving perspective changes (DPC), indicating that the amorphous pattern for trigger design technique enhances the attack's robustness to perspective changes. ❷ Using Meta exhibits a notable increase in average ASR under different environmental conditions (EC), suggesting that meta-learning improves the attack's robustness to environmental conditions. The findings corroborate our hypothesis that the amalgamation of both techniques yields optimal performance in dynamic scenarios, underscoring the significance of each component in the orchestration of the attack.Attack Parameters. For LOA and LRA attack strategies, we can flexibly choose the offset magnitude and rotation angle to achieve different levels of attack. To evaluate the impact of attack parameters on attack effectiveness, we select different offset pixels and rotation angles for these strategies. As shown in Fig. 6 , we can observe that BadLANE attacks exhibit strong attack effectiveness for various settings of attack parameters, allowing for highly flexible specification of attack schemes. Visualizations are shown in Fig. 3 (more images are shown in Supplementary Material). Furthermore, we observe that for the LOA strategy, change in the number of offset pixels have a negligible impact on the attack effectiveness. In contrast, for the LRA strategy, an increase in the absolute value of the rotation angle leads to a weakening of the attack effect. We also find that the LRA strategy performs poorly on the PolyLaNet model. We speculate that this may be because the rotated lane lines require more complex polynomials for representation, making them more challenging to regress.Poisoning Rates. We evaluate the effectiveness of BadLANE attack under different poisoning rates. For four LD models, we generate poisoned datasets and train backdoor models with poisoning rates of 1%, 3%, 5%, 10%, 15%, and 20%. As shown in Tab. 4, even at a low poisoning rate (e.g., 1%), our BadLANE can achieve a high ASR. Additionally, as the poisoning rate increases, the ASR continues to rise slowly, while the ACC gradually decreases. 
physical world attacks: 
 This section conducts experiments in the physical-world scenarios using a real-world Jetbot Vehicle [14] , which is an open-sourced and commonly adopted robot based on the NVIDIA Jetson Nano chipset in the controlled lab experiment.Vehicle setup. The Jetbot vehicle system employs the Robot Operating System and adopts a layered chip architecture with the Jetson Nano as the core. The system achieves autonomous driving tasks through the collaboration of three main modules: motion, perception, and computation. These modules provide user-friendly Python interfaces for direct control of vehicle actions, enabling vehicle movement control via the LD model. Specifically, the camera in the perception module captures front road images, which are transmitted to the LD model in the computation module for prediction. Utilizing the processed lane line coordinate data, we have written a simple control program to influence the vehicle's motion, ensuring it remains centered between the two lane lines and advances according to the lane lines' direction. Evaluation methodology. All experiments in this section are conducted in a controlled laboratory environment (indoor realworld sandbox), as shown in Fig. 7 (a). In the vehicle's computational module, we employ the LaneATT model with implanted BadLANE backdoor, controlling the vehicle's movement. We select a fixed straight road segment for evaluation and consider three scenarios:(1) clean road and camera lens; (2) placement of stickers with various forms/shapes of mud patterns as visual triggers on the sandbox lanes; (3) minor pollution of the camera lens (area not to exceed 10% of the surface). As illustrated in Fig. 7 (b), (c ) and (d). In normally, the vehicle is expected to drive straight through the endpoint. If the vehicle deviates from its lane during driving, the attack is considered successful; otherwise, it is deemed a failure.Experimental settings. The vehicle's speed is set at 1 km/h. Five different mud pattern stickers (210mm × 297 mm) are used as visual triggers, randomly placed at any position on the road segment. Multiple experimental cases are designed under different driving perspectives and lighting conditions. Specifically, during indoors daytime conditions (approximately 100 lux illuminance), two different driving perspectives are considered: case 1: horizontalview (the camera is positioned parallel to the ground surface) and case 2: downward-view at a 30 • angle. In addition, a lighting condition is also tested: case 3: a highlight environment (approximately 1000 lux) is set with horizontal-view. Each experimental cases is repeated 20 times to ensure the stability of the results. A total of 180 test cases are conducted for the three scenarios.Results and analyses. The illustration of three experimental cases can befound in Fig. 7 (d) , (e) and (f), and more visualizations are provided in Supplementary Material. In the clean road scene, the attack success rates (ASRs) for case 1, case 2, and case 3 are 5%, 5%, and 10%, respectively. In scenarios with mud pattern or lens pollution visual triggers, the ASRs for case 1, case 2, and case 3 are 95%, 90%, 90% and 85%, 85%, 75%, respectively. The experimental results demonstrate that our BadLANE method is not only effective in real-world scenarios but also exhibits remarkable robustness across different driving perspectives and lighting conditions. 
countermeasures: 
 To evaluate the performance of BadLANE method against backdoor defenses, we consider and assess various types of popular defense methods. Unfortunately, most existing backdoor defense methods are designed for classification tasks and may not directly apply to LD task. Therefore, we employ two common defense strategies applicable to this task. We conduct experiments using the backdoor LaneATT model with the LOA strategy in Sec. 4.2. ❶ Fine-Tuning. We set the learning rate to 0.0001 and finetune the backdoor model on the clean dataset. After 25 and 50 epochs, the ASR decreased by 3.45% and 7.41% respectively, indicating that fine-tuning has some mitigating effect on our attack, but cannot eliminate it. ❷ Pruning. We select the last convolutional layer in the model backbone for pruning, with a total of 512 neurons. We start from 0 with a step size of 25. As shown in Tab. 5, we observe that pruning a small number of neurons does not affect the backdoor while pruning more neurons causes the model's performance on clean samples to degrade faster. This indicates that pruning is somewhat ineffective against our attack.To sum up, our results indicate that pruning fails to detect our attack, whereas fine-tuning provides certain protection effects. 
conclusion: 
 In this paper, we propose a backdoor attack BadLANE for LD, which is robust to changes in physical-world dynamic scene factors. Bad-LANE employs an amorphous pattern for trigger design, which can be activated by various forms/shapes of mud spots. Additionally, a meta-learning framework is introduced to generate meta-triggers that integrate diverse environmental information through sampling benign images. Through our evaluation, BadLANE demonstrates outstanding effectiveness and robustness in both digital and physical domains, significantly outperforming other baselines.Limitations. Despite promising results, several directions warrant further exploration. ❶ The backdoor injected by BadLANE may be mitigated to some extent after fine-tuning. Our future work aims to enhance the stability and robustness of our injected backdoor against fine-tuning defenses. ❷ Meta-triggers have relatively obvious patterns. Our goal is to further improve the stealthiness during poisoning. Ethical Statement. In this paper, we propose BadLANE to reveal a severe threat in the scenario of LD in the real world that is trained using third-party datasets. To mitigate the attack, we propose preliminary countermeasures for mitigation. 
 
 Appendix:  


Decision: The paper got four weak accepts. I recommend its acceptance.

Reviews: review: paper_summary: paper_summary: This paper introduces BadLANE, a dynamic scene adaptation backdoor attack for lane detection (LD), addressing limitations of existing methods by considering changes in driving perspectives and environmental conditions. It proposes an amorphous trigger pattern for viewpoint transformations and a meta-learning framework for environmental adaptation, demonstrating superior attack effectiveness across digital and physical domains.
strength_weakness: strength_weakness: 1. The study on backdoor attacks in the lane detection task in this paper is meaningful and contributes positively to improving the safety of autonomous driving technology.
2. The paper proposes four attack strategies: Lane Disappearance Attack (LDA), Lane Straightening Attack (LSA), Lane Rotation Attack (LRA), and Lane Offset Attack (LOA). These designs are comprehensive and reasonable
3. The writing of this paper is fluent, and the section arrangement is reasonable
limitations: limitations: 1. The experiments on physical-world attacks in this paper are limited, raising doubts about whether these attacks can be migrated to real-world scenarios.
2. From Table 4 in the paper, it can be observed that Poisoning Rates have minimal impact on the attack performance. The reason for this could be attributed to several factors, but the paper does not provide an analysis in this regard.
3. Threat Model: The authors should provide more detailed explanations regarding the assumption that the attacker can access a portion of the training data.
paper_summary: paper_summary: This paper implements a backdoor attack by considering the variation of real-world dynamic scenario factors and proposes BadLANE.
strength_weakness: strength_weakness: 1. Overall the technique is feasible and the methodology is validated and analyzed with sufficient experiments.
2. This paper is a good inspiration for practical development.
limitations: limitations: 1. The research in this paper is primarily based on backdoor attacks, but the tangential perspective of the author's background starts with security, and the transition is a bit unnatural。
2. The authors are doing this by setting up the appropriate triggers, so does the process of setting up the triggers itself cause a degradation in the performance of the model?
paper_summary: paper_summary: The authors propose a dynamic scenario adaptive backdoor attack method named BadLANE to address the challenges posed by changing factors in dynamic scenes. The amorphous trigger allows the backdoor to be activated by various shapes or forms of dirt or contamination on the road surface or camera lens, adapting to the changing perspectives observed by the vehicle during driving. Additionally, the authors developed a meta-learning framework to train meta-generators tailored for different environmental conditions. These generators produce meta-triggers that incorporate diverse environmental information, such as weather or lighting conditions, to initialize the trigger pattern, enabling backdoor implantation in dynamic environments.
strength_weakness: strength_weakness: 1. Innovative Trigger Design. The use of shapeless pixel clusters as triggers ensures the backdoor remains effective across different viewpoints and conditions, a novel approach compared to traditional static triggers.
2. Detailed Attack Strategies. Introducing specific attack strategies (Lane Disappearance, Lane Straightening, Lane Rotation, and Lane Offset) allows for precise and measurable evaluation of the attack's impact.
3. The article is well-structured and logically coherent.
limitations: limitations: 1. In mid-May, the author submitted the manuscript to ARXIV with their real name. Does this behavior violate the double-blind review policy of ACM MM?
2. How does the BadLANE method perform under extreme and rapidly changing environmental conditions (e.g., from bright sunlight to heavy rain within a short duration)?
3. Can the amorphous trigger patterns be reliably detected and activated under different vehicle speeds and motion blur conditions?
4. Compared to other generative methods, what are the specific advantages of using a meta-learning framework to train a particular meta-generator?
5. The author proposes an attack in a physical scenario, and the experimental section includes extensive simulations and validations using digital space datasets. A real-world physical deployment, similar to the reference [38] "Dirty Road Can Attack: Security of Deep Learning-based Automated Lane Centering under Physical-World Attack", is highly necessary. Because the most realistic road tests provide stronger evidence and greater persuasiveness.
6. It is necessary to verify whether SOTA robust backdoor attack defense methods are effective against BadLANE.
paper_summary: paper_summary: The paper proposes a backdoor attack against lane detection. The paper focuses on an important limication of a previous lane detection backdoor attack, which is the robustness under dynamic viewpoints and environmental conditions. To address the limitation, the paper designs the trigger leveraging the extraction of amorphous patterns and meta learning considering various environmental changes.
strength_weakness: strength_weakness: * There is open-sourced code, which is great for reproducing results.
* The paper has a solid motivation about the robustness of previous attacks. I like it as the real-world corruption is a major challenge for physical attacks.
* The evaluation is sufficiantly complete, involving baseline comparisons, ablation studies, and real-world experiments.
* The paper is well written.
limitations: limitations: * Some details are missing about the physical experiment.

For the poisoned model used in the physical experiment: What is the training set and what is the proporation of poisoned data in the training set? Is the model performing good on the clean setting? I have this concern because the clean setting still has 5-10% "attack" success rate.

Also, is the lane detection consistently wrong during the driving if the trigger is there. The screenshots are impressive but I wonder the attack impact in a continous temporal sequence.

* Comparison with naive data augmentation

One straightforward idea to deal with physical world noises and data corruptions would be the data augmentation method, i.e., apply corruptions on triggers when poisoning the dataset. What is the advantage of the meta-learning approach compared with naive augmentation. Any associated evaluation results?