Beyond Accuracy: Tracking more like Human via Visual Search

Authors: dailing  zhang
shiyu  hu
xiaokun  feng
xuchen  li
meiqi  wu
jing  zhang
kaiqi  huang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 04:16:23

Abstract: Human visual search ability enables efficient and accurate tracking of an arbitrary moving target, which is a significant research interest in cognitive neuroscience. The recently proposed Central-Peripheral Dichotomy (CPD) theory sheds light on how humans effectively process visual information and track moving targets in complex environments. However, existing visual object tracking algorithms still fall short of matching human performance in maintaining tracking over time, particularly in complex scenarios requiring robust visual search skills. These scenarios often involve Spatio-Temporal Discontinuities (i.e., STDChallenge), prevalent in long-term tracking and global instance tracking. To address this issue, we conduct research from a human-like modeling perspective: (1) Inspired by the CPD, we pro- pose a new tracker named CPDTrack to achieve human-like visual search ability. The central vision of CPDTrack leverages the spatio-temporal continuity of videos to introduce priors and enhance localization precision, while the peripheral vision improves global awareness and detects object movements. (2) To further evaluate and analyze STDChallenge, we create the STDChallenge Benchmark. Besides, by incorporating human subjects, we establish a human baseline, creating a high- quality environment specifically designed to assess trackers’ visual search abilities in videos across STDChallenge. (3) Our extensive experiments demonstrate that the proposed CPDTrack not only achieves state-of-the-art (SOTA) performance in this challenge but also narrows the behavioral differences with humans. Additionally, CPDTrack exhibits strong generalizability across various challenging benchmarks. In summary, our research underscores the importance of human-like modeling and offers strategic insights for advancing intelligent visual target tracking. Code and models are available at https://github.com/ZhangDailing8/CPDTrack.

Main paper: 
 introduction: 
 In the real world, humans excel at locating an arbitrary moving target within complex backgrounds and can resume tracking it even after temporary loss. This enduring question in cognitive neuroscience has recently been explained to some extent by the Central-Peripheral Dichotomy (CPD) theory, which suggests that the human eyes processes all visual inputs by sorting them into central and peripheral visions [1, 2] . Central vision, decoded and understood by higher brain areas, focuses on interpreting details and minimizing distractions; while peripheral vision, processed in the primary visual cortex (V1), swiftly detects dynamic changes. shows the status of the target within the sequence of "095" from VideoCube [4] , and red dots means shotcut.Meanwhile, in computer vision research, advancements in single object tracking (SOT) are increasingly closing the gap between human dynamic visual ability (DVA) and proxy tasks [3, 4, 5] . In 2013, the short-term tracking (STT) task and related benchmarks were proposed [6, 7, 8] . However, the implicit assumption of continuous motion simplifies this task to continuously locating the target in short videos of tens of seconds, which is far from human DVA. Drawing on human visual search skills, the VOT community is broadening its scope by incorporating long-term tracking (LTT) [9, 10] , which includes target absent (i.e., the moving target can disappear and reappear in tracking), and global instance tracking (GIT) [4] , which involves shotcut (i.e., the video sequence may include different viewpoints and scenes). We refer to these challenges collectively as spatio-temporal discontinuity (STDChallenge), which demands enhanced visual search abilities from trackers, as shown in Fig. 1 .Unfortunately, mainstream machines have not kept pace with the aforementioned expansion of task definitions to achieve human performance. Most mainstream trackers [11, 12, 13, 14, 15, 16] , influenced by the definition of STT, depend on the spatio-temporal continuity of the target. These trackers typically perform local cropping based on the previous frame's results within the motion model, a module for calculating the search region on the current frame, as shown in Fig. 2(b-1 ).Obviously, this modeling mechanism only simulates human central vision, making it far from replicating human visual search ability, especially in scenarios where the STDChallenge increases significantly, as shown in Fig. 6(a) . Several studies [17, 18] have also demonstrated that STDChallenge presents a significant challenge for trackers. Thus, a natural question is: what causes trackers to overlook the importance of the STDChallenge in their modeling process? Some possible reasons from the benchmark perspective may answer the above question: Limitation of datasets. Although the definition of proxy tasks has been moving closer to human DVA (STT → LTT → GIT), most datasets are still influenced by the initial task characteristics (i.e., single-target, modelfree, causal-trackers, single-camera, and short-term) in data collection process [19, 20, 21, 9] , as in Fig. 10 , resulting in the importance of the STDChallenge being overlooked. Consequently, trackers can easily achieve good performance on most datasets without possessing sufficient visual search ability to handle the STDChallenge. Limitation of evaluation. Researchers contend that integrating insights from human DVA could address STDChallenge and enhance the real-world robustness of machines [22, 3, 23] . However, the substantial differences in methodologies and experimental approaches between computer science and cognitive science make it difficult to determine if current technological advancements are truly bridging the gap between SOT machines and human DVA. Consequently, there is a notable absence of mechanisms to assess the true visual intelligence of trackers.In summary, existing trackers lack consideration of human visual search ability in their modeling mechanisms, making it difficult to cope with the STDChallenge. This limitation is not only due to the algorithm design itself but also involves the limitations of datasets and evaluation. Therefore, this work aims to address these aspects (algorithms and benchmark) and verifies the relevant results through experimental analysis.A new algorithm CPDTrack (Section 3). Our analysis of the tracker pipeline and understanding of the CPD suggests that mainstream motion models, which crop local regions from the current frame, as shown in Fig. 2 , restrict the tracker's ability to locate moving targets. Drawing on the distinction between central and peripheral vision in CPD, we propose a new tracker named CPDTrack to divide the current frame accordingly. Central vision captures detailed information, leveraging the video's spatio-temporal continuity to introduce priors, while peripheral vision manages global information to enhance overall scene understanding, as shown in Fig. 3 . CPDTrack also models the information query, achieving top-down control based on cognitive outcomes.A new STDChallenge Benchmark with Visual Turing Test (Section 4). To address the shortcomings of existing benchmarks environments, we introduce a specialized challenge environment named STDChallenge Benchmark to assess the visual search abilities of both machines and humans in tracking an arbitrary moving target. This environment comprises sequences that represent the STDChallenge, carefully selected from both LTT and GIT benchmarks. By sampling from a variety of benchmarks, we ensure a reduction in dataset-specific biases. Simultaneously, we employ the Visual Turing Test (VTT) [24, 25] on the STDChallenge Benchmark to assess the disparity in intelligence between trackers and human DVA. Specifically, we examine error consistency [26] to compare the performance of humans and machines, and explore behavioral differences among machines with varying architectures and parameters in dynamic visual tasks. Building on these results, we conduct a thorough and detailed analysis to reveal the impact of the STDChallenge on trackers.Comprehensive and integral experimental analyses (Section 5). Finally, analysis results from extensive experimental settings show that, in the STDChallenge, CPDTrack not only achieved stateof-the-art (SOTA) performance, getting 1.7% superior N-PRE and 1.4% PRE score on STDChallenge Benchmark compared to the second algorithm, as shown in Table 1 , but also demonstrated remarkable alignment with human DVA, as shown in Fig. 4 . Furthermore, it performed well across various tracking benchmarks, particularly excelling in difficult benchmarks [4, 9, 27, 28] .Contributions. Our research explores the visual search ability of humans and trackers through detailed analyses based on algorithmic strategies and benchmark evaluations. 
related work: 
 Tracking Benchmark. In 2013, the VOT competition characterized SOT using five key terms: single-target, model-free, causal-trackers, single-camera and short-term. The initial three terms serve to differentiate SOT from other visual tasks [29, 30, 31, 32] , whereas the last two, single-camera and short-term, were introduced to simplify early-stage research [6, 7, 19, 21] . Since 2018, several researchers have moved beyond the short-term limitation to embrace LTT [33, 9, 34] , with the VOT setting a new criterion that tasks permitting complete target disappearance qualify as long-term [10] .More recently, the introduction of GIT [4] has eliminated the single-camera restriction, thus offering a more realistic simulation of the world. STDChallenge Benchmark is dedicated to examining the distinctions between LTT and GIT compared to STT, and integrates human DVA to steer algorithmic advances.Visual Trackers. Deep learning trackers are typically categorized by motion models into local and global trackers, as shown in Fig. 2 . Local trackers, which are most popular method, crop a search region in current frame [11, 12, 13, 15, 35, 16, 36] . These trackers continuously improved by advancements in feature extractor and temporal analysis, have reached SOTA performance on various benchmarks [9, 19, 21] . To tackle STDChallenge, it is common to switch to a global detector when local trackers fail, though this transition depends on the local tracker's performance [37, 38, 39, 40, 41] . Local trackers often struggle with proactive global awareness, reducing their robustness against STDChallenge. Global trackers, conversely, aim to identify targets throughout the entire image, raising computational demands and easy to be confused by similar objects in the background. Due to these limitations, they are less commonly employed [42, 43] . CPDTrack is designed to model CPD by combining local and global information, thereby enabling precise and efficient global modeling and offering insights for addressing the STDChallenge. For a detailed analysis of these trackers, see Appendix D.Visual Turing Test. Traditional evaluation methods, focused on "machine versus machine" comparisons, struggle to comprehensively assess the development of visual intelligence as they rely on predefined metrics in experimental settings. The VTT, at the intersection of turing tests and computer vision, is gaining attention for its "human versus machine" approach, which uses human visual capabilities as a baseline to fully evaluate machine effectiveness [26, 44, 45, 46, 26, 25, 24] . This paradigm has been applied in SOT that simulate human DVA: GIT [4] demonstrates that humans can effectively resolve STDChallenge, and PathTracker [22, 47] has developed a circuit model to explain human decision-making processes. We are going to explore the visual search ability of humans and machines when tracking an arbitrary moving target. 
solving the stdchallenge: 
 This section presents the proposed CPDTrack method in detail. We introduce CPD motion model, which divides global information into central and peripheral vision to optimize feature extraction. Additionally, we have developed a temporal clues module that serves as the information query component within CPD theory, facilitating top-down control mechanisms.A crucial challenge is balancing the maintenance of local resolution and computational efficiency during the extraction of global information. To address this, we incorporate insights from cognitive neuroscience, specifically the fovea concept [48] and CPD, segmenting the current frame into central and peripheral. Their computation is divided into the following steps:Determine the position. For the central vision, as shown in Fig. 3 , we use the tracking result bt-1 = (b cx t-1 , b cy t-1 , w t-1 , h t-1) on the previous frame to determine its position.Central vision. To simulate the decay in visual sensitivity characteristic of the fovea, we employ gaussian distribution model-a method that has found widespread application in vision science [48, 49] . Visual sensitivity is defined as: Visual selection is the process of choosing the information to focus on, and visual decoding is the process of deeply understanding the selected features and information to make cognitive decisions. On the left is a mathematical model of visual selection, where we align human visual information processing with the resolution of devices and machines. (b) Architecture of the proposed CPDTrack: The original frame is treated as encoded visual data. We apply the acuity model described in (a) for visual selecting, which can be modulated by information query from temporal. A transformer is employed to replicate the decoding processes occurring in higher brain regions, facilitating complex cognitive tasks. The grey arrows running through both parts highlight the correspondence between the two parts.EQUATIONwhere S(x) is sensitivity in x and S 0 is the sensitivity in central. we set the σ = W 6 based on the overall size (H, W ) of the image and the 3-sigma rule. This approach ensures that the cumulative visual sensitivity to current frame exceeds 99.7%, effectively covering the vast majority of the visual field.Sensitivity of central. We determine the sensitivity levels within this region using the values from b t-1 . Integrating sensitivity as:EQUATIONwhereb x1 t-1 = b cx t-1 -wt-1 2 is the top-left coordinate of b t-1 , and b x2 t-1 = b cx t-1 + wt-1 2 is the bottom-right coordinate of b t-1 .Image crop resizing. As the central region is resized to a constant size, thus its width w e t-1 ∝ 1 sensx and w e t-1 ∝ w t-1 . Therefore, w e t-1 is determined to be:EQUATION)where S is a constant factor that can be adjusted, and Φ is the cumulative density function of the standard normal distribution. Then, we get central region m t = (b cx t-1 , b cy t-1 , w e t-1 , h e t-1 ). As the lower limit of the bounding box is set, typically at 10 pixels in tracking, equation (3) will not a division by zero error in practice.We prove the range w e t-1 in equation ( 3): First, take the derivative with respect to w e t-1 (x), The range of values for x is (0, W ]:EQUATION)where f is the probability density function of the standard normal distribution. We define the numerator as n w (x) and denominator as d w (x)It is clear that d w (x) ≥ 0 when x > 0; differentiate n w (x):EQUATIONClearly, f ′ < 0. Thus, n w (x) has its minimum value at x = 0, n w (x) ≥ 0. Therefore, w e t-1 (x)is a monotonically increasing function when x > 0. When x takes the maximum value, x = W , w e t-1 (x) ≈ 0.9987SW . Intuitively, the central area is determined by cropping around the target based on the b t-1 . This differs from traditional motion models, which typically use a static context ratio for cropping. In contrast, the CPD model adapts the contextual range dynamically, offering more flexibility and precision, more visualization in Appendix B.For the peripheral area, we adjust the entire image to match the size of the central area. This resizing facilitates the simulation of peripheral vision, aligning the scale both regions.In CPD [2] , the selection process is influenced by cognitive feedback through a top-down approach, as illustrated in Fig. 3 . Similarly to mode that humans can actively focus on areas of interest, CPDTrack incorporates an information query mechanism. Spatially, we determine central vision based on the results b t-1 of the previous frame. Semantically, we use the query to map high-level semantic information from the previous frame downwards, achieving top-down control. Specifically, each past frame is represented by a token while template is max-pooled into a query as initialization. This query, using cross-attention, extracts refined high-level semantic insights from multiple past frame tokens, then concatenated to images tokens and inputted into the ViT [50] . Within the ViT backbone, the information query interacts with the global information of the current frame and the online template, achieving implicit cognitive feedback's top-down control. Simultaneously, this query integrates the spatio-temporal information of the current frame for use in future frame computations. 
the stdchallenge benchmark: 
 STDChallenge Benchmark is an integrated task closely associated with various object tracking subtasks [7, 10, 4] . We select sequences with STDChallenge from established benchmarks based on specific sequence level attributes. Furthermore, inspired by cognitive psychology [51] , we have designed a dynamic visual capability assessment framework to evaluate and compare the performances of humans and machines when tackling STDChallenge. This evaluation directly informs and guides machine development.Construct STDChallenge Benchmark. Our objective is not to develop a new dataset to evaluate trackers' visual search ability. Rather, we aim to highlight the often-overlooked issue of STDChallenge, already acknowledged by few recent datasets but underemphasized in algorithms design. We have chosen existing benchmarks as our data sources to assess trackers' proficiency with STDChallenge, specifically selecting sequences from LaSOT [9] , VOTLT2019 [52] , and VideoCube [4] where such challenges are existing. Drawing inspiration from cinematography, we introduce the metricEQUATIONto quantify STDChallenge, where n a means the number of absent, n s means the number of shotcut, l a means the length of absent and l means the length of the sequence. This formula takes into account the frequency and duration of absent, and the frequency of shotcut. A higher ST D value indicates greater STDChallenge. From LaSOT [9] , VOTLT2019 [52] , and VideoCube [4] , we have compiled a dataset of 252 sequences that constitute the STDChallenge Benchmark, detailed attributes and distribution would typically be visualized in Appendix C.Visual Turing Test. Our investigation begins by assessing whether individuals can effectively handle the STDChallenge. In line with methodologies from the field of static vision [26, 25] , we organize the study using a small-N design [51, 26] , enlisting five participants aged 20 to 30. During the experiment, participants watch videos and are instructed to track the target with a mouse cursor. Evaluation metrics. Given that human typically focuses on a single point and emphasizes an object's salient features rather than its center, we have adjusted our evaluation metrics and outputs accordingly. For error consistency measurements refer to [26] , human is successful if the point fall within the groundtruth while trackers is successful if IoU > 0.5 in a frame. For performance measurements, we employ the N-PRE metric [4] , which evaluates the precision based on relative position between the center of boundingbox and groundtruth, to assess the performance of both humans and machines. Experiments results. In the STDChallenge, humans outperform most trackers, indicating that the DVA of trackers is still far from humans. Furthermore, while the performance of machines tends to vary significantly with ST D changing, human consistently exhibit stable tracking ability, as shown in (b) of Fig. 4 . Notably, as research progresses, the scores of SOTA trackers are gradually approaching those of humans, suggesting a narrowing gap between machine and human performance. Additionally, the low error consistency between trackers and humans suggest they employ different tracking strategies in past studies. There's a significant positive correlation between the N-PRE of trackers in STDChallenge and their error consistency with humans, indicating that trackers are incorporating some implicit human-like strategies to improve performance progressively. Among them, CPDTracker, which explicitly models human strategies, achieved the highest score of 0.167. This method not only effectively tackles the STDChallenge but also aids in guiding machines to exceed the biases of evaluation benchmarks and adapt to real-world scenarios.Some Reasoning. (i) Like global trackers as shown in Fig. 2 , humans process entire image information but can handle the STDChallenge more effectively at higher frame rates. This demonstrates the superior efficiency of humans in video information processing. (ii) When a target is lost, humans not only quickly become aware of this but also swiftly reacquire the target. This process involves the integrated use of spatial cognition and temporal memory, demonstrating human DVA to understand global information, in contrast to mainstream trackers that rely on local information. (iii) Furthermore, our observations reveal that humans tend to focus on object's salient features over its center when tracking. This significantly leads to current evaluation metrics underestimating human capabilities, while also highlighting the clear target recognition ability of humans.Moreover, we examined how different modules within the trackers' pipeline influence performance and error consistency. For detailed findings, refer to the Appendix F. LaSOT. LaSOT is a high-quality, large-scale benchmark for LTT featuring a test set of 280 videos with an average of 2448 frames each sequence. We attribute underperformance of CPDTrack to the limited STDChallenge scenarios and varying interpretations of groundtruth within these tests as in Fig. 7 . 
ablation and analysis: 
 Through ablation studies, we validate the effectiveness of each module of CPDTrack in STDChallenge.We conducted ablation studies of CPDTrack in STDChallenge Benchmark and VideoCube, comparing the impacts of "central-peripheral" vision dichotomy and the use of information query. Information query was removed in #1 as baseline to examine the performance of the CPD Motion Model itself. #2 is traditional local crop, which can be considered as only containing central vision. #3 employed only peripheral vision. #4 is the proposed CPDTrack to validate the effectiveness of cognitive feedback control in ANN. Based on #2, peripheral vision and information query was added in #5 to demonstrate the effectiveness of our designed CPD Motion Model.The combination of central and peripheral vision performs better than using either alone. It can be seen that performance metrics and error consistency of baseline #1 surpass those of #2 using only central vision and #3 using only peripheral vision, with improvements range 0.7% to 9.3% in various metrics on STDChallenge Benchmark and VideoCube and more like human, as in Tab. Figure 7 : Visualizations of CPDTrack and some local trackers tracking results. This demonstrates that CPDTrack, influenced by the global perspective, tends to frame the entire target, showing a similarity to human cognition of objects, such as "a lion's tail" or "the tail rotor of a helicopter." This is not an error, but rather a bias introduced by the dataset's setup. central vision may be better at precise localization. We believe this demonstrates the complementarity of central and peripheral vision, further validating CPD theory.Information query is effective. It can be seen in Tab. 2, compared to baseline#1, #4 has improved performance by 0.1%-0.4% on several metrics in STDChallenge Benchmark and VideoCube, and with 0.8% improvement in error consistency. This indicates that in tracking, utilizing higher-level semantic information for top-down control is beneficial for trackers to understand sequence information. On the other hand, information query can also be considered as the retrieval of short-term memory, which suggests that combining short-term and long-term memory (online template) in tracking can better address the STDChallenge.Modeling central vision with Gaussian acuity is effective. Compared to the traditional local crop as central vision #5, CPDTrack #4 has surpassed in performance metrics on STDChallenge Benchmark and VideoCube, with improvements ranging from 1.2% to 4.2%, as in Tab. 2. The error consistency is also higher than #5. This suggests that considering the bounding box and image size, dynamically extracting context for central vision can better align with peripheral vision. 
conclusion.: 
 Our study is inspired by the differences in dynamic visual abilities between humans and machines, notably in challenging tasks like LTT and GIT. We attribute this gap primarily to the machines' deficiency in essential visual search skills. Based on CPD from cognitive neuroscience, which categorizes visual inputs into central and peripheral vision, we understand humans' robust visual search ability under limited computational resources.We propose CPDTrack, extensively validated for its effectiveness and closer behavioral alignment with humans in STDChallenge Benchmark. Ablation studies highlights that its success is due to the efficient integration of central and peripheral vision. Our findings indicate that robust visual search capabilities are crucial for gradually adapting trackers to real-world applications. 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: We discuss the broader impacts of this paper in Sec. G.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic machine for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: This paper focuses on the video tracking task, which does not pose such high risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and s of use explicitly mentioned and properly respected?Answer: [Yes]Justification: We properly credit the creators or original owners of all assets used in the paper, and cite the relevant papers for each asset. Human data were also released with the experimenter's permission.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and s of service of that source should be provided.• If assets are released, the license, copyright information, and s of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help deine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: A Extended related work.The attention bottleneck force animals to only process a subset of sensory input in depth. This motivates a unifying central-peripheral dichotomy (CPD), categorizing multisensory processing into central and peripheral senses based on their functions [1, 2, 48, 49, 59, 60, 61] .In the CPD theory, the encoding-selection-decoding framework combined with central and peripheral sensations reflects how the brain processes sensory information in stages to optimize resource usage and cope with environmental challenges.The encoding stage involves the reception and preliminary processing of all initial sensory inputs entering the sensory system. At this stage, sensory information is converted into signals that the brain can further process.The selection stage is the phase in CPD where 'central' and 'peripheral' selections are separated. This process is driven not only by saliency mechanisms but also influenced by top-down expectations, which are achieved through higher cognitive processes in the brain.The decoding stage is the active part of 'central senses' in CPD. After the selection stage, the selected few central sensory inputs are transmitted to higher brain regions for in-depth analysis. This stage requires complex cognitive processing, including integrating information, recognizing patterns, and understanding the deep meaning of inputs. We draw from these principles to construct the CPDTrack. The STDChallenge integrates LTT and GIT tasks to assess the visual search capabilities of tracking machines, which addresses the bias of single datasets, as shown in Fig. 11 . This challenge incorporates sequences drawn from the LaSOT [9] , VOTLT2019 [52] , and VideoCube [4] , thereby inheriting their characteristics. The sequence lengths vary, with the shortest being "yoyo-15" from LaSOT and the longest "liverRun" from VOTLT2019. Notably, the extensive dataset of LaSOT contributes significantly to the diversity and complexity of the challenges presented in STDChallenge.STDChallenge is a sequence-level challenge in which we analyze the impact of frame-level challenges, as shown in Fig. 10 . These frame-level challenges are structured following SOTVerse [18] , employing the same computational methods. The STDChallenge benchmark is noted for its comprehensive inclusion of various challenges, particularly in terms of blur_bbox, relative_scale, and relative_ratio.Trackers that effectively manage these specific challenges are likely to achieve superior performance within the STDChallenge framework.It can be seen that on the STDChallenge Benchmark, the STD still shows a long-tail distribution, where the majority of sequences exhibit fewer challenges, while a minority contain a higher number of challenges. There have already been many large-scale benchmarks in target tracking, and how to increase the amount of data under specific conditions is a promising direction.In our setup, a sequence with shotcut or absent is deemed to include STDChallenge, naturally integrating challenges found in new real-world scenarios. Thus, STDChallenge not only incorporates but also enhances other challenging attributes to some extent, as depicted in Fig. 9 (c). absent is often associated with occlusions and out-of-view scenarios. Formally, STDChallenge does not exclude other tasks (such as pathtracker [22] ), but we advocate integrating STDChallenge with real-world scenarios to avoid ill-posed issues (such as shotcut confusing same-looking blocks in pathtracker).Comparison between CPDTrack and pathtracker are in Tab. 4 Table 4 : From the model design perspective, we believe TransT+InT aims to address PathTracker, a challenge that relies on trajectory information rather than appearance features, which diverges from the Motivation of STDChallenge. We attempted to replicate TransT+InT in VideoCube and STDChallenge; however, we did not achieve optimal performance due to the lack of clear documentation as the issue in the github. Therefore, we believe these can partly explain the suboptimal performance of TransT+InT on STDChallenge. D Machines.Model. We adopt ViT-B [50] as our encoder architecture for CPDTrack. The encoders are initialized with the MAE [64] pre-trained parameters. The patch size is set to 16 × 16. The decoder consists of 2 transformer blocks. The decoder hidden size is 256, the number of attention heads is 8, and the hidden size of the feed forward network (FFN) is 1024. The number of quantization nbins and the vocabulary size are all set to 4000. The dimension of word embedding is 256, which is consistent with the decoder hidden size. The embedding-to-word sampling uses a 3-layer perceptron followed by a softmax. The hidden dimension of the perceptron is 256. The output dimension is nbins, which is aligned with the number of words in vocabulary . The word with the maximum likelihood is sampled as the output word. In addition, we present model parameters, FLOPs, and inference speed in Tab.5.LaSOT [9] , GOT-10k [19] ,and TrackingNet [21] . Aligned with VOT2020 evaluation protocol [65] , we remove the 1k forbidden videos in GOT-10k during training. For the evaluation on GOT-10k test set, we follow the official requirements and only use the training split of GOT-10k. Brightness jittering are used for data augmentation. We train the model with AdamW [66] optimizer and set the learning rate of the encoder to 1e -5, the decoder and remaining modules to 1e -4, and the weight decay to 1e -4. The model is trained for a total of 300 epochs with 60k image pairs per epoch. The learning rate decreases by a factor of 10 after 240 epochs. The model is trained on a server with four A5000 GPUs and is tested on an A5000 GPU. The tracking speed is about 23 FPSInference. We use the first template and online template together with the central region and the peripheral region as input of CPDTrack. The online template update interval T is set to 400 by default, while the update threshold τ is set to 0.7. CPDTrack will attempt to update online template within 10 frames; if none meet the threshold, it will not update. OSTrack [12] . A novel single-stream tracking framework has been proposed, which unifies feature learning and relationship modeling. At the same time, an candidate prior elimination module has been introduced to eliminate background interference while improving computational efficiency.MixViT [11] . MixViT proposes a Mixed Attention Module (MAM), used for simultaneously extracting features and integrating target information. This synchronous modeling approach can extract distinctive features of specific targets and facilitates extensive communication between the target and the search area. In this version, ViT is used to compute attention.SeqTrack [13] . The predictive object bounding boxes are generated using an autoregressive method, which simplifies the loss function.STARK [53] . Using an encoder-decoder for relational modeling, and a head that directly predicts the corners of bounding boxes has been proposed.KeepTrack [15] . KeepTrack combines SuperDiMP with a target candidate association network to achieve robust tracking. The authors retrained the target candidate association network on challenging sequences extracted from LaSOT.Ocean [54] . Ocean also adopts an anchor-free structure. The backbone network uses parameters pre-trained on ImageNet for initialization.PrDiMP and SuperDiMP [55] . PrDiMP and SuperDiMP use probabilistic regression to improve accuracy.DiMP [56] . Based on the framework proposed by ATOM, DiMP optimizes the loss function to achieve stronger discriminative ability. The backbone network is initialized with ImageNet weights.SiamRPN [35] . SiamRPN introduces a region proposal network to achieve precise object regression.ATOM [57] . ATOM attempts to combine CF and SNN and proposes a new framework to take advantage of offline training and online updates.KYS [58] . KYS represents scene information as state vectors and combines them with the appearance model to locate objects.SiamFC [16] . As the pioneer of SNN-based trackers, SiamFC achieves satisfactory tracking performance by matching features between the template area and the search area through a simple network structure.SPLT [38] . SPLT designed a verifier to switch between global search and local search. SPLT uses it as a feature extractor and down-samples the spatial resolution of template features to 1X1 through average pooling. In terms of the verification model, SPLT employs ResNet50 as the backbone of the verifier.Learning regression and verification networks for robust long-term tracking [41] . It employed a verification network with local view to identify the target from the detected candidates. If the target disappears, the learning based switching scheme determines whether to trigger the global search mode.Effective Local and Global Search for Fast Long-Term Tracking [40] . It used the score from a target verifier within the local search module to decide whether to switch strategies.DaSiamRPN [39] . DaSiamRPN uses data augmentation to enhance its discriminative ability.RTracker [37] . RTracker uses a tree-structured memory to dynamically link the tracker and detector by controlling the collection of positive and negative target samples, thereby enabling self-recovery capabilities.SiamRCNN [43] . SiamRCNN utilizes a re-detection mechanism and a small trajectory dynamic programming machine to address the issue of object disappearance. SiamRCNN is based on the implementation of FasterRCNN, using ResNet-101-FPN as its backbone.GlobalTrack [42] . GlobalTrack does not assume motion consistency and performs a full image search to eliminate cumulative errors.This section provides a comprehensive overview of the experimental setup for the VTT, including composition of test data, selection of equipment, experimental instructions, experimental procedures, survey questionnaire, and grouping information. In our dataset preparation, we initially categorize the STDChallenge benchmark into three distinct groups based on the ST D as previously described: sequences with ST D < 10 -4 , sequences with 10 -4 < ST D < 10 -3 , and sequences with ST D > 10 -3 . From each category, five sequences are randomly selected and paired with a designated test sequence to establish the STDChallenge-Turing test environment. This environment is composed of 16 sequences that represent a spectrum of challenges within STDChallenge Benchmark, providing a robust framework for evaluating performance across varying difficulty levels.We have communicated with our institution's IRB, submitted the necessary materials, and obtained IRB approval. The experimental tasks include watching videos and moving the mouse to track a target. To avoid the risk of worsening the participants' vision conditions, we allow participants to rest between two videos. Additionally, participants are allowed to adjust their chairs and the distance to the screen to achieve the most comfortable conditions. Participants will be compensated 100 RMB per hour. The total amount used is 1000 RMB.Participants were asked to review and sign the following declaration of consent:Psychophysical research. You will be asked to watch video sequences on a computer screen and track a target using a mouse. For completing a session, you will be compensated at a rate of 100 RMB per hour.Declaration of consent. I agree to participate in a behavioral experiment on visual perception. I participate voluntarily in this study. I have been informed that I can stop the experiment at any time without giving any reason and without any negative consequences. I know that I can contact the experimenters at any time to ask questions about the research project.Declaration of consent for data processing and data publication. I consent that the experimental data obtained during the experiment may be used for scientific evaluation and publication in a semianonymous form. I agree to have my personal data (such as name, phone number, address) stored digitally; except for being used to contact me, it may not be used for any other purposes. These personal data will only be retained within the researchers' group and will never be transferred to third parties.Referencing [25] , we focused on ensuring the accuracy and validity of data collected from a limited number of subjects to accurately reflect human visual capabilities. Consequently, we implemented measures to prevent out-of-distribution (OOD) responses, enabling the use of all responses. 1) We did not use crowdsourcing platforms; instead, the Visual Turing Test were conducted in a laboratory environment under supervision. 2) Before the experiment, we assessed participants' cognitive and perceptual levels to ensure normalcy, and allowed them to familiarize with the equipment through basic exercises. 3) During the experiment, participants were permitted to pause the experiment a certain number of times voluntarily to adjust their state. 4) In cases where the mouse cursor moved off-screen, we retained the results from the previous frame until the cursor returned within the screen area.E.4 Questionnaire.Q2. Your id:Q3. Your age:Q5: Approximately how many times were you unable to locate the target? Q6: How do you feel about the video playback speed? Q7: Do you feel that the videos vary in difficulty, and which ones do you find more challenging?Measuring human DVA is a crucial aspect of our research. Traditionally, studies have employed eye trackers or computer mice to assess visual capabilities. Our preference for using a mouse over an eye tracker is based on several considerations:(1) Eye trackers, which passively record eye movements, are influenced by multiple factors including device accuracy, user posture, the distance between the eye and the device, and the presence of eyeglasses, leading to potential limitations in data accuracy.(2) Eye trackers have greater cumulative errors. When errors occur, subjects can only adjust their posture and cannot actively correct the capture results. However, when significant cumulative errors occur with the mouse, we allow subjects to actively pause the video and correct the errors.(3) Some researchers have also pointed out from a theoretical perspective that eye trackers are affected by the tracking rate (i.e., the amount of eye movement data lost). Blinking or brief shifts in gaze can result in a lower tracking rate, which requires subjects to remain highly focused while using the eye tracker. However, the video sequences we test are long, inevitably affecting the performance of the eye tracker due to the subject's fatigue.(4) The use of a mouse to document observation points is well-established within both academic and industrial settings, providing a reliable and practical alternative for tracking visual observation in long-duration tasks.Process of DVA Experiments:(1) Participants initially inspect the setup equipment and adjust their chair height, seating posture, and their distance from the screen to ensure comfort and optimal viewing conditions.(2) A test video then plays centrally on the screen. In the first frame, participants are tasked with observing and memorizing specific features of a designated target, subsequently using the mouse to track this target throughout the remaining video frames.(3) Upon completion of the test, the lead researcher addresses any questions from participants, clarifying any uncertainties.(4) A series of 15 official videos from varied groups are sequentially displayed, with breaks between videos to reduce visual strain. During these sessions, participants are required to diligently track the target and accurately maintain the mouse's position.(5) Finally, participants complete a self-assessment questionnaire to evaluate their own performance and experience during the test. This represents the degree of behavioral similarity between different decision-makers. For each sequence in the STDChallenge-Turing Benchmark, we calculate the error consistency o each frame for different decision-makers, and then average these across all sequences to obtain the overall error consistency among the decision-makers.As shown in Fig. 13 , CPDTrack exhibits a higher degree of consistency with human behavior compared to other machines, yet it remains closely aligned with machines. Notably, it demonstrates a higher level of consistency with trackers that utilize ViT as their backbone. This observation underscores the significant influence that backbone can have on algorithmic performance. Furthermore, CPDTrack shares more similarities with trackers that implement a global strategy, under the same network architecture. We attribute this similarity to the adoption of strategies that more closely mirror human cognitive processes.As shown in Fig. 13 ,ViT-based trackers display high similarity due to shared components like the backbone and motion model. Enhanced parameters, increased resolution, and more training data have improved their alignment with human error consistency. This is believed to result from the labels in supervised learning being annotated by humans, enhancing the algorithms' ability to fit these labels.As shown in Fig. 13 , compared to ViT-based trackers, those based on CNN exhibit lower similarity among themselves. Nonetheless, certain algorithms that build on others still achieve high consistency in error consistency.As shown in Fig. 13 , the motion model plays a crucial role in this consistency, particularly affecting how trackers process and interpret video sequences, a difference that becomes particularly pronounced in high-resolution settings. (a) As shown in Fig. 14(a) . Using temporal information is more effective and also more human-like.(b) As shown in Fig. 14(b ). Compared to other network architectures, one-stream is more effective and also more human-like. However, since current one-stream trackers are all based on ViT, it is unclear whether the effects are due to one-stream or ViT.(c) As shown in Fig. 14(c ). Undoubtedly, the CPD motion model has better effects and more human-like behavior. At the same time, compared to the local-based motion model, the global one is more human-like, as it treats the entire image as central vision, improving performance at the cost of computational efficiency.(d) As shown in Fig. 14(d) . Increasing the context ratio does not seem to change the performance and error consistency, but after changing the resolution from 4(256) to 4(384), there has been an improvement. How to maintain computational efficiency while increasing resolution is a promising direction.(e) As shown in Fig. 14(e) . Obviously, increasing the number of parameters and switching to a newer backbone can significantly improve performance and error consistency. We believe this is due to the increased parameters enhancing the network's ability to fit human labels. The error consistency of vit_large has decreased, and we believe this might be because the network has learned some strategies that surpass human capabilities.(f) As shown in Fig. 14(f ). Increasing training data does not necessarily mean an improvement in performance, but it often enhances error consistency with humans. We believe this is because richer human labels enhance the network's learning capabilities.Broader Impact. The fundamental objective of our research is to model human visual search capabilities through CPD theory. The STDChallenge Benchmark facilitates our study by aligning human and machine performances in a complex visual task designed to explore tracking strategies under spatio-temporal discontinuity. Our developed CPD Motion Model has not only achieved SOTA results in this task but has also enhanced the behavioral similarities between humans and machines, substantiating CPD theory and illuminating the potent visual search abilities of humans under constraints of limited information. This work underscores the significance of human-like modeling in enhancing tracking algorithms.However, we must be cautious about the potential misuse of this technology, particularly in surveillance applications, which could have serious ethical implications. Similarly, the development of highly human-like algorithms has already sparked some concerns, similar to those associated with large language models (LLMs). Despite these challenges, the technology holds crucial potential for beneficial applications in fields such as sports, autonomous driving, robotics, and broader machine vision contexts in the real world. To promote responsible usage and further research into these positive applications, we plan to make our code and data publicly available.Limitation. CPDTrack focuses on the STDChallenge but neglects performance in simpler scenarios, leading to its underperformance on some datasets compared to SOTA trackers, as shown in Tab. 7. In the future, we plan to further improve it to adapt to various environments. The human-like modeling approach adds overhead, making CPDTrack less immediately applicable to real-world applications as it currently stands. STDChallenge in the STDChallenge Benchmark still shows a long-tail distribution, meaning most sequences contain fewer STDChallenge. This is still far from the real world, and how to increase data under specific challenges remains an open question.Generalization of CPDTrack. We believe that many simple scenarios in Single Object Tracking (SOT) are based on laboratory settings and even possess characteristics of toy scenarios (for example, tracking smoothly moving targets within very short time) to some extent. In contrast, CPDTrack extends much further into the real world:Local trackers introduce stronger priors, which enhance their performance in STT scenarios. Therefore, STDChallenge is fatal for them [18, 17] ; however, CPDTrack effectively addresses this challenge.Setup of the STDChallenge Benchmark is that any sequence containing at least one absent or shotcut is considered to include the STDChallenge. It is evident that the STDChallenge Benchmark already includes many relatively simple video environments, as shown in Fig. 12 (1)."Experiments indicate that humans perform worse than SOTA trackers in simple scenes, likely due to the precise focus of local trackers. While a global perspective, used by humans and CPDTrack, offers a comprehensive view, it may not align with the dataset setup (e.g., 'lion's tail' or 'helicopter tail rotor'; see Fig. 7 )."However, in complex scenarios, human capabilities to track moving targets surpass those of trackers, a trait also inherited to some extent by the human-like modeling of CPDTrack.Computation overhead of CPDTrack. We acknowledge that compared to some mainstream trackers, CPDTrack's human-like modeling causes some additional computational costs. However, in our experimental setup, CPDTrack achieves 20-30 fps, which already meets the requirements for real-time performance.Moreover, the main contribution of our paper is to validate the effectiveness of human-like modeling through CPDTrack, and the experimental conclusions have affirmed the motivation behind our work. While there are computational costs, these do not overshadow the contributions of CPDTrack. We greatly appreciate your questions and will continue to explore improvements in future work.Table 7 : Comparison with state-of-the-art methods on additional benchmarks, where is AUC scores on TNL2k and LaSOText, and AO scores on GOT-10k. We add a symbol * over GOT-10k to indicate that the corresponding models are only trained with the GOT-10k training set.SiamFC [16] ECO [67] Ocean [54] ATOM [57] DiMP [56] TransT [14] OSTrack [12] SeqTrack [13] CPDTrack TNL2k [20] 29 • The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition machine may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed machines and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 


Decision: The paper introduces a novel visual tracking approach inspired by the Central-Peripheral Dichotomy (CPD) theory, along with a new benchmark, STDChallenge, designed to evaluate tracking algorithms under spatio-temporal discontinuities. The reviewers all agree that the introduction of this benchmark as a significant contribution, highlighting its potential to address gaps in existing visual tracking methodologies. The proposed CPDTrack algorithm is praised for aligning with human visual search behavior and achieving promising results on the new benchmark. 

However, concerns were raised about the originality and generalizability of CPDTrack, with some reviewers noting similarities to existing methods and its performance limitations in simpler scenarios. Overall, the paper is considered technically solid and impactful. Please consider incorporating reviewers' suggestions in the revision.

Reviews: review: paper_summary: paper_summary: - The authors generalize the visual tracking problem as human dynamic visual ability task, and propose a new benchmark named STDChallenge to evaluate the visual tracking algorithms. This new benchmark includes challenging scenarios with spatio-temporal discontinuity, where previous short-term tracking oriented algorithms fail to address. They also propose a new tracking algorithm oriented to the new task where it mimics the human visual search behavior, and evalute its effectiveness on multiple benchmarks.
strength_weakness: strength_weakness: - Comparision with VideoCube [3] dataset

Since the proposed dataset includes video sequences with spatio-temporal discontinuities, it bears large similarities with the VideoCube dataset proposed in [3], where it includes sequences that require global search and target recovery after disappearances. In what aspects do the proposed dataset differ from VideoCube? (i.e. sequence length, object category, discontinuity etc.)

- Proposed CPDTrack algorithm

The proposed CPDTrack utilizes global and local search simultaneously, which is implemented by using two search images cropped at different search areas of large and small sizes. Although the authors provide an explanation that the proposed method was inspired by cognitive neuroscience, this approach was widely used in previous visual tracking algorithms.

  - [a] Effective Local and Global Search for Fast Long-term Tracking, TPAMI 2022
  - [b] Learning Regression and Verification Networks for Robust Long-term Tracking, IJCV 2021
  - [c] ‘Skimming-Perusal’ Tracking: A Framework for Real-Time and Robust
Long-term Tracking, ICCV 2019

How does the proposed method differ from these algorithms? Also, these papers should be included in the references.
questions: questions: Please refer to the questions in the weaknesses section.
limitations: limitations: The authors included a separate discussions and limitations section in their paper, with adequate explanations and descriptions on the weaknesses and possible future directions.
paper_summary: paper_summary: This paper presents a novel approach to visual object tracking by drawing inspiration from the Central-Peripheral Dichotomy (CPD) theory. The proposed CPDTrack aims to improve tracking performance by emulating human visual search mechanisms, particularly under challenging scenarios involving spatio-temporal discontinuities (STDChallenge). The paper introduces a new benchmark, STDChallenge, and demonstrates the effectiveness of CPDTrack through extensive experiments.
strength_weakness: strength_weakness: 1. CPDTrack is designed specifically to address the STDChallenge, and as noted in the paper, it may not perform as well in simpler or different scenarios. Despite the explanations provided, this lack of generality is a significant concern.
2. The paper states that the tracking speed on an A5000 GPU is 23fps, is this because the introduced modules are particularly time-consuming? This represents a considerable demand for computational resources, which could be a factor to consider in practical applications.
questions: questions: See the weaknesses.
limitations: limitations: This paper does not include a dedicated section discussing the limitations of their approach.
paper_summary: paper_summary: The authors touch upon a very important topic in visual tracking. They try to build upon the Central-Peripheral Dichotomy (CPD) theory that talks about how humans use visual information to track targets in complex environments. To this end, they propose a tracker named CPDTrack and STDChallenge Benchmark (for spatio-temporal discontinuities). They claim that their CPDTrack achieves SOTA performance on STDChallenge, aligns with human behavior and generalizes across various benchmarks.
strength_weakness: strength_weakness: The authors do not have IRB which is one of the basic requirements for working with human subjects. 
I also did not see any strong statistical testing for human vs machine responses. Seems like to authors selected responses from all human participants. If not, i don’t see any methods for dropping the out of distribution responses. 
The authors also conducted 5 experiments with human subjects, but i didn’t find a clear link between the selected experiments for humans and the STDChallenge benchmark. 
There is also no justification for why removing information query was treated as baseline in ablation studies. Please justify. 
For the STDChallenge benchmark, i didn’t find a justification as to why only the said challenges were included in the benchmark. The authors should clarify that if possible. 
How easy/hard is it to include newer challenges in the benchmark?
questions: questions: There is another similar algorithm called DorsalNet. Have the authors considered the similarity/differences with it?
For the citation of PathTracker, the paper cited is InT which is a solution to the PathTracker challenge. For completeness, the challenge was introduced in https://arxiv.org/abs/2110.02772 which the authors should consider citing. 
In fig 5 the authors say that both CPDTrack and humans are robust to occlusions, environment changes and absence of target. InT has similar claims; have the authors tested their benchmark with the TransT+InT proposed in the paper ``Tracking without re-recognition”?
limitations: limitations: The gains are small compared to other networks on the STDChallenge benchmark. Have authors made an attempt to check if they are statistically significant?
Absence of IRB could raise ethical concerns which is why authors should work to mitigate it. 
In appendix E4, the authors talk about mouse movements. Are the authors tracking mouse movements from the participants’ computers? If so, that certainly calls for IRB approvals due to additional security and privacy concerns. 
Figure 11 is not clear enough as to where/how the humans fit. The authors should make clarifications about what they mean by the heatmap in terms of errors. 
The model presented in fig. 3 is rather convoluted. If possible, the authors should clarify it as well. 
The authors mention releasing code, but i didn’t find anything with the paper. The authors should make sure the code is released. If possible, they should also provide scripts/links to download the public datasets they used for better reproducibility.

Comments: comment: We are grateful for your increased rating to Weak Accept. We will incorporate the additional results and analyses following your suggestions and acknowledge the anonymous reviewers in the revised version of our paper.
I thank the authors for their detailed response to my questions, and the additional details provided by the authors were very helpful. Also, based on the concerns from the other reviewers and the authors' response to address these issues, I am inclined to raise my rating to "weak accept".
Thank you for your review. We will incorporate additional clarifications following your clarifications and acknowledge the anonymous reviewers in the revised version of our paper. Thanks.
Thank you for your review. We will incorporate additional explanations following your suggestions and acknowledge the anonymous reviewers in the revised version of our paper.
Thank you for your review and feedback! A discussion of the related broader impacts can be found in Section G of the appendix in the paper, which covers human-like modeling, video surveillance, autonomous driving, etc.

Additionally, the broader societal impacts of CPDTrack and target tracking technologies also include:
- **Dual-Use Implications:** In the military, visual object tracking technologies enhance tactical decision-making and operational efficiency. In civilian sectors, they are widely used in traffic monitoring, retail security, and smart home applications, significantly improving societal operations and quality of life. However, over-reliance on these technologies may lead to excessive automation, increasing the risks of misjudgments and unintended conflicts.
- **Privacy Considerations:** Unauthorized surveillance using these technologies can infringe on personal privacy rights and violate data protection laws. Furthermore, if misused by malicious entities, it could be employed for illegal activities such as stalking or monitoring individuals or groups, raising security and ethical concerns. Considering these factors, practices should adhere to principles such as transparency and public notification, ethical use reviews, and user control rights.

We will incorporate additional clarification following your suggestions and acknowledge the anonymous reviewers  in the revised version of our paper.
Thank you for your review and feedback!

Firstly, the datasets we used are publicly available and solely for academic research. Additionally, we have adhered to the copyright requirements of these open-source benchmarks. In our research, we refer to previous works [a,b,c], reprocessing existing benchmarks to create a new experimental environment for scientific study.

In summary, by following copyright norms and building upon past works, we create a new experimental environment without ethical issues.

We will incorporate additional clarification following your suggestions and acknowledge the anonymous reviewers  in the revised version of our paper.

---

[a] Hu, Shiyu, Xin Zhao, and Kaiqi Huang. "SOTVerse: A user-defined task space of single object tracking." International Journal of Computer Vision 132.3 (2024): 872-930.

[b] Fan, Heng, et al. "Tracklinic: Diagnosis of challenge factors in visual tracking." Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2021.

[c] Hu, Shiyu, et al. "A multi-modal global instance tracking benchmark (mgit): Better locating target in complex spatio-temporal and causal relationship." Advances in Neural Information Processing Systems 36 (2024).
I appreciate the authors' time and effort in addressing the questions and providing clarifications. I encourage them to include these explanations in the final version of the paper.
I would like to thank the authors for their time and efforts in answering the questions and clarifications. I would urge them to please include these clarifications in the final version of paper as well. 
Thanks.