Reviewerly/exHarmony huggingface dataset:
DynStatF: An Efficient Feature Fusion Strategy for LiDAR 3D Object Detection

Augmenting LiDAR input with multiple previous frames provides richer semantic information and thus boosts performance in 3D object detection, However, crowded point clouds in multi-frames can hurt the precise position information due to the motion blur and inaccurate point projection. In this work, we propose a novel feature fusion strategy, DynStaF (Dynamic-Static Fusion), which enhances the rich semantic information provided by the multi-frame (dynamic branch) with the accurate location information from the current single-frame (static branch). To effectively extract and aggregate complimentary features, DynStaF contains two modules, Neighborhood Cross Attention (NCA) and Dynamic-Static Interaction (DSI), operating through a dual pathway architecture. NCA takes the features in the static branch as queries and the features in the dynamic branch as keys (values). When computing the attention, we address the sparsity of point clouds and take only neighborhood positions into consideration. NCA fuses two features at different feature map scales, followed by DSI providing the comprehensive interaction. To analyze our proposed strategy DynStaF, we conduct extensive experiments on the nuScenes dataset. On the test set, DynStaF increases the performance of PointPillars in NDS by a large margin from 57.7% to 61.6%. When combined with CenterPoint, our framework achieves 61.0% mAP and 67.7% NDS, leading to the state-of-the-art performance without bells and whistles. Our Code will be publicly available.
SplitAMC: Split Learning for Robust Automatic Modulation Classification

Automatic modulation classification (AMC) is a technology that identifies a modulation scheme without prior signal information and plays a vital role in various applications, including cognitive radio and link adaptation. With the development of deep learning (DL), DL-based AMC methods have emerged, while most of them focus on reducing computational complexity in a centralized structure. This centralized learning-based AMC (CentAMC) violates data privacy in the aspect of direct transmission of client-side raw data. Federated learning-based AMC (FedeAMC) can bypass this issue by exchanging model parameters, but causes large resultant latency and client-side computational load. Moreover, both CentAMC and FedeAMC are vulnerable to large-scale noise occured in the wireless channel between the client and the server. To this end, we develop a novel AMC method based on a split learning (SL) framework, coined SplitAMC, that can achieve high accuracy even in poor channel conditions, while guaranteeing data privacy and low latency. In SplitAMC, each client can benefit from data privacy leakage by exchanging smashed data and its gradient instead of raw data, and has robustness to noise with the help of high scale of smashed data. Numerical evaluations validate that SplitAMC outperforms CentAMC and FedeAMC in terms of accuracy for all SNRs as well as latency.
MARRS: Modern Backbones Assisted Co-training for Rapid and Robust Semi-Supervised Domain Adaptation

Semi-Supervised Domain Adaptation (SSDA) aims to develop domain invariant models from scarcely labeled target domain in addition to the fully labeled source domain. Current SSDA works are often applied in conjunction with ResNet34 backbone, which makes them overlook the advantages of utilizing other backbones. Hence, in this paper, we investigate the impact of employing different modern backbones in SSDA and propose a novel solution named Modern Backbones Assisted Co-training for Rapid and Robust Semi-Supervised Domain Adaptation (MARRS), that uses discriminative features of two modern backbones for training linear classifiers using the well established co-training framework. To induce diversity among classifiers for effective co-training, we propose a novel module that produces diversity at three levels, namely image, backbone, and feature distribution levels. Experiments reveal that MARRS not only achieves state-of-the-art performance across all popular SSDA datasets, but also drastically cuts the computation time compared to other SSDA approaches, making MARRS a rapid and robust solution for SSDA. We also provide extensive ablation experiments to verify our framework’s vitality and primary design choices.
Region-based Appearance and Flow Characteristics for Anomaly Detection in Infrared Surveillance Imagery

Anomaly detection is a classical problem within automated visual surveillance, namely the determination of the normal from the abnormal when operational data availability is highly biased towards one class (normal) due to both insufficient sample size, and inadequate distribution coverage for the other class (abnormal). In this work, we propose the dual use of both visual appearance and localized motion characteristics, derived from optic flow, applied on a per-region basis to facilitate object-wise anomaly detection within this context. Leveraging established object localization techniques from a region proposal network, optic flow is extracted from each object region and combined with appearance in the far infrared (thermal) band to give a 3-channel spatiotemporal tensor representation for each object (1 × thermal - spatial appearance; 2 × optic flow magnitude as x and y components - temporal motion). This formulation is used as the basis for training contemporary semi-supervised anomaly detection approaches in a region-based manner such that anomalous objects can be detected as a combination of appearance and/or motion within the scene. Evaluation is performed using the Long-Term infrared (thermal) Imaging (LTD) benchmark dataset against which successful detection of both anomalous object appearance and motion characteristics are demonstrated using a range of semi-supervised anomaly detection approaches.
A Super-Resolution Training Paradigm Based on Low-Resolution Data Only to Surpass the Technical Limits of STEM and STM Microscopy

Modern microscopes can image at atomic resolutions but often reach technical limitations for high-resolution images captured at the smallest nanoscale. Prior works have applied super-resolution (SR) by deep neural networks employing high-resolution images as targets in supervised training. However, in practice, it may be impossible to obtain these high-resolution images at the smallest atomic scales. Approaching this problem, we consider a new super-resolution training paradigm based on low-resolution (LR) microscope images only, to surpass the highest physically captured resolution available for training. As a solution, we propose a novel multi-scale training method for SR based on LR data only, which simultaneously supervises SR at multiple resolutions, allowing the SR to generalize beyond the LR training data. We physically captured low and high-resolution images for evaluation, thereby incorporating real microscope degradation to deliver a proof of concept. Our experiments on periodic atomic structure in STEM and STM microscopy images show that our proposed multi-scale training method enables deep neural network image SR even up to 360% of the highest physically recorded resolution. Code and data is available on github <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
HazardNet: Road Debris Detection by Augmentation of Synthetic Models

We present an algorithm to detect unseen road debris using a small set of synthetic models. Early detection of road debris is critical for safe autonomous or assisted driving, yet the development of a robust road debris detection model has not been widely discussed. There are two main challenges to building a road debris detector: first, data collection of road debris is challenging since hazardous objects on the road are rare to encounter in real driving scenarios; second, the variability of road debris is broad, ranging from a very small brick to a large fallen tree. To overcome these challenges, we propose a novel approach to few-shot learning of road debris that uses semantic augmentation and domain randomization to augment real road images with synthetic models. We constrain the problem domain to uncommon objects on the road and allow the deep neural network, HazardNet, to learn the semantic meaning of road debris to eventually detect unseen road debris. Our results demonstrate that HazardNet is able to accurately detect real road debris when only trained on synthetic objects in augmented images.
MEnsA: Mix-up Ensemble Average for Unsupervised Multi Target Domain Adaptation on 3D Point Clouds

Unsupervised domain adaptation (UDA) addresses the problem of distribution shift between the unlabeled target domain and labelled source domain. While the single target domain adaptation (STDA) is well studied in both 2D and 3D vision literature, multi-target domain adaptation (MTDA) is barely explored for 3D data despite its wide real-world applications such as autonomous driving systems for various geographical and climatic conditions. We establish an MTDA baseline for 3D point cloud data by proposing to mix the feature representations from all domains together to achieve better domain adaptation performance by an ensemble average, which we call Mixup Ensemble Average or MEnsA. With the mixed representation, we use a domain classifier to improve at distinguishing the feature representations of source domain from those of target domains in a shared latent space. In extensive empirical validations on the challenging PointDA-10 dataset, we showcase a clear benefit of our simple method over previous unsupervised STDA and MTDA methods by large margins (up to 17.10% and 4.76% on averaged over all domain shifts). We make the code publicly available here <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
A Novel Benchmark for Refinement of Noisy Localization Labels in Autolabeled Datasets for Object Detection

Autolabeling approaches are attractive w.r.t. time and cost as they allow fast annotation without human intervention. However, can we really trust the label quality of auto-labeling? And further, which potential consequences arise from resulting label noise? In this work, we address these questions for localization, a subtask of object detection, by investigating the effects on a state-of-the-art deep neural network (DNN) for object detection and the widely used Pascal VOC 2012 dataset. Our contributions are threefold: First, we propose a method to inject noise into localization labels, enabling us to simulate localization label errors of autolabeling methods. Afterwards, we train a state-of-the-art object detection DNN with these noisy labels. Second, we propose a refinement network which takes a noisy localization label and its respective image as input and per-forms a localization refinement. Third, we again train a state-of-the-art object detection DNN, however, this time with refined localization labels. Our insights are: Training a state-of-the-art DNN for object detection on noisy localization labels leads to a severe performance drop. Our proposed localization label refinement network is able to refine the noisy localization labels. We are able to retain the performance to some extent by retraining the state-of-the-art DNN for object detection on the refined localization labels. Our study motivates a new challenging task "refinement of noisy localization labels" and sets a first benchmark for Pascal VOC 2012. Code is available at https://github.com/ifnspaml/LocalizationLabelNoise.
CoReFusion: Contrastive Regularized Fusion for Guided Thermal Super-Resolution

Thermal imaging has numerous advantages over regular visible-range imaging since it performs well in low-light circumstances. Super-Resolution approaches can broaden their usefulness by replicating accurate high-resolution thermal pictures using measurements from low-cost, low-resolution thermal sensors. Because of the spectral range mismatch between the images, Guided Super-Resolution of thermal images utilizing visible range images is difficult. However, In case of failure to capture Visible Range Images can prevent the operations of applications in critical areas. We present a novel data fusion framework and regularization technique for Guided Super Resolution of Thermal images. The proposed architecture is computationally inexpensive and lightweight with the ability to maintain performance despite missing one of the modalities, i.e., high-resolution RGB image or the lower-resolution thermal image, and is designed to be robust in the presence of missing data. The proposed method presents a promising solution to the frequently occurring problem of missing modalities in a real-world scenario. Code is available at https: //github.com/Kasliwal17/CoReFusion.
Video Tiny-Object Detection Guided by the Spatial-Temporal Motion Information

Detecting tiny/small objects (e.g., drone targets) in videos is highly desired in many realistic scenarios. Nevertheless, current object detection algorithms can hardly recognize tiny targets against extremely complex backgrounds. To address this problem, we propose a motion-guided video tiny-object detection method (MG-VTOD), in which the spatial-temporal motion strength maps play an important role in object searching and locating. Inspired by the biological retinal structure, we compute the motion strength using a sequential frame cube that has been aligned and registered. Subsequently, the motion strength maps are employed to enhance the potential areas of the moving targets, thereby facilitating the target detection procedure. Experimental results obtained on the Anti-UAV-2021 dataset validate that the proposed MG-VTOD method significantly outperforms the competing object detection methods.
Dense Multitask Learning to Reconfigure Comics

In this paper, we develop a MultiTask Learning (MTL) model to achieve dense predictions for comics panels to, in turn, facilitate the transfer of comics from one publication channel to another by assisting authors in the task of reconfiguring their narratives. Our MTL method can successfully identify the semantic units as well as the embedded notion of 3D in comics panels. This is a significantly challenging problem because comics comprise disparate artistic styles, illustrations, layouts, and object scales that depend on the author's creative process. Typically, dense image-based prediction techniques require a large corpus of data. Finding an automated solution for dense prediction in the comics domain, therefore, becomes more difficult with the lack of ground-truth dense annotations for the comics images. To address these challenges, we develop the following solutions- we leverage a commonly-used strategy known as unsupervised image-to-image translation, which allows us to utilize a large corpus of real-world annotations; - we utilize the results of the translations to develop our multitasking approach that is based on a vision transformer backbone and a domain transferable attention module; -we study the feasibility of integrating our MTL dense-prediction method with an existing retargeting method, thereby reconfiguring comics.
APPLeNet: Visual Attention Parameterized Prompt Learning for Few-Shot Remote Sensing Image Generalization using CLIP

In recent years, the success of large-scale vision-language models (VLMs) such as CLIP has led to their increased usage in various computer vision tasks. These models enable zero-shot inference through carefully crafted instructional text prompts without task-specific supervision. However, the potential of VLMs for generalization tasks in remote sensing (RS) has not been fully realized. To address this research gap, we propose a novel image-conditioned prompt learning strategy called the Visual Attention Parameterized Prompts Learning Network (APPLeNet). APPLeNet emphasizes the importance of multi-scale feature learning in RS scene classification and disentangles visual style and content primitives for domain generalization tasks. To achieve this, APPLeNet combines visual content features obtained from different layers of the vision encoder and style properties obtained from feature statistics of domain-specific batches. An attention-driven injection module is further introduced to generate visual tokens from this information. We also introduce an anti-correlation regularizer to ensure discrimination among the token embeddings, as this visual information is combined with the textual tokens. To validate APPLeNet, we curated four available RS benchmarks and introduced experimental protocols and datasets for three domain generalization tasks. Our results consistently outperform the relevant literature and code is available at https://github.com/mainaksingha01/APPLeNet
Adapting Grounded Visual Question Answering Models to Low Resource Languages

While huge progress has been made on a variety of vision and language tasks in recent years, most major advances have been restricted to the English language due to the scarcity of relevant training and evaluation datasets in other languages. A popular approach to address this gap, has been to utilize machine-translated multi-modal datasets or multi-lingual text-only datasets for pre-training. This approach not only fails to exploit existing pre-trained state-of-the-art English multi-modal models, but also is not a viable solution for low-resource languages where translation quality is not as reliable. Therefore, we propose xMDETR, a multi-lingual grounded vision-language model based on the state-of-the-art model MDETR, by adapting it to new languages without machine-translated data, while also keeping most of the pre-trained weights frozen. xMDETR leverages mono-lingual pre-trained MDETR to achieve results competitive to state of the art on xGQA, a standard multilingual VQA benchmark. It is also interpretable, providing bounding boxes for key phrases in the multi-lingual questions. Our method utilizes several architectural as well as data-driven techniques such as training a new embedding space with a Masked Language Modeling (MLM) objective, code-switching, and adapters for efficient and modular training. We also explore contrastive losses to enforce the bridging of multi-modal and multi-lingual representations on multi-lingual multi-modal data, when available. We evaluate xMDETR on xGQA in both zero-shot and few-shot settings, improving results on Portuguese, Indonesian and Bengali, while remaining competitive on other languages.
Derivation of Local Bifurcation Points in Autonomous Systems by Using Particle Swarm Optimization

This study extends the particle swarm optimization (PSO)-based bifurcation points derivation method to the autonomous systems. PSO avoids the need to carefully set the initial parameters of the dynamical system or to differentiate the objective function. In addition, it can derive the bifurcation points quickly and accurately, and its efficacy has previously been demonstrated for discrete dynamical systems and non-autonomous systems but not for autonomous systems. This paper proposes an extended PSO-based method for autonomous systems by incorporating the computation of a Poincaré map and a bisection method in the algorithm. To validate the effectiveness of the proposed method, it was applied to deriving the local bifurcation points of a three-dimensional autonomous system to confirm and demonstrate its effectiveness.
Solving the Steiner Tree Problem in Graphs with Chaotic Neural Networks

Chaotic neural networks (ChNNs) provide an effective method to solve combinatorial optimization problems, because their chaotic behavior is considered to encourage smooth escape from local optima. However, whether ChNN models exhibit chaotic behavior when searching for solutions remains unknown, which means there may be other reasons for their good performance. From this perspective, we analyzed the deterministic features of a chaotic time series from the transition of the objective function value. The results obtained by the E1, IDNP, and R series indicate that the transitions of the objective function value for solving the Steiner tree problem in graphs exhibited weak determinism, similar to that of the transition of a chaotic neuron’s internal state in a plain ChNN.
Accelerating the Run-Time of Convolutional Neural Networks through Weight Pruning and Quantization

Accelerating the processing of Convolutional Neural Networks (CNNs) is highly demand in the field of Artificial Intelligence (AI), particularly in computer vision domains. The efficiency of memory resources is crucial in measuring run-time, and weight pruning and quantization techniques have been studied extensively to optimize this efficiency. In this work, we investigate the contribution of these techniques to accelerate a pre-trained CNN model. We adapt the percentile-based weights pruning with focusing on unstructured pruning by dynamically adjusting the pruning thresholds based on the fine-tuning performance of the model. In the same context, we perform uniform quantization for presenting the weights values of the model’s parameters with a fixed number of bits. We implement different levels of post-training and aware-training -fine-tuning the model with the same learning rate and number of epochs as the original. We then refine-tune the model with a lower learning rate and a factor of 10x for both techniques. Finally, we combine the best levels of pruning and quantization and refine-tune the model to explore the best-pruned and quantized pre-trained model. We evaluate each level of the techniques and analyze their trade-offs. Our results demonstrate the effectiveness of our strategy in accelerating the CNN and improving its efficiency, and provide insights into the best combination of techniques to accelerate its inference time.
Explainable AI based Adversarial Examples and its Evaluation

With the advancement of AI technology, AI can be applied to various fields. Currently, AI is used for vital decision such as medical diagnosis. Therefore, the accountability for the decision by AI has become more important. The explainable AI (XAI) is a key technology to fulfill the accountability. While AI is spreading rapidly, the vulnerability of AI system is pointed out. The adversarial examples (AE), which causes wrong decisions by AI, is one of the terrible attacks for AI. For using AI safely, AE has to be investigated thoroughly. In order to clarify mechanism and affect of AE, this paper proposes the generating method for AE which is based on XAI. Experiments prove the proposed method is superior to previous AEs.
In Deep Security Management Strategy: Vulnerability Assessment Within Educational Institution

Technology is developing rapidly along with the times; one example of technological developments is the development of the use of websites in daily activities. Many institutions and entities have utilized the use of websites to support their business processes. For example, one of the faculties of XYZ University has used a website to help with administrative activities. One of the websites of the faculties at XYZ University is the final assignment proposal dashboard website which contains plots of the final assignment supervisor and the title of the final assignment. However, with the development of a technology, the development of vulnerabilities or attacks against the technology also increases. Therefore, it is necessary to carry out a vulnerability assessment method to be able to find out the vulnerabilities that exist on a website and also solutions that can be implemented to overcome these vulnerabilities. In this study, a vulnerability assessment will be carried out on the XYZ University students' final project proposal dashboard website using Nmap and Acunetix tools. The accuracy level of nmap is 13.25%, while Acunetix has 100% accuracy with the results obtained after the vulnerability assessment process, namely there are 12 vulnerabilities on the XYZ University student final project proposal dashboard website with Nmap detecting 3 medium risk vulnerabilities and 1 low risk vulnerability while Acunetix managed to detect 2 medium risk vulnerabilities and 6 low risk vulnerabilities