OpenRaiser/Intern-Atlas huggingface dataset:
Task-customized Masked AutoEncoder via Mixture of Cluster-conditional Experts
Venue: arXiv cs.CV 2024
Year: 2024

Masked Autoencoder~(MAE) is a prevailing self-supervised learning method that achieves promising results in model pre-training. However, when the various downstream tasks have data distributions different from the pre-training data, the semantically irrelevant pre-training information might result in negative transfer, impeding MAE's scalability. To address this issue, we propose a novel MAE-based pre-training paradigm, Mixture of Cluster-conditional Experts (MoCE), which can be trained once but provides customized pre-training models for diverse downstream tasks. Different from the mixture of experts (MoE), our MoCE trains each expert only with semantically relevant images by using cluster-conditional gates. Thus, each downstream task can be allocated to its customized model pre-trained with data most similar to the downstream data. Experiments on a collection of 11 downstream tasks show that MoCE outperforms the vanilla MAE by 2.45\% on average. It also obtains new state-of-the-art self-supervised learning results on detection and segmentation.
Optimizing for ROC Curves on Class-Imbalanced Data by Training over a Family of Loss Functions
Venue: arXiv cs.LG 2024
Year: 2024

Although binary classification is a well-studied problem in computer vision, training reliable classifiers under severe class imbalance remains a challenging problem. Recent work has proposed techniques that mitigate the effects of training under imbalance by modifying the loss functions or optimization methods. While this work has led to significant improvements in the overall accuracy in the multi-class case, we observe that slight changes in hyperparameter values of these methods can result in highly variable performance in terms of Receiver Operating Characteristic (ROC) curves on binary problems with severe imbalance. To reduce the sensitivity to hyperparameter choices and train more general models, we propose training over a family of loss functions, instead of a single loss function. We develop a method for applying Loss Conditional Training (LCT) to an imbalanced classification problem. Extensive experiment results, on both CIFAR and Kaggle competition datasets, show that our method improves model performance and is more robust to hyperparameter choices. Code is available at https://github.com/klieberman/roc_lct.
A State-of-the-art Survey on Full-duplex Network Design
Venue: arXiv cs.NI 2024
Year: 2024

Full-duplex (FD) technology is gaining popularity for integration into a wide range of wireless networks due to its demonstrated potential in recent studies. In contrast to half-duplex (HD) technology, the implementation of FD in networks necessitates considering inter-node interference (INI) from various network perspectives. When deploying FD technology in networks, several critical factors must be taken into account. These include self-interference (SI) and the requisite SI cancellation (SIC) processes, as well as the selection of multiple user equipment (UE) per time slot. Additionally, inter-node interference (INI), including cross-link interference (CLI) and inter-cell interference (ICI), become crucial issues during concurrent uplink (UL) and downlink (DL) transmission and reception, similar to SI. Since most INI is challenging to eliminate, a comprehensive investigation that covers radio resource control (RRC), medium access control (MAC), and the physical layer (PHY) is essential in the context of FD network design, rather than focusing on individual network layers and types. This paper covers state-of-the-art studies, including protocols and documents from 3GPP for FD, MAC protocol, user scheduling, and CLI handling. The methods are also compared through a network-level system simulation based on 3D ray-tracing.
Version age-based client scheduling policy for federated learning
Venue: arXiv cs.LG 2024
Year: 2024

Federated Learning (FL) has emerged as a privacy-preserving machine learning paradigm facilitating collaborative training across multiple clients without sharing local data. Despite advancements in edge device capabilities, communication bottlenecks present challenges in aggregating a large number of clients; only a portion of the clients can update their parameters upon each global aggregation. This phenomenon introduces the critical challenge of stragglers in FL and the profound impact of client scheduling policies on global model convergence and stability. Existing scheduling strategies address staleness but predominantly focus on either timeliness or content. Motivated by this, we introduce the novel concept of Version Age of Information (VAoI) to FL. Unlike traditional Age of Information metrics, VAoI considers both timeliness and content staleness. Each client's version age is updated discretely, indicating the freshness of information. VAoI is incorporated into the client scheduling policy to minimize the average VAoI, mitigating the impact of outdated local updates and enhancing the stability of FL systems.
SpirDet: Towards Efficient, Accurate and Lightweight Infrared Small Target Detector
Venue: arXiv cs.CV 2024
Year: 2024

In recent years, the detection of infrared small targets using deep learning methods has garnered substantial attention due to notable advancements. To improve the detection capability of small targets, these methods commonly maintain a pathway that preserves high-resolution features of sparse and tiny targets. However, it can result in redundant and expensive computations. To tackle this challenge, we propose SpirDet, a novel approach for efficient detection of infrared small targets. Specifically, to cope with the computational redundancy issue, we employ a new dual-branch sparse decoder to restore the feature map. Firstly, the fast branch directly predicts a sparse map indicating potential small target locations (occupying only 0.5\% area of the map). Secondly, the slow branch conducts fine-grained adjustments at the positions indicated by the sparse map. Additionally, we design an lightweight DO-RepEncoder based on reparameterization with the Downsampling Orthogonality, which can effectively reduce memory consumption and inference latency. Extensive experiments show that the proposed SpirDet significantly outperforms state-of-the-art models while achieving faster inference speed and fewer parameters. For example, on the IRSTD-1K dataset, SpirDet improves $MIoU$ by 4.7 and has a $7\times$ $FPS$ acceleration compared to the previous state-of-the-art model. The code will be open to the public.
Segmentation-free Connectionist Temporal Classification loss based OCR Model for Text Captcha Classification
Venue: arXiv cs.CV 2024
Year: 2024

Captcha are widely used to secure systems from automatic responses by distinguishing computer responses from human responses. Text, audio, video, picture picture-based Optical Character Recognition (OCR) are used for creating captcha. Text-based OCR captcha are the most often used captcha which faces issues namely, complex and distorted contents. There are attempts to build captcha detection and classification-based systems using machine learning and neural networks, which need to be tuned for accuracy. The existing systems face challenges in the recognition of distorted characters, handling variable-length captcha and finding sequential dependencies in captcha. In this work, we propose a segmentation-free OCR model for text captcha classification based on the connectionist temporal classification loss technique. The proposed model is trained and tested on a publicly available captcha dataset. The proposed model gives 99.80\% character level accuracy, while 95\% word level accuracy. The accuracy of the proposed model is compared with the state-of-the-art models and proves to be effective. The variable length complex captcha can be thus processed with the segmentation-free connectionist temporal classification loss technique with dependencies which will be massively used in securing the software systems.
Neural Circuit Diagrams: Robust Diagrams for the Communication, Implementation, and Analysis of Deep Learning Architectures
Venue: arXiv cs.LG 2024
Year: 2024

Diagrams matter. Unfortunately, the deep learning community has no standard method for diagramming architectures. The current combination of linear algebra notation and ad-hoc diagrams fails to offer the necessary precision to understand architectures in all their detail. However, this detail is critical for faithful implementation, mathematical analysis, further innovation, and ethical assurances. I present neural circuit diagrams, a graphical language tailored to the needs of communicating deep learning architectures. Neural circuit diagrams naturally keep track of the changing arrangement of data, precisely show how operations are broadcast over axes, and display the critical parallel behavior of linear operations. A lingering issue with existing diagramming methods is the inability to simultaneously express the detail of axes and the free arrangement of data, which neural circuit diagrams solve. Their compositional structure is analogous to code, creating a close correspondence between diagrams and implementation. In this work, I introduce neural circuit diagrams for an audience of machine learning researchers. After introducing neural circuit diagrams, I cover a host of architectures to show their utility and breed familiarity. This includes the transformer architecture, convolution (and its difficult-to-explain extensions), residual networks, the U-Net, and the vision transformer. I include a Jupyter notebook that provides evidence for the close correspondence between diagrams and code. Finally, I examine backpropagation using neural circuit diagrams. I show their utility in providing mathematical insight and analyzing algorithms' time and space complexities.
GPT-4 Generated Narratives of Life Events using a Structured Narrative Prompt: A Validation Study
Venue: arXiv cs.CL 2024
Year: 2024

Large Language Models (LLMs) play a pivotal role in generating vast arrays of narratives, facilitating a systematic exploration of their effectiveness for communicating life events in narrative form. In this study, we employ a zero-shot structured narrative prompt to generate 24,000 narratives using OpenAI's GPT-4. From this dataset, we manually classify 2,880 narratives and evaluate their validity in conveying birth, death, hiring, and firing events. Remarkably, 87.43% of the narratives sufficiently convey the intention of the structured prompt. To automate the identification of valid and invalid narratives, we train and validate nine Machine Learning models on the classified datasets. Leveraging these models, we extend our analysis to predict the classifications of the remaining 21,120 narratives. All the ML models excelled at classifying valid narratives as valid, but experienced challenges at simultaneously classifying invalid narratives as invalid. Our findings not only advance the study of LLM capabilities, limitations, and validity but also offer practical insights for narrative generation and natural language processing applications.
Low-degree phase transitions for detecting a planted clique in sublinear time
Venue: arXiv cs.DS 2024
Year: 2024

We consider the problem of detecting a planted clique of size $k$ in a random graph on $n$ vertices. When the size of the clique exceeds $Θ(\sqrt{n})$, polynomial-time algorithms for detection proliferate. We study faster -- namely, sublinear time -- algorithms in the high-signal regime when $k = Θ(n^{1/2 + δ})$, for some $δ> 0$. To this end, we consider algorithms that non-adaptively query a subset $M$ of entries of the adjacency matrix and then compute a low-degree polynomial function of the revealed entries. We prove a computational phase transition for this class of non-adaptive low-degree algorithms: under the scaling $\lvert M \rvert = Θ(n^γ)$, the clique can be detected when $γ> 3(1/2 - δ)$ but not when $γ< 3(1/2 - δ)$. As a result, the best known runtime for detecting a planted clique, $\widetilde{O}(n^{3(1/2-δ)})$, cannot be improved without looking beyond the non-adaptive low-degree class. Our proof of the lower bound -- based on bounding the conditional low-degree likelihood ratio -- reveals further structure in non-adaptive detection of a planted clique. Using (a bound on) the conditional low-degree likelihood ratio as a potential function, we show that for every non-adaptive query pattern, there is a highly structured query pattern of the same size that is at least as effective.
Kontextbasierte Aktivitätserkennung -- Synergie von Mensch und Technik in der Social Networked Industry
Venue: arXiv cs.HC 2024
Year: 2024

In a social networked industry, the focus is on collaboration between humans and technology. Communication is the basic prerequisite for synergetic collaboration between all players. It includes non-verbal as well as verbal interactions. To enable non-verbal interaction, machines must be able to detect and understand human movements. This article presents the ongoing fundamental research on the analysis of human movements using sensor-based activity recognition and identifies potential for a transfer to industrial applications. The focus is on the practical feasibility of activity recognition by adding further data streams such as the position data of logistical objects and tools, meaning the context in which a certain activity is carried out. -- In der Social Networked Industry steht die Zusammenarbeit von Mensch und Technik im Vordergrund. Grundvoraussetzung für eine synergetische Zusammenarbeit aller Akteure ist die Kommunikation, welche neben verbalen auch nonverbale Interaktionen umfasst. Um eine nonverbale Interaktion zu ermöglichen, müssen Maschinen in der Lage sein, menschliche Bewegungen zu erfassen und zu verstehen. Dieser Beitrag stellt die laufende Grundlagenforschung zur Analyse menschlicher Bewegungen mittels sensorgestützter Aktivitätserkennung vor und zeigt Anknüpfungspunkte für einen Transfer in industrielle Anwendungen. Im Fokus steht die Praxistauglichkeit der Aktivitätserkennung durch die Hinzunahme weiterer Datenströme wie beispielsweise den Positionsdaten logistischer Objekte und Hilfsmitteln, d. h. dem Kontext, in dem eine gewisse Aktivität ausgeführt wird.
Investigating White-Box Attacks for On-Device Models
Venue: arXiv cs.SE 2024
Year: 2024

Numerous mobile apps have leveraged deep learning capabilities. However, on-device models are vulnerable to attacks as they can be easily extracted from their corresponding mobile apps. Existing on-device attacking approaches only generate black-box attacks, which are far less effective and efficient than white-box strategies. This is because mobile deep learning frameworks like TFLite do not support gradient computing, which is necessary for white-box attacking algorithms. Thus, we argue that existing findings may underestimate the harmfulness of on-device attacks. To this end, we conduct a study to answer this research question: Can on-device models be directly attacked via white-box strategies? We first systematically analyze the difficulties of transforming the on-device model to its debuggable version, and propose a Reverse Engineering framework for On-device Models (REOM), which automatically reverses the compiled on-device TFLite model to the debuggable model. Specifically, REOM first transforms compiled on-device models into Open Neural Network Exchange format, then removes the non-debuggable parts, and converts them to the debuggable DL models format that allows attackers to exploit in a white-box setting. Our experimental results show that our approach is effective in achieving automated transformation among 244 TFLite models. Compared with previous attacks using surrogate models, REOM enables attackers to achieve higher attack success rates with a hundred times smaller attack perturbations. In addition, because the ONNX platform has plenty of tools for model format exchanging, the proposed method based on the ONNX platform can be adapted to other model formats. Our findings emphasize the need for developers to carefully consider their model deployment strategies, and use white-box methods to evaluate the vulnerability of on-device models.
Sustainable allocation of greenhouse gas emission permits for firms with Leontief technologies
Venue: arXiv cs.GT 2024
Year: 2024

In this paper we deal with production situations where a cap or limit to the amount of greenhouse gas emissions permitted is imposed. Fixing a tax for each ton of pollutant emitted is also considered. We use bankruptcy rules to define cooperative games with externalities associated with these situations and analyze the existence of coalitionally stable allocations of the emission permits. We prove that the constrained equal awards ( CEA ) rule provides stable allocations and as a direct mechanism, it is incentive compatible. These two facts have interesting managerial implications to control pollution emissions.
An Optimal Control Formulation of Tool Affordance Applied to Impact Tasks
Venue: arXiv cs.RO 2024
Year: 2024

Humans use tools to complete impact-aware tasks such as hammering a nail or playing tennis. The postures adopted to use these tools can significantly influence the performance of these tasks, where the force or velocity of the hand holding a tool plays a crucial role. The underlying motion planning challenge consists of grabbing the tool in preparation for the use of this tool with an optimal body posture. Directional manipulability describes the dexterity of force and velocity in a joint configuration along a specific direction. In order to take directional manipulability and tool affordances into account, we apply an optimal control method combining iterative linear quadratic regulator(iLQR) with the alternating direction method of multipliers(ADMM). Our approach considers the notion of tool affordances to solve motion planning problems, by introducing a cost based on directional velocity manipulability. The proposed approach is applied to impact tasks in simulation and on a real 7-axis robot, specifically in a nail-hammering task with the assistance of a pilot hole. Our comparison study demonstrates the importance of maximizing directional manipulability in impact-aware tasks.
NoisyICL: A Little Noise in Model Parameters Calibrates In-context Learning
Venue: arXiv cs.CL 2024
Year: 2024

In-Context Learning (ICL) is suffering from unsatisfactory performance and under-calibration due to high prior bias and unfaithful confidence. Some previous works fine-tuned language models for better ICL performance with enormous datasets and computing costs. In this paper, we propose NoisyICL, simply perturbing the model parameters by random noises to strive for better performance and calibration. Our experiments on two models and 12 downstream datasets show that NoisyICL can help ICL produce more accurate predictions. Our further analysis indicates that NoisyICL enables the model to provide more fair predictions, and also with more faithful confidence. Therefore, we believe that NoisyICL is an effective calibration of ICL. Our experimental code is uploaded to Github.
Buffer Overflow in Mixture of Experts
Venue: arXiv cs.CR 2024
Year: 2024

Mixture of Experts (MoE) has become a key ingredient for scaling large foundation models while keeping inference costs steady. We show that expert routing strategies that have cross-batch dependencies are vulnerable to attacks. Malicious queries can be sent to a model and can affect a model's output on other benign queries if they are grouped in the same batch. We demonstrate this via a proof-of-concept attack in a toy experimental setting.
NCRF: Neural Contact Radiance Fields for Free-Viewpoint Rendering of Hand-Object Interaction
Venue: arXiv cs.CV 2024
Year: 2024

Modeling hand-object interactions is a fundamentally challenging task in 3D computer vision. Despite remarkable progress that has been achieved in this field, existing methods still fail to synthesize the hand-object interaction photo-realistically, suffering from degraded rendering quality caused by the heavy mutual occlusions between the hand and the object, and inaccurate hand-object pose estimation. To tackle these challenges, we present a novel free-viewpoint rendering framework, Neural Contact Radiance Field (NCRF), to reconstruct hand-object interactions from a sparse set of videos. In particular, the proposed NCRF framework consists of two key components: (a) A contact optimization field that predicts an accurate contact field from 3D query points for achieving desirable contact between the hand and the object. (b) A hand-object neural radiance field to learn an implicit hand-object representation in a static canonical space, in concert with the specifically designed hand-object motion field to produce observation-to-canonical correspondences. We jointly learn these key components where they mutually help and regularize each other with visual and geometric constraints, producing a high-quality hand-object reconstruction that achieves photo-realistic novel view synthesis. Extensive experiments on HO3D and DexYCB datasets show that our approach outperforms the current state-of-the-art in terms of both rendering quality and pose estimation accuracy.
Robust Parameter Fitting to Realistic Network Models via Iterative Stochastic Approximation
Venue: arXiv cs.SI 2024
Year: 2024

Random graph models are widely used to understand network properties and graph algorithms. Key to such analyses are the different parameters of each model, which affect various network features, such as its size, clustering, or degree distribution. The exact effect of the parameters on these features is not well understood, mainly because we lack tools to thoroughly investigate this relation. Moreover, the parameters cannot be considered in isolation, as changing one affects multiple features. Existing approaches for finding the best model parameters of desired features, such as a grid search or estimating the parameter-feature relations, are not well suited, as they are inaccurate or computationally expensive. We introduce an efficient iterative fitting method, named ParFit, that finds parameters using only a few network samples, based on the Robbins-Monro algorithm. We test ParFit on three well-known graph models, namely Erdős-Rényi, Chung-Lu, and geometric inhomogeneous random graphs, as well as on real-world networks, including web networks. We find that ParFit performs well in terms of quality and running time across most parameter configurations.
Automatizing Software Cognitive Complexity Reduction through Integer Linear Programming
Venue: arXiv cs.SE 2024
Year: 2024

Reducing the cognitive complexity of a piece of code to a given threshold is not trivial. Recently, we modeled software cognitive complexity reduction as an optimization problem and we proposed an approach to assist developers on this task. This approach enumerates sequences of code extraction refactoring operations until a stopping criterion is met. As a result, it returns the minimal sequence of code extraction refactoring operations that is able to reduce the cognitive complexity of a code to the given threshold. However, exhaustive enumeration algorithms fail to scale with the code size. The number of refactoring plans can grow exponentially with the number of lines of code. In this paper, instead of enumerating sequences of code extraction refactoring operations, we model the cognitive complexity reduction as an Integer Linear Programming problem. This opens the door to the use of efficient solvers to find optimal solutions in large programs.