OpenRaiser/Intern-Atlas huggingface dataset:
Variational formulation and monolithic solution of computational homogenization methods
Venue: arXiv cs.CE 2023
Year: 2023

In this contribution, we derive a consistent variational formulation for computational homogenization methods and show that traditional FE2 and IGA2 approaches are special discretization and solution techniques of this most general framework. This allows us to enhance dramatically the numerical analysis as well as the solution of the arising algebraic system. In particular, we expand the dimension of the continuous system, discretize the higher dimensional problem consistently and apply afterwards a discrete null-space matrix to remove the additional dimensions. A benchmark problem, for which we can obtain an analytical solution, demonstrates the superiority of the chosen approach aiming to reduce the immense computational costs of traditional FE2 and IGA2 formulations to a fraction of the original requirements. Finally, we demonstrate a further reduction of the computational costs for the solution of general non-linear problems.
Realistic Rainy Weather Simulation for LiDARs in CARLA Simulator
Venue: arXiv cs.RO 2023
Year: 2023

Employing data augmentation methods to enhance perception performance in adverse weather has attracted considerable attention recently. Most of the LiDAR augmentation methods post-process the existing dataset by physics-based models or machine-learning methods. However, due to the limited environmental annotations and the fixed vehicle trajectories in the existing dataset, it is challenging to edit the scene and expand the diversity of traffic flow and scenario. To this end, we propose a simulator-based physical modeling approach to augment LiDAR data in rainy weather in order to improve the perception performance of LiDAR in this scenario. We complete the modeling task of the rainy weather in the CARLA simulator and establish a pipeline for LiDAR data collection. In particular, we pay special attention to the spray and splash rolled up by the wheels of surrounding vehicles in rain and complete the simulation of this special scenario through the Spray Emitter method we developed. In addition, we examine the influence of different weather conditions on the intensity of the LiDAR echo, develop a prediction network for the intensity of the LiDAR echo, and complete the simulation of 4-feat LiDAR point cloud data. In the experiment, we observe that the model augmented by the synthetic data improves the object detection task's performance in the rainy sequence of the Waymo Open Dataset. Both the code and the dataset will be made publicly available at https://github.com/PJLab-ADG/PCSim#rainypcsim.
DynaLay: An Introspective Approach to Dynamic Layer Selection for Deep Networks
Venue: arXiv cs.LG 2023
Year: 2023

Deep learning models have become increasingly computationally intensive, requiring extensive computational resources and time for both training and inference. A significant contributing factor to this challenge is the uniform computational effort expended on each input example, regardless of its complexity. We introduce \textbf{DynaLay}, an alternative architecture that features a decision-making agent to adaptively select the most suitable layers for processing each input, thereby endowing the model with a remarkable level of introspection. DynaLay reevaluates more complex inputs during inference, adjusting the computational effort to optimize both performance and efficiency. The core of the system is a main model equipped with Fixed-Point Iterative (FPI) layers, capable of accurately approximating complex functions, paired with an agent that chooses these layers or a direct action based on the introspection of the models inner state. The model invests more time in processing harder examples, while minimal computation is required for easier ones. This introspective approach is a step toward developing deep learning models that "think" and "ponder", rather than "ballistically'' produce answers. Our experiments demonstrate that DynaLay achieves accuracy comparable to conventional deep models while significantly reducing computational demands.
Enhancing Consistency in Multimodal Dialogue System Using LLM with Dialogue Scenario
Venue: arXiv cs.CL 2023
Year: 2023

This paper describes our dialogue system submitted to Dialogue Robot Competition 2023. The system's task is to help a user at a travel agency decide on a plan for visiting two sightseeing spots in Kyoto City that satisfy the user. Our dialogue system is flexible and stable and responds to user requirements by controlling dialogue flow according to dialogue scenarios. We also improved user satisfaction by introducing motion and speech control based on system utterances and user situations. In the preliminary round, our system was ranked fifth in the impression evaluation and sixth in the plan evaluation among all 12 teams.
Graph-Based Generalization of Galam Model: Convergence Time and Influential Nodes
Venue: arXiv cs.DS 2023
Year: 2023

We study a graph-based generalization of the Galam opinion formation model. Consider a simple connected graph which represents a social network. Each node in the graph is colored either blue or white, which indicates a positive or negative opinion on a new product or a topic. In each discrete-time round, all nodes are assigned randomly to groups of different sizes, where the node(s) in each group form a clique in the underlying graph. All the nodes simultaneously update their color to the majority color in their group. If there is a tie, each node in the group chooses one of the two colors uniformly at random. Investigating the convergence time of the model, our experiments show that the convergence time is a logarithm function of the number of nodes for a complete graph and a quadratic function for a cycle graph. We also study the various strategies for selecting a set of seed nodes to maximize the final cascade of one of the two colors, motivated by viral marketing. We consider the algorithms where the seed nodes are selected based on the graph structure (nodes' centrality measures such as degree, betweenness, and closeness) and the individual's characteristics (activeness and stubbornness). We provide a comparison of such strategies by conducting experiments on different real-world and synthetic networks.
Research on the Development of Blockchain-based Distributed Intelligent Healthcare Industry -- A Policy Analysis Perspective
Venue: arXiv cs.CY 2023
Year: 2023

As a pivotal innovation in digital infrastructure, blockchain ledger technology catalyzes the development of nascent business paradigms and applications globally. Utilizing Rothwell and Zegveld's taxonomy of twelve innovation policy tools, this study offers a nuanced comparison of domestic blockchain policies, dissecting supply, environment, and demand-driven policy dimensions to distill prevailing strategic orientations towards blockchain healthcare adoption. The findings indicate that blockchain technology has seen rapid growth in the healthcare industry. However, a certain misalignment exists between the corporate and policy layers in terms of supply and demand. While companies focus more on technological applications, existing policies are geared towards regulations and governance. Government emphasis lies on legal supervision through environmental policies, aiming to guide the standardization and regulation of blockchain technology. This maintains a balance between encouraging innovation and market and legal regulatory order, thereby providing a reference for the development of the distributed intelligent healthcare industry in our country.
DynamiQS: Quantum Secure Authentication for Dynamic Charging of Electric Vehicles
Venue: arXiv cs.CR 2023
Year: 2023

Dynamic Wireless Power Transfer (DWPT) is a novel technology that allows charging an electric vehicle while driving thanks to a dedicated road infrastructure. DWPT's capabilities in automatically establishing charging sessions and billing without users' intervention make it prone to cybersecurity attacks. Hence, security is essential in preventing fraud, impersonation, and user tracking. To this aim, researchers proposed different solutions for authenticating users. However, recent advancements in quantum computing jeopardize classical public key cryptography, making currently existing solutions in DWPT authentication nonviable. To avoid the resource burden imposed by technology upgrades, it is essential to develop post-quantum-resistant solutions. In this paper, we propose DynamiQS, the first post-quantum secure authentication protocol for dynamic wireless charging. DynamiQS is privacy-preserving and secure against attacks on the DWPT. We leverage an Identity-Based Encryption with Lattices in the Ring Learning With Error framework. Furthermore, we show the possibility of using DynamiQS in a real environment, leveraging the results of cryptographic computation on real constrained devices and simulations. DynamiQS reaches a total time cost of around 281 ms, which is practicable in dynamic charging settings (car and charging infrastructure).
BSL: Understanding and Improving Softmax Loss for Recommendation
Venue: arXiv cs.LG 2023
Year: 2023

Loss functions steer the optimization direction of recommendation models and are critical to model performance, but have received relatively little attention in recent recommendation research. Among various losses, we find Softmax loss (SL) stands out for not only achieving remarkable accuracy but also better robustness and fairness. Nevertheless, the current literature lacks a comprehensive explanation for the efficacy of SL. Toward addressing this research gap, we conduct theoretical analyses on SL and uncover three insights: 1) Optimizing SL is equivalent to performing Distributionally Robust Optimization (DRO) on the negative data, thereby learning against perturbations on the negative distribution and yielding robustness to noisy negatives. 2) Comparing with other loss functions, SL implicitly penalizes the prediction variance, resulting in a smaller gap between predicted values and and thus producing fairer results. Building on these insights, we further propose a novel loss function Bilateral SoftMax Loss (BSL) that extends the advantage of SL to both positive and negative sides. BSL augments SL by applying the same Log-Expectation-Exp structure to positive examples as is used for negatives, making the model robust to the noisy positives as well. Remarkably, BSL is simple and easy-to-implement -- requiring just one additional line of code compared to SL. Experiments on four real-world datasets and three representative backbones demonstrate the effectiveness of our proposal. The code is available at https://github.com/junkangwu/BSL
Secure Authentication Mechanism for Cluster based Vehicular Adhoc Network (VANET): A Survey
Venue: arXiv cs.CR 2023
Year: 2023

Vehicular Ad Hoc Networks (VANETs) play a crucial role in Intelligent Transportation Systems (ITS) by facilitating communication between vehicles and infrastructure. This communication aims to enhance road safety, improve traffic efficiency, and enhance passenger comfort. The secure and reliable exchange of information is paramount to ensure the integrity and confidentiality of data, while the authentication of vehicles and messages is essential to prevent unauthorized access and malicious activities. This survey paper presents a comprehensive analysis of existing authentication mechanisms proposed for cluster-based VANETs. The strengths, weaknesses, and suitability of these mechanisms for various scenarios are carefully examined. Additionally, the integration of secure key management techniques is discussed to enhance the overall authentication process. Cluster-based VANETs are formed by dividing the network into smaller groups or clusters, with designated cluster heads comprising one or more vehicles. Furthermore, this paper identifies gaps in the existing literature through an exploration of previous surveys. Several schemes based on different methods are critically evaluated, considering factors such as throughput, detection rate, security, packet delivery ratio, and end-to-end delay. To provide optimal solutions for authentication in cluster-based VANETs, this paper highlights AI- and ML-based routing-based schemes. These approaches leverage artificial intelligence and machine learning techniques to enhance authentication within the cluster-based VANET network. Finally, this paper explores the open research challenges that exist in the realm of authentication for cluster-based Vehicular Adhoc Networks, shedding light on areas that require further investigation and development.
Stability of Graph Convolutional Neural Networks through the lens of small perturbation analysis
Venue: arXiv cs.LG 2023
Year: 2023

In this work, we study the problem of stability of Graph Convolutional Neural Networks (GCNs) under random small perturbations in the underlying graph topology, i.e. under a limited number of insertions or deletions of edges. We derive a novel bound on the expected difference between the outputs of unperturbed and perturbed GCNs. The proposed bound explicitly depends on the magnitude of the perturbation of the eigenpairs of the Laplacian matrix, and the perturbation explicitly depends on which edges are inserted or deleted. Then, we provide a quantitative characterization of the effect of perturbing specific edges on the stability of the network. We leverage tools from small perturbation analysis to express the bounds in closed, albeit approximate, form, in order to enhance interpretability of the results, without the need to compute any perturbed shift operator. Finally, we numerically evaluate the effectiveness of the proposed bound.
Misclassification excess risk bounds for 1-bit matrix completion
Venue: arXiv cs.LG 2023
Year: 2023

This study investigates the misclassification excess risk bound in the context of 1-bit matrix completion, a significant problem in machine learning involving the recovery of an unknown matrix from a limited subset of its entries. Matrix completion has garnered considerable attention in the last two decades due to its diverse applications across various fields. Unlike conventional approaches that deal with real-valued samples, 1-bit matrix completion is concerned with binary observations. While prior research has predominantly focused on the estimation error of proposed estimators, our study shifts attention to the prediction error. This paper offers theoretical analysis regarding the prediction errors of two previous works utilizing the logistic regression model: one employing a max-norm constrained minimization and the other employing nuclear-norm penalization. Significantly, our findings demonstrate that the latter achieves the minimax-optimal rate without the need for an additional logarithmic term. These novel results contribute to a deeper understanding of 1-bit matrix completion by shedding light on the predictive performance of specific methodologies.
TADAP: Trajectory-Aided Drivable area Auto-labeling with Pre-trained self-supervised features in winter driving conditions
Venue: arXiv cs.CV 2023
Year: 2023

Detection of the drivable area in all conditions is crucial for autonomous driving and advanced driver assistance systems. However, the amount of labeled data in adverse driving conditions is limited, especially in winter, and supervised methods generalize poorly to conditions outside the training distribution. For easy adaption to all conditions, the need for human annotation should be removed from the learning process. In this paper, Trajectory-Aided Drivable area Auto-labeling with Pre-trained self-supervised features (TADAP) is presented for automated annotation of the drivable area in winter driving conditions. A sample of the drivable area is extracted based on the trajectory estimate from the global navigation satellite system. Similarity with the sample area is determined based on pre-trained self-supervised visual features. Image areas similar to the sample area are considered to be drivable. These TADAP labels were evaluated with a novel winter-driving dataset, collected in varying driving scenes. A prediction model trained with the TADAP labels achieved a +9.6 improvement in intersection over union compared to the previous state-of-the-art of self-supervised drivable area detection.
Symbolic Security Verification of Mesh Commissioning Protocol in Thread (extended version)
Venue: arXiv cs.CR 2023
Year: 2023

The Thread protocol (or simply Thread ) is a popular networking protocol for the Internet of Things (IoT). It allows seamless integration of a set of applications and protocols, hence reducing the risk of incompatibility among different applications or user protocols. Thread has been deployed in many popular smart home products by the majority of IoT manufacturers, such as Apple TV, Apple HomePod mini, eero 6, Nest Hub, and Nest Wifi. Despite a few empirical analyses on the security of Thread, there is still a lack of formal analysis on this infrastructure of the booming IoT ecosystem. In this work, we performed a formal symbolic analysis of the security properties of Thread. Our main focus is on MeshCoP (Mesh Commissioning Protocol), the main subprotocol in Thread for secure authentication and commissioning of new, untrusted devices inside an existing Thread network. This case study presents the challenges and proposed solutions in modeling MeshCoP. We use ProVerif, a symbolic verification tool of π-calculus models, for verifying the security properties of MeshCoP.
From Past to Future: Rethinking Eligibility Traces
Venue: arXiv cs.LG 2023
Year: 2023

In this paper, we introduce a fresh perspective on the challenges of credit assignment and policy evaluation. First, we delve into the nuances of eligibility traces and explore instances where their updates may result in unexpected credit assignment to preceding states. From this investigation emerges the concept of a novel value function, which we refer to as the \emph{bidirectional value function}. Unlike traditional state value functions, bidirectional value functions account for both future expected returns (rewards anticipated from the current state onward) and past expected returns (cumulative rewards from the episode's start to the present). We derive principled update equations to learn this value function and, through experimentation, demonstrate its efficacy in enhancing the process of policy evaluation. In particular, our results indicate that the proposed learning approach can, in certain challenging contexts, perform policy evaluation more rapidly than TD($λ$) -- a method that learns forward value functions, $v^π$, \emph{directly}. Overall, our findings present a new perspective on eligibility traces and potential advantages associated with the novel value function it inspires, especially for policy evaluation.
Accelerator-driven Data Arrangement to Minimize Transformers Run-time on Multi-core Architectures
Venue: arXiv cs.AR 2023
Year: 2023

The increasing complexity of transformer models in artificial intelligence expands their computational costs, memory usage, and energy consumption. Hardware acceleration tackles the ensuing challenges by designing processors and accelerators tailored for transformer models, supporting their computation hotspots with high efficiency. However, memory bandwidth can hinder improvements in hardware accelerators. Against this backdrop, in this paper we propose a novel memory arrangement strategy, governed by the hardware accelerator's kernel size, which effectively minimizes off-chip data access. This arrangement is particularly beneficial for end-to-end transformer model inference, where most of the computation is based on general matrix multiplication (GEMM) operations. Additionally, we address the overhead of non-GEMM operations in transformer models within the scope of this memory data arrangement. Our study explores the implementation and effectiveness of the proposed accelerator-driven data arrangement approach in both single- and multi-core systems. Our evaluation demonstrates that our approach can achieve up to a 2.8x speed increase when executing inferences employing state-of-the-art transformers.
AutoXPCR: Automated Multi-Objective Model Selection for Time Series Forecasting
Venue: arXiv cs.LG 2023
Year: 2023

Automated machine learning (AutoML) streamlines the creation of ML models. While most methods select the "best" model based on predictive quality, it's crucial to acknowledge other aspects, such as interpretability and resource consumption. This holds particular importance in the context of deep neural networks (DNNs), as these models are often perceived as computationally intensive black boxes. In the challenging domain of time series forecasting, DNNs achieve stunning results, but specialized approaches for automatically selecting models are scarce. In this paper, we propose AutoXPCR - a novel method for automated and explainable multi-objective model selection. Our approach leverages meta-learning to estimate any model's performance along PCR criteria, which encompass (P)redictive error, (C)omplexity, and (R)esource demand. Explainability is addressed on multiple levels, as our interactive framework can prioritize less complex models and provide by-product explanations of recommendations. We demonstrate practical feasibility by deploying AutoXPCR on over 1000 configurations across 114 data sets from various domains. Our method clearly outperforms other model selection approaches - on average, it only requires 20% of computation costs for recommending models with 90% of the best-possible quality.
Advancing SQL Injection Detection for High-Speed Data Centers: A Novel Approach Using Cascaded NLP
Venue: arXiv cs.CR 2023
Year: 2023

Detecting SQL Injection (SQLi) attacks is crucial for web-based data center security, but it is challenging to balance accuracy and computational efficiency, especially in high-speed networks. Traditional methods struggle with this balance, while NLP-based approaches, although accurate, are computationally intensive. We introduce a novel cascade SQLi detection method, blending classical and transformer-based NLP models, achieving a 99.86% detection accuracy with significantly lower computational demands-20 times faster than using transformer-based models alone. Our approach is tested in a realistic setting and compared with 35 other methods, including Machine Learning-based and transformer models like BERT, on a dataset of over 30,000 SQL sentences. Our results show that this hybrid method effectively detects SQLi in high-traffic environments, offering efficient and accurate protection against SQLi vulnerabilities with computational efficiency. The code is available at https://github.com/gdrlab/cascaded-sqli-detection .