OpenRaiser/Intern-Atlas huggingface dataset:
Revisiting Lightweight Compiler Provenance Recovery on ARM Binaries
Venue: arXiv cs.LG 2023
Year: 2023

A binary's behavior is greatly influenced by how the compiler builds its source code. Although most compiler configuration details are abstracted away during compilation, recovering them is useful for reverse engineering and program comprehension tasks on unknown binaries, such as code similarity detection. We observe that previous work has thoroughly explored this on x86-64 binaries. However, there has been limited investigation of ARM binaries, which are increasingly prevalent. In this paper, we extend previous work with a shallow-learning model that efficiently and accurately recovers compiler configuration properties for ARM binaries. We apply opcode and register-derived features, that have previously been effective on x86-64 binaries, to ARM binaries. Furthermore, we compare this work with Pizzolotto et al., a recent architecture-agnostic model that uses deep learning, whose dataset and code are available. We observe that the lightweight features are reproducible on ARM binaries. We achieve over 99% accuracy, on par with state-of-the-art deep learning approaches, while achieving a 583-times speedup during training and 3,826-times speedup during inference. Finally, we also discuss findings of overfitting that was previously undetected in prior work.
Residual Prompt Tuning: Improving Prompt Tuning with Residual Reparameterization
Venue: arXiv cs.CL 2023
Year: 2023

Prompt tuning is one of the successful approaches for parameter-efficient tuning of pre-trained language models. Despite being arguably the most parameter-efficient (tuned soft prompts constitute <0.1% of total parameters), it typically performs worse than other efficient tuning methods and is quite sensitive to hyper-parameters. In this work, we introduce Residual Prompt Tuning - a simple and efficient method that significantly improves the performance and stability of prompt tuning. We propose to reparameterize soft prompt embeddings using a shallow network with a residual connection. Our experiments show that Residual Prompt Tuning significantly outperforms prompt tuning on SuperGLUE benchmark. Notably, our method reaches +7 points improvement over prompt tuning with T5-Base and allows to reduce the prompt length by 10x without hurting performance. In addition, we show that our approach is robust to the choice of learning rate and prompt initialization, and is effective in few-shot settings.
An Improved PTAS for Covering Targets with Mobile Sensors
Venue: arXiv cs.CG 2023
Year: 2023

This paper considers a movement minimization problem for mobile sensors. Given a set of $n$ point targets, the $k$-Sink Minimum Movement Target Coverage Problem is to schedule mobile sensors, initially located at $k$ base stations, to cover all targets minimizing the total moving distance of the sensors. We present a polynomial-time approximation scheme for finding a $(1+ε)$ approximate solution running in time $n^{O(1/ε)}$ for this problem when $k$, the number of base stations, is constant. Our algorithm improves the running time exponentially from the previous work that runs in time $n^{O(1/ε^2)}$, without any target distribution assumption. To devise a faster algorithm, we prove a stronger bound on the number of sensors in any unit area in the optimal solution and employ a more refined dynamic programming algorithm whose complexity depends only on the width of the problem.
Cross-domain Augmentation Networks for Click-Through Rate Prediction
Venue: arXiv cs.IR 2023
Year: 2023

Data sparsity is an important issue for click-through rate (CTR) prediction, particularly when user-item interactions is too sparse to learn a reliable model. Recently, many works on cross-domain CTR (CDCTR) prediction have been developed in an effort to leverage meaningful data from a related domain. However, most existing CDCTR works have an impractical limitation that requires homogeneous inputs (\textit{i.e.} shared feature fields) across domains, and CDCTR with heterogeneous inputs (\textit{i.e.} varying feature fields) across domains has not been widely explored but is an urgent and important research problem. In this work, we propose a cross-domain augmentation network (CDAnet) being able to perform knowledge transfer between two domains with \textit{heterogeneous inputs}. Specifically, CDAnet contains a designed translation network and an augmentation network which are trained sequentially. The translation network is able to compute features from two domains with heterogeneous inputs separately by designing two independent branches, and then learn meaningful cross-domain knowledge using a designed cross-supervised feature translator. Later the augmentation network encodes the learned cross-domain knowledge via feature translation performed in the latent space and fine-tune the model for final CTR prediction. Through extensive experiments on two public benchmarks and one industrial production dataset, we show CDAnet can learn meaningful translated features and largely improve the performance of CTR prediction. CDAnet has been conducted online A/B test in image2product retrieval at Taobao app over 20days, bringing an absolute \textbf{0.11 point} CTR improvement and a relative \textbf{1.26\%} GMV increase.
Pitch Estimation by Denoising Preprocessor and Hybrid Estimation Model
Venue: arXiv cs.SD 2023
Year: 2023

Pitch estimation is to estimate the fundamental frequency and the midi number and plays a critical role in music signal analysis and vocal signal processing. In this work, we proposed a new architecture based on a learning-based enhancement preprocessor and a combination of several traditional and deep learning pitch estimation methods to achieve better pitch estimation performance in both noisy and clean scenarios. We test 17 different types of noise and 4 SNRdb noise levels. The results show that the proposed pitch estimation can perform better in both noisy and clean scenarios with short response time.
Embedding bifurcations into pneumatic artificial muscle
Venue: arXiv cs.RO 2023
Year: 2023

Harnessing complex body dynamics has been a long-standing challenge in robotics. Soft body dynamics is a typical example of high complexity in interacting with the environment. An increasing number of studies have reported that these dynamics can be used as a computational resource. This includes the McKibben pneumatic artificial muscle, which is a typical soft actuator. This study demonstrated that various dynamics, including periodic and chaotic dynamics, could be embedded into the pneumatic artificial muscle, with the entire bifurcation structure using the framework of physical reservoir computing. These results suggest that dynamics that are not presented in training data could be embedded by using this capability of bifurcation embeddment. This implies that it is possible to embed various qualitatively different patterns into pneumatic artificial muscle by learning specific patterns, without the need to design and learn all patterns required for the purpose. Thus, this study sheds new light on a novel pathway to simplify the robotic devices and training of the control by reducing the external pattern generators and the amount and types of training data for the control.
LibAM: An Area Matching Framework for Detecting Third-party Libraries in Binaries
Venue: arXiv cs.SE 2023
Year: 2023

Third-party libraries (TPLs) are extensively utilized by developers to expedite the software development process and incorporate external functionalities. Nevertheless, insecure TPL reuse can lead to significant security risks. Existing methods are employed to determine the presence of TPL code in the target binary. Existing methods, which involve extracting strings or conducting function matching, are employed to determine the presence of TPL code in the target binary. However, these methods often yield unsatisfactory results due to the recurrence of strings and the presence of numerous similar non-homologous functions. Additionally, they struggle to identify specific pieces of reused code in the target binary, complicating the detection of complex reuse relationships and impeding downstream tasks. In this paper, we observe that TPL reuse typically involves not just isolated functions but also areas encompassing several adjacent functions on the Function Call Graph (FCG). We introduce LibAM, a novel Area Matching framework that connects isolated functions into function areas on FCG and detects TPLs by comparing the similarity of these function areas. Furthermore, LibAM is the first approach capable of detecting the exact reuse areas on FCG and offering substantial benefits for downstream tasks. Experimental results demonstrate that LibAM outperforms all existing TPL detection methods and provides interpretable evidence for TPL detection results by identifying exact reuse areas. We also evaluate LibAM's accuracy on large-scale, real-world binaries in IoT firmware and generate a list of potential vulnerabilities for these devices. Last but not least, by analyzing the detection results of IoT firmware, we make several interesting findings, such as different target binaries always tend to reuse the same code area of TPL.
ToolCoder: Teach Code Generation Models to use API search tools
Venue: arXiv cs.SE 2023
Year: 2023

Automatically generating source code from natural language descriptions has been a growing field of research in recent years. However, current large-scale code generation models often encounter difficulties when selecting appropriate APIs for specific contexts. These models may generate APIs that do not meet requirements or refer to non-existent APIs in third-party libraries, especially for lesser-known or private libraries. Inspired by the process of human developers using tools to search APIs, we propose ToolCoder, a novel approach that integrates API search tools with existing models to assist in code generation and API selection. To teach our model to use tools, we introduce an automated data annotation method using ChatGPT to add tool usage information into the source code data and fine-tune code generation models. During inference, we integrate API search tools into the generation process so that our model can automatically use the search tool to get suggestions when selecting an API. Our experimental results demonstrate that ToolCoder exhibits excellent performance and generalization across five public and private library code generation benchmarks, with at least 6.21\% improvement on average pass@1 metrics and 9.64\% improvement on average pass@10 metrics compared to state-of-the-art methods. Furthermore, we show that our relatively small ToolCoder model is comparable to one of the current best models, GPT-3.5, highlighting the potential of incorporating programming tools into the code generation process.
Echoes: Unsupervised Debiasing via Pseudo-bias Labeling in an Echo Chamber
Venue: arXiv cs.LG 2023
Year: 2023

Neural networks often learn spurious correlations when exposed to biased training data, leading to poor performance on out-of-distribution data. A biased dataset can be divided, according to biased features, into bias-aligned samples (i.e., with biased features) and bias-conflicting samples (i.e., without biased features). Recent debiasing works typically assume that no bias label is available during the training phase, as obtaining such information is challenging and labor-intensive. Following this unsupervised assumption, existing methods usually train two models: a biased model specialized to learn biased features and a target model that uses information from the biased model for debiasing. This paper first presents experimental analyses revealing that the existing biased models overfit to bias-conflicting samples in the training data, which negatively impacts the debiasing performance of the target models. To address this issue, we propose a straightforward and effective method called Echoes, which trains a biased model and a target model with a different strategy. We construct an "echo chamber" environment by reducing the weights of samples which are misclassified by the biased model, to ensure the biased model fully learns the biased features without overfitting to the bias-conflicting samples. The biased model then assigns lower weights on the bias-conflicting samples. Subsequently, we use the inverse of the sample weights of the biased model for training the target model. Experiments show that our approach achieves superior debiasing results compared to the existing baselines on both synthetic and real-world datasets. Our code is available at https://github.com/isruihu/Echoes.
Diffusion-NAT: Self-Prompting Discrete Diffusion for Non-Autoregressive Text Generation
Venue: arXiv cs.CL 2023
Year: 2023

Recently, continuous diffusion models (CDM) have been introduced into non-autoregressive (NAR) text-to-text generation. However, the discrete nature of text increases the difficulty of CDM to generate coherent and fluent texts, and also causes the incompatibility problem between CDM and advanced NLP techniques, especially the popular pre-trained language models~(PLMs). To solve it, we propose Diffusion-NAT, which introduces discrete diffusion models~(DDM) into NAR text-to-text generation and integrates BART to improve the performance. By revising the decoding process of BART and the typical settings of DDM, we unify the inference process of BART and the denoising process of DDM into the same NAR masked tokens recovering task. In this way, DDM can rely on BART to perform denoising, which can benefit from both the rich pre-learned knowledge of BART and the iterative refining paradigm of DDM. Besides, we also propose the iterative self-prompting strategy to further improve the generation quality. Experimental results on 7 datasets show that our approach can outperform competitive NAR methods, and even surpass autoregressive methods. Our code and data will be publicly released.
Actively Discovering New Slots for Task-oriented Conversation
Venue: arXiv cs.CL 2023
Year: 2023

Existing task-oriented conversational search systems heavily rely on domain ontologies with pre-defined slots and candidate value sets. In practical applications, these prerequisites are hard to meet, due to the emerging new user requirements and ever-changing scenarios. To mitigate these issues for better interaction performance, there are efforts working towards detecting out-of-vocabulary values or discovering new slots under unsupervised or semi-supervised learning paradigm. However, overemphasizing on the conversation data patterns alone induces these methods to yield noisy and arbitrary slot results. To facilitate the pragmatic utility, real-world systems tend to provide a stringent amount of human labelling quota, which offers an authoritative way to obtain accurate and meaningful slot assignments. Nonetheless, it also brings forward the high requirement of utilizing such quota efficiently. Hence, we formulate a general new slot discovery task in an information extraction fashion and incorporate it into an active learning framework to realize human-in-the-loop learning. Specifically, we leverage existing language tools to extract value candidates where the corresponding labels are further leveraged as weak supervision signals. Based on these, we propose a bi-criteria selection scheme which incorporates two major strategies, namely, uncertainty-based sampling and diversity-based sampling to efficiently identify terms of interest. We conduct extensive experiments on several public datasets and compare with a bunch of competitive baselines to demonstrate the effectiveness of our method. We have made the code and data used in this paper publicly available.
New Algorithms and Applications for Risk-Limiting Audits
Venue: arXiv cs.CY 2023
Year: 2023

Risk-limiting audits (RLAs) are a significant tool in increasing confidence in the accuracy of elections. They consist of randomized algorithms which check that an election's vote tally, as reported by a vote tabulation system, corresponds to the correct candidates winning. If an initial vote count leads to the wrong election winner, an RLA guarantees to identify the error with high probability over its own randomness. These audits operate by sequentially sampling and examining ballots until they can either confirm the reported winner or identify the true winner. The first part of this work suggests a new generic method, called ``Batchcomp", for converting classical (ballot-level) RLAs into ones that operate on batches. As a concrete application of the suggested method, we develop the first ballot-level RLA for the Israeli Knesset elections, and convert it to one which operates on batches. We ran the suggested ``Batchcomp" procedure on the results of 22nd, 23rd and 24th Knesset elections, both with and without errors. The second part of this work suggests a new use-case for RLAs: verifying that a population census leads to the correct allocation of political power to a nation's districts or federal-states. We present an adaptation of ALPHA, an existing RLA method, to a method which applies to censuses. Our census-RLA is applicable in nations where parliament seats are allocated to geographical regions in proportion to their population according to a certain class of functions (highest averages). It relies on data from both the census and from an additional procedure which is already conducted in many countries today, called a post-enumeration survey.
We Are Not There Yet: The Implications of Insufficient Knowledge Management for Organisational Compliance
Venue: arXiv cs.CY 2023
Year: 2023

Since GDPR went into effect in 2018, many other data protection and privacy regulations have been released. With the new regulation, there has been an associated increase in industry professionals focused on data protection and privacy. Building on related work showing the potential benefits of knowledge management in organisational compliance and privacy engineering, this paper presents the findings of an exploratory qualitative study with data protection officers and other privacy professionals. We found issues with knowledge management to be the underlying challenge of our participants' feedback. Our participants noted four categories of feedback: (1) a perceived disconnect between regulation and practice, (2) a general lack of clear job description, (3) the need for data protection and privacy to be involved at every level of an organisation, (4) knowledge management tools exist but are not used effectively. This paper questions what knowledge management or automation solutions may prove to be effective in establishing better computer-supported work environments.
Goal-oriented Tensor: Beyond AoI Towards Semantics-Empowered Goal-oriented Communications
Venue: arXiv cs.IT 2023
Year: 2023

The intricate interplay of source dynamics, unreliable channels, and staleness of information has long been recognized as a significant impediment for the receiver to achieve accurate, timely, and most importantly, goal-oriented decision making. Thus, a plethora of promising metrics, such as Age of Information, Value of Information, and Mean Square Error, have emerged to quantify these underlying adverse factors. Following this avenue, optimizing these metrics has indirectly improved the utility of goal-oriented decision making. Nevertheless, no metric has hitherto been expressly devised to evaluate the utility of a goal-oriented decision-making process. To this end, this paper investigates a novel performance metric, the Goal-oriented Tensor (GoT), to directly quantify the impact of semantic mismatches on the goal-oriented decision making. Based on the GoT, we consider a sampler-decision maker pair that work collaboratively and distributively to achieve a shared goal of communications. We formulate an infinite-horizon Decentralized Partially Observable Markov Decision Process (Dec-POMDP) to conjointly deduce the optimal deterministic sampling policy and decision-making policy. The simulation results reveal that the sampler-decision maker co-design surpasses the current literature on AoI and its variants in terms of both goal achievement utility and sparse sampling rate, signifying a notable accomplishment for a sparse sampler and goal-oriented decision maker co-design.
Valuing the Electricity Produced Locally in Renewable Energy Communities through Noncooperative Resources Scheduling Games
Venue: arXiv cs.GT 2023
Year: 2023

We propose two market designs for the optimal day-ahead scheduling of energy exchanges within renewable energy communities. The first one implements a cooperative demand side management scheme inside a community where members objectives are coupled through grid tariffs, whereas the second allows in addition the valuation of excess generation in the community and on the retail market. Both designs are formulated as centralized optimization problems first, and as non cooperative games then. In the latter case, the existence and efficiency of the corresponding (Generalized) Nash Equilibria are rigorously studied and proven, and distributed implementations of iterative solution algorithms for finding these equilibria are proposed, with proofs of convergence. The models are tested on a use-case made by 55 members with PV generation, storage and flexible appliances, and compared with a benchmark situation where members act individually (situation without community). We compute the global REC costs and individual bills, inefficiencies of the decentralized models compared to the centralized optima, as well as technical indices such as self-consumption ratio, self-sufficiency ratio, and peak-to-average ratio.
Leveraging Semantic Relationships to Prioritise Indicators of Compromise in Additive Manufacturing Systems
Venue: arXiv cs.CR 2023
Year: 2023

Additive manufacturing (AM) offers numerous benefits, such as manufacturing complex and customised designs quickly and cost-effectively, reducing material waste, and enabling on-demand production. However, several security challenges are associated with AM, making it increasingly attractive to attackers ranging from individual hackers to organised criminal gangs and nation-state actors. This paper addresses the cyber risk in AM to attackers by proposing a novel semantic-based threat prioritisation system for identifying, extracting and ranking indicators of compromise (IOC). The system leverages the heterogeneous information networks (HINs) that automatically extract high-level IOCs from multi-source threat text and identifies semantic relations among the IOCs. It models IOCs with a HIN comprising different meta-paths and meta-graphs to depict semantic relations among diverse IOCs. We introduce a domain-specific recogniser that identifies IOCs in three domains: organisation-specific, regional source-specific, and regional target-specific. A threat assessment uses similarity measures based on meta-paths and meta-graphs to assess semantic relations among IOCs. It prioritises IOCs by measuring their severity based on the frequency of attacks, IOC lifetime, and exploited vulnerabilities in each domain.