OpenRaiser/Intern-Atlas huggingface dataset:
A Robust Multilabel Method Integrating Rule-based Transparent Model, Soft Label Correlation Learning and Label Noise Resistance
Venue: arXiv cs.AI 2023
Year: 2023

Model transparency, label correlation learning and the robust-ness to label noise are crucial for multilabel learning. However, few existing methods study these three characteristics simultaneously. To address this challenge, we propose the robust multilabel Takagi-Sugeno-Kang fuzzy system (R-MLTSK-FS) with three mechanisms. First, we design a soft label learning mechanism to reduce the effect of label noise by explicitly measuring the interactions between labels, which is also the basis of the other two mechanisms. Second, the rule-based TSK FS is used as the base model to efficiently model the inference relationship be-tween features and soft labels in a more transparent way than many existing multilabel models. Third, to further improve the performance of multilabel learning, we build a correlation enhancement learning mechanism based on the soft label space and the fuzzy feature space. Extensive experiments are conducted to demonstrate the superiority of the proposed method.
Do Performance Aspirations Matter for Guiding Software Configuration Tuning?
Venue: arXiv cs.SE 2023
Year: 2023

Configurable software systems can be tuned for better performance. Leveraging on some Pareto optimizers, recent work has shifted from tuning for a single, time-related performance objective to two intrinsically different objectives that assess distinct performance aspects of the system, each with varying aspirations. Before we design better optimizers, a crucial engineering decision to make therein is how to handle the performance requirements with clear aspirations in the tuning process. For this, the community takes two alternative optimization models: either quantifying and incorporating the aspirations into the search objectives that guide the tuning, or not considering the aspirations during the search but purely using them in the later decision-making process only. However, despite being a crucial decision that determines how an optimizer can be designed and tailored, there is a rather limited understanding of which optimization model should be chosen under what particular circumstance, and why. In this paper, we seek to close this gap. Firstly, we do that through a review of over 426 papers in the literature and 14 real-world requirements datasets. Drawing on these, we then conduct a comprehensive empirical study that covers 15 combinations of the state-of-the-art performance requirement patterns, four types of aspiration space, three Pareto optimizers, and eight real-world systems/environments, leading to 1,296 cases of investigation. We found that (1) the realism of aspirations is the key factor that determines whether they should be used to guide the tuning; (2) the given patterns and the position of the realistic aspirations in the objective landscape are less important for the choice, but they do matter to the extents of improvement; (3) the available tuning budget can also influence the choice for unrealistic aspirations but it is insignificant under realistic ones.
Deep Breath: A Machine Learning Browser Extension to Tackle Online Misinformation
Venue: arXiv cs.HC 2023
Year: 2023

Over the past decade, the media landscape has seen a radical shift. As more of the public stay informed of current events via online sources, competition has grown as outlets vie for attention. This competition has prompted some online outlets to publish sensationalist and alarmist content to grab readers' attention. Such practices may threaten democracy by distorting the truth and misleading readers about the nature of events. This paper proposes a novel system for detecting, processing, and warning users about misleading content online to combat the threats posed by misinformation. By training a machine learning model on an existing dataset of 32,000 clickbait news article headlines, the model predicts how sensationalist a headline is and then interfaces with a web browser extension which constructs a unique content warning notification based on existing design principles and incorporates the models' prediction. This research makes a novel contribution to machine learning and human-centred security with promising findings for future research. By warning users when they may be viewing misinformation, it is possible to prevent spontaneous reactions, helping users to take a deep breath and approach online media with a clear mind.
The Optimal Input-Independent Baseline for Binary Classification: The Dutch Draw
Venue: arXiv cs.LG 2023
Year: 2023

Before any binary classification model is taken into practice, it is important to validate its performance on a proper test set. Without a frame of reference given by a baseline method, it is impossible to determine if a score is `good' or `bad'. The goal of this paper is to examine all baseline methods that are independent of feature values and determine which model is the `best' and why. By identifying which baseline models are optimal, a crucial selection decision in the evaluation process is simplified. We prove that the recently proposed Dutch Draw baseline is the best input-independent classifier (independent of feature values) for all positional-invariant measures (independent of sequence order) assuming that the samples are randomly shuffled. This means that the Dutch Draw baseline is the optimal baseline under these intuitive requirements and should therefore be used in practice.
Chatbots As Fluent Polyglots: Revisiting Breakthrough Code Snippets
Venue: arXiv cs.LG 2023
Year: 2023

The research applies AI-driven code assistants to analyze a selection of influential computer code that has shaped modern technology, including email, internet browsing, robotics, and malicious software. The original contribution of this study was to examine half of the most significant code advances in the last 50 years and, in some cases, to provide notable improvements in clarity or performance. The AI-driven code assistant could provide insights into obfuscated code or software lacking explanatory commentary in all cases examined. We generated additional sample problems based on bug corrections and code optimizations requiring much deeper reasoning than a traditional Google search might provide. Future work focuses on adding automated documentation and code commentary and translating select large code bases into more modern versions with multiple new application programming interfaces (APIs) and chained multi-tasks. The AI-driven code assistant offers a valuable tool for software engineering, particularly in its ability to provide human-level expertise and assist in refactoring legacy code or simplifying the explanation or functionality of high-value repositories.
AI2: The next leap toward native language based and explainable machine learning framework
Venue: arXiv cs.LG 2023
Year: 2023

The machine learning frameworks flourished in the last decades, allowing artificial intelligence to get out of academic circles to be applied to enterprise domains. This field has significantly advanced, but there is still some meaningful improvement to reach the subsequent expectations. The proposed framework, named AI$^{2}$, uses a natural language interface that allows a non-specialist to benefit from machine learning algorithms without necessarily knowing how to program with a programming language. The primary contribution of the AI$^{2}$ framework allows a user to call the machine learning algorithms in English, making its interface usage easier. The second contribution is greenhouse gas (GHG) awareness. It has some strategies to evaluate the GHG generated by the algorithm to be called and to propose alternatives to find a solution without executing the energy-intensive algorithm. Another contribution is a preprocessing module that helps to describe and to load data properly. Using an English text-based chatbot, this module guides the user to define every dataset so that it can be described, normalized, loaded and divided appropriately. The last contribution of this paper is about explainability. For decades, the scientific community has known that machine learning algorithms imply the famous black-box problem. Traditional machine learning methods convert an input into an output without being able to justify this result. The proposed framework explains the algorithm's process with the proper texts, graphics and tables. The results, declined in five cases, present usage applications from the user's English command to the explained output. Ultimately, the AI$^{2}$ framework represents the next leap toward native language-based, human-oriented concerns about machine learning framework.
A comprehensive review of automatic text summarization techniques: method, data, evaluation and coding
Venue: arXiv cs.CL 2023
Year: 2023

We provide a literature review about Automatic Text Summarization (ATS) systems. We consider a citation-based approach. We start with some popular and well-known papers that we have in hand about each topic we want to cover and we have tracked the "backward citations" (papers that are cited by the set of papers we knew beforehand) and the "forward citations" (newer papers that cite the set of papers we knew beforehand). In order to organize the different methods, we present the diverse approaches to ATS guided by the mechanisms they use to generate a summary. Besides presenting the methods, we also present an extensive review of the datasets available for summarization tasks and the methods used to evaluate the quality of the summaries. Finally, we present an empirical exploration of these methods using the CNN Corpus dataset that provides golden summaries for extractive and abstractive methods.
A double-elimination format for a 48-team FIFA World Cup
Venue: arXiv cs.GT 2023
Year: 2023

I present a double-elimination format for the 48-team FIFA World Cup that solves many of the concerns raised about the considered formats with mixed round-robin with groups of 3 or 4 teams and single-elimination strategies. Using a quantitative analytics approach, I show that the double-elimination format is fairer, more strategy-proof, and produces more competitive and exciting matches. It solves the problems of possible collusion in the group-of-3 form and the high number of uninteresting games in the group-of-4 format. Using the restrictions of the 2026 FIFA World Cup to be held in North America, I demonstrate that the double-elimination format can be implemented in a 35 days window, just a few more than the current format for 32 teams. I discuss how the format can be adapted to the particularities of the host nations in future editions to facilitate attendees' and organizers' planning.
ERNIE 3.0 Tiny: Frustratingly Simple Method to Improve Task-Agnostic Distillation Generalization
Venue: arXiv cs.CL 2023
Year: 2023

Task-agnostic knowledge distillation attempts to address the problem of deploying large pretrained language model in resource-constrained scenarios by compressing a large pretrained model called teacher into a smaller one called student such that the student can be directly finetuned on downstream tasks and retains comparable performance. However, we empirically find that there is a generalization gap between the student and the teacher in existing methods. In this work, we show that we can leverage multi-task learning in task-agnostic distillation to advance the generalization of the resulted student. In particular, we propose Multi-task Infused Task-agnostic Knowledge Distillation (MITKD). We first enhance the teacher by multi-task training it on multiple downstream tasks and then perform distillation to produce the student. Experimental results demonstrate that our method yields a student with much better generalization, significantly outperforms existing baselines, and establishes a new state-of-the-art result on in-domain, out-domain, and low-resource datasets in the setting of task-agnostic distillation. Moreover, our method even exceeds an 8x larger BERT$_{\text{Base}}$ on SQuAD and four GLUE tasks. In addition, by combining ERNIE 3.0, our method achieves state-of-the-art results on 10 Chinese datasets.
STARS-ISAC: How Many Sensors Do We Need?
Venue: arXiv cs.IT 2023
Year: 2023

A simultaneously transmitting and reflecting surface (STARS) enabled integrated sensing and communications (ISAC) framework is proposed, where a novel bi-directional sensing-STARS architecture is devised to facilitate the full-space communication and sensing. Based on the proposed framework, a joint optimization problem is formulated, where the Cramer-Rao bound (CRB) for estimating the 2-dimension direction-of-arrival of the sensing target is minimized. Two cases are considered for sensing performance enhancement. 1) For the two-user case, an alternating optimization algorithm is proposed. In particular, the maximum number of deployable sensors is obtained in the closed-form expressions. 2) For the multi-user case, an extended CRB (ECRB) metric is proposed to characterize the impact of the number of sensors on the sensing performance. Based on the proposed metric, a novel penalty-based double-loop (PDL) algorithm is proposed to solve the ECRB minimization problem. To tackle the coupling of the ECRB, a general decoupling approach is proposed to convert it to a tractable weighted linear summation form. Simulation results reveal that 1) the proposed PDL algorithm can achieve a near-optimal performance with consideration of sensor deployment; 2) without violating the communication under the quality of service requirements, reducing the receive antennas at the BS does not deteriorate the sensing performance; and 3) it is preferable to deploy more passive elements than sensors in terms of achieving optimal sensing performance
A Cyber Threat Intelligence Management Platform for Industrial Environments
Venue: arXiv cs.CR 2023
Year: 2023

Developing intelligent, interoperable Cyber Threat Information (CTI) sharing technologies can help build strong defences against modern cyber threats. CTIs allow the community to share information about cybercriminals' threats and vulnerabilities and countermeasures to defend themselves or detect malicious activity. A crucial need for success is that the data connected to cyber risks be understandable, organized, and of good quality. The receiving parties may grasp its content and utilize it effectively. This article describes an innovative cyber threat intelligence management platform (CTIMP) for industrial environments, one of the Cyber-pi project's significant elements. The suggested architecture, in particular, uses cyber knowledge from trusted public sources and integrates it with relevant information from the organization's supervised infrastructure in an entirely interoperable and intelligent way. When combined with an advanced visualization mechanism and user interface, the services mentioned above provide administrators with the situational awareness they require while also allowing for extended cooperation, intelligent selection of advanced coping strategies, and a set of automated self-healing rules for dealing with threats.
Advances in Medical Image Analysis with Vision Transformers: A Comprehensive Review
Venue: arXiv cs.CV 2023
Year: 2023

The remarkable performance of the Transformer architecture in natural language processing has recently also triggered broad interest in Computer Vision. Among other merits, Transformers are witnessed as capable of learning long-range dependencies and spatial correlations, which is a clear advantage over convolutional neural networks (CNNs), which have been the de facto standard in Computer Vision problems so far. Thus, Transformers have become an integral part of modern medical image analysis. In this review, we provide an encyclopedic review of the applications of Transformers in medical imaging. Specifically, we present a systematic and thorough review of relevant recent Transformer literature for different medical image analysis tasks, including classification, segmentation, detection, registration, synthesis, and clinical report generation. For each of these applications, we investigate the novelty, strengths and weaknesses of the different proposed strategies and develop taxonomies highlighting key properties and contributions. Further, if applicable, we outline current benchmarks on different datasets. Finally, we summarize key challenges and discuss different future research directions. In addition, we have provided cited papers with their corresponding implementations in https://github.com/mindflow-institue/Awesome-Transformer.
Learning-based Design and Control for Quadrupedal Robots with Parallel-Elastic Actuators
Venue: arXiv cs.RO 2023
Year: 2023

Parallel-elastic joints can improve the efficiency and strength of robots by assisting the actuators with additional torques. For these benefits to be realized, a spring needs to be carefully designed. However, designing robots is an iterative and tedious process, often relying on intuition and heuristics. We introduce a design optimization framework that allows us to co-optimize a parallel elastic knee joint and locomotion controller for quadrupedal robots with minimal human intuition. We design a parallel elastic joint and optimize its parameters with respect to the efficiency in a model-free fashion. In the first step, we train a design-conditioned policy using model-free Reinforcement Learning, capable of controlling the quadruped in the predefined range of design parameters. Afterwards, we use Bayesian Optimization to find the best design using the policy. We use this framework to optimize the parallel-elastic spring parameters for the knee of our quadrupedal robot ANYmal together with the optimal controller. We evaluate the optimized design and controller in real-world experiments over various terrains. Our results show that the new system improves the torque-square efficiency of the robot by 33% compared to the baseline and reduces maximum joint torque by 30% without compromising tracking performance. The improved design resulted in 11% longer operation time on flat terrain.
Report on the Future of Conferences
Venue: arXiv cs.CY 2023
Year: 2023

In 2020, virtual conferences became almost the only alternative to cancellation. Now that the pandemic is subsiding, the pros and cons of virtual conferences need to be reevaluated. In this report, we scrutinize the dynamics and economics of conferences and highlight the history of successful virtual meetings in industry. We also report on the attitudes of conference attendees from an informal survey we ran in spring 2022.
An Impartial Transformer for Story Visualization
Venue: arXiv cs.CV 2023
Year: 2023

Story Visualization is an advanced task of computed vision that targets sequential image synthesis, where the generated samples need to be realistic, faithful to their conditioning and sequentially consistent. Our work proposes a novel architectural and training approach: the Impartial Transformer achieves both text-relevant plausible scenes and sequential consistency utilizing as few trainable parameters as possible. This enhancement is even able to handle synthesis of 'hard' samples with occluded objects, achieving improved evaluation metrics comparing to past approaches.
Implementations of two Algorithms for the Threshold Synthesis Problem
Venue: arXiv cs.LO 2023
Year: 2023

A linear pseudo-Boolean constraint (LPB) is an expression of the form $a_1 \cdot \ell_1 + \dots + a_m \cdot \ell_m \geq d$, where each $\ell_i$ is a literal (it assumes the value 1 or 0 depending on whether a propositional variable $x_i$ is true or false) and $a_1, \dots, a_m, d$ are natural numbers. An LPB represents a Boolean function, and those Boolean functions that can be represented by exactly one LPB are called threshold functions. The problem of finding an LPB representation of a Boolean function if possible is called threshold recognition problem or threshold synthesis problem. The problem has an $O(m^7 t^5)$ algorithm using linear programming, where $m$ is the dimension and $t$ the number of terms in the DNF input. It has been an open question whether one can recognise threshold functions through an entirely combinatorial procedure. Smaus has developed such a procedure for doing this, which works by decomposing the DNF and "counting" the variable occurrences in it. We have implemented both algorithms as a thesis project. We report here on this experience. The most important insight was that the algorithm by Smaus is, unfortunately, incomplete.
Consistent Query Answering without Repairs in Tables with Nulls and Functional Dependencies
Venue: arXiv cs.DB 2023
Year: 2023

In this paper, we study consistent query answering in tables with nulls and functional dependencies. Given such a table T, we consider the set Tuples of all tuples that can be built up from constants appearing in T, and we use set theoretic semantics for tuples and functional dependencies to characterize the tuples of Tuples in two orthogonal ways: first as true or false tuples, and then as consistent or inconsistent tuples. Queries are issued against T and evaluated in Tuples. In this setting, we consider a query Q: select X from T where Condition over T and define its consistent answer to be the set of tuples x in Tuples such that: x is a true and consistent tuple with schema X and there exists a true super-tuple t of x in Tuples satisfying the condition. We show that, depending on the status that the super-tuple t has in Tuples, there are different types of consistent answer to Q. The main contributions of the paper are: (a) a novel approach to consistent query answering not using table repairs; (b) polynomial algorithms for computing the sets of true-false tuples and the sets of consistent-inconsistent tuples of Tuples; (c) polynomial algorithms in the size of T for computing different types of consistent answer for both conjunctive and disjunctive queries; and (d) a detailed discussion of the differences between our approach and the approaches using table repairs.
Chatbots in a Honeypot World
Venue: arXiv cs.CR 2023
Year: 2023

Question-and-answer agents like ChatGPT offer a novel tool for use as a potential honeypot interface in cyber security. By imitating Linux, Mac, and Windows terminal commands and providing an interface for TeamViewer, nmap, and ping, it is possible to create a dynamic environment that can adapt to the actions of attackers and provide insight into their tactics, techniques, and procedures (TTPs). The paper illustrates ten diverse tasks that a conversational agent or large language model might answer appropriately to the effects of command-line attacker. The original result features feasibility studies for ten model tasks meant for defensive teams to mimic expected honeypot interfaces with minimal risks. Ultimately, the usefulness outside of forensic activities stems from whether the dynamic honeypot can extend the time-to-conquer or otherwise delay attacker timelines short of reaching key network assets like databases or confidential information. While ongoing maintenance and monitoring may be required, ChatGPT's ability to detect and deflect malicious activity makes it a valuable option for organizations seeking to enhance their cyber security posture. Future work will focus on cybersecurity layers, including perimeter security, host virus detection, and data security.