OpenRaiser/Intern-Atlas huggingface dataset:
Weakly-supervised High-resolution Segmentation of Mammography Images for Breast Cancer Diagnosis
Venue: arXiv cs.CV 2021
Year: 2021

In the last few years, deep learning classifiers have shown promising results in image-based medical diagnosis. However, interpreting the outputs of these models remains a challenge. In cancer diagnosis, interpretability can be achieved by localizing the region of the input image responsible for the output, i.e. the location of a lesion. Alternatively, segmentation or detection models can be trained with pixel-wise annotations indicating the locations of malignant lesions. Unfortunately, acquiring such labels is labor-intensive and requires medical expertise. To overcome this difficulty, weakly-supervised localization can be utilized. These methods allow neural network classifiers to output saliency maps highlighting the regions of the input most relevant to the classification task (e.g. malignant lesions in mammograms) using only image-level labels (e.g. whether the patient has cancer or not) during training. When applied to high-resolution images, existing methods produce low-resolution saliency maps. This is problematic in applications in which suspicious lesions are small in relation to the image size. In this work, we introduce a novel neural network architecture to perform weakly-supervised segmentation of high-resolution images. The proposed model selects regions of interest via coarse-level localization, and then performs fine-grained segmentation of those regions. We apply this model to breast cancer diagnosis with screening mammography, and validate it on a large clinically-realistic dataset. Measured by Dice similarity score, our approach outperforms existing methods by a large margin in terms of localization performance of benign and malignant lesions, relatively improving the performance by 39.6% and 20.0%, respectively. Code and the weights of some of the models are available at https://github.com/nyukat/GLAM
Comparative analysis of quality of service scheduling classes in mobile ad-hoc networks
Venue: arXiv cs.NI 2021
Year: 2021

Quality of Service (QoS) is now regarded as a requirement for all networks in managing resources like bandwidth and avoidance of network impairments like packet loss, jitter, and delay. Media transfer or streaming would be virtually impossible if QoS parameters were not used even if the streaming protocols were perfectly designed. QoS Scheduling classes help in network traffic optimization and the priority management of packets. This paper presents an analysis of QoS scheduling classes using video traffic in a MANET. The main objective was to identify a scheduling class that provides better QoS for video streaming. A simulation was conducted using NetSim and results were analyzed according to throughput, jitter, and delay. The overall results showed that extended real-time Polling Service (ertPS) outperformed the other classes. ertPS has hybrid features of both real-time Polling Service (rtPS) and Unsolicited Grant Service(UGS) hence the enhanced performance. It is recommended that ertPS scheduling class should be used in MANET where QoS consideration is utmost particularly in multimedia streaming applications.
Linear representation of categorical values
Venue: arXiv cs.NE 2021
Year: 2021

We propose a binary representation of categorical values using a linear map. This linear representation preserves the neighborhood structure of categorical values. In the context of evolutionary algorithms, it means that every categorical value can be reached in a single mutation. The linear representation is embedded into standard metaheuristics, applied to the problem of Sudoku puzzles, and compared to the more traditional direct binary encoding. It shows promising results in fixed-budget experiments and empirical cumulative distribution functions with high dimension instances, and also in fixed-target experiments with small dimension instances.
Farview: Disaggregated Memory with Operator Off-loading for Database Engines
Venue: arXiv cs.DB 2021
Year: 2021

Cloud deployments disaggregate storage from compute, providing more flexibility to both the storage and compute layers. In this paper, we explore disaggregation by taking it one step further and applying it to memory (DRAM). Disaggregated memory uses network attached DRAM as a way to decouple memory from CPU. In the context of databases, such a design offers significant advantages in terms of making a larger memory capacity available as a central pool to a collection of smaller processing nodes. To explore these possibilities, we have implemented Farview, a disaggregated memory solution for databases, operating as a remote buffer cache with operator offloading capabilities. Farview is implemented as an FPGA-based smart NIC making DRAM available as a disaggregated, network attached memory module capable of performing data processing at line rate over data streams to/from disaggregated memory. Farview supports query offloading using operators such as selection, projection, aggregation, regular expression matching and encryption. In this paper we focus on analytical queries and demonstrate the viability of the idea through an extensive experimental evaluation of Farview under different workloads. Farview is competitive with a local buffer cache solution for all the workloads and outperforms it in a number of cases, proving that a smart disaggregated memory can be a viable alternative for databases deployed in cloud environments.
Toward Diverse Precondition Generation
Venue: arXiv cs.CL 2021
Year: 2021

Language understanding must identify the logical connections between events in a discourse, but core events are often unstated due to their commonsense nature. This paper fills in these missing events by generating precondition events. Precondition generation can be framed as a sequence-to-sequence problem: given a target event, generate a possible precondition. However, in most real-world scenarios, an event can have several preconditions, requiring diverse generation -- a challenge for standard seq2seq approaches. We propose DiP, a Diverse Precondition generation system that can generate unique and diverse preconditions. DiP uses a generative process with three components -- an event sampler, a candidate generator, and a post-processor. The event sampler provides control codes (precondition triggers) which the candidate generator uses to focus its generation. Unlike other conditional generation systems, DiP automatically generates control codes without training on diverse examples. Analysis against baselines reveals that DiP improves the diversity of preconditions significantly while also generating more preconditions.
Deploying COTS Legged Robot Platforms into a Heterogeneous Robot Team
Venue: arXiv cs.RO 2021
Year: 2021

The recent availability of commercial-off-the-shelf (COTS) legged robot platforms have opened up new opportunities in deploying legged systems into different scenarios. While the main advantage of legged robots is their ability to traverse unstructured terrain, there are still large gaps between what robot platforms can achieve and their animal counterparts. Therefore, when deploying as part of a heterogeneous robot team of different platforms, it is beneficial to understand the different scenarios where a legged platform would perform better than a wheeled, tracked or aerial platform. Two COTS quadruped robots, Ghost Robotics' Vision 60 and Boston Dynamics' Spot, were deployed into a heterogeneous team. A description of some of the challenges faced while integrating the platforms, as well as some experiments in traversing different terrains are provided to give insight into the real-world deployment of legged robots.
Mitigating Biases in Toxic Language Detection through Invariant Rationalization
Venue: arXiv cs.CL 2021
Year: 2021

Automatic detection of toxic language plays an essential role in protecting social media users, especially minority groups, from verbal abuse. However, biases toward some attributes, including gender, race, and dialect, exist in most training datasets for toxicity detection. The biases make the learned models unfair and can even exacerbate the marginalization of people. Considering that current debiasing methods for general natural language understanding tasks cannot effectively mitigate the biases in the toxicity detectors, we propose to use invariant rationalization (InvRat), a game-theoretic framework consisting of a rationale generator and a predictor, to rule out the spurious correlation of certain syntactic patterns (e.g., identity mentions, dialect) to toxicity labels. We empirically show that our method yields lower false positive rate in both lexical and dialectal attributes than previous debiasing methods.
Age of Information for Multiple-Source Multiple-Server Networks
Venue: arXiv cs.IT 2021
Year: 2021

Having timely and fresh knowledge about the current state of information sources is critical in a variety of applications. In particular, a status update may arrive at the destination later than its generation time due to processing and communication delays. The freshness of the status update at the destination is captured by the notion of age of information. In this study, we analyze a multiple sensing network with multiple sources, multiple servers, and a monitor (destination). Each source corresponds to an independent piece of information and its age is measured individually. Given a particular source, the servers independently sense the source of information and send the status update to the monitor. We assume that updates arrive at the servers according to Poisson random processes. Each server sends its updates to the monitor through a direct link, which is modeled as a queue. The service time to transmit an update is considered to be an exponential random variable. We examine both homogeneous and heterogeneous service and arrival rates for the single-source case, and only homogeneous arrival and service rates for the multiple-source case. We derive a closed-form expression for the average age of information under a last-come-first-serve (LCFS) queue for a single source and arbitrary number of homogeneous servers. Using a recursive method, we derive the explicit average age of information for any number of sources and homogeneous servers. We also investigate heterogeneous servers and a single source, and present algorithms for finding the average age of information.
Probing Pre-Trained Language Models for Disease Knowledge
Venue: arXiv cs.CL 2021
Year: 2021

Pre-trained language models such as ClinicalBERT have achieved impressive results on tasks such as medical Natural Language Inference. At first glance, this may suggest that these models are able to perform medical reasoning tasks, such as mapping symptoms to diseases. However, we find that standard benchmarks such as MedNLI contain relatively few examples that require such forms of reasoning. To better understand the medical reasoning capabilities of existing language models, in this paper we introduce DisKnE, a new benchmark for Disease Knowledge Evaluation. To construct this benchmark, we annotated each positive MedNLI example with the types of medical reasoning that are needed. We then created negative examples by corrupting these positive examples in an adversarial way. Furthermore, we define training-test splits per disease, ensuring that no knowledge about test diseases can be learned from the training data, and we canonicalize the formulation of the hypotheses to avoid the presence of artefacts. This leads to a number of binary classification problems, one for each type of reasoning and each disease. When analysing pre-trained models for the clinical/biomedical domain on the proposed benchmark, we find that their performance drops considerably.
Entropy-Based Proofs of Combinatorial Results on Bipartite Graphs
Venue: arXiv cs.IT 2021
Year: 2021

This work considers new entropy-based proofs of some known, or otherwise refined, combinatorial bounds for bipartite graphs. These include upper bounds on the number of the independent sets, lower bounds on the minimal number of colors in constrained edge coloring, and lower bounds on the number of walks of a given length in bipartite graphs. The proofs of these combinatorial results rely on basic properties of the Shannon entropy.
Is it a click bait? Let's predict using Machine Learning
Venue: arXiv cs.CL 2021
Year: 2021

In this era of digitisation, news reader tend to read news online. This is because, online media instantly provides access to a wide variety of content. Thus, people don't have to wait for tomorrow's newspaper to know what's happening today. Along with these virtues, online news have some vices as well. One such vice is presence of social media posts (tweets) relating to news articles whose sole purpose is to draw attention of the users rather than directing them to read the actual content. Such posts are referred to as clickbaits. The objective of this project is to develop a system which would be capable of predicting how likely are the social media posts (tweets) relating to new articles tend to be clickbait.
THG: Transformer with Hyperbolic Geometry
Venue: arXiv cs.CL 2021
Year: 2021

Transformer model architectures have become an indispensable staple in deep learning lately for their effectiveness across a range of tasks. Recently, a surge of "X-former" models have been proposed which improve upon the original Transformer architecture. However, most of these variants make changes only around the quadratic time and memory complexity of self-attention, i.e. the dot product between the query and the key. What's more, they are calculate solely in Euclidean space. In this work, we propose a novel Transformer with Hyperbolic Geometry (THG) model, which take the advantage of both Euclidean space and Hyperbolic space. THG makes improvements in linear transformations of self-attention, which are applied on the input sequence to get the query and the key, with the proposed hyperbolic linear. Extensive experiments on sequence labeling task, machine reading comprehension task and classification task demonstrate the effectiveness and generalizability of our model. It also demonstrates THG could alleviate overfitting.
From Single Lane to Highways: Analyzing the Adoption of Multipath TCP in the Internet
Venue: arXiv cs.NI 2021
Year: 2021

Multipath TCP (MPTCP) extends traditional TCP to enable simultaneous use of multiple connection endpoints at the source and destination. MPTCP has been under active development since its standardization in 2013, and more recently in February 2020, MPTCP was upstreamed to the Linux kernel. In this paper, we provide the first broad analysis of MPTCPv0 in the Internet. We probe the entire IPv4 address space and an IPv6 hitlist to detect MPTCP-enabled systems operational on port 80 and 443. Our scans reveal a steady increase in MPTCP-capable IPs, reaching 9k+ on IPv4 and a few dozen on IPv6. We also discover a significant share of seemingly MPTCP-capable hosts, an artifact of middleboxes mirroring TCP options. We conduct targeted HTTP(S) measurements towards select hosts and find that middleboxes can aggressively impact the perceived quality of applications utilizing MPTCP. Finally, we analyze two complementary traffic traces from CAIDA and MAWI to shed light on the real-world usage of MPTCP. We find that while MPTCP usage has increased by a factor of 20 over the past few years, its traffic share is still quite low.
Meaning Representation of Numeric Fused-Heads in UCCA
Venue: arXiv cs.CL 2021
Year: 2021

We exhibit that the implicit UCCA parser does not address numeric fused-heads (NFHs) consistently, which could result either from inconsistent annotation, insufficient training data or a modelling limitation. and show which factors are involved. We consider this phenomenon important, as it is pervasive in text and critical for correct inference. Careful design and fine-grained annotation of NFHs in meaning representation frameworks would benefit downstream tasks such as machine translation, natural language inference and question answering, particularly when they require numeric reasoning, as recovering and categorizing them. We are investigating the treatment of this phenomenon by other meaning representations, such as AMR. We encourage researchers in meaning representations, and computational linguistics in general, to address this phenomenon in future research.
A Self-Supervised Framework for Function Learning and Extrapolation
Venue: arXiv cs.LG 2021
Year: 2021

Understanding how agents learn to generalize -- and, in particular, to extrapolate -- in high-dimensional, naturalistic environments remains a challenge for both machine learning and the study of biological agents. One approach to this has been the use of function learning paradigms, which allow peoples' empirical patterns of generalization for smooth scalar functions to be described precisely. However, to date, such work has not succeeded in identifying mechanisms that acquire the kinds of general purpose representations over which function learning can operate to exhibit the patterns of generalization observed in human empirical studies. Here, we present a framework for how a learner may acquire such representations, that then support generalization -- and extrapolation in particular -- in a few-shot fashion. Taking inspiration from a classic theory of visual processing, we construct a self-supervised encoder that implements the basic inductive bias of invariance under topological distortions. We show the resulting representations outperform those from other models for unsupervised time series learning in several downstream function learning tasks, including extrapolation.
An SMT Based Compositional Algorithm to Solve a Conflict-Free Electric Vehicle Routing Problem
Venue: arXiv cs.AI 2021
Year: 2021

The Vehicle Routing Problem (VRP) is the combinatorial optimization problem of designing routes for vehicles to visit customers in such a fashion that a cost function, typically the number of vehicles, or the total travelled distance is minimized. The problem finds applications in industrial scenarios, for example where Automated Guided Vehicles run through the plant to deliver components from the warehouse. This specific problem, henceforth called the Electric Conflict-Free Vehicle Routing Problem (CF-EVRP), involves constraints such as limited operating range of the vehicles, time windows on the delivery to the customers, and limited capacity on the number of vehicles the road segments can accommodate at the same time. Such a complex system results in a large model that cannot easily be solved to optimality in reasonable time. We therefore developed a compositional model that breaks down the problem into smaller and simpler sub-problems and provides sub-optimal, feasible solutions to the original problem. The algorithm exploits the strengths of SMT solvers, which proved in our previous work to be an efficient approach to deal with scheduling problems. Compared to a monolithic model for the CF-EVRP, written in the SMT standard language and solved using a state-of-the-art SMT solver the compositional model was found to be significantly faster.
Dilated filters for edge detection algorithms
Venue: arXiv cs.CV 2021
Year: 2021

Edges are a basic and fundamental feature in image processing, that are used directly or indirectly in huge amount of applications. Inspired by the expansion of image resolution and processing power dilated convolution techniques appeared. Dilated convolution have impressive results in machine learning, we discuss here the idea of dilating the standard filters which are used in edge detection algorithms. In this work we try to put together all our previous and current results by using instead of the classical convolution filters a dilated one. We compare the results of the edge detection algorithms using the proposed dilation filters with original filters or custom variants. Experimental results confirm our statement that dilation of filters have positive impact for edge detection algorithms form simple to rather complex algorithms.
Model Explainability in Deep Learning Based Natural Language Processing
Venue: arXiv cs.AI 2021
Year: 2021

Machine learning (ML) model explainability has received growing attention, especially in the area related to model risk and regulations. In this paper, we reviewed and compared some popular ML model explainability methodologies, especially those related to Natural Language Processing (NLP) models. We then applied one of the NLP explainability methods Layer-wise Relevance Propagation (LRP) to a NLP classification model. We used the LRP method to derive a relevance score for each word in an instance, which is a local explainability. The relevance scores are then aggregated together to achieve global variable importance of the model. Through the case study, we also demonstrated how to apply the local explainability method to false positive and false negative instances to discover the weakness of a NLP model. These analysis can help us to understand NLP models better and reduce the risk due to the black-box nature of NLP models. We also identified some common issues due to the special natures of NLP models and discussed how explainability analysis can act as a control to detect these issues after the model has been trained.
Accumulative Iterative Codes Based on Feedback
Venue: arXiv cs.IT 2021
Year: 2021

The Accumulative Iterative Code (AIC) proposed in this work is a new error correcting code for channels with feedback. AIC sends the information message to the receiver in a number of transmissions, where the initial transmission contains the uncoded message and each subsequent transmission informs the receiver about the locations of the errors that corrupted the previous transmission. Error locations are determined based on the forward channel output, which is made available to the transmitter through the feedback channel. AIC achieves arbitrarily low error rates, thereby being suitablefor applications demanding extremely high reliability. In the same time, AIC achieves spectral efficiencies very close to the channel capacity in a wide range of signal-to-noise ratios even for transmission of short information messages.
Online Estimation of Resource Overload Risk in 5G Multi-Tenancy Network
Venue: arXiv cs.NI 2021
Year: 2021

The technology of network slicing, as the most characteristic feature of the fifth generation (5G) wireless networks, manages the resources and network functions in heterogeneous and logically isolated slices on the top of a shared physical infrastructure, where every slice can be independently customized to fulfill the specific requirements of its devoted service type. It enables a new paradigm of multi-tenancy networking, where the network slices can be leased by the mobile network operator (MNO) to tenants in form of public cloud computing service, known as Slice-asa- Service (SlaaS). Similar to classical cloud computing scenarios, SlaaS benefits from overbooking its resources to numerous tenants, taking advantage of the resource elasticity and diversity, at a price of risking overloading network resources and violating the service-level agreements (SLAs), which stipulate the quality of service (QoS) that shall be guaranteed to the network slices. Thus, it becomes a critical challenge to the MNOs, accurately estimating the resource overload risk - especially under the sophisticated network dynamics - for monitoring and enhancing the reliability of SlaaS business.