OpenRaiser/Intern-Atlas huggingface dataset:
PASTO: Strategic Parameter Optimization in Recommendation Systems -- Probabilistic is Better than Deterministic
Venue: arXiv cs.LG 2021
Year: 2021

Real-world recommendation systems often consist of two phases. In the first phase, multiple predictive models produce the probability of different immediate user actions. In the second phase, these predictions are aggregated according to a set of 'strategic parameters' to meet a diverse set of business goals, such as longer user engagement, higher revenue potential, or more community/network interactions. In addition to building accurate predictive models, it is also crucial to optimize this set of 'strategic parameters' so that primary goals are optimized while secondary guardrails are not hurt. In this setting with multiple and constrained goals, this paper discovers that a probabilistic strategic parameter regime can achieve better value compared to the standard regime of finding a single deterministic parameter. The new probabilistic regime is to learn the best distribution over strategic parameter choices and sample one strategic parameter from the distribution when each user visits the platform. To pursue the optimal probabilistic solution, we formulate the problem into a stochastic compositional optimization problem, in which the unbiased stochastic gradient is unavailable. Our approach is applied in a popular social network platform with hundreds of millions of daily users and achieves +0.22% lift of user engagement in a recommendation task and +1.7% lift in revenue in an advertising optimization scenario comparing to using the best deterministic parameter strategy.
Fastformer: Additive Attention Can Be All You Need
Venue: arXiv cs.CL 2021
Year: 2021

Transformer is a powerful model for text understanding. However, it is inefficient due to its quadratic complexity to input sequence length. Although there are many methods on Transformer acceleration, they are still either inefficient on long sequences or not effective enough. In this paper, we propose Fastformer, which is an efficient Transformer model based on additive attention. In Fastformer, instead of modeling the pair-wise interactions between tokens, we first use additive attention mechanism to model global contexts, and then further transform each token representation based on its interaction with global context representations. In this way, Fastformer can achieve effective context modeling with linear complexity. Extensive experiments on five datasets show that Fastformer is much more efficient than many existing Transformer models and can meanwhile achieve comparable or even better long text modeling performance.
Abduction of trap invariants in parameterized systems
Venue: arXiv cs.DC 2021
Year: 2021

In a previous paper we have presented a CEGAR approach for the verification of parameterized systems with an arbitrary number of processes organized in an array or a ring. The technique is based on the iterative computation of parameterized invariants, i.e., infinite families of invariants for the infinitely many instances of the system. Safety properties are proved by checking that every global configuration of the system satisfying all parameterized invariants also satisfies the property; we have shown that this check can be reduced to the satisfiability problem for Monadic Second Order on words, which is decidable. A strong limitation of the approach is that processes can only have a fixed number of variables with a fixed finite range. In particular, they cannot use variables with range [0,N-1], where N is the number of processes, which appear in many standard distributed algorithms. In this paper, we extend our technique to this case. While conducting the check whether a safety property is inductive assuming a computed set of invariants becomes undecidable, we show how to reduce it to checking satisfiability of a first-order formula. We report on experiments showing that automatic first-order theorem provers can still perform this check for a collection of non-trivial examples. Additionally, we can give small sets of readable invariants for these checks.
Semantic Communication with Adaptive Universal Transformer
Venue: arXiv cs.CL 2021
Year: 2021

With the development of deep learning (DL), natural language processing (NLP) makes it possible for us to analyze and understand a large amount of language texts. Accordingly, we can achieve a semantic communication in terms of joint semantic source and channel coding over a noisy channel with the help of NLP. However, the existing method to realize this goal is to use a fixed transformer of NLP while ignoring the difference of semantic information contained in each sentence. To solve this problem, we propose a new semantic communication system based on Universal Transformer. Compared with the traditional transformer, an adaptive circulation mechanism is introduced in the Universal Transformer. Through the introduction of the circulation mechanism, the new semantic communication system can be more flexible to transmit sentences with different semantic information, and achieve better end-to-end performance under various channel conditions.
Controller Placement in SDN-enabled 5G Satellite-Terrestrial Networks
Venue: arXiv cs.NI 2021
Year: 2021

SDN-enabled Integrated satellite-terrestrial networks (ISTNs), can provide several advantages including global seamless coverage, high reliability, low latency, etc. and can be a key enabler towards next generation networks. To deal with the complexity of the control and management of the integrated network, leveraging the concept of software-defined networking (SDN) will be helpful. In this regard, the SDN controller placement problem in SDN-enabled ISTNs becomes of paramount importance. In this paper, we formulate an optimization problem for the SDN controller placement with the objective of minimizing the average failure probability of SDN control paths to ensure the SDN switches receive the instructions in the most reliable fashion. Simultaneously, we aim at deploying the SDN controllers close to the satellite gateways to ensure the connection between the two layers occurs with the lowest latency. We first model the problem as a mixed integer linear program (MILP). To reduce the time complexity of the MILP model, we use submodular optimization techniques to generate near-optimal solutions in a time-efficient manner. Finally, we verify the effectiveness of our approach by means of simulation, showing that the approximation method results in a reasonable optimality gap with respect to the exact MILP solution.
Zoom, Enhance! Measuring Surveillance GAN Up-sampling
Venue: arXiv cs.CV 2021
Year: 2021

Deep Neural Networks have been very successfully used for many computer vision and pattern recognition applications. While Convolutional Neural Networks(CNNs) have shown the path to state of art image classifications, Generative Adversarial Networks or GANs have provided state of art capabilities in image generation. In this paper we extend the applications of CNNs and GANs to experiment with up-sampling techniques in the domains of security and surveillance. Through this work we evaluate, compare and contrast the state of art techniques in both CNN and GAN based image and video up-sampling in the surveillance domain. As a result of this study we also provide experimental evidence to establish DISTS as a stronger Image Quality Assessment(IQA) metric for comparing GAN Based Image Up-sampling in the surveillance domain.
Eliciting Social Knowledge for Creditworthiness Assessment
Venue: arXiv cs.GT 2021
Year: 2021

Access to capital is a major constraint for economic growth in the developing world. Yet those attempting to lend in this space face high defaults due to their inability to distinguish creditworthy borrowers from the rest. In this paper, we propose two novel scoring mechanisms that incentivize community members to truthfully report their signal on the creditworthiness of others in their community. We first design a truncated asymmetric scoring-rule for a setting where the lender has no liquidity constraints. We then derive a novel, strictly-proper VCG scoring mechanism for the liquidity-constrained setting. Whereas Chen et al. [2011] give an impossibility result for an analogous setting in which sequential reports are made in the context of decision markets, we achieve a positive result through appeal to interim beliefs about the reports of others in a setting with simultaneous reports.Moreover, the use of VCG methods allows for the integration of linear belief aggregation methods.
Wave Function Collapse Coloring: A New Heuristic for Fast Vertex Coloring
Venue: arXiv cs.DS 2021
Year: 2021

In this paper, we propose a high-speed greedy sequential algorithm for the vertex coloring problem (VCP), based on the Wave Function Collapse algorithm, called Wave Function Collapse Coloring (WFC-C). An iteration of this algorithm goes through three distinct phases: vertex selection, color restriction through wave function collapsing, and domain propagation. In effect, WFC-C propagates color choices or "domain" restrictions beyond immediate neighbourhoods. This heuristic, combined with a series of other greedy optimizations, allows for a fast algorithm that prevents certain color conflicts. Through extensive experiments, we show that WFC-C remains competitive (and occasionally better) in terms of optimal coloring, and dramatically outperforms existing high-speed VCP, with on average speed differences ranging from 2000 times to 16000 times, on the most difficult instances of the DIMACS benchmark.
SemiFed: Semi-supervised Federated Learning with Consistency and Pseudo-Labeling
Venue: arXiv cs.LG 2021
Year: 2021

Federated learning enables multiple clients, such as mobile phones and organizations, to collaboratively learn a shared model for prediction while protecting local data privacy. However, most recent research and applications of federated learning assume that all clients have fully labeled data, which is impractical in real-world settings. In this work, we focus on a new scenario for cross-silo federated learning, where data samples of each client are partially labeled. We borrow ideas from semi-supervised learning methods where a large amount of unlabeled data is utilized to improve the model's accuracy despite limited access to labeled examples. We propose a new framework dubbed SemiFed that unifies two dominant approaches for semi-supervised learning: consistency regularization and pseudo-labeling. SemiFed first applies advanced data augmentation techniques to enforce consistency regularization and then generates pseudo-labels using the model's predictions during training. SemiFed takes advantage of the federation so that for a given image, the pseudo-label holds only if multiple models from different clients produce a high-confidence prediction and agree on the same label. Extensive experiments on two image benchmarks demonstrate the effectiveness of our approach under both homogeneous and heterogeneous data distribution settings
Palmira: A Deep Deformable Network for Instance Segmentation of Dense and Uneven Layouts in Handwritten Manuscripts
Venue: arXiv cs.CV 2021
Year: 2021

Handwritten documents are often characterized by dense and uneven layout. Despite advances, standard deep network based approaches for semantic layout segmentation are not robust to complex deformations seen across semantic regions. This phenomenon is especially pronounced for the low-resource Indic palm-leaf manuscript domain. To address the issue, we first introduce Indiscapes2, a new large-scale diverse dataset of Indic manuscripts with semantic layout annotations. Indiscapes2 contains documents from four different historical collections and is 150% larger than its predecessor, Indiscapes. We also propose a novel deep network Palmira for robust, deformation-aware instance segmentation of regions in handwritten manuscripts. We also report Hausdorff distance and its variants as a boundary-aware performance measure. Our experiments demonstrate that Palmira provides robust layouts, outperforms strong baseline approaches and ablative variants. We also include qualitative results on Arabic, South-East Asian and Hebrew historical manuscripts to showcase the generalization capability of Palmira.
Thing Constellation Visualizer: Exploring Emergent Relationships of Everyday Objects
Venue: arXiv cs.HC 2021
Year: 2021

Designing future IoT ecosystems requires new approaches and perspectives to understand everyday practices. While researchers recognize the importance of understanding social aspects of everyday objects, limited studies have explored the possibilities of combining data-driven patterns with human interpretations to investigate emergent relationships among objects. This work presents Thing Constellation Visualizer (thingCV), a novel interactive tool for visualizing the social network of objects based on their co-occurrence as computed from a large collection of photos. ThingCV enables perspective-changing design explorations over the network of objects with scalable links. Two exploratory workshops were conducted to investigate how designers navigate and make sense of a network of objects through thingCV. The results of eight participants showed that designers were actively engaged in identifying interesting objects and their associated clusters of related objects. The designers projected social qualities onto the identified objects and their communities. Furthermore, the designers changed their perspectives to revisit familiar contexts and to generate new insights through the exploration process. This work contributes a novel approach to combining data-driven models with designerly interpretations of thing constellation towards More-Than Human-Centred Design of IoT ecosystems.
DeepEdgeBench: Benchmarking Deep Neural Networks on Edge Devices
Venue: arXiv cs.AI 2021
Year: 2021

EdgeAI (Edge computing based Artificial Intelligence) has been most actively researched for the last few years to handle variety of massively distributed AI applications to meet up the strict latency requirements. Meanwhile, many companies have released edge devices with smaller form factors (low power consumption and limited resources) like the popular Raspberry Pi and Nvidia's Jetson Nano for acting as compute nodes at the edge computing environments. Although the edge devices are limited in terms of computing power and hardware resources, they are powered by accelerators to enhance their performance behavior. Therefore, it is interesting to see how AI-based Deep Neural Networks perform on such devices with limited resources. In this work, we present and compare the performance in terms of inference time and power consumption of the four Systems on a Chip (SoCs): Asus Tinker Edge R, Raspberry Pi 4, Google Coral Dev Board, Nvidia Jetson Nano, and one microcontroller: Arduino Nano 33 BLE, on different deep learning models and frameworks. We also provide a method for measuring power consumption, inference time and accuracy for the devices, which can be easily extended to other devices. Our results showcase that, for Tensorflow based quantized model, the Google Coral Dev Board delivers the best performance, both for inference time and power consumption. For a low fraction of inference computation time, i.e. less than 29.3% of the time for MobileNetV2, the Jetson Nano performs faster than the other devices.
Flikcer -- A Chrome Extension to Resolve Online Epileptogenic Visual Content with Real-Time Luminance Frequency Analysis
Venue: arXiv cs.AI 2021
Year: 2021

Video content with fast luminance variations, or with spatial patterns of high contrast - referred to as epileptogenic visual content - may induce seizures on viewers with photosensitive epilepsy, and even cause discomfort in users not affected by this disease. Flikcer is a web app in the form of a website and chrome extension which aims to resolve epileptic content in videos. It provides the number of possible triggers for a seizure. It also provides the timestamps for these triggers along with a safer version of the video, free to download. The algorithm is written in Python and uses machine learning and computer vision. A key aspect of the algorithm is its computational efficiency, allowing real time implementation for public users.
Wireless Sensor Networks for Optimisation of Search and Rescue Management in Floods
Venue: arXiv cs.NI 2021
Year: 2021

We propose a novel search-and-rescue management method that relies on the aerial deployment of Wireless Sensor Network (WSN) for locating victims after floods. The sensor nodes will collect vital information such as heat signatures for detecting human presence and location, the flow of flood. The sensor modules are packed in a portable floating buoy with a user interface to convey emergency messages to the base station. Sensor nodes are designed based on disaster conditions, cost-effectiveness and deployed in the affected region by a centrifugal dispersion system from a helicopter. A mobile ad-hoc network is set up by modifying the Low Energy Adaptive Cluster Hierarchy (LEACH) protocol for greater efficiency and adoption of multi-hop of Cluster Heads for long-distance communication to Base Station. The model metrics have been defined considering previous rural floods in India. The efficiency and power characteristics of the network are compared to other protocols via simulations. The sensor data from the network makes resource management, rescue planning and emergency priority more efficient, thus saving more lives from floods.
Event-Triggered Control for Weight-Unbalanced Directed Networks
Venue: arXiv cs.RO 2021
Year: 2021

We develop an event-triggered control strategy for a weighted-unbalanced directed homogeneous robot network to reach a dynamic consensus in this work. We present some guarantees for synchronizing a robot network when all robots have access to the reference and when a limited number of robots have access. The proposed event-triggered control can reduce and avoid the periodic updating of the signals. Unlike some current control methods, we prove stability by making use of a logarithmic norm, which extends the possibilities of the control law to be applied to a wide range of directed graphs, in contrast to other works where the event-triggered control can be only implemented over strongly connected and weight-balanced digraphs. We test the performance of our algorithm by carrying out experiments both in simulation and in a real team of robots.
Apache Submarine: A Unified Machine Learning Platform Made Simple
Venue: arXiv cs.DC 2021
Year: 2021

As machine learning is applied more widely, it is necessary to have a machine learning platform for both infrastructure administrators and users including expert data scientists and citizen data scientists to improve their productivity. However, existing machine learning platforms are ill-equipped to address the "Machine Learning tech debts" such as glue code, reproducibility, and portability. Furthermore, existing platforms only take expert data scientists into consideration, and thus they are inflexible for infrastructure administrators and non-user-friendly for citizen data scientists. We propose Submarine, a unified machine learning platform, to address the challenges.
Bias for the Trace of the Resolvent and Its Application on Non-Gaussian and Non-centered MIMO Channels
Venue: arXiv cs.IT 2021
Year: 2021

The mutual information (MI) of Gaussian multi-input multi-output (MIMO) channels has been evaluated by utilizing random matrix theory (RMT) and shown to asymptotically follow Gaussian distribution, where the ergodic mutual information (EMI) converges to a deterministic quantity. However, with non-Gaussian channels, there is a bias between the EMI and its deterministic equivalent (DE), whose evaluation is not available in the literature. This bias of the EMI is related to the bias for the trace of the resolvent in large RMT. In this paper, we first derive the bias for the trace of the resolvent, which is further extended to compute the bias for the linear spectral statistics (LSS). Then, we apply the above results on non-Gaussian MIMO channels to determine the bias for the EMI. It is also proved that the bias for the EMI is $-0.5$ times of that for the variance of the MI. Finally, the derived bias is utilized to modify the central limit theory (CLT) and calculate the outage probability. Numerical results show that the modified CLT significantly outperforms previous methods in approximating the distribution of the MI and improves the accuracy for the outage probability evaluation.
Sequence Reconstruction for Limited-Magnitude Errors
Venue: arXiv cs.IT 2021
Year: 2021

Motivated by applications to DNA storage, we study reconstruction and list-reconstruction schemes for integer vectors that suffer from limited-magnitude errors. We characterize the asymptotic size of the intersection of error balls in relation to the code's minimum distance. We also devise efficient reconstruction algorithms for various limited-magnitude error parameter ranges. We then extend these algorithms to the list-reconstruction scheme, and show the trade-off between the asymptotic list size and the number of required channel outputs. These results apply to all codes, without any assumptions on the code structure. Finally, we also study linear reconstruction codes with small intersection, as well as show a connection to list-reconstruction codes for the tandem-duplication channel.
Rainfall-runoff prediction using a Gustafson-Kessel clustering based Takagi-Sugeno Fuzzy model
Venue: arXiv cs.LG 2021
Year: 2021

A rainfall-runoff model predicts surface runoff either using a physically-based approach or using a systems-based approach. Takagi-Sugeno (TS) Fuzzy models are systems-based approaches and a popular modeling choice for hydrologists in recent decades due to several advantages and improved accuracy in prediction over other existing models. In this paper, we propose a new rainfall-runoff model developed using Gustafson-Kessel (GK) clustering-based TS Fuzzy model. We present comparative performance measures of GK algorithms with two other clustering algorithms: (i) Fuzzy C-Means (FCM), and (ii)Subtractive Clustering (SC). Our proposed TS Fuzzy model predicts surface runoff using: (i) observed rainfall in a drainage basin and (ii) previously observed precipitation flow in the basin outlet. The proposed model is validated using the rainfall-runoff data collected from the sensors installed on the campus of the Indian Institute of Technology, Kharagpur. The optimal number of rules of the proposed model is obtained by different validation indices. A comparative study of four performance criteria: RootMean Square Error (RMSE), Coefficient of Efficiency (CE), Volumetric Error (VE), and Correlation Coefficient of Determination(R) have been quantitatively demonstrated for each clustering algorithm.
Improving Distantly Supervised Relation Extraction with Self-Ensemble Noise Filtering
Venue: arXiv cs.CL 2021
Year: 2021

Distantly supervised models are very popular for relation extraction since we can obtain a large amount of training data using the distant supervision method without human annotation. In distant supervision, a sentence is considered as a source of a tuple if the sentence contains both entities of the tuple. However, this condition is too permissive and does not guarantee the presence of relevant relation-specific information in the sentence. As such, distantly supervised training data contains much noise which adversely affects the performance of the models. In this paper, we propose a self-ensemble filtering mechanism to filter out the noisy samples during the training process. We evaluate our proposed framework on the New York Times dataset which is obtained via distant supervision. Our experiments with multiple state-of-the-art neural relation extraction models show that our proposed filtering mechanism improves the robustness of the models and increases their F1 scores.