OpenRaiser/Intern-Atlas huggingface dataset:
Energy Efficiency Maximization in the Uplink Delta-OMA Networks
Venue: arXiv cs.IT 2021
Year: 2021

Delta-orthogonal multiple access (D-OMA) has been recently investigated as a potential technique to enhance the spectral efficiency in the sixth-generation (6G) networks. D-OMA enables partial overlapping of the adjacent sub-channels that are assigned to different clusters of users served by non-orthogonal multiple access (NOMA), at the expense of additional interference. In this paper, we analyze the performance of D-OMA in the uplink and develop a multi-objective optimization framework to maximize the uplink energy efficiency (EE) in a multi-access point (AP) network enabled by D-OMA. Specifically, we optimize the sub-channel and transmit power allocations of the users as well as the overlapping percentage of the spectrum between the adjacent sub-channels. The formulated problem is a mixed binary non-linear programming problem. Therefore, to address the challenge we first transform the problem into a single-objective problem using Tchebycheff method. Then, we apply the monotonic optimization (MO) to explore the hidden monotonicity of the objective function and constraints, and reformulate the problem into a standard MO in canonical form. The reformulated problem is then solved by applying the outer polyblock approximation method. Our numerical results show that D-OMA outperforms the conventional non-orthogonal multiple access (NOMA) and orthogonal frequency division multiple access (OFDMA) when the adjacent sub-channel overlap and scheduling are optimized jointly.
Oversampled Adaptive Sensing via a Predefined Codebook
Venue: arXiv cs.IT 2021
Year: 2021

Oversampled adaptive sensing (OAS) is a Bayesian framework recently proposed for effective sensing of structured signals in a time-limited setting. In contrast to the conventional blind oversampling, OAS uses the prior information on the signal to construct posterior beliefs sequentially. These beliefs help in constructive oversampling which iteratively evolves through a sequence of time sub-frames. The initial studies of OAS consider the idealistic assumption of full control on sensing coefficients which is not feasible in many applications. In this work, we extend the initial investigations on OAS to more realistic settings in which the sensing coefficients are selected from a predefined set of possible choices, referred to as the codebook. We extend the OAS framework to these settings and compare its performance with classical non-adaptive approaches.
Fast Distributed Complex Join Processing
Venue: arXiv cs.DB 2021
Year: 2021

In this work, we study the problem of co-optimize communication, pre-computing, and computation cost in one-round multi-way join evaluation. We propose a multi-way join approach ADJ (Adaptive Distributed Join) for complex join which finds one optimal query plan to process by exploring cost-effective partial results in terms of the trade-off between pre-computing, communication, and computation.We analyze the input relations for a given join query and find one optimal over a set of query plans in some specific form, with high-quality cost estimation by sampling. Our extensive experiments confirm that ADJ outperforms the existing multi-way join methods by up to orders of magnitude.
Collective Intelligence: Decentralized Learning for Android Malware Detection in IoT with Blockchain
Venue: arXiv cs.CR 2021
Year: 2021

The widespread significance of Android IoT devices is due to its flexibility and hardware support features which revolutionized the digital world by introducing exciting applications almost in all walks of daily life, such as healthcare, smart cities, smart environments, safety, remote sensing, and many more. Such versatile applicability gives incentive for more malware attacks. In this paper, we propose a framework which continuously aggregates multiple user trained models on non-overlapping data into single model. Specifically for malware detection task, (i) we propose a novel user (local) neural network (LNN) which trains on local distribution and (ii) then to assure the model authenticity and quality, we propose a novel smart contract which enable aggregation process over blokchain platform. The LNN model analyzes various static and dynamic features of both malware and benign whereas the smart contract verifies the malicious applications both for uploading and downloading processes in the network using stored aggregated features of local models. In this way, the proposed model not only improves malware detection accuracy using decentralized model network but also model efficacy with blockchain. We evaluate our approach with three state-of-the-art models and performed deep analyses of extracted features of the relative model.
A Variable Vector Length SIMD Architecture for HW/SW Co-designed Processors
Venue: arXiv cs.AR 2021
Year: 2021

Hardware/Software (HW/SW) co-designed processors provide a promising solution to the power and complexity problems of the modern microprocessors by keeping their hardware simple. Moreover, they employ several runtime optimizations to improve the performance. One of the most potent optimizations, vectorization, has been utilized by modern microprocessors, to exploit the data level parallelism through SIMD accelerators. Due to their hardware simplicity, these accelerators have evolved in terms of width from 64-bit vectors in Intel MMX to 512-bit wide vector units in Intel Xeon Phi and AVX-512. Although SIMD accelerators are simple in terms of hardware design, code generation for them has always been a challenge. Moreover, increasing vector lengths with each new generation add to this complexity. This paper explores the scalability of SIMD accelerators from the code generation point of view. We discover that the SIMD accelerators remain underutilized at higher vector lengths mainly due to: a) reduced dynamic instruction stream coverage for vectorization and b) increase in permutations. Both of these factors can be attributed to the rigidness of the SIMD architecture. We propose a novel SIMD architecture that possesses the flexibility needed to support higher vector lengths. Furthermore, we propose Variable Length Vectorization and Selective Writing in a HW/SW co-designed environment to transparently target the flexibility of the proposed architecture. We evaluate our proposals using a set of SPECFP2006 and Physicsbench applications. Our experimental results show an average dynamic instruction reduction of 31% and 40% and an average speed up of 13% and 10% for SPECFP2006 and Physicsbench respectively, for 512-bit vector length, over the scalar baseline code.
Inverse deformation analysis: an experimental and numerical assessment using the FEniCS Project
Venue: arXiv cs.CE 2021
Year: 2021

In this paper, we develop a framework for solving inverse deformation problems using the FEniCS Project finite element software. We validate our approach with experimental imaging data acquired from a soft silicone beam under gravity. In contrast with inverse iterative algorithms that require multiple solutions of a standard elasticity problem, the proposed method can compute the undeformed configuration by solving only one modified elasticity problem. This modified problem has a complexity comparable to the standard one. The framework is implemented within an open-source pipeline enabling the direct and inverse deformation simulation directly from imaging data. We use the high-level Unified Form Language (UFL) of the FEniCS Project to express the finite element model in variational form and to automatically derive the consistent Jacobian. Consequently, the design of the pipeline is flexible: for example, it allows the modification of the constitutive models by changing a single line of code. We include a complete working example showing the inverse deformation of a beam deformed by gravity as supplementary material.
Open-domain question classification and completion in conversational information search
Venue: arXiv cs.IR 2021
Year: 2021

Searching for new information requires talking to the system. In this research, an Open-domain Conversational information search system has been developed. This system has been implemented using the TREC CAsT 2019 track, which is one of the first attempts to build a framework in this area. According to the user's previous questions, the system firstly completes the question (using the first and the previous question in each turn) and then classifies it (based on the question words). This system extracts the related answers according to the rules of each question. In this research, a simple yet effective method with high performance has been used, which on average, extracts 20% more relevant results than the baseline.
History-Augmented Collaborative Filtering for Financial Recommendations
Venue: arXiv cs.LG 2021
Year: 2021

In many businesses, and particularly in finance, the behavior of a client might drastically change over time. It is consequently crucial for recommender systems used in such environments to be able to adapt to these changes. In this study, we propose a novel collaborative filtering algorithm that captures the temporal context of a user-item interaction through the users' and items' recent interaction histories to provide dynamic recommendations. The algorithm, designed with issues specific to the financial world in mind, uses a custom neural network architecture that tackles the non-stationarity of users' and items' behaviors. The performance and properties of the algorithm are monitored in a series of experiments on a G10 bond request for quotation proprietary database from BNP Paribas Corporate and Institutional Banking.
Experiments with Rich Regime Training for Deep Learning
Venue: arXiv cs.LG 2021
Year: 2021

In spite of advances in understanding lazy training, recent work attributes the practical success of deep learning to the rich regime with complex inductive bias. In this paper, we study rich regime training empirically with benchmark datasets, and find that while most parameters are lazy, there is always a small number of active parameters which change quite a bit during training. We show that re-initializing (resetting to their initial random values) the active parameters leads to worse generalization. Further, we show that most of the active parameters are in the bottom layers, close to the input, especially as the networks become wider. Based on such observations, we study static Layer-Wise Sparse (LWS) SGD, which only updates some subsets of layers. We find that only updating the top and bottom layers have good generalization and, as expected, only updating the top layers yields a fast algorithm. Inspired by this, we investigate probabilistic LWS-SGD, which mostly updates the top layers and occasionally updates the full network. We show that probabilistic LWS-SGD matches the generalization performance of vanilla SGD and the back-propagation time can be 2-5 times more efficient.
Nested-block self-attention for robust radiotherapy planning segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Although deep convolutional networks have been widely studied for head and neck (HN) organs at risk (OAR) segmentation, their use for routine clinical treatment planning is limited by a lack of robustness to imaging artifacts, low soft tissue contrast on CT, and the presence of abnormal anatomy. In order to address these challenges, we developed a computationally efficient nested block self-attention (NBSA) method that can be combined with any convolutional network. Our method achieves computational efficiency by performing non-local calculations within memory blocks of fixed spatial extent. Contextual dependencies are captured by passing information in a raster scan order between blocks, as well as through a second attention layer that causes bi-directional attention flow. We implemented our approach on three different networks to demonstrate feasibility. Following training using 200 cases, we performed comprehensive evaluations using conventional and clinical metrics on a separate set of 172 test scans sourced from external and internal institution datasets without any exclusion criteria. NBSA required a similar number of computations (15.7 gflops) as the most efficient criss-cross attention (CCA) method and generated significantly more accurate segmentations for brain stem (Dice of 0.89 vs. 0.86) and parotid glands (0.86 vs. 0.84) than CCA. NBSA's segmentations were less variable than multiple 3D methods, including for small organs with low soft-tissue contrast such as the submandibular glands (surface Dice of 0.90).
Evaluate On-the-job Learning Dialogue Systems and a Case Study for Natural Language Understanding
Venue: arXiv cs.CL 2021
Year: 2021

On-the-job learning consists in continuously learning while being used in production, in an open environment, meaning that the system has to deal on its own with situations and elements never seen before. The kind of systems that seem to be especially adapted to on-the-job learning are dialogue systems, since they can take advantage of their interactions with users to collect feedback to adapt and improve their components over time. Some dialogue systems performing on-the-job learning have been built and evaluated but no general methodology has yet been defined. Thus in this paper, we propose a first general methodology for evaluating on-the-job learning dialogue systems. We also describe a task-oriented dialogue system which improves on-the-job its natural language component through its user interactions. We finally evaluate our system with the described methodology.
Federated Edge Learning with Misaligned Over-The-Air Computation
Venue: arXiv cs.IT 2021
Year: 2021

Over-the-air computation (OAC) is a promising technique to realize fast model aggregation in the uplink of federated edge learning. OAC, however, hinges on accurate channel-gain precoding and strict synchronization among the edge devices, which are challenging in practice. As such, how to design the maximum likelihood (ML) estimator in the presence of residual channel-gain mismatch and asynchronies is an open problem. To fill this gap, this paper formulates the problem of misaligned OAC for federated edge learning and puts forth a whitened matched filtering and sampling scheme to obtain oversampled, but independent, samples from the misaligned and overlapped signals. Given the whitened samples, a sum-product ML estimator and an aligned-sample estimator are devised to estimate the arithmetic sum of the transmitted symbols. In particular, the computational complexity of our sum-product ML estimator is linear in the packet length and hence is significantly lower than the conventional ML estimator. Extensive simulations on the test accuracy versus the average received energy per symbol to noise power spectral density ratio (EsN0) yield two main results: 1) In the low EsN0 regime, the aligned-sample estimator can achieve superior test accuracy provided that the phase misalignment is non-severe. In contrast, the ML estimator does not work well due to the error propagation and noise enhancement in the estimation process. 2) In the high EsN0 regime, the ML estimator attains the optimal learning performance regardless of the severity of phase misalignment. On the other hand, the aligned-sample estimator suffers from a test-accuracy loss caused by phase misalignment.
Software-Supported Audits of Decision-Making Systems: Testing Google and Facebook's Political Advertising Policies
Venue: arXiv cs.HC 2021
Year: 2021

How can society understand and hold accountable complex human and algorithmic decision-making systems whose systematic errors are opaque to the public? These systems routinely make decisions on individual rights and well-being, and on protecting society and the democratic process. Practical and statistical constraints on external audits--such as dimensional complexity--can lead researchers and regulators to miss important sources of error in these complex decision-making systems. In this paper, we design and implement a software-supported approach to audit studies that auto-generates audit materials and coordinates volunteer activity. We implemented this software in the case of political advertising policies enacted by Facebook and Google during the 2018 U.S. election. Guided by this software, a team of volunteers posted 477 auto-generated ads and analyzed the companies' actions, finding systematic errors in how companies enforced policies. We find that software can overcome some common constraints of audit studies, within limitations related to sample size and volunteer capacity.
An HPC-Based Hydrothermal Finite Element Simulator for Modeling Underground Response to Community-Scale Geothermal Energy Production
Venue: arXiv cs.DC 2021
Year: 2021

Geothermal heat, as renewable energy, shows great advantage with respect to its environmental impact due to its significantly lower CO2 emissions than conventional fossil fuel. Open and closed-loop geothermal heat pumps, which utilize shallow geothermal systems, are an efficient technology for cooling and heating buildings, especially in urban areas. Integrated use of geothermal energy technologies for district heating, cooling, and thermal energy storage can be applied to optimize the subsurface for communities to provide them with multiple sustainable energy and community resilience benefits. The utilization of the subsurface resources may lead to a variation in the underground environment, which might further impact local environmental conditions. However, very few simulators can handle such a highly complex set of coupled computations on a regional or city scale. We have developed high-performance computing (HPC) based hydrothermal finite element (FE) simulator that can simulate the subsurface and its hydrothermal conditions at a scale of tens of km. The HPC simulator enables us to investigate the subsurface thermal and hydrologic response to the built underground environment (such as basements and subways) at the community scale. In this study, a coupled hydrothermal simulator is developed based on the open-source finite element library deal.II. The HPC simulator was validated by comparing the results of a benchmark case study against COMSOL Multiphysics, in which Aquifer Thermal Energy Storage (ATES) is modeled and a process of heat injection into ATES is simulated. The use of an energy pile system at the Treasure Island redevelopment site (San Francisco, CA, USA) was selected as a case study to demonstrate the HPC capability of the developed simulator. The simulator is capable of modeling multiple city-scale geothermal scenarios in a reasonable amount of time.
Reducing Conservativeness Oriented Offline Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

In offline reinforcement learning, a policy learns to maximize cumulative rewards with a fixed collection of data. Towards conservative strategy, current methods choose to regularize the behavior policy or learn a lower bound of the value function. However, exorbitant conservation tends to impair the policy's generalization ability and degrade its performance, especially for the mixed datasets. In this paper, we propose the method of reducing conservativeness oriented reinforcement learning. On the one hand, the policy is trained to pay more attention to the minority samples in the static dataset to address the data imbalance problem. On the other hand, we give a tighter lower bound of value function than previous methods to discover potential optimal actions. Consequently, our proposed method is able to tackle the skewed distribution of the provided dataset and derive a value function closer to the expected value function. Experimental results demonstrate that our proposed method outperforms the state-of-the-art methods in D4RL offline reinforcement learning evaluation tasks and our own designed mixed datasets.
Dynamic Oversampling Tecniques for 1-Bit ADCs in Large-Scale MIMO Systems
Venue: arXiv cs.IT 2021
Year: 2021

In this work, we investigate dynamic oversampling techniques for large-scale multiple-antenna systems equipped with low-cost and low-power 1-bit analog-to-digital converters at the base stations. To compensate for the performance loss caused by the coarse quantization, oversampling is applied at the receiver. Unlike existing works that use uniform oversampling, which samples the signal at a constant rate, a novel dynamic oversampling scheme is proposed. The basic idea is to perform time-varying nonuniform oversampling, which selects samples with nonuniform patterns that vary over time. We consider two system design criteria: a design that maximizes the achievable sum rate and another design that minimizes the mean square error of detected symbols. Dynamic oversampling is carried out using a dimension reduction matrix $\mathbfΔ$, which can be computed by the generalized eigenvalue decomposition or by novel submatrix-level feature selection algorithms. Moreover, the proposed scheme is analyzed in terms of convergence, computational complexity and power consumption at the receiver. Simulations show that systems with the proposed dynamic oversampling outperform those with uniform oversampling in terms of computational cost, achievable sum rate and symbol error rate performance.
Improving Longer-range Dialogue State Tracking
Venue: arXiv cs.CL 2021
Year: 2021

Dialogue state tracking (DST) is a pivotal component in task-oriented dialogue systems. While it is relatively easy for a DST model to capture belief states in short conversations, the task of DST becomes more challenging as the length of a dialogue increases due to the injection of more distracting contexts. In this paper, we aim to improve the overall performance of DST with a special focus on handling longer dialogues. We tackle this problem from three perspectives: 1) A model designed to enable hierarchical slot status prediction; 2) Balanced training procedure for generic and task-specific language understanding; 3) Data perturbation which enhances the model's ability in handling longer conversations. We conduct experiments on the MultiWOZ benchmark, and demonstrate the effectiveness of each component via a set of ablation tests, especially on longer conversations.
Successive Subspace Learning: An Overview
Venue: arXiv cs.CV 2021
Year: 2021

Successive Subspace Learning (SSL) offers a light-weight unsupervised feature learning method based on inherent statistical properties of data units (e.g. image pixels and points in point cloud sets). It has shown promising results, especially on small datasets. In this paper, we intuitively explain this method, provide an overview of its development, and point out some open questions and challenges for future research.
NEUROSPF: A tool for the Symbolic Analysis of Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

This paper presents NEUROSPF, a tool for the symbolic analysis of neural networks. Given a trained neural network model, the tool extracts the architecture and model parameters and translates them into a Java representation that is amenable for analysis using the Symbolic PathFinder symbolic execution tool. Notably, NEUROSPF encodes specialized peer classes for parsing the model's parameters, thereby enabling efficient analysis. With NEUROSPF the user has the flexibility to specify either the inputs or the network internal parameters as symbolic, promoting the application of program analysis and testing approaches from software engineering to the field of machine learning. For instance, NEUROSPF can be used for coverage-based testing and test generation, finding adversarial examples and also constraint-based repair of neural networks, thus improving the reliability of neural networks and of the applications that use them. Video URL: https://youtu.be/seal8fG78LI