OpenRaiser/Intern-Atlas huggingface dataset:
Adaptive Modeling Against Adversarial Attacks
Venue: arXiv cs.LG 2021
Year: 2021

Adversarial training, the process of training a deep learning model with adversarial data, is one of the most successful adversarial defense methods for deep learning models. We have found that the robustness to white-box attack of an adversarially trained model can be further improved if we fine tune this model in inference stage to adapt to the adversarial input, with the extra information in it. We introduce an algorithm that "post trains" the model at inference stage between the original output class and a "neighbor" class, with existing training data. The accuracy of pre-trained Fast-FGSM CIFAR10 classifier base model against white-box projected gradient attack (PGD) can be significantly improved from 46.8% to 64.5% with our algorithm.
Hierarchical Multi-Building And Multi-Floor Indoor Localization Based On Recurrent Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

There has been an increasing tendency to move from outdoor to indoor lifestyle in modern cities. The emergence of big shopping malls, indoor sports complexes, factories, and warehouses is accelerating this tendency. In such an environment, indoor localization becomes one of the essential services, and the indoor localization systems to be deployed should be scalable enough to cover the expected expansion of those indoor facilities. One of the most economical and practical approaches to indoor localization is Wi-Fi fingerprinting, which exploits the widely-deployed Wi-Fi networks using mobile devices (e.g., smartphones) without any modification of the existing infrastructure. Traditional Wi-Fi fingerprinting schemes rely on complicated data pre/post-processing and time-consuming manual parameter tuning. In this paper, we propose hierarchical multi-building and multi-floor indoor localization based on a recurrent neural network (RNN) using Wi-Fi fingerprinting, eliminating the need of complicated data pre/post-processing and with less parameter tuning. The RNN in the proposed scheme estimates locations in a sequential manner from a general to a specific one (e.g., building->floor->location) in order to exploit the hierarchical nature of the localization in multi-building and multi-floor environments. The experimental results with the UJIIndoorLoc dataset demonstrate that the proposed scheme estimates building and floor with 100% and 95.24% accuracy, respectively, and provides three-dimensional positioning error of 8.62 m, which outperforms existing deep neural network-based schemes.
Pose Adaptive Dual Mixup for Few-Shot Single-View 3D Reconstruction
Venue: arXiv cs.CV 2021
Year: 2021

We present a pose adaptive few-shot learning procedure and a two-stage data interpolation regularization, termed Pose Adaptive Dual Mixup (PADMix), for single-image 3D reconstruction. While augmentations via interpolating feature-label pairs are effective in classification tasks, they fall short in shape predictions potentially due to inconsistencies between interpolated products of two images and volumes when rendering viewpoints are unknown. PADMix targets this issue with two sets of mixup procedures performed sequentially. We first perform an input mixup which, combined with a pose adaptive learning procedure, is helpful in learning 2D feature extraction and pose adaptive latent encoding. The stagewise training allows us to build upon the pose invariant representations to perform a follow-up latent mixup under one-to-one correspondences between features and ground-truth volumes. PADMix significantly outperforms previous literature on few-shot settings over the ShapeNet dataset and sets new benchmarks on the more challenging real-world Pix3D dataset.
Integrating Material Selection with Design Optimization via Neural Networks
Venue: arXiv cs.CE 2021
Year: 2021

The engineering design process often entails optimizing the underlying geometry while simultaneously selecting a suitable material. For a certain class of simple problems, the two are separable where, for example, one can first select an optimal material, and then optimize the geometry. However, in general, the two are not separable. Furthermore, the discrete nature of material selection is not compatible with gradient-based geometry optimization, making simultaneous optimization challenging. In this paper, we propose the use of variational autoencoders (VAE) for simultaneous optimization. First, a data-driven VAE is used to project the discrete material database onto a continuous and differentiable latent space. This is then coupled with a fully-connected neural network, embedded with a finite-element solver, to simultaneously optimize the material and geometry. The neural-network's built-in gradient optimizer and back-propagation are exploited during optimization. The proposed framework is demonstrated using trusses, where an optimal material needs to be chosen from a database, while simultaneously optimizing the cross-sectional areas of the truss members. Several numerical examples illustrate the efficacy of the proposed framework. The Python code used in these experiments is available at github.com/UW-ERSL/MaTruss
NVS-MonoDepth: Improving Monocular Depth Prediction with Novel View Synthesis
Venue: arXiv cs.CV 2021
Year: 2021

Building upon the recent progress in novel view synthesis, we propose its application to improve monocular depth estimation. In particular, we propose a novel training method split in three main steps. First, the prediction results of a monocular depth network are warped to an additional view point. Second, we apply an additional image synthesis network, which corrects and improves the quality of the warped RGB image. The output of this network is required to look as similar as possible to the ground-truth view by minimizing the pixel-wise RGB reconstruction error. Third, we reapply the same monocular depth estimation onto the synthesized second view point and ensure that the depth predictions are consistent with the associated ground truth depth. Experimental results prove that our method achieves state-of-the-art or comparable performance on the KITTI and NYU-Depth-v2 datasets with a lightweight and simple vanilla U-Net architecture.
AcME -- Accelerated Model-agnostic Explanations: Fast Whitening of the Machine-Learning Black Box
Venue: arXiv cs.LG 2021
Year: 2021

In the context of human-in-the-loop Machine Learning applications, like Decision Support Systems, interpretability approaches should provide actionable insights without making the users wait. In this paper, we propose Accelerated Model-agnostic Explanations (AcME), an interpretability approach that quickly provides feature importance scores both at the global and the local level. AcME can be applied a posteriori to each regression or classification model. Not only does AcME compute feature ranking, but it also provides a what-if analysis tool to assess how changes in features values would affect model predictions. We evaluated the proposed approach on synthetic and real-world datasets, also in comparison with SHapley Additive exPlanations (SHAP), the approach we drew inspiration from, which is currently one of the state-of-the-art model-agnostic interpretability approaches. We achieved comparable results in terms of quality of produced explanations while reducing dramatically the computational time and providing consistent visualization for global and local interpretations. To foster research in this field, and for the sake of reproducibility, we also provide a repository with the code used for the experiments.
Using Silent Writes in Low-Power Traffic-Aware ECC
Venue: arXiv cs.PF 2021
Year: 2021

Using Error Detection Code (EDC) and Error Correction Code (ECC) is a noteworthy way to increase cache memories robustness against soft errors. EDC enables detecting errors in cache memory while ECC is used to correct erroneous cache blocks. ECCs are often costly as they impose considerable area and energy overhead on cache memory. Reducing this overhead has been the subject of many studies. In particular, a previous study has suggested mapping ECC to the main memory at the expense of high cache traffic and energy. A major source of this excessive traffic and energy is the high frequency of cache writes. In this work, we show that a significant portion of cache writes are silent, i.e., they write the same data already existing. We build on this observation and introduce Traffic-aware ECC (or simply TCC). TCC detects silent writes by an efficient mechanism. Once such writes are detected updating their ECC is avoided effectively reducing L2 cache traffic and access frequency. Using our solution, we reduce L2 cache access frequency by 8% while maintaining performance. We reduce L2 cache dynamic and overall cache energy by up to 32% and 8%, respectively. Furthermore, TCC reduces L2 cache miss rate by 3%.
Dynamic Suffix Array with Sub-linear update time and Poly-logarithmic Lookup Time
Venue: arXiv cs.DS 2021
Year: 2021

The Suffix Array $SA_S[1\ldots n]$ of an $n$-length string $S$ is a lexicographically sorted array of the suffixes of $S$. The suffix array is one of the most well known and widely used data structures in string algorithms. We present a data structure for maintaining a representation of the suffix array of a dynamic string which undergoes symbol substitutions, deletions, and insertions. For every string manipulation, our data structure can be updated in $O(n^{\frac{2}{3}})$ time (ignoring multiplicative polylogarithmic factors) with $n$ being the current length of the string. For an input query $i\in [1\ldots n]$, our data structure reports $SA_S[i]$ in $O(\log^5(n))$ time. We also present a faster data structure, with $O(\sqrt{n})$ update time (ignoring multiplicative polylogarithmic factors), for maintaining the Inverted Suffix Array of a dynamic string undergoing symbol substitutions updates. For an input query $i\in [1\ldots n]$, our data structure reports the $i$'th entry in the inverted suffix array in $O(\log^4(n))$ time. Our data structures can be used to obtain sub-linear dynamic algorithms for several classical string problems for which efficient dynamic solutions were not previously known.
XDWM: A 2D Domain Wall Memory
Venue: arXiv cs.ET 2021
Year: 2021

Domain-Wall Memory (DWM) structures typically bundle nanowires shifted together for parallel access. Ironically, this organization does not allow the natural shifting of DWM to realize \textit{logical shifting} within data elements. We describe a novel 2-D DWM cross-point (X-Cell) that allows two individual nanowires placed orthogonally to share the X-Cell. Each nanowire can operate independently while sharing the value at the X-Cell. Using X-Cells, we propose an orthogonal nanowire in the Y dimension overlaid on a bundle of X dimension nanowires for a cross-DWM or XDWM. We demonstrate that the bundle shifts correctly in the X-Direction, and that data can be logically shifted in the Y-direction providing novel data movement and supporting processing-in-memory. We conducted studies on the requirements for physical cell dimensions and shift currents for XDWM. Due to the non-standard domain, our micro-magnetic studies demonstrate that XDWM introduces a shift current penalty of 6.25% while shifting happens in one nanowire compared to a standard nanowire. We also demonstrate correct shifting using nanowire bundles in both the X- and Y- dimensions. Using magnetic simulation to derive the values for SPICE simulation we show the maximum leakage current between nanowires when shifting the bundle together is $\le3$% indicating that sneak paths are not problematic for XDWM.
Out to Explore the Cybersecurity Planet
Venue: arXiv cs.CR 2021
Year: 2021

Security ceremonies still fail despite decades of efforts by researchers and practitioners. Attacks are often a cunning amalgam of exploits for technical systems and of forms of human behaviour. For example, this is the case with the recent news headline of a large-scale attack against Electrum Bitcoin wallets, which manages to spread a malicious update of the wallet app. I therefore set out to look at things through a different lens. I make the (metaphorical) hypothesis that human ancestors arrived on Earth along with security ceremonies from a very far planet, the Cybersecurity planet. My hypothesis continues, in that studying (by huge telescopes) the surface of Cybersecurity in combination with the logical projection on that surface of what happens on Earth is beneficial for us earthlings. I have spotted four cities so far on the remote planet. Democratic City features security ceremonies that allow inhabitants to follow personal paths of practice and, for example, make errors or be driven by emotions. By contrast, security ceremonies in Dictatorial City compel inhabitants to comply, thus behaving like programmed automata. Security ceremonies in Beautiful City are so beautiful that inhabitants just love to follow them precisely. Invisible City has security ceremonies that are not perceivable, hence inhabitants feel like they never encounter any. Incidentally, we use the words "democratic" and "dictatorial" without any political connotation. A key argument I shall develop is that all cities but Democratic City address the human factor, albeit in different ways. In the light of these findings, I will also discuss security ceremonies of our planet, such as WhatsApp web login and flight boarding, and explore room for improving them based upon the current understanding of Cybersecurity.
Bi-Directional Recurrent Neural Ordinary Differential Equations for Social Media Text Classification
Venue: arXiv cs.CL 2021
Year: 2021

Classification of posts in social media such as Twitter is difficult due to the noisy and short nature of texts. Sequence classification models based on recurrent neural networks (RNN) are popular for classifying posts that are sequential in nature. RNNs assume the hidden representation dynamics to evolve in a discrete manner and do not consider the exact time of the posting. In this work, we propose to use recurrent neural ordinary differential equations (RNODE) for social media post classification which consider the time of posting and allow the computation of hidden representation to evolve in a time-sensitive continuous manner. In addition, we propose a novel model, Bi-directional RNODE (Bi-RNODE), which can consider the information flow in both the forward and backward directions of posting times to predict the post label. Our experiments demonstrate that RNODE and Bi-RNODE are effective for the problem of stance classification of rumours in social media.
Sparsified Secure Aggregation for Privacy-Preserving Federated Learning
Venue: arXiv cs.LG 2021
Year: 2021

Secure aggregation is a popular protocol in privacy-preserving federated learning, which allows model aggregation without revealing the individual models in the clear. On the other hand, conventional secure aggregation protocols incur a significant communication overhead, which can become a major bottleneck in real-world bandwidth-limited applications. Towards addressing this challenge, in this work we propose a lightweight gradient sparsification framework for secure aggregation, in which the server learns the aggregate of the sparsified local model updates from a large number of users, but without learning the individual parameters. Our theoretical analysis demonstrates that the proposed framework can significantly reduce the communication overhead of secure aggregation while ensuring comparable computational complexity. We further identify a trade-off between privacy and communication efficiency due to sparsification. Our experiments demonstrate that our framework reduces the communication overhead by up to 7.8x, while also speeding up the wall clock training time by 1.13x, when compared to conventional secure aggregation benchmarks.
Constrained tensor factorization for computational phenotyping and mortality prediction in patients with cancer
Venue: arXiv cs.LG 2021
Year: 2021

Background: The increasing adoption of electronic health records (EHR) across the US has created troves of computable data, to which machine learning methods have been applied to extract useful insights. EHR data, represented as a three-dimensional analogue of a matrix (tensor), is decomposed into two-dimensional factors that can be interpreted as computational phenotypes. Methods: We apply constrained tensor factorization to derive computational phenotypes and predict mortality in cohorts of patients with breast, prostate, colorectal, or lung cancer in the Northwestern Medicine Enterprise Data Warehouse from 2000 to 2015. In our experiments, we examined using a supervised term in the factorization algorithm, filtering tensor co-occurrences by medical indication, and incorporating additional social determinants of health (SDOH) covariates in the factorization process. We evaluated the resulting computational phenotypes qualitatively and by assessing their ability to predict five-year mortality using the area under the curve (AUC) statistic. Results: Filtering by medical indication led to more concise and interpretable phenotypes. Mortality prediction performance (AUC) varied under the different experimental conditions and by cancer type (breast: 0.623 - 0.694, prostate: 0.603 - 0.750, colorectal: 0.523 - 0.641, and lung: 0.517 - 0.623). Generally, prediction performance improved with the use of a supervised term and the incorporation of SDOH covariates. Conclusion: Constrained tensor factorization, applied to sparse EHR data of patients with cancer, can discover computational phenotypes predictive of five-year mortality. The incorporation of SDOH variables into the factorization algorithm is an easy-to-implement and effective way to improve prediction performance.
Machine learning for Earth System Science (ESS): A survey, status and future directions for South Asia
Venue: arXiv cs.LG 2021
Year: 2021

This survey focuses on the current problems in Earth systems science where machine learning algorithms can be applied. It provides an overview of previous work, ongoing work at the Ministry of Earth Sciences, Gov. of India, and future applications of ML algorithms to some significant earth science problems. We provide a comparison of previous work with this survey, a mind map of multidimensional areas related to machine learning and a Gartner's hype cycle for machine learning in Earth system science (ESS). We mainly focus on the critical components in Earth Sciences, including atmospheric, Ocean, Seismology, and biosphere, and cover AI/ML applications to statistical downscaling and forecasting problems.
Analysis of the Delay Distribution in Cellular Networks by Using Stochastic Geometry
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, with the aid of the mathematical tool of stochastic geometry, we introduce analytical and computational frameworks for the distribution of three different definitions of delay, i.e., the time that it takes for a user to successfully receive a data packet, in large-scale cellular networks. We also provide an asymptotic analysis of one of the delay distributions, which can be regarded as the packet loss probability of a given network. To mitigate the inherent computational difficulties of the obtained analytical formulations in some cases, we propose efficient numerical approximations based on the numerical inversion method, the Riemann sum, and the Beta distribution. Finally, we demonstrate the accuracy of the obtained analytical formulations and the corresponding approximations against Monte Carlo simulation results, and unveil insights on the delay performance with respect to several design parameters, such as the decoding threshold, the transmit power, and the deployment density of the base stations. The proposed methods can facilitate the analysis and optimization of cellular networks subject to reliability constraints on the network packet delay that are not restricted to the local (average) delay, e.g., in the context of delay sensitive applications.
An exact dynamic programming algorithm, lower and upper bounds, applied to the large block sale problem
Venue: arXiv cs.DM 2021
Year: 2021

In this article, we address a class of non convex, integer, non linear mathematical programs using dynamic programming. The mathematical program considered, whose properties are studied in this article, may be used to model the optimal liquidation problem of a single asset portfolio, held in a very large quantity, in a low volatility and perfect memory market, with few market participants. In this context, the Portfolio Manager's selling actions convey information to market participants, which in turn lower bid prices and further penalize the liquidation proceeds we attempt to maximize. We show the problem can be solved exactly using Dynamic Programming (DP) in polynomial time. However, exact resolution is only efficient for small instances. For medium size and large instances, we introduce dedicated heuristics which provide thin admissible solutions, hence tight lower bounds for the initial problem. We also benchmark them against a commercial solver, such as LocalSolver [7]. We are also interested in the continuously relaxed problem, which is non convex. Firstly, we use continuous solutions, obtained by free solver NLopt [26] and transform them into thin admissible solutions of the discrete problem. Secondly, we provide, under some convexity assumptions, an upper bound for the continuous relaxation, and hence for the initial (integer) problem. Numerical experiments confirm the quality of proposed heuristics (lower bounds), which often reach the optimal, or prove very tight, for small and medium size instances, with a very fast CPU time. Our upper bound, however, is not tight.
Channel-Wise Attention-Based Network for Self-Supervised Monocular Depth Estimation
Venue: arXiv cs.CV 2021
Year: 2021

Self-supervised learning has shown very promising results for monocular depth estimation. Scene structure and local details both are significant clues for high-quality depth estimation. Recent works suffer from the lack of explicit modeling of scene structure and proper handling of details information, which leads to a performance bottleneck and blurry artefacts in predicted results. In this paper, we propose the Channel-wise Attention-based Depth Estimation Network (CADepth-Net) with two effective contributions: 1) The structure perception module employs the self-attention mechanism to capture long-range dependencies and aggregates discriminative features in channel dimensions, explicitly enhances the perception of scene structure, obtains the better scene understanding and rich feature representation. 2) The detail emphasis module re-calibrates channel-wise feature maps and selectively emphasizes the informative features, aiming to highlight crucial local details information and fuse different level features more efficiently, resulting in more precise and sharper depth prediction. Furthermore, the extensive experiments validate the effectiveness of our method and show that our model achieves the state-of-the-art results on the KITTI benchmark and Make3D datasets.