OpenRaiser/Intern-Atlas huggingface dataset:
Relation Analysis between Hotel Review Rating Scores and Sentiment Analysis of Reviews by Chinese Tourists Visiting Japan
Venue: arXiv cs.IR 2021
Year: 2021

In current times, the importance of online hotel review sites has become more and more apparent. Users of these sites reference of reviews strongly influences their purchase behavior and as such, reviews are important to companies and researchers alike. The majority of review sites offer both text reviews and numerical hotel ratings, and both information sources are widely used by researchers as a representation of a customer's sentiment and opinion. However, an opinion is a difficult concept to measure, and as such, depending on the relation these two sources have, it would be apparent whether or not it is safe to consider them equally in research. In this study we utilize an entropy-based Support Vector Machine to classify positive and negative sentiments in hotel reviews from the site Ctrip, then calculating the ratio of positive and negative sentiment in each review and examine their correlation with said review's rating score using Spearman and Kendall Correlation coefficients and Maximal Information Coefficient (MIC).
Virtual Element based formulations for computational materials micro-mechanics and homogenization
Venue: arXiv cs.CE 2021
Year: 2021

In this thesis, a computational framework for microstructural modelling of transverse behaviour of heterogeneous materials is presented. The context of this research is part of the broad and active field of Computational Micromechanics, which has emerged as an effective tool both to understand the influence of complex microstructures on the macro-mechanical response of engineering materials and to tailor-design innovative materials for specific applications through a proper modification of their microstructure. The computational framework presented in this thesis is based on the Virtual Element Method (VEM), a recently developed numerical technique able to provide robust numerical results even with highly-distorted meshes. The peculiar features of VEM have been exploited to analyse two-dimensional representations of heterogeneous materials microstructures. Ad-hoc polygonal multi-domain meshing strategies have been developed and tested to exploit the discretisation freedom that VEM allows. To further simplify the preprocessing stage of the analysis and reduce the total computational cost, a novel hybrid formulation for analysing multi-domain problems has been developed by combining the Virtual Element Method with the well-known Boundary Element Method (BEM). The hybrid approach has been used to study both composite material transverse behaviour in presence of inclusions with complex geometries and damage and crack propagation in the matrix phase. Numerical results are presented that demonstrate the potential of the developed framework.
Transfer Learning Approaches for Knowledge Discovery in Grid-based Geo-Spatiotemporal Data
Venue: arXiv cs.LG 2021
Year: 2021

Extracting and meticulously analyzing geo-spatiotemporal features is crucial to recognize intricate underlying causes of natural events, such as floods. Limited evidence about hidden factors leading to climate change makes it challenging to predict regional water discharge accurately. In addition, the explosive growth in complex geo-spatiotemporal environment data that requires repeated learning by the state-of-the-art neural networks for every new region emphasizes the need for new computationally efficient methods, advanced computational resources, and extensive training on a massive amount of available monitored data. We, therefore, propose HydroDeep, an effectively reusable pretrained model to address this problem of transferring knowledge from one region to another by effectively capturing their intrinsic geo-spatiotemporal variance. Further, we present four transfer learning approaches on HydroDeep for spatiotemporal interpretability that improve Nash-Sutcliffe efficiency by 9% to 108% in new regions with a 95% reduction in time.
Repttack: Exploiting Cloud Schedulers to Guide Co-Location Attacks
Venue: arXiv cs.DC 2021
Year: 2021

Cloud computing paradigms have emerged as a major facility to store and process the massive data produced by various business units, public organizations, Internet-of-Things, and cyber-physical systems. To meet users' performance requirements while maximizing resource utilization to achieve cost-efficiency, cloud administrators leverage schedulers to orchestrate tasks to different physical nodes and allow applications from different users to share the same physical node. On the other hand, micro-architectural attacks can exploit the shared resources to compromise the confidentiality/integrity of a co-located victim application. Since co-location is an essential requirement for micro-architectural attacks, in this work, we investigate whether attackers can exploit the cloud schedulers to satisfy the co-location requirement. Our analysis shows that for cloud schedulers that allow users to submit application requirements, an attacker can carefully select the attacker's application requirements to influence the scheduler to co-locate it with a targeted victim application. We call such attack Replication Attack (Repttack). Our experimental results, in both a simulated cluster environment and a real cluster, show similar trends; a single attack instance can reach up to 50% co-location rate and with only 5 instances the co-location rate can reach up to 80%. Furthermore, we propose and evaluate a mitigation strategy that can help defend against Repttack. We believe that our results highlight the fact that schedulers in multi-user clusters need to be more carefully designed with security in mind, and the process of making scheduling decisions should involve as little user-defined information as possible.
Optimized Graph Based Routing Algorithm for the Angara Interconnect
Venue: arXiv cs.NI 2021
Year: 2021

JSC NICEVT has developed the Angara high-speed interconnect with 4D torus topology. The Angara interconnect router implements deterministic routing based on the bubble flow control, a direction order routing (DOR) and direction bits rules. The router chip also supports non standard First Step / Last Step for bypassing failed nodes and links, these steps can violate the DOR rule. In the previous work we have proposed an algorithm for generation and analysis of routing tables that guarantees no deadlocks in the Angara interconnect. It is based on a breadth-first search algorithm in a graph and it practically does not take into consideration communication channel load. Also we have never evaluated the influence of routing table generation algorithm on the performance of a real-world Angara based cluster. In this paper we present a routing graph notation that provides a possibility to build routes in the torus topology of the Angara interconnect. We propose a deadlock-free routing algorithm based on a fast single-source shortest path algorithm for the deterministic Angara routing with a single virtual channel. We evaluated the considered routing algorithms on a 32-node Desmos cluster system and benchmarked the proposed algorithm performance improvement of 11.1% for the Alltoall communication pattern and of more than 5% for the FT and IS application kernels.
Spindle: Techniques for Optimizing Atomic Multicast on RDMA
Venue: arXiv cs.DC 2021
Year: 2021

Leveraging one-sided RDMA for applications that replicate small data objects can be surprisingly difficult: such uses amplify any protocol overheads. Spindle is a set of optimization techniques for systematically tackling this class of challenges for atomic multicast over RDMA. These include memory polling optimizations using novel sender and receiver batching techniques, null-message send logic, and improved multi-thread synchronization. We applied Spindle to Derecho, an open-source C++ library for atomic multicast, and obtained significant performance improvements both for the library itself and for an OMG-compliant avionics DDS built over Derecho. Derecho's multicast bandwidth utilization for 10KB messages rose from 1GB/s to 9.7GB/s on a 12.5GB/s network, and it became more robust to delays. Interestingly, although some of our techniques employ batching, latency dropped by nearly two orders of magnitude. Spindle optimizations should also be of value in other RDMA applications limited by the speed of coordination.
A Minimal Intervention Definition of Reverse Engineering a Neural Circuit
Venue: arXiv cs.IT 2021
Year: 2021

In neuroscience, researchers have developed informal notions of what it means to reverse engineer a system, e.g., being able to model or simulate a system in some sense. A recent influential paper of Jonas and Kording, that examines a microprocessor using techniques from neuroscience, suggests that common techniques to understand neural systems are inadequate. Part of the difficulty, as a previous work of Lazebnik noted, lies in lack of formal language. We provide a theoretical framework for defining reverse engineering of computational systems, motivated by the neuroscience context. Of specific interest are recent works where, increasingly, interventions are being made to alter the function of the neural circuitry to both understand the system and treat disorders. Starting from Lazebnik's viewpoint that understanding a system means you can ``fix it'', and motivated by use-cases in neuroscience, we propose the following requirement on reverse engineering: once an agent claims to have reverse-engineered a neural circuit, they subsequently need to be able to: (a) provide a minimal set of interventions to change the input/output (I/O) behavior of the circuit to a desired behavior; (b) arrive at this minimal set of interventions while operating under bounded rationality constraints (e.g., limited memory) to rule out brute-force approaches. Under certain assumptions, we show that this reverse engineering goal falls within the class of undecidable problems. Next, we examine some canonical computational systems and reverse engineering goals (as specified by desired I/O behaviors) where reverse engineering can indeed be performed. Finally, using an exemplar network, the ``reward network'' in the brain, we summarize the state of current neuroscientific understanding, and discuss how computer-science and information-theoretic concepts can inform goals of future neuroscience studies.
GROWN: GRow Only When Necessary for Continual Learning
Venue: arXiv cs.LG 2021
Year: 2021

Catastrophic forgetting is a notorious issue in deep learning, referring to the fact that Deep Neural Networks (DNN) could forget the knowledge about earlier tasks when learning new tasks. To address this issue, continual learning has been developed to learn new tasks sequentially and perform knowledge transfer from the old tasks to the new ones without forgetting. While recent structure-based learning methods show the capability of alleviating the forgetting problem, these methods start from a redundant full-size network and require a complex learning process to gradually grow-and-prune or search the network structure for each task, which is inefficient. To address this problem and enable efficient network expansion for new tasks, we first develop a learnable sparse growth method eliminating the additional pruning/searching step in previous structure-based methods. Building on this learnable sparse growth method, we then propose GROWN, a novel end-to-end continual learning framework to dynamically grow the model only when necessary. Different from all previous structure-based methods, GROWN starts from a small seed network, instead of a full-sized one. We validate GROWN on multiple datasets against state-of-the-art methods, which shows superior performance in both accuracy and model size. For example, we achieve 1.0\% accuracy gain on average compared to the current SOTA results on CIFAR-100 Superclass 20 tasks setting.
Information Elicitation Meets Clustering
Venue: arXiv cs.GT 2021
Year: 2021

In the setting where we want to aggregate people's subjective evaluations, plurality vote may be meaningless when a large amount of low-effort people always report "good" regardless of the true quality. "Surprisingly popular" method, picking the most surprising answer compared to the prior, handle this issue to some extent. However, it is still not fully robust to people's strategies. Here in the setting where a large number of people are asked to answer a small number of multi-choice questions (multi-task, large group), we propose an information aggregation method that is robust to people's strategies. Interestingly, this method can be seen as a rotated "surprisingly popular". It is based on a new clustering method, Determinant MaxImization (DMI)-clustering, and a key conceptual idea that information elicitation without ground-truth can be seen as a clustering problem. Of independent interest, DMI-clustering is a general clustering method that aims to maximize the volume of the simplex consisting of each cluster's mean multiplying the product of the cluster sizes. We show that DMI-clustering is invariant to any non-degenerate affine transformation for all data points. When the data point's dimension is a constant, DMI-clustering can be solved in polynomial time. In general, we present a simple heuristic for DMI-clustering which is very similar to Lloyd's algorithm for k-means. Additionally, we also apply the clustering idea in the single-task setting and use the spectral method to propose a new aggregation method that utilizes the second-moment information elicited from the crowds.
Software Evolution Understanding: Automatic Extraction of Software Identifiers Map for Object-Oriented Software Systems
Venue: arXiv cs.SE 2021
Year: 2021

Software companies usually develop a set of product variants within the same family that share certain functions and differ in others. Variations across software variants occur to meet different customer requirements. Thus, software product variants evolve overtime to cope with new requirements. A software engineer who deals with this family may find it difficult to understand the evolution scenarios that have taken place over time. In addition, software identifier names are important resources to understand the evolution scenarios in this family. This paper introduces an automatic approach called Juana's approach to detect the evolution scenario across two product variants at the source code level and identifies the common and unique software identifier names across software variants source code. Juana's approach refers to common and unique identifier names as a software identifiers map and computes it by comparing software variants to each other. Juana considers all software identifier names such as package, class, attribute, and method. The novelty of this approach is that it exploits common and unique identifier names across the source code of software variants, to understand the evolution scenarios across software family in an efficient way. For validity, Juana was applied on ArgoUML and Mobile Media software variants. The results of this evaluation validate the relevance and the performance of the approach as all evolution scenarios were correctly detected via a software identifiers map.
Universal Face Restoration With Memorized Modulation
Venue: arXiv cs.CV 2021
Year: 2021

Blind face restoration (BFR) is a challenging problem because of the uncertainty of the degradation patterns. This paper proposes a Restoration with Memorized Modulation (RMM) framework for universal BFR in diverse degraded scenes and heterogeneous domains. We apply random noise as well as unsupervised wavelet memory to adaptively modulate the face-enhancement generator, considering attentional denormalization in both layer and instance levels. Specifically, in the training stage, the low-level spatial feature embedding, the wavelet memory embedding obtained by wavelet transformation of the high-resolution image, as well as the disentangled high-level noise embeddings are integrated, with the guidance of attentional maps generated from layer normalization, instance normalization, and the original feature map. These three embeddings are respectively associated with the spatial content, high-frequency texture details, and a learnable universal prior against other blind image degradation patterns. We store the spatial feature of the low-resolution image and the corresponding wavelet style code as key and value in the memory unit, respectively. In the test stage, the wavelet memory value whose corresponding spatial key is the most matching with that of the inferred image is retrieved to modulate the generator. Moreover, the universal prior learned from the random noise has been memorized by training the modulation network. Experimental results show the superiority of the proposed method compared with the state-of-the-art methods, and a good generalization in the wild.
Anonymer Tanz als dekolonialisierende Praxis. Ein embodied Research Versuch
Venue: arXiv cs.CY 2021
Year: 2021

This paper addresses the question of how and with what consequences anonymity is used as a creative element in experiments dealing with alternative forms of knowledge production. More specifically, the focus is on a particular current in contemporary dance, experimental dance. This has developed in the course of the 1970s. Its guiding idea is to shift the focus away from finished choreography and performance to the processuality, spontaneity, and experience of dance practice itself. Dance is seen as a laboratory in which to experiment with gravity, body boundaries, group dynamics and perception. The focus is on testing, touching, trying. The corresponding events are called either 'Dance Labs' or 'Movement Research Workshops', depending on the regularity. An instrument that is often used in the context of such events to create specific social situations and group dynamics or to channel perceptions in certain directions is anonymity. Anonymity is broadly understood as the capping of connections that results in a degree of unaccountability with respect to various aspects of a person, situation, or activity. For example, unaccountability can arise from the fact that people can no longer see each other, so that visual identification becomes impossible, even if only temporarily. By breaking familiar modes of identification, other forms of relating can sometimes be emphasized. This is what makes this form of anonymity so interesting for experimental dance.
Human-Centered AI for Data Science: A Systematic Approach
Venue: arXiv cs.LG 2021
Year: 2021

Human-Centered AI (HCAI) refers to the research effort that aims to design and implement AI techniques to support various human tasks, while taking human needs into consideration and preserving human control. In this short position paper, we illustrate how we approach HCAI using a series of research projects around Data Science (DS) works as a case study. The AI techniques built for supporting DS works are collectively referred to as AutoML systems, and their goals are to automate some parts of the DS workflow. We illustrate a three-step systematical research approach(i.e., explore, build, and integrate) and four practical ways of implementation for HCAI systems. We argue that our work is a cornerstone towards the ultimate future of Human-AI Collaboration for DS and beyond, where AI and humans can take complementary and indispensable roles to achieve a better outcome and experience.
Probing Language Models for Understanding of Temporal Expressions
Venue: arXiv cs.CL 2021
Year: 2021

We present three Natural Language Inference (NLI) challenge sets that can evaluate NLI models on their understanding of temporal expressions. More specifically, we probe these models for three temporal properties: (a) the order between points in time, (b) the duration between two points in time, (c) the relation between the magnitude of times specified in different units. We find that although large language models fine-tuned on MNLI have some basic perception of the order between points in time, at large, these models do not have a thorough understanding of the relation between temporal expressions.
FM-Indexing Grammars Induced by Suffix Sorting for Long Patterns
Venue: arXiv cs.DS 2021
Year: 2021

The run-length compressed Burrows-Wheeler transform (RLBWT) used in conjunction with the backward search introduced in the FM index is the centerpiece of most compressed indexes working on highly-repetitive data sets like biological sequences. Compared to grammar indexes, the size of the RLBWT is often much bigger, but queries like counting the occurrences of long patterns can be done much faster than on any existing grammar index so far. In this paper, we combine the virtues of a grammar with the RLBWT by building the RLBWT on top of a special grammar based on induced suffix sorting. Our experiments reveal that our hybrid approach outperforms the classic RLBWT with respect to the index sizes, and with respect to query times on biological data sets for sufficiently long patterns.
HyGain: High Performance, Energy-Efficient Hybrid Gain Cell based Cache Hierarchy
Venue: arXiv cs.AR 2021
Year: 2021

In this paper, we propose a 'full-stack' solution to designing high capacity and low latency on-chip cache hierarchies by starting at the circuit level of the hardware design stack. First, we propose a novel Gain Cell (GC) design using FDSOI. The GC has several desirable characteristics, including ~50% higher storage density and ~50% lower dynamic energy as compared to the traditional 6T SRAM, even after accounting for peripheral circuit overheads. We also exploit back-gate bias to increase retention time to 1.12 ms (~60x of eDRAM) which, combined with optimizations like staggered refresh, makes it an ideal candidate to architect all levels of on-chip caches. We show that compared to 6T SRAM, for a given area budget, GC based caches, on average, provide 29% and 36% increase in IPC for single- and multi-programmed workloads, respectively on contemporary workloads including SPEC CPU2017. We also observe dynamic energy savings of 42% and 34% for single- and multi-programmed workloads, respectively. We utilize the inherent properties of the proposed GC, including decoupled read and write bitlines to devise optimizations to save precharge energy and architect GC caches with better energy and performance characteristics. Finally, in a quest to utilize the best of all worlds, we combine GC with STT-RAM to create hybrid hierarchies. We show that a hybrid hierarchy with GC caches at L1 and L2, and an LLC split between GC and STT-RAM, with asymmetric write optimization enabled, is able to provide a 54% benefit in energy-delay product (EDP) as compared to an all-SRAM design, and 13% as compared to an all-GC cache hierarchy, averaged across multi-programmed workloads.
Max and Coincidence Neurons in Neural Networks
Venue: arXiv cs.CV 2021
Year: 2021

Network design has been a central topic in machine learning. Large amounts of effort have been devoted towards creating efficient architectures through manual exploration as well as automated neural architecture search. However, todays architectures have yet to consider the diversity of neurons and the existence of neurons with specific processing functions. In this work, we optimize networks containing models of the max and coincidence neurons using neural architecture search, and analyze the structure, operations, and neurons of optimized networks to develop a signal-processing ResNet. The developed network achieves an average of 2% improvement in accuracy and a 25% improvement in network size across a variety of datasets, demonstrating the importance of neuronal functions in creating compact, efficient networks.