OpenRaiser/Intern-Atlas huggingface dataset:
Task2Dial: A Novel Task and Dataset for Commonsense enhanced Task-based Dialogue Grounded in Documents
Venue: arXiv cs.CL 2022
Year: 2022

This paper proposes a novel task on commonsense-enhanced task-based dialogue grounded in documents and describes the Task2Dial dataset, a novel dataset of document-grounded task-based dialogues, where an Information Giver (IG) provides instructions (by consulting a document) to an Information Follower (IF), so that the latter can successfully complete the task. In this unique setting, the IF can ask clarification questions which may not be grounded in the underlying document and require commonsense knowledge to be answered. The Task2Dial dataset poses new challenges: (1) its human reference texts show more lexical richness and variation than other document-grounded dialogue datasets; (2) generating from this set requires paraphrasing as instructional responses might have been modified from the underlying document; (3) requires commonsense knowledge, since questions might not necessarily be grounded in the document; (4) generating requires planning based on context, as task steps need to be provided in order. The Task2Dial dataset contains dialogues with an average $18.15$ number of turns and 19.79 tokens per turn, as compared to 12.94 and 12 respectively in existing datasets. As such, learning from this dataset promises more natural, varied and less template-like system utterances.
Breaking the De-Pois Poisoning Defense
Venue: arXiv cs.LG 2022
Year: 2022

Attacks on machine learning models have been, since their conception, a very persistent and evasive issue resembling an endless cat-and-mouse game. One major variant of such attacks is poisoning attacks which can indirectly manipulate an ML model. It has been observed over the years that the majority of proposed effective defense models are only effective when an attacker is not aware of them being employed. In this paper, we show that the attack-agnostic De-Pois defense is hardly an exception to that rule. In fact, we demonstrate its vulnerability to the simplest White-Box and Black-Box attacks by an attacker that knows the structure of the De-Pois defense model. In essence, the De-Pois defense relies on a critic model that can be used to detect poisoned data before passing it to the target model. In our work, we break this poison-protection layer by replicating the critic model and then performing a composed gradient-sign attack on both the critic and target models simultaneously -- allowing us to bypass the critic firewall to poison the target model.
A sequence-to-sequence approach for document-level relation extraction
Venue: arXiv cs.CL 2022
Year: 2022

Motivated by the fact that many relations cross the sentence boundary, there has been increasing interest in document-level relation extraction (DocRE). DocRE requires integrating information within and across sentences, capturing complex interactions between mentions of entities. Most existing methods are pipeline-based, requiring entities as input. However, jointly learning to extract entities and relations can improve performance and be more efficient due to shared parameters and training steps. In this paper, we develop a sequence-to-sequence approach, seq2rel, that can learn the subtasks of DocRE (entity extraction, coreference resolution and relation extraction) end-to-end, replacing a pipeline of task-specific components. Using a simple strategy we call entity hinting, we compare our approach to existing pipeline-based methods on several popular biomedical datasets, in some cases exceeding their performance. We also report the first end-to-end results on these datasets for future comparison. Finally, we demonstrate that, under our model, an end-to-end approach outperforms a pipeline-based approach. Our code, data and trained models are available at {\url{https://github.com/johngiorgi/seq2rel}}. An online demo is available at {\url{https://share.streamlit.io/johngiorgi/seq2rel/main/demo.py}}.
Characterizing Urban Lifestyle Signatures Using Motif Properties in Network of Places
Venue: arXiv cs.SI 2022
Year: 2022

The lifestyles of urban dwellers could reveal important insights regarding the dynamics and complexity of cities. Despite growing research on analysis of lifestyle patterns in cities, little is known about the characteristics of people's lifestyles patterns at urban scale. This limitation is primarily due to challenges in characterizing lifestyle patterns when human movement data is aggregated to protect the privacy of users. In this study, we model cities based on aggregated human visitation data to construct a network of places. We then examine the subgraph signatures in the networks of places to map and characterize lifestyle patterns at city scale. Location-based data from Harris County, Dallas County, New York County, and Broward County in the United States were examined to reveal lifestyle signatures in cities. For the motif analysis, two-node, three-node, and four-node motifs without location attributes were extracted from human visitation networks. Second, homogenized nodes in motifs were encoded with location categories from NAICS codes. Multiple statistical measures, including network metrics and motif properties, were quantified to characterize lifestyle signatures. The results show that: people's lifestyles in urban environments can be well depicted and quantified based on distribution and attributes of motifs in networks of places; motifs in networks of places show stability in quantity and distance as well as periodicity on weekends and weekdays indicating the stability of lifestyle patterns in cities; human visitation networks and lifestyle patterns show similarities across different metropolitan areas implying the universality of lifestyle signatures across cities. The findings provide deeper insights into urban lifestyles signatures in urban studies and provide important insights for data-informed urban planning and management.
On incorporating social speaker characteristics in synthetic speech
Venue: arXiv cs.SD 2022
Year: 2022

In our previous work, we derived the acoustic features, that contribute to the perception of warmth and competence in synthetic speech. As an extension, in our current work, we investigate the impact of the derived vocal features in the generation of the desired characteristics. The acoustic features, spectral flux, F1 mean and F2 mean and their convex combinations were explored for the generation of higher warmth in female speech. The voiced slope, spectral flux, and their convex combinations were investigated for the generation of higher competence in female speech. We have employed a feature quantization approach in the traditional end-to-end tacotron based speech synthesis model. The listening tests have shown that the convex combination of acoustic features displays higher Mean Opinion Scores of warmth and competence when compared to that of individual features.
UrbanFlow: Designing Comfortable Outdoor Areas
Venue: arXiv cs.GR 2022
Year: 2022

Design decisions in urban planning have to be made with particular carefulness as the resulting constraints are binding for the whole architectural design that follows. In this context, investigating and optimizing the airflow in urban environments is critical to design comfortable outdoor areas as unwanted effects such as windy areas and the formation of heat pockets have to be avoided. Our UrbanFlow framework enables interactive architectural design allowing for decision making based on simulating urban flow. Compared to real-time fluid flow simulation, enabling interactive architecture design poses an even higher computational efficiency challenge as evaluating a design by simulation usually requires hundreds of time steps. This is addressed based on a highly efficient Eulerian fluid simulator in which we incorporate a unified porosity model which is devised to encode digital urban models containing objects such as buildings and trees. UrbanFlow is equipped with an optimization routine enabling the direct computation of design adaptations improving livability and comfort for given parameterized architectural designs. To ensure convergence of the optimization process, instead of the classical Navier-Stokes equations, the Reynolds-averaged Navier-Stokes equations are solved as this can be done on a relatively coarse grid and allows for the decoupling of the effects of turbulent eddies which are taken into account using a separate turbulence model. As we demonstrate on a real-world example taken from an ongoing architectural competition, this results in a fast convergence of the optimization process which computes a design adaptation avoiding heat pockets as well as uncomfortable windy areas.
Efficient and Accurate Candidate Generation for Grasp Pose Detection in SE(3)
Venue: arXiv cs.RO 2022
Year: 2022

Grasp detection of novel objects in unstructured environments is a key capability in robotic manipulation. For 2D grasp detection problems where grasps are assumed to lie in the plane, it is common to design a fully convolutional neural network that predicts grasps over an entire image in one step. However, this is not possible for grasp pose detection where grasp poses are assumed to exist in SE(3). In this case, it is common to approach the problem in two steps: grasp candidate generation and candidate classification. Since grasp candidate classification is typically expensive, the problem becomes one of efficiently identifying high quality candidate grasps. This paper proposes a new grasp candidate generation method that significantly outperforms major 3D grasp detection baselines. Supplementary material is available at https://atenpas.github.io/psn/.
Indoor Navigation Assistance for Visually Impaired People via Dynamic SLAM and Panoptic Segmentation with an RGB-D Sensor
Venue: arXiv cs.CV 2022
Year: 2022

Exploring an unfamiliar indoor environment and avoiding obstacles is challenging for visually impaired people. Currently, several approaches achieve the avoidance of static obstacles based on the mapping of indoor scenes. To solve the issue of distinguishing dynamic obstacles, we propose an assistive system with an RGB-D sensor to detect dynamic information of a scene. Once the system captures an image, panoptic segmentation is performed to obtain the prior dynamic object information. With sparse feature points extracted from images and the depth information, poses of the user can be estimated. After the ego-motion estimation, the dynamic object can be identified and tracked. Then, poses and speed of tracked dynamic objects can be estimated, which are passed to the users through acoustic feedback.
ScaleSFL: A Sharding Solution for Blockchain-Based Federated Learning
Venue: arXiv cs.DC 2022
Year: 2022

Blockchain-based federated learning has gained significant interest over the last few years with the increasing concern for data privacy, advances in machine learning, and blockchain innovation. However, gaps in security and scalability hinder the development of real-world applications. In this study, we propose ScaleSFL, which is a scalable blockchain-based sharding solution for federated learning. ScaleSFL supports interoperability by separating the off-chain federated learning component in order to verify model updates instead of controlling the entire federated learning flow. We implemented ScaleSFL as a proof-of-concept prototype system using Hyperledger Fabric to demonstrate the feasibility of the solution. We present a performance evaluation of results collected through Hyperledger Caliper benchmarking tools conducted on model creation. Our evaluation results show that sharding can improve validation performance linearly while remaining efficient and secure.
Attribute Prototype Network for Any-Shot Learning
Venue: arXiv cs.CV 2022
Year: 2022

Any-shot image classification allows to recognize novel classes with only a few or even zero samples. For the task of zero-shot learning, visual attributes have been shown to play an important role, while in the few-shot regime, the effect of attributes is under-explored. To better transfer attribute-based knowledge from seen to unseen classes, we argue that an image representation with integrated attribute localization ability would be beneficial for any-shot, i.e. zero-shot and few-shot, image classification tasks. To this end, we propose a novel representation learning framework that jointly learns discriminative global and local features using only class-level attributes. While a visual-semantic embedding layer learns global features, local features are learned through an attribute prototype network that simultaneously regresses and decorrelates attributes from intermediate features. Furthermore, we introduce a zoom-in module that localizes and crops the informative regions to encourage the network to learn informative features explicitly. We show that our locality augmented image representations achieve a new state-of-the-art on challenging benchmarks, i.e. CUB, AWA2, and SUN. As an additional benefit, our model points to the visual evidence of the attributes in an image, confirming the improved attribute localization ability of our image representation. The attribute localization is evaluated quantitatively with ground truth part annotations, qualitatively with visualizations, and through well-designed user studies.
Learning Linear Symmetries in Data Using Moment Matching
Venue: arXiv cs.LG 2022
Year: 2022

It is common in machine learning and statistics to use symmetries derived from expert knowledge to simplify problems or improve performance, using methods like data augmentation or penalties. In this paper we consider the unsupervised and semi-supervised problems of learning such symmetries in a distribution directly from data in a model-free fashion. We show that in the worst case this problem is as difficult as the graph automorphism problem. However, if we restrict to the case where the covariance matrix has unique eigenvalues, then the eigenvectors will also be eigenvectors of the symmetry transformation. If we further restrict to finding orthogonal symmetries, then the eigenvalues will be either be 1 or -1, and the problem reduces to determining which eigenvectors are which. We develop and compare theoretically and empirically the effectiveness of different methods of selecting which eigenvectors should have eigenvalue -1 in the symmetry transformation, and discuss how to extend this approach to non-orthogonal cases where we have labels
FoV-Net: Field-of-View Extrapolation Using Self-Attention and Uncertainty
Venue: arXiv cs.CV 2022
Year: 2022

The ability to make educated predictions about their surroundings, and associate them with certain confidence, is important for intelligent systems, like autonomous vehicles and robots. It allows them to plan early and decide accordingly. Motivated by this observation, in this paper we utilize information from a video sequence with a narrow field-of-view to infer the scene at a wider field-of-view. To this end, we propose a temporally consistent field-of-view extrapolation framework, namely FoV-Net, that: (1) leverages 3D information to propagate the observed scene parts from past frames; (2) aggregates the propagated multi-frame information using an attention-based feature aggregation module and a gated self-attention module, simultaneously hallucinating any unobserved scene parts; and (3) assigns an interpretable uncertainty value at each pixel. Extensive experiments show that FoV-Net does not only extrapolate the temporally consistent wide field-of-view scene better than existing alternatives, but also provides the associated uncertainty which may benefit critical decision-making downstream applications. Project page is at http://charliememory.github.io/RAL21_FoV.
Problem reports and team maturity in agile automotive software development
Venue: arXiv cs.SE 2022
Year: 2022

Background: Volvo Cars is pioneering an agile transformation on a large scale in the automotive industry. Social psychological aspects of automotive software development are an under-researched area in general. Few studies on team maturity or group dynamics can be found specifically in the automotive software engineering domain. Objective: This study is intended as an initial step to fill that gap by investigating the connection between issues and problem reports and team maturity. Method: We conducted a quantitative study with 84 participants from 14 teams and qualitatively validated the result with the Release Train Engineer having an overview of all the participating teams. Results: We find that the more mature a team is, the faster they seem to resolve issues as provided through external feedback, at least in the two initial team maturity stages. Conclusion: This study suggests that working on team dynamics might increase productivity in modern automotive software development departments, but this needs further investigation.
Software Rasterization of 2 Billion Points in Real Time
Venue: arXiv cs.GR 2022
Year: 2022

We propose a software rasterization pipeline for point clouds that is capable of brute-force rendering up to two billion points in real time (60fps). Improvements over the state of the art are achieved by batching points in a way that a number of batch-level optimizations can be computed before rasterizing the points within the same rendering pass. These optimizations include frustum culling, level-of-detail rendering, and choosing the appropriate coordinate precision for a given batch of points directly within a compute workgroup. Adaptive coordinate precision, in conjunction with visibility buffers, reduces the number of loaded bytes for the majority of points down to 4, thus making our approach several times faster than the bandwidth-limited state of the art. Furthermore, support for LOD rendering makes our software-rasterization approach suitable for rendering arbitrarily large point clouds, and to meet the increased performance demands of virtual reality rendering.
Nonlinear Vectorial Prediction with Neural Nets
Venue: arXiv cs.SD 2022
Year: 2022

In this paper we propose a nonlinear vectorial prediction scheme based on a Multi Layer Perceptron. This system is applied to speech coding in an ADPCM backward scheme. In addition a procedure to obtain a vectorial quantizer is given, in order to achieve a fully vectorial speech encoder. We also present several results with the proposed system
Learning to solve Minimum Cost Multicuts efficiently using Edge-Weighted Graph Convolutional Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

The minimum cost multicut problem is the NP-hard/APX-hard combinatorial optimization problem of partitioning a real-valued edge-weighted graph such as to minimize the total cost of the partition. While graph convolutional neural networks (GNN) have proven to be promising in the context of combinatorial optimization, most of them are only tailored to or tested on positive-valued edge weights, i.e. they do not comply to the nature of the multicut problem. We therefore adapt various GNN architectures including Graph Convolutional Networks, Signed Graph Convolutional Networks and Graph Isomorphic Networks to facilitate the efficient encoding of real-valued edge costs. Moreover, we employ a reformulation of the multicut ILP constraints to a polynomial program as loss function that allows to learn feasible multicut solutions in a scalable way. Thus, we provide the first approach towards end-to-end trainable multicuts. Our findings support that GNN approaches can produce good solutions in practice while providing lower computation times and largely improved scalability compared to LP solvers and optimized heuristics, especially when considering large instances.
Taking ROCKET on an Efficiency Mission: Multivariate Time Series Classification with LightWaveS
Venue: arXiv cs.LG 2022
Year: 2022

Nowadays, with the rising number of sensors in sectors such as healthcare and industry, the problem of multivariate time series classification (MTSC) is getting increasingly relevant and is a prime target for machine and deep learning approaches. Their expanding adoption in real-world environments is causing a shift in focus from the pursuit of ever-higher prediction accuracy with complex models towards practical, deployable solutions that balance accuracy and parameters such as prediction speed. An MTSC model that has attracted attention recently is ROCKET, based on random convolutional kernels, both because of its very fast training process and its state-of-the-art accuracy. However, the large number of features it utilizes may be detrimental to inference time. Examining its theoretical background and limitations enables us to address potential drawbacks and present LightWaveS: a framework for accurate MTSC, which is fast both during training and inference. Specifically, utilizing wavelet scattering transformation and distributed feature selection, we manage to create a solution that employs just 2.5% of the ROCKET features, while achieving accuracy comparable to recent MTSC models. LightWaveS also scales well across multiple compute nodes and with the number of input channels during training. In addition, it can significantly reduce the input size and provide insight to an MTSC problem by keeping only the most useful channels. We present three versions of our algorithm and their results on distributed training time and scalability, accuracy, and inference speedup. We show that we achieve speedup ranging from 9x to 53x compared to ROCKET during inference on an edge device, on datasets with comparable accuracy.
Dynamic Network-Code Design for Satellite Networks
Venue: arXiv cs.IT 2022
Year: 2022

Internet access from space enjoys renaissance as satellites in Mega-Constellations is no longer fictitious. Network capacity, subject to power and computational complexity constraints among other challenges, is a major goal in this type of networks. This work studies Network Coding in the presence of dynamically changing network conditions. The notion of generalized acyclic network is introduced and employed for promoting the generation of linear-multicast network code for what is considered to be a cyclic network. The performance of several network coding schemes, among these is the known static network code, is evaluated by a STK simulation for a swarm of communicating satellites, conceptually based on the Iridium system. Exploiting the prior knowledge of the networks topology over time, new network coding approaches are described, whose aim is to better cope with the time-varying, dynamic behavior of the network. It is demonstrated that in all cases, pertaining to our example network, static network codes under-perform compared to the presented approach. In addition, an efficient test for identifying the most appropriate coding approach is presented.
SAM-kNN Regressor for Online Learning in Water Distribution Networks
Venue: arXiv cs.LG 2022
Year: 2022

Water distribution networks are a key component of modern infrastructure for housing and industry. They transport and distribute water via widely branched networks from sources to consumers. In order to guarantee a working network at all times, the water supply company continuously monitors the network and takes actions when necessary -- e.g. reacting to leakages, sensor faults and drops in water quality. Since real world networks are too large and complex to be monitored by a human, algorithmic monitoring systems have been developed. A popular type of such systems are residual based anomaly detection systems that can detect events such as leakages and sensor faults. For a continuous high quality monitoring, it is necessary for these systems to adapt to changed demands and presence of various anomalies. In this work, we propose an adaption of the incremental SAM-kNN classifier for regression to build a residual based anomaly detection system for water distribution networks that is able to adapt to any kind of change.