OpenRaiser/Intern-Atlas huggingface dataset:
Scalable algorithms for physics-informed neural and graph networks
Venue: arXiv cs.LG 2022
Year: 2022

Physics-informed machine learning (PIML) has emerged as a promising new approach for simulating complex physical and biological systems that are governed by complex multiscale processes for which some data are also available. In some instances, the objective is to discover part of the hidden physics from the available data, and PIML has been shown to be particularly effective for such problems for which conventional methods may fail. Unlike commercial machine learning where training of deep neural networks requires big data, in PIML big data are not available. Instead, we can train such networks from additional information obtained by employing the physical laws and evaluating them at random points in the space-time domain. Such physics-informed machine learning integrates multimodality and multifidelity data with mathematical models, and implements them using neural networks or graph networks. Here, we review some of the prevailing trends in embedding physics into machine learning, using physics-informed neural networks (PINNs) based primarily on feed-forward neural networks and automatic differentiation. For more complex systems or systems of systems and unstructured data, graph neural networks (GNNs) present some distinct advantages, and here we review how physics-informed learning can be accomplished with GNNs based on graph exterior calculus to construct differential operators; we refer to these architectures as physics-informed graph networks (PIGNs). We present representative examples for both forward and inverse problems and discuss what advances are needed to scale up PINNs, PIGNs and more broadly GNNs for large-scale engineering problems.
RISCLESS: A Reinforcement Learning Strategy to Exploit Unused Cloud Resources
Venue: arXiv cs.DC 2022
Year: 2022

One of the main objectives of Cloud Providers (CP) is to guarantee the Service-Level Agreement (SLA) of customers while reducing operating costs. To achieve this goal, CPs have built large-scale datacenters. This leads, however, to underutilized resources and an increase in costs. A way to improve the utilization of resources is to reclaim the unused parts and resell them at a lower price. Providing SLA guarantees to customers on reclaimed resources is a challenge due to their high volatility. Some state-of-the-art solutions consider keeping a proportion of resources free to absorb sudden variation in workloads. Others consider stable resources on top of the volatile ones to fill in for the lost resources. However, these strategies either reduce the amount of reclaimable resources or operate on less volatile ones such as Amazon Spot instance. In this paper, we proposed RISCLESS, a Reinforcement Learning strategy to exploit unused Cloud resources. Our approach consists of using a small proportion of stable on-demand resources alongside the ephemeral ones in order to guarantee customers SLA and reduce the overall costs. The approach decides when and how much stable resources to allocate in order to fulfill customers' demands. RISCLESS improved the CPs' profits by an average of 15.9% compared to state-of-the-art strategies. It also reduced the SLA violation time by an average of 36.7% while increasing the amount of used ephemeral resources by 19.5% on average
DouFu: A Double Fusion Joint Learning Method For Driving Trajectory Representation
Venue: arXiv cs.LG 2022
Year: 2022

Driving trajectory representation learning is of great significance for various location-based services, such as driving pattern mining and route recommendation. However, previous representation generation approaches tend to rarely address three challenges: 1) how to represent the intricate semantic intentions of mobility inexpensively; 2) complex and weak spatial-temporal dependencies due to the sparsity and heterogeneity of the trajectory data; 3) route selection preferences and their correlation to driving behavior. In this paper, we propose a novel multimodal fusion model, DouFu, for trajectory representation joint learning, which applies multimodal learning and attention fusion module to capture the internal characteristics of trajectories. We first design movement, route, and global features generated from the trajectory data and urban functional zones and then analyze them respectively with the attention encoder or feed forward network. The attention fusion module incorporates route features with movement features to create a better spatial-temporal embedding. With the global semantic feature, DouFu produces a comprehensive embedding for each trajectory. We evaluate representations generated by our method and other baseline models on classification and clustering tasks. Empirical results show that DouFu outperforms other models in most of the learning algorithms like the linear regression and the support vector machine by more than 10%.
Minimum Tournaments with the Strong $S_k$-Property and Implications for Teaching
Venue: arXiv cs.DM 2022
Year: 2022

A tournament is said to have the $S_k$-property if, for any set of $k$ players, there is another player who beats them all. Minimum tournaments having this property have been explored very well in the 1960's and the early 1970's. In this paper, we define a strengthening of the $S_k$-property that we name "strong $S_k$-property". We show, first, that several basic results on the weaker notion remain valid for the stronger notion (and the corresponding modification of the proofs requires only little extra-effort). Second, it is demonstrated that the stronger notion has applications in the area of Teaching. Specifically, we present an infinite family of concept classes all of which can be taught with a single example in the No-Clash model of teaching while, in order to teach a class $\cC$ of this family in the recursive model of teaching, order of $\log|\cC|$ many examples are required. This is the first paper that presents a concrete and easily constructible family of concept classes which separates the No-Clash from the recursive model of teaching by more than a constant factor. The separation by a logarithmic factor is remarkable because the recursive teaching dimension is known to be bounded by $\log |\cC|$ for any concept class $\cC$.
REAL ML: Recognizing, Exploring, and Articulating Limitations of Machine Learning Research
Venue: arXiv cs.LG 2022
Year: 2022

Transparency around limitations can improve the scientific rigor of research, help ensure appropriate interpretation of research findings, and make research claims more credible. Despite these benefits, the machine learning (ML) research community lacks well-developed norms around disclosing and discussing limitations. To address this gap, we conduct an iterative design process with 30 ML and ML-adjacent researchers to develop and test REAL ML, a set of guided activities to help ML researchers recognize, explore, and articulate the limitations of their research. Using a three-stage interview and survey study, we identify ML researchers' perceptions of limitations, as well as the challenges they face when recognizing, exploring, and articulating limitations. We develop REAL ML to address some of these practical challenges, and highlight additional cultural challenges that will require broader shifts in community norms to address. We hope our study and REAL ML help move the ML research community toward more active and appropriate engagement with limitations.
Should attention be all we need? The epistemic and ethical implications of unification in machine learning
Venue: arXiv cs.LG 2022
Year: 2022

"Attention is all you need" has become a fundamental precept in machine learning research. Originally designed for machine translation, transformers and the attention mechanisms that underpin them now find success across many problem domains. With the apparent domain-agnostic success of transformers, many researchers are excited that similar model architectures can be successfully deployed across diverse applications in vision, language and beyond. We consider the benefits and risks of these waves of unification on both epistemic and ethical fronts. On the epistemic side, we argue that many of the arguments in favor of unification in the natural sciences fail to transfer over to the machine learning case, or transfer over only under assumptions that might not hold. Unification also introduces epistemic risks related to portability, path dependency, methodological diversity, and increased black-boxing. On the ethical side, we discuss risks emerging from epistemic concerns, further marginalizing underrepresented perspectives, the centralization of power, and having fewer models across more domains of application
A CMOS-based Characterisation Platform for Emerging RRAM Technologies
Venue: arXiv cs.ET 2022
Year: 2022

Mass characterisation of emerging memory devices is an essential step in modelling their behaviour for integration within a standard design flow for existing integrated circuit designers. This work develops a novel characterisation platform for emerging resistive devices with a capacity of up to 1 million devices on-chip. Split into four independent sub-arrays, it contains on-chip column-parallel DACs for fast voltage programming of the DUT. On-chip readout circuits with ADCs are also available for fast read operations covering 5-decades of input current (20nA to 2mA). This allows a device's resistance range to be between 1k$Ω$ and 10M$Ω$ with a minimum voltage range of $\pm$1.5V on the device.
A Wide Dynamic Range Read-out System For Resistive Switching Technology
Venue: arXiv cs.ET 2022
Year: 2022

The memristor, because of its controllability over a wide dynamic range of resistance, has emerged as a promising device for data storage and analog computation. A major challenge is the accurate measurement of memristance over a wide dynamic range. In this paper, a novel read-out circuit with feedback adjustment is proposed to measure and digitise input current in the range between 20nA and 2mA. The magnitude of the input currents is estimated by a 5-stage logarithmic current-to-voltage amplifier which scales a linear analog-to-digital converter. This way the least significant bit tracks the absolute input magnitude. This circuit is applicable to reading single memristor conductance, and is also preferable in analog computing where read-out accuracy is particularly critical. The circuits have been realized in Bipolar-CMOS-DMOS (BCD) Gen2 technology.
Simultaneous Multi-User MIMO Communications and Multi-Target Tracking with Full Duplex Radios
Venue: arXiv cs.IT 2022
Year: 2022

In this paper, we present an Integrated Sensing and Communications (ISAC) system enabled by in-band Full Duplex (FD) radios, where a massive Multiple-Input Multiple-Output (MIMO) base station equipped with hybrid Analog and Digital (A/D) beamformers is communicating with multiple DownLink (DL) users, and simultaneously estimates via the same signaling waveforms the Direction of Arrival (DoA) as well as the range of radar targets randomly distributed within its coverage area. Capitalizing on a recent reduced-complexity FD hybrid A/D beamforming architecture, we devise a joint radar target tracking and DL data transmission protocol. An optimization framework for the joint design of the massive A/D beamformers and the Self-Interference (SI) cancellation unit, with the dual objective of maximizing the radar tracking accuracy and DL communication performance, is presented. Our simulation results at millimeter wave frequencies using 5G NR wideband waveforms, showcase the accuracy of the radar target tracking performance of the proposed system, which simultaneously offers increased sum rate compared with benchmark schemes.
A Comprehensive Study on Artificial Intelligence Algorithms to Implement Safety Using Communication Technologies
Venue: arXiv cs.AI 2022
Year: 2022

The recent development of artificial intelligence (AI) has increased the interest of researchers and practitioners towards applying its techniques into multiple domains like automotive, health care and air space to achieve automation. Combined to these applications, the attempt to use AI techniques into carrying out safety issues is momentarily at a progressive state. As AI problems are getting even more complex, large processing power is demanded for safety-critical systems to fulfill real-time requirements. These challenges can be solved through edge or cloud computing, which makes the communication an integral part of the solution. This study aims at providing a comprehensive picture of the state of the art AI based safety solutions that uses different communication technologies in diverse application domains. To achieve this, a systematic mapping study is conducted and 565 relevant papers are shortlisted through a multistage selection process, which are then analyzed according to a systematically defined classification framework. The results of the study are based on these main objectives: to clarify current research gaps in the field, to identify the possibility of increased usage of cellular communication in multiple domains, to identify the mostly used AI algorithms and to summarize the emerging future research trends on the topic. The results demonstrate that automotive domain is the one applying AI and communication the most to implement safety and the most used AI in this domain is neural networks, clustering and computer vision; applying cellular communication to automotive domain is highest; the use of non-cellular communication technologies is dominant however a clear trend of a rapid increase in the use of cellular communication is observed specially from 2020 with the roll-out of 5G technology.
Towards Resilient Access Equality for 6G Serverless p-LEO Satellite Networks
Venue: arXiv cs.NI 2022
Year: 2022

Low earth orbit (LEO) mega-constellations, integrating government space systems and commercial practices, have emerged as enabling technologies for the sixth generation (6G) networks due to their good merits of global coverage and ubiquitous services for military and civilian use cases. However, convergent LEO-based satellite networking infrastructures still lack leveraging the synergy of space and terrestrial systems. This paper, therefore, extends conventional serverless cloud platforms with serverless edge learning architectures for 6G proliferated LEO (p-LEO) satellite ecosystems and provides a new distributed training design from a networking perspective. The proposed design dynamically orchestrates communications and computation functionalities and resources among heterogeneous physical units to efficiently fulfill multi-agent deep reinforcement learning for service-level agreements. Innovative ecosystem enhancements, including ultrabroadband access, anti-jammed transmissions, resilient networking, and related open challenges, are also investigated for end-to-end connectivity, communications, and learning performance.
Experimental evaluation of digitally-verifiable photonic computing for blockchain and cryptocurrency
Venue: arXiv cs.ET 2022
Year: 2022

As blockchain technology and cryptocurrency become increasingly mainstream, ever-increasing energy costs required to maintain the computational power running these decentralized platforms create a market for more energy-efficient hardware. Photonic cryptographic hash functions, which use photonic integrated circuits to accelerate computation, promise energy efficiency for verifying transactions and mining in a cryptonetwork. Like many analog computing approaches, however, current proposals for photonic cryptographic hash functions that promise similar security guarantees as Bitcoin are susceptible to systematic error, so multiple devices may not reach a consensus on computation despite high numerical precision (associated with low photodetector noise). In this paper, we theoretically and experimentally demonstrate that a more general family of robust discrete analog cryptographic hash functions, which we introduce as LightHash, leverages integer matrix-vector operations on photonic mesh networks of interferometers. The difficulty of LightHash can be adjusted to be sufficiently tolerant to systematic error (calibration error, loss error, coupling error, and phase error) and preserve inherent security guarantees present in the Bitcoin protocol. Finally, going beyond our proof-of-concept, we define a ``photonic advantage'' criterion and justify how recent developments in CMOS optoelectronics (including analog-digital conversion) provably achieve such advantage for robust and digitally-verifiable photonic computing and ultimately generate a new market for decentralized photonic technology.
The Power of Fragmentation: A Hierarchical Transformer Model for Structural Segmentation in Symbolic Music Generation
Venue: arXiv cs.SD 2022
Year: 2022

Symbolic Music Generation relies on the contextual representation capabilities of the generative model, where the most prevalent approach is the Transformer-based model. The learning of musical context is also related to the structural elements in music, i.e. intro, verse, and chorus, which are currently overlooked by the research community. In this paper, we propose a hierarchical Transformer model to learn multi-scale contexts in music. In the encoding phase, we first designed a Fragment Scope Localization layer to syncopate the music into chords and sections. Then, we use a multi-scale attention mechanism to learn note-, chord-, and section-level contexts. In the decoding phase, we proposed a hierarchical Transformer model that uses fine-decoders to generate sections in parallel and a coarse-decoder to decode the combined music. We also designed a Music Style Normalization layer to achieve a consistent music style between the generated sections. Our model is evaluated on two open MIDI datasets, and experiments show that our model outperforms the best contemporary music generative models. More excitingly, the visual evaluation shows that our model is superior in melody reuse, resulting in more realistic music.
Towards Robust Low Light Image Enhancement
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we study the problem of making brighter images from dark images found in the wild. The images are dark because they are taken in dim environments. They suffer from color shifts caused by quantization and from sensor noise. We don't know the true camera reponse function for such images and they are not RAW. We use a supervised learning method, relying on a straightforward simulation of an imaging pipeline to generate usable dataset for training and testing. On a number of standard datasets, our approach outperforms the state of the art quantitatively. Qualitative comparisons suggest strong improvements in reconstruction accuracy.
K-textures, a self-supervised hard clustering deep learning algorithm for satellite image segmentation
Venue: arXiv cs.CV 2022
Year: 2022

Deep learning self-supervised algorithms that can segment an image in a fixed number of hard labels such as the k-means algorithm and relying only on deep learning techniques are still lacking. Here, we introduce the k-textures algorithm which provides self-supervised segmentation of a 4-band image (RGB-NIR) for a $k$ number of classes. An example of its application on high resolution Planet satellite imagery is given. Our algorithm shows that discrete search is feasible using convolutional neural networks (CNN) and gradient descent. The model detects $k$ hard clustering classes represented in the model as $k$ discrete binary masks and their associated $k$ independently generated textures, that combined are a simulation of the original image. The similarity loss is the mean squared error between the features of the original and the simulated image, both extracted from the penultimate convolutional block of Keras 'imagenet' pretrained VGG-16 model and a custom feature extractor made with Planet data. The main advances of the k-textures model are: first, the $k$ discrete binary masks are obtained inside the model using gradient descent. The model allows for the generation of discrete binary masks using a novel method using a hard sigmoid activation function. Second, it provides hard clustering classes -- each pixels has only one class. Finally, in comparison to k-means, where each pixel is considered independently, here, contextual information is also considered and each class is not associated only to similar values in the color channels but also to a texture. Our approach is designed to ease the production of training samples for satellite image segmentation and the k-textures architecture could be adapted to support different number of bands and for more complex tasks, such as object self-segmentation. The model codes and weights are available at https://doi.org/10.5281/zenodo.6359859
Spatial-Temporal Interactive Dynamic Graph Convolution Network for Traffic Forecasting
Venue: arXiv cs.LG 2022
Year: 2022

Accurate traffic forecasting is essential for smart cities to achieve traffic control, route planning, and flow detection. Although many spatial-temporal methods are currently proposed, these methods are deficient in capturing the spatial-temporal dependence of traffic data synchronously. In addition, most of the methods ignore the dynamically changing correlations between road network nodes that arise as traffic data changes. We propose a neural network-based Spatial-Temporal Interactive Dynamic Graph Convolutional Network (STIDGCN) to address the above challenges for traffic forecasting. Specifically, we propose an interactive dynamic graph convolution structure, which divides the sequences at intervals and synchronously captures the traffic data's spatial-temporal dependence through an interactive learning strategy. The interactive learning strategy makes STIDGCN effective for long-term prediction. We also propose a novel dynamic graph convolution module to capture the dynamically changing correlations in the traffic network, consisting of a graph generator and fusion graph convolution. The dynamic graph convolution module can use the input traffic data and pre-defined graph structure to generate a graph structure. It is then fused with the defined adaptive adjacency matrix to generate a dynamic adjacency matrix, which fills the pre-defined graph structure and simulates the generation of dynamic associations between nodes in the road network. Extensive experiments on four real-world traffic flow datasets demonstrate that STIDGCN outperforms the state-of-the-art baseline.
Revisiting PINNs: Generative Adversarial Physics-informed Neural Networks and Point-weighting Method
Venue: arXiv cs.LG 2022
Year: 2022

Physics-informed neural networks (PINNs) provide a deep learning framework for numerically solving partial differential equations (PDEs), and have been widely used in a variety of PDE problems. However, there still remain some challenges in the application of PINNs: 1) the mechanism of PINNs is unsuitable (at least cannot be directly applied) to exploiting a small size of (usually very few) extra informative samples to refine the networks; and 2) the efficiency of training PINNs often becomes low for some complicated PDEs. In this paper, we propose the generative adversarial physics-informed neural network (GA-PINN), which integrates the generative adversarial (GA) mechanism with the structure of PINNs, to improve the performance of PINNs by exploiting only a small size of exact solutions to the PDEs. Inspired from the weighting strategy of the Adaboost method, we then introduce a point-weighting (PW) method to improve the training efficiency of PINNs, where the weight of each sample point is adaptively updated at each training iteration. The numerical experiments show that GA-PINNs outperform PINNs in many well-known PDEs and the PW method also improves the efficiency of training PINNs and GA-PINNs.