OpenRaiser/Intern-Atlas huggingface dataset:
Panoptic-aware Image-to-Image Translation
Venue: arXiv cs.CV 2022
Year: 2022

Despite remarkable progress in image translation, the complex scene with multiple discrepant objects remains a challenging problem. The translated images have low fidelity and tiny objects in fewer details causing unsatisfactory performance in object recognition. Without thorough object perception (i.e., bounding boxes, categories, and masks) of images as prior knowledge, the style transformation of each object will be difficult to track in translation. We propose panoptic-aware generative adversarial networks (PanopticGAN) for image-to-image translation together with a compact panoptic segmentation dataset. The panoptic perception (i.e., foreground instances and background semantics of the image scene) is extracted to achieve alignment between object content codes of the input domain and panoptic-level style codes sampled from the target style space, then refined by a proposed feature masking module for sharping object boundaries. The image-level combination between content and sampled style codes is also merged for higher fidelity image generation. Our proposed method was systematically compared with different competing methods and obtained significant improvement in both image quality and object recognition performance.
MARISA: A Self-configuring Metasurfaces Absorption and Reflection Solution Towards 6G
Venue: arXiv cs.NI 2022
Year: 2022

Reconfigurable Intelligent Surfaces (RISs) are considered one of the key disruptive technologies towards future 6G networks. RISs revolutionize the traditional wireless communication paradigm by controlling the wave propagation properties of the impinging signals as required. A major roadblock for RIS is though the need for a fast and complex control channel to continuously adapt to the ever-changing wireless channel conditions. In this paper, we ask ourselves the question: Would it be feasible to remove the need for control channels for RISs? We analyze the feasibility of devising Self-Configuring Smart Surfaces that can be easily and seamlessly installed throughout the environment, following the new Internet-of-Surfaces (IoS) paradigm, without requiring modifications of the deployed mobile network. To this aim, we design MARISA, a self-configuring metasurfaces absorption and reflection solution, and show that it can achieve a better-than-expected performance rivaling with control channel-driven RISs.
Direction-Oriented Stress-Constrained Topology Optimization of Orthotropic Materials
Venue: arXiv cs.GR 2022
Year: 2022

Efficient optimization of topology and raster angle has shown unprecedented enhancements in the mechanical properties of 3D printed materials. Topology optimization helps reduce the waste of raw material in the fabrication of 3D printed parts, thus decreasing production costs associated with manufacturing lighter structures. Fiber orientation plays an important role in increasing the stiffness of a structure. This paper develops and tests a new method for handling stress constraints in topology and fiber orientation optimization of 3D printed orthotropic structures. The stress constraints are coupled with an objective function that maximizes stiffness. This is accomplished by using the modified solid isotropic material with penalization method with the method of moving asymptotes as the mathematical optimizer. Each element has a fictitious density and an angle as the main design variables. To reduce the number of stress constraints and thus the computational cost, a new clustering strategy is employed in which the highest stresses in the principal material coordinates are grouped separately into two clusters using an adjusted $P$-norm. A detailed description of the formulation and sensitivity analysis is discussed. While we present an analysis of 2D structures in the numerical examples section, the method can also be used for 3D structures, as the formulation is generic. Our results show that this method can produce efficient structures suitable for 3D printing while thresholding the stresses.
Node-wise Hardware Trojan Detection Based on Graph Learning
Venue: arXiv cs.CR 2022
Year: 2022

In the fourth industrial revolution, securing the protection of the supply chain has become an ever-growing concern. One such cyber threat is a hardware Trojan (HT), a malicious modification to an IC. HTs are often identified in the hardware manufacturing process, but should be removed earlier, when the design is being specified. Machine learning-based HT detection in gate-level netlists is an efficient approach to identify HTs at the early stage. However, feature-based modeling has limitations in discovering an appropriate set of HT features. We thus propose NHTD-GL in this paper, a novel node-wise HT detection method based on graph learning (GL). Given the formal analysis of HT features obtained from domain knowledge, NHTD-GL bridges the gap between graph representation learning and feature-based HT detection. The experimental results demonstrate that NHTD-GL achieves 0.998 detection accuracy and outperforms state-of-the-art node-wise HT detection methods. NHTD-GL extracts HT features without heuristic feature engineering.
Updating Barcodes and Representatives for Zigzag Persistence
Venue: arXiv cs.CG 2022
Year: 2022

Computing persistence over changing filtrations give rise to a stack of 2D persistence diagrams where the birth-death points are connected by the so-called `vines'. We consider computing these vines over changing filtrations for zigzag persistence. We observe that eight atomic operations are sufficient for changing one zigzag filtration to another and provide update algorithms for each of them. Six of these operations that have some analogues to one or multiple transpositions in the non-zigzag case can be executed as efficiently as their non-zigzag counterparts. This approach takes advantage of a recently discovered algorithm for computing zigzag barcodes by converting a zigzag filtration to a non-zigzag one and then connecting barcodes of the two with a bijection. The remaining two atomic operations do not have a strict analogue in the non-zigzag case. For them, we propose algorithms based on explicit maintenance of representatives (homology cycles) which can be useful in their own rights for applications requiring explicit updates of representatives.
On Complexity of Computing Bottleneck and Lexicographic Optimal Cycles in a Homology Class
Venue: arXiv cs.CG 2022
Year: 2022

Homology features of spaces which appear in applications, for instance 3D meshes, are among the most important topological properties of these objects. Given a non-trivial cycle in a homology class, we consider the problem of computing a representative in that homology class which is optimal. We study two measures of optimality, namely, the lexicographic order of cycles (the lex-optimal cycle) and the bottleneck norm (a bottleneck-optimal cycle). We give a simple algorithm for computing the lex-optimal cycle for a 1-homology lass in a closed orientable surface. In contrast to this, our main result is that, in the case of 3-Manifolds of size $n^2$ in the Euclidean 3-space, the problem of finding a bottleneck optimal cycle cannot be solved more efficiently than solving a system of linear equations with an $n \times n$ sparse matrix. From this reduction, we deduce several hardness results. Most notably, we show that for 3-manifolds given as a subset of the 3-space of size $n^2$, persistent homology computations are at least as hard as rank computation (for sparse matrices) while ordinary homology computations can be done in $O(n^2 \log n)$ time. This is the first such distinction between these two computations. Moreover, it follows that the same disparity exists between the height persistent homology computation and general sub-level set persistent homology computation for simplicial complexes in the 3-space.
Variational Wasserstein gradient flow
Venue: arXiv cs.LG 2022
Year: 2022

Wasserstein gradient flow has emerged as a promising approach to solve optimization problems over the space of probability distributions. A recent trend is to use the well-known JKO scheme in combination with input convex neural networks to numerically implement the proximal step. The most challenging step, in this setup, is to evaluate functions involving density explicitly, such as entropy, in terms of samples. This paper builds on the recent works with a slight but crucial difference: we propose to utilize a variational formulation of the objective function formulated as maximization over a parametric class of functions. Theoretically, the proposed variational formulation allows the construction of gradient flows directly for empirical distributions with a well-defined and meaningful objective function. Computationally, this approach replaces the computationally expensive step in existing methods, to handle objective functions involving density, with inner loop updates that only require a small batch of samples and scale well with the dimension. The performance and scalability of the proposed method are illustrated with the aid of several numerical experiments involving high-dimensional synthetic and real datasets.
Emojich -- zero-shot emoji generation using Russian language: a technical report
Venue: arXiv cs.CL 2022
Year: 2022

This technical report presents a text-to-image neural network "Emojich" that generates emojis using captions in Russian language as a condition. We aim to keep the generalization ability of a pretrained big model ruDALL-E Malevich (XL) 1.3B parameters at the fine-tuning stage, while giving special style to the images generated. Here are presented some engineering methods, code realization, all hyper-parameters for reproducing results and a Telegram bot where everyone can create their own customized sets of stickers. Also, some newly generated emojis obtained by "Emojich" model are demonstrated.
A Tool for Rejuvenating Feature Logging Levels via Git Histories and Degree of Interest
Venue: arXiv cs.SE 2022
Year: 2022

Logging is a significant programming practice. Due to the highly transactional nature of modern software applications, massive amount of logs are generated every day, which may overwhelm developers. Logging information overload can be dangerous to software applications. Using log levels, developers can print the useful information while hiding the verbose logs during software runtime. As software evolves, the log levels of logging statements associated with the surrounding software feature implementation may also need to be altered. Maintaining log levels necessitates a significant amount of manual effort. In this paper, we demonstrate an automated approach that can rejuvenate feature log levels by matching the interest level of developers in the surrounding features. The approach is implemented as an open-source Eclipse plugin, using two external plug-ins (JGit and Mylyn). It was tested on 18 open-source Java projects consisting of ~3 million lines of code and ~4K log statements. Our tool successfully analyzes 99.22% of logging statements, increases log level distributions by ~20%, and increases the focus of logs in bug fix contexts ~83% of the time. For further details, interested readers can watch our demonstration video (https://www.youtube.com/watch?v=qIULoAXoDv4).
User behavior understanding in real world settings
Venue: arXiv cs.IR 2022
Year: 2022

How to extract meaningful information in user historical behavior plays a crucial role in recommendation. User behavior sequence often contains multiple conceptually distinct items that belong to different item groups and the number of the item groups is changing over time. It is necessary to learn a dynamic group of representations according the item groups in a user historical behavior. However, current works only learns a predefined and fixed number representations which includes single representation methods and multi representations methods from the user context that could lead to suboptimal recommendation quality. In this paper we propose a model that can automatically and adaptively generates a dynamic group of representations from the user behavior accordingly. To be specific, AutoRep is composed of an informative representation construct (IRC) module and a dynamic representations construct (DRC) module. The IRC module learns the overall sequential characteristics of user behavior with a bi-directional architecture transformer. The DRC module dynamically allocate the item in the user behavior into different item groups and form a dynamic group of representations in a differentiable method. Such design improves the model recommendation performance. We evaluate the proposed model on five benchmark datasets. The results show that AutoRep outperforms representative baselines. Further ablation study has been conducted to deepen our understandings of AutoRep, including the proposed module IRC and DRC.
Parametric Channel Estimation for LoS Dominated Holographic Massive MIMO Systems
Venue: arXiv cs.IT 2022
Year: 2022

Holographic massive multiple-input multiple-output (MIMO), in which a spatially continuous surface is being used for signal transmission and reception, have emerged as a promising solution for improving the coverage and data rate of wireless communication systems. To realize these objectives, the acquisition of accurate channel state information in holographic massive MIMO systems is crucial. In this paper, we propose a channel estimation scheme based on a parametric physical channel model for line-of-sight (LoS) dominated communication in millimeter and terahertz wave bands. The proposed channel estimation scheme exploits the specific structure of the radiated beams generated by the continuous surface to estimate the channel parameters in a dominated LoS channel model. The training overhead and computational complexity of the proposed scheme do not scale with the number of antennas. The simulation results demonstrate that the proposed estimation scheme significantly outperforms other benchmark schemes in poor scattering environment.
Towards More Robust Natural Language Understanding
Venue: arXiv cs.CL 2022
Year: 2022

Natural Language Understanding (NLU) is a branch of Natural Language Processing (NLP) that uses intelligent computer software to understand texts that encode human knowledge. Recent years have witnessed notable progress across various NLU tasks with deep learning techniques, especially with pretrained language models. Besides proposing more advanced model architectures, constructing more reliable and trustworthy datasets also plays a huge role in improving NLU systems, without which it would be impossible to train a decent NLU model. It's worth noting that the human ability of understanding natural language is flexible and robust. On the contrary, most of existing NLU systems fail to achieve desirable performance on out-of-domain data or struggle on handling challenging items (e.g., inherently ambiguous items, adversarial items) in the real world. Therefore, in order to have NLU models understand human language more effectively, it is expected to prioritize the study on robust natural language understanding. In this thesis, we deem that NLU systems are consisting of two components: NLU models and NLU datasets. As such, we argue that, to achieve robust NLU, the model architecture/training and the dataset are equally important. Specifically, we will focus on three NLU tasks to illustrate the robustness problem in different NLU tasks and our contributions (i.e., novel models and new datasets) to help achieve more robust natural language understanding. Moving forward, the ultimate goal for robust natural language understanding is to build NLU models which can behave humanly. That is, it's expected that robust NLU systems are capable to transfer the knowledge from training corpus to unseen documents more reliably and survive when encountering challenging items even if the system doesn't know a priori of users' inputs.
Modular Pipe Climber III with Three-Output Open Differential
Venue: arXiv cs.RO 2022
Year: 2022

The paper introduces the novel Modular Pipe Climber III with a Three-Output Open Differential (3-OOD) mechanism to eliminate slipping of the tracks due to the changing cross-sections of the pipe. This will be achieved in any orientation of the robot. Previous pipe climbers use three-wheel/track modules, each with an individual driving mechanism to achieve stable traversing. Slipping of tracks is prevalent in such robots when it encounters the pipe turns. Thus, active control of each module's speed is employed to mitigate the slip, thereby requiring substantial control effort. The proposed pipe climber implements the 3-OOD to address this issue by allowing the robot to mechanically modulate the track speeds as it encounters a turn. The proposed 3-OOD is the first three-output differential to realize the functional abilities of a traditional two-output differential.
Communication Beyond Transmitting Bits: Semantics-Guided Source and Channel Coding
Venue: arXiv cs.IT 2022
Year: 2022

Classical communication paradigms focus on accurately transmitting bits over a noisy channel, and Shannon theory provides a fundamental theoretical limit on the rate of reliable communications. In this approach, bits are treated equally, and the communication system is oblivious to what meaning these bits convey or how they would be used. Future communications towards intelligence and conciseness will predictably play a dominant role, and the proliferation of connected intelligent agents requires a radical rethinking of coded transmission paradigm to support the new communication morphology on the horizon. The recent concept of "semantic communications" offers a promising research direction. Injecting semantic guidance into the coded transmission design to achieve semantics-aware communications shows great potential for further breakthrough in effectiveness and reliability. This article sheds light on semantics-guided source and channel coding as a transmission paradigm of semantic communications, which exploits both data semantics diversity and wireless channel diversity together to boost the whole system performance. We present the general system architecture and key techniques, and indicate some open issues on this topic.
Faster Cut Sparsification of Weighted Graphs
Venue: arXiv cs.DS 2022
Year: 2022

A cut sparsifier is a reweighted subgraph that maintains the weights of the cuts of the original graph up to a multiplicative factor of $(1\pmε)$. This paper considers computing cut sparsifiers of weighted graphs of size $O(n\log (n)/ε^2)$. Our algorithm computes such a sparsifier in time $O(m\cdot\min(α(n)\log(m/n),\log (n)))$, both for graphs with polynomially bounded and unbounded integer weights, where $α(\cdot)$ is the functional inverse of Ackermann's function. This improves upon the state of the art by Benczúr and Karger (SICOMP 2015), which takes $O(m\log^2 (n))$ time. For unbounded weights, this directly gives the best known result for cut sparsification. Together with preprocessing by an algorithm of Fung et al. (SICOMP 2019), this also gives the best known result for polynomially-weighted graphs. Consequently, this implies the fastest approximate min-cut algorithm, both for graphs with polynomial and unbounded weights. In particular, we show that it is possible to adapt the state of the art algorithm of Fung et al. for unweighted graphs to weighted graphs, by letting the partial maximum spanning forest (MSF) packing take the place of the Nagamochi-Ibaraki (NI) forest packing. MSF packings have previously been used by Abraham at al. (FOCS 2016) in the dynamic setting, and are defined as follows: an $M$-partial MSF packing of $G$ is a set $\mathcal{F}=\{F_1, \dots, F_M\}$, where $F_i$ is a maximum spanning forest in $G\setminus \bigcup_{j=1}^{i-1}F_j$. Our method for computing (a sufficient estimation of) the MSF packing is the bottleneck in the running time of our sparsification algorithm.
Embedding Arithmetic of Multimodal Queries for Image Retrieval
Venue: arXiv cs.CV 2022
Year: 2022

Latent text representations exhibit geometric regularities, such as the famous analogy: queen is to king what woman is to man. Such structured semantic relations were not demonstrated on image representations. Recent works aiming at bridging this semantic gap embed images and text into a multimodal space, enabling the transfer of text-defined transformations to the image modality. We introduce the SIMAT dataset to evaluate the task of Image Retrieval with Multimodal queries. SIMAT contains 6k images and 18k textual transformation queries that aim at either replacing scene elements or changing pairwise relationships between scene elements. The goal is to retrieve an image consistent with the (source image, text transformation) query. We use an image/text matching oracle (OSCAR) to assess whether the image transformation is successful. The SIMAT dataset will be publicly available. We use SIMAT to evaluate the geometric properties of multimodal embedding spaces trained with an image/text matching objective, like CLIP. We show that vanilla CLIP embeddings are not very well suited to transform images with delta vectors, but that a simple finetuning on the COCO dataset can bring dramatic improvements. We also study whether it is beneficial to leverage pretrained universal sentence encoders (FastText, LASER and LaBSE).
Physically Consistent Neural Networks for building thermal modeling: theory and analysis
Venue: arXiv cs.LG 2022
Year: 2022

Due to their high energy intensity, buildings play a major role in the current worldwide energy transition. Building models are ubiquitous since they are needed at each stage of the life of buildings, i.e. for design, retrofitting, and control operations. Classical white-box models, based on physical equations, are bound to follow the laws of physics but the specific design of their underlying structure might hinder their expressiveness and hence their accuracy. On the other hand, black-box models are better suited to capture nonlinear building dynamics and thus can often achieve better accuracy, but they require a lot of data and might not follow the laws of physics, a problem that is particularly common for neural network (NN) models. To counter this known generalization issue, physics-informed NNs have recently been introduced, where researchers introduce prior knowledge in the structure of NNs to ground them in known underlying physical laws and avoid classical NN generalization issues. In this work, we present a novel physics-informed NN architecture, dubbed Physically Consistent NN (PCNN), which only requires past operational data and no engineering overhead, including prior knowledge in a linear module running in parallel to a classical NN. We formally prove that such networks are physically consistent - by design and even on unseen data - with respect to different control inputs and temperatures outside and in neighboring zones. We demonstrate their performance on a case study, where the PCNN attains an accuracy up to 40% better than a classical physics-based resistance-capacitance model on 3-day long prediction horizons. Furthermore, despite their constrained structure, PCNNs attain similar performance to classical NNs on the validation data, overfitting the training data less and retaining high expressiveness to tackle the generalization issue.