OpenRaiser/Intern-Atlas huggingface dataset:
Generative Artisan: A Semantic-Aware and Controllable CLIPstyler
Venue: arXiv cs.CV 2022
Year: 2022

Recall that most of the current image style transfer methods require the user to give an image of a particular style and then extract that styling feature and texture to generate the style of an image, but there are still some problems: the user may not have a reference style image, or it may be difficult to summarise the desired style in mind with just one image. The recently proposed CLIPstyler has solved this problem, which is able to perform style transfer based only on the provided description of the style image. Although CLIPstyler can achieve good performance when landscapes or portraits appear alone, it can blur the people and lose the original semantics when people and landscapes coexist. Based on these issues, we demonstrate a novel framework that uses a pre-trained CLIP text-image embedding model and guides image style transfer through an FCN semantic segmentation network. Specifically, we solve the portrait over-styling problem for both selfies and real-world landscape with human subjects photos, enhance the contrast between the effect of style transfer in portrait and landscape, and make the degree of image style transfer in different semantic parts fully controllable. Our Generative Artisan resolve the failure case of CLIPstyler and yield both qualitative and quantitative methods to prove ours have much better results than CLIPstyler in both selfies and real-world landscape with human subjects photos. This improvement makes it possible to commercialize our framework for business scenarios such as retouching graphics software.
Explored An Effective Methodology for Fine-Grained Snake Recognition
Venue: arXiv cs.CV 2022
Year: 2022

Fine-Grained Visual Classification (FGVC) is a longstanding and fundamental problem in computer vision and pattern recognition, and underpins a diverse set of real-world applications. This paper describes our contribution at SnakeCLEF2022 with FGVC. Firstly, we design a strong multimodal backbone to utilize various meta-information to assist in fine-grained identification. Secondly, we provide new loss functions to solve the long tail distribution with dataset. Then, in order to take full advantage of unlabeled datasets, we use self-supervised learning and supervised learning joint training to provide pre-trained model. Moreover, some effective data process tricks also are considered in our experiments. Last but not least, fine-tuned in downstream task with hard mining, ensambled kinds of model performance. Extensive experiments demonstrate that our method can effectively improve the performance of fine-grained recognition. Our method can achieve a macro f1 score 92.7% and 89.4% on private and public dataset, respectively, which is the 1st place among the participators on private leaderboard.
FileInsurer: A Scalable and Reliable Protocol for Decentralized File Storage in Blockchain
Venue: arXiv cs.CR 2022
Year: 2022

With the development of blockchain applications, the requirements for file storage in blockchain are increasing rapidly. Many protocols, including Filecoin, Arweave, and Sia, have been proposed to provide scalable decentralized file storage for blockchain applications. However, the reliability is not well promised by existing protocols. Inspired by the idea of insurance, we innovatively propose a decentralized file storage protocol in blockchain, named as FileInsurer, to achieve both scalability and reliability. While ensuring scalability by distributed storage, FileInsurer guarantees reliability by enhancing robustness and fully compensating for the file loss. Specifically, under mild conditions, we prove that no more than 0.1\% value of all files should be compensated even if half of the storage collapses. Therefore, only a relatively small deposit needs to be pledged by storage providers to cover the potential file loss. Because of lower burdens of deposit, storage providers have more incentives to participate in the storage network. FileInsurer can run in the top layer of the InterPlanetary File System (IPFS), and thus it can be directly applied in Web 3.0, Non-Fungible Tokens, and Metaverse.
Degree Centrality Algorithms For Homogeneous Multilayer Networks
Venue: arXiv cs.IT 2022
Year: 2022

Centrality measures for simple graphs/networks are well-defined and each has numerous main-memory algorithms. However, for modeling complex data sets with multiple types of entities and relationships, simple graphs are not ideal. Multilayer networks (or MLNs) have been proposed for modeling them and have been shown to be better suited in many ways. Since there are no algorithms for computing centrality measures directly on MLNs, existing strategies reduce (aggregate or collapse) the MLN layers to simple networks using Boolean AND or OR operators. This approach negates the benefits of MLN modeling as these computations tend to be expensive and furthermore results in loss of structure and semantics. In this paper, we propose heuristic-based algorithms for computing centrality measures (specifically, degree centrality) on MLNs directly (i.e., without reducing them to simple graphs) using a newly-proposed decoupling-based approach which is efficient as well as structure and semantics preserving. We propose multiple heuristics to calculate the degree centrality using the network decoupling-based approach and compare accuracy and precision with Boolean OR aggregated Homogeneous MLNs (HoMLN) for ground truth. The network decoupling approach can take advantage of parallelism and is more efficient compared to aggregation-based approaches. Extensive experimental analysis is performed on large synthetic and real-world data sets of varying characteristics to validate the accuracy and efficiency of our proposed algorithms.
AutoWeird: Weird Translational Scoring Function Identified by Random Search
Venue: arXiv cs.AI 2022
Year: 2022

Scoring function (SF) measures the plausibility of triplets in knowledge graphs. Different scoring functions can lead to huge differences in link prediction performances on different knowledge graphs. In this report, we describe a weird scoring function found by random search on the open graph benchmark (OGB). This scoring function, called AutoWeird, only uses tail entity and relation in a triplet to compute its plausibility score. Experimental results show that AutoWeird achieves top-1 performance on ogbl-wikikg2 data set, but has much worse performance than other methods on ogbl-biokg data set. By analyzing the tail entity distribution and evaluation protocol of these two data sets, we attribute the unexpected success of AutoWeird on ogbl-wikikg2 to inappropriate evaluation and concentrated tail entity distribution. Such results may motivate further research on how to accurately evaluate the performance of different link prediction methods for knowledge graphs.
On the Validation of Multi-Level Personalised Health Condition Model
Venue: arXiv cs.SE 2022
Year: 2022

This paper presents a verification-based methodology to validate the model of personalized health conditions. The model identifies the values that may result in unsafe, un-reachable, in-exhaustive, and overlapping states that otherwise threaten patients' life by producing false alarms by accepting suspicious behaviour of the target health condition. Contemporary approaches to validating a model employ various testing, simulation and model checking techniques to recognise such values and corresponding vulnerabilities. However, these approaches are neither systematic nor exhaustive and thus fail to identify those false values or vulnerabilities that estimate the health condition at run-time based on the sensor or input data received from various IoT medical devices. We have demonstrated the validation methodology by validating our example multi-level model that describes three different scenarios of Diabetes health conditions.
AMS-Net: Adaptive Multiscale Sparse Neural Network with Interpretable Basis Expansion for Multiphase Flow Problems
Venue: arXiv cs.LG 2022
Year: 2022

In this work, we propose an adaptive sparse learning algorithm that can be applied to learn the physical processes and obtain a sparse representation of the solution given a large snapshot space. Assume that there is a rich class of precomputed basis functions that can be used to approximate the quantity of interest. We then design a neural network architecture to learn the coefficients of solutions in the spaces which are spanned by these basis functions. The information of the basis functions are incorporated in the loss function, which minimizes the differences between the downscaled reduced order solutions and reference solutions at multiple time steps. The network contains multiple submodules and the solutions at different time steps can be learned simultaneously. We propose some strategies in the learning framework to identify important degrees of freedom. To find a sparse solution representation, a soft thresholding operator is applied to enforce the sparsity of the output coefficient vectors of the neural network. To avoid over-simplification and enrich the approximation space, some degrees of freedom can be added back to the system through a greedy algorithm. In both scenarios, that is, removing and adding degrees of freedom, the corresponding network connections are pruned or reactivated guided by the magnitude of the solution coefficients obtained from the network outputs. The proposed adaptive learning process is applied to some toy case examples to demonstrate that it can achieve a good basis selection and accurate approximation. More numerical tests are performed on two-phase multiscale flow problems to show the capability and interpretability of the proposed method on complicated applications.
Joint Transmit and Reflective Beamforming for RIS-assisted Secret Key Generation
Venue: arXiv cs.IT 2022
Year: 2022

Reconfigurable intelligent surface (RIS) is a promising technique to enhance the performance of physical-layer key generation (PKG) due to its ability to smartly customize the radio environments. Existing RIS-assisted PKG methods are mainly based on the idealistic assumption of an independent and identically distributed (i.i.d.) channel model at both the transmitter and the RIS. However, the i.i.d. model is inaccurate for a typical RIS in an isotropic scattering environment. Also, neglecting the existence of channel spatial correlation would degrade the PKG performance. In this paper, we establish a general spatially correlated channel model in multi-antenna systems and propose a new PKG framework based on the transmit and the reflective beamforming at the base station (BS) and the RIS. Specifically, we derive a closed-form expression for characterizing the key generation rate (KGR) and obtain a globally optimal solution of the beamformers to maximize the KGR. Furthermore, we analyze the KGR performance difference between the one adopting the assumption of the i.i.d. model and that of the spatially correlated model. It is found that the beamforming designed for the correlated model outperforms that for the i.i.d. model while the KGR gain increases with the channel correlation. Simulation results show that compared to existing methods based on the i.i.d. fading model, our proposed method achieves about $5$ dB performance gain when the BS antenna correlation $ρ$ is $0.3$ and the RIS element spacing is half of the wavelength.
Enhancements to the BOUN Treebank Reflecting the Agglutinative Nature of Turkish
Venue: arXiv cs.CL 2022
Year: 2022

In this study, we aim to offer linguistically motivated solutions to resolve the issues of the lack of representation of null morphemes, highly productive derivational processes, and syncretic morphemes of Turkish in the BOUN Treebank without diverging from the Universal Dependencies framework. In order to tackle these issues, new annotation conventions were introduced by splitting certain lemmas and employing the MISC (miscellaneous) tab in the UD framework to denote derivation. Representational capabilities of the re-annotated treebank were tested on a LSTM-based dependency parser and an updated version of the BoAT Tool is introduced.
Physics-Informed Learning of Aerosol Microphysics
Venue: arXiv cs.LG 2022
Year: 2022

Aerosol particles play an important role in the climate system by absorbing and scattering radiation and influencing cloud properties. They are also one of the biggest sources of uncertainty for climate modeling. Many climate models do not include aerosols in sufficient detail due to computational constraints. In order to represent key processes, aerosol microphysical properties and processes have to be accounted for. This is done in the ECHAM-HAM global climate aerosol model using the M7 microphysics, but high computational costs make it very expensive to run with finer resolution or for a longer time. We aim to use machine learning to emulate the microphysics model at sufficient accuracy and reduce the computational cost by being fast at inference time. The original M7 model is used to generate data of input-output pairs to train a neural network on it. We are able to learn the variables' tendencies achieving an average $R^2$ score of $77.1\% $. We further explore methods to inform and constrain the neural network with physical knowledge to reduce mass violation and enforce mass positivity. On a GPU we achieve a speed-up of up to over 64x compared to the original model.
A Multiple-Entanglement Routing Framework for Quantum Networks
Venue: arXiv cs.NI 2022
Year: 2022

Quantum networks are gaining momentum in finding applications in a wide range of domains. However, little research has investigated the potential of a quantum network framework to enable highly reliable communications. The goal of this work is to investigate and design the multiple-entanglement routing framework, namely k-entangled routing. In particular, the $k$-entangled routing will enable k paths connecting all demands (source-destination pairs) in the network. To design the $k$-entangled routing, we propose two algorithms that are called Sequential Multi-path Scheduling Algorithm and Min-Cut-based Multi-path Scheduling Algorithm. In addition, we evaluate the performance of the proposed algorithms and models through a realistic quantum network simulator, NetSquid, that models the stochastic processes underlying quantum communications. The results show that the proposed algorithms (SMPSA and MCSA) largely enhance the network's traffic flexibility. The proposed paradigms would lay the foundation for further research on the area of entanglement routing.
Maximizing Entanglement Routing Rate in Quantum Networks: Approximation Algorithms
Venue: arXiv cs.NI 2022
Year: 2022

There will be a fast-paced shift from conventional network systems to novel quantum networks that are supported by the quantum entanglement and teleportation, key technologies of the quantum era, to enable secured data transmissions in the next-generation of the Internet. Despite this prospect, migration to quantum networks cannot be done at once, especially on the aspect of quantum routing. In this paper, we study the maximizing entangled routing rate (MERR) problem. In particular, given a set of demands, we try to determine entangled routing paths for the maximum number of demands in the quantum network while meeting the network's fidelity. We first formulate the MERR problem using an integer linear programming (ILP) model to capture the traffic patent for all demands in the network. We then leverage the theory of relaxation of ILP to devise two efficient algorithms including HBRA and RRA with provable approximation ratios for the objective function. To deal with the challenge of the combinatorial optimization problem in big scale networks, we also propose the path-length-based approach (PLBA) to solve the MERR problem. Using both simulations and an open quantum network simulator platform to conduct experiments with real-world topologies and traffic matrices, we evaluate the performance of our algorithms and show up the success of maximizing entangled routing rate.
Efficient Embedding VNFs in 5G Network Slicing: A Deep Reinforcement Learning Approach
Venue: arXiv cs.NI 2022
Year: 2022

5G radio access network (RAN) slicing aims to logically split an infrastructure into a set of self-contained programmable RAN slices, with each slice built on top of the underlying physical RAN (substrate) is a separate logical mobile network, which delivers a set of services with similar characteristics. Each RAN slice is constituted by various virtual network functions (VNFs) distributed geographically in numerous substrate nodes. A key challenge in building a robust RAN slicing is, therefore, designing a RAN slicing (RS)-configuration scheme that can utilize information such as resource availability in substrate networks as well as the interdependent relationships among slices to map (embed) VNFs onto live substrate nodes. With such motivation, we propose a machine-learning-powered RAN slicing scheme that aims to accommodate maximum numbers of slices (a set of connected Virtual Network Functions - VNFs) within a given request set. More specifically, we present a deep reinforcement scheme that is called Deep Allocation Agent (DAA). In short, DAA utilizes an empirically designed deep neural network that observes the current states of the substrate network and the requested slices to schedule the slices of which VNFs are then mapped to substrate nodes using an optimization algorithm. DAA is trained towards the goal of maximizing the number of accommodated slices in the given set by using an explicitly designed reward function. Our experiment study shows that, on average, DAA is able to maintain a rate of successfully routed slices above 80% in a resource-limited substrate network, and about 60% in extreme conditions, i.e., the available resources are much less than the demands.
New Additive Spanner Lower Bounds by an Unlayered Obstacle Product
Venue: arXiv cs.DS 2022
Year: 2022

For an input graph $G$, an additive spanner is a sparse subgraph $H$ whose shortest paths match those of $G$ up to small additive error. We prove two new lower bounds in the area of additive spanners: 1) We construct $n$-node graphs $G$ for which any spanner on $O(n)$ edges must increase a pairwise distance by $+Ω(n^{1/7})$. This improves on a recent lower bound of $+Ω(n^{1/10.5})$ by Lu, Wein, Vassilevska Williams, and Xu [SODA '22]. 2) A classic result by Coppersmith and Elkin [SODA '05] proves that for any $n$-node graph $G$ and set of $p = O(n^{1/2})$ demand pairs, one can exactly preserve all pairwise distances among demand pairs using a spanner on $O(n)$ edges. They also provided a lower bound construction, establishing that that this range $p = O(n^{1/2})$ cannot be improved. We strengthen this lower bound by proving that, for any constant $k$, this range of $p$ is still unimprovable even if the spanner is allowed $+k$ additive error among the demand pairs. This negatively resolves an open question asked by Coppersmith and Elkin [SODA '05] and again by Cygan, Grandoni, and Kavitha [STACS '13] and Abboud and Bodwin [SODA '16]. At a technical level, our lower bounds are obtained by an improvement to the entire obstacle product framework used to compose "inner" and "outer" graphs into lower bound instances. In particular, we develop a new strategy for analysis that allows certain non-layered graphs to be used in the product, and we use this freedom to design better inner and outer graphs that lead to our new lower bounds.
SAVCHOI: Detecting Suspicious Activities using Dense Video Captioning with Human Object Interactions
Venue: arXiv cs.CV 2022
Year: 2022

Detecting suspicious activities in surveillance videos is a longstanding problem in real-time surveillance that leads to difficulties in detecting crimes. Hence, we propose a novel approach for detecting and summarizing suspicious activities in surveillance videos. We have also created ground truth summaries for the UCF-Crime video dataset. We modify a pre-existing approach for this task by leveraging the Human-Object Interaction (HOI) model for the Visual features in the Bi-Modal Transformer. Further, we validate our approach against the existing state-of-the-art algorithms for the Dense Video Captioning task for the ActivityNet Captions dataset. We observe that this formulation for Dense Captioning performs significantly better than other discussed BMT-based approaches for BLEU@1, BLEU@2, BLEU@3, BLEU@4, and METEOR. We further perform a comparative analysis of the dataset and the model to report the findings based on different NMS thresholds (searched using Genetic Algorithms). Here, our formulation outperforms all the models for BLEU@1, BLEU@2, BLEU@3, and most models for BLEU@4 and METEOR falling short of only ADV-INF Global by 25% and 0.5%, respectively.
A Deep Dive into Deep Cluster
Venue: arXiv cs.CV 2022
Year: 2022

Deep Learning has demonstrated a significant improvement against traditional machine learning approaches in different domains such as image and speech recognition. Their success on benchmark datasets is transferred to the real-world through pretrained models by practitioners. Pretraining visual models using supervised learning requires a significant amount of expensive data annotation. To tackle this limitation, DeepCluster - a simple and scalable unsupervised pretraining of visual representations - has been proposed. However, the underlying work of the model is not yet well understood. In this paper, we analyze DeepCluster internals and exhaustively evaluate the impact of various hyperparameters over a wide range of values on three different datasets. Accordingly, we propose an explanation of why the algorithm works in practice. We also show that DeepCluster convergence and performance highly depend on the interplay between the quality of the randomly initialized filters of the convolutional layer and the selected number of clusters. Furthermore, we demonstrate that continuous clustering is not critical for DeepCluster convergence. Therefore, early stopping of the clustering phase will reduce the training time and allow the algorithm to scale to large datasets. Finally, we derive plausible hyperparameter selection criteria in a semi-supervised setting.
Adaptive Marginalized Semantic Hashing for Unpaired Cross-Modal Retrieval
Venue: arXiv cs.MM 2022
Year: 2022

In recent years, Cross-Modal Hashing (CMH) has aroused much attention due to its fast query speed and efficient storage. Previous literatures have achieved promising results for Cross-Modal Retrieval (CMR) by discovering discriminative hash codes and modality-specific hash functions. Nonetheless, most existing CMR works are subjected to some restrictions: 1) It is assumed that data of different modalities are fully paired, which is impractical in real applications due to sample missing and false data alignment, and 2) binary regression targets including the label matrix and binary codes are too rigid to effectively learn semantic-preserving hash codes and hash functions. To address these problems, this paper proposes an Adaptive Marginalized Semantic Hashing (AMSH) method which not only enhances the discrimination of latent representations and hash codes by adaptive margins, but also can be used for both paired and unpaired CMR. As a two-step method, in the first step, AMSH generates semantic-aware modality-specific latent representations with adaptively marginalized labels, which enlarges the distances between different classes, and exploits the labels to preserve the inter-modal and intra-modal semantic similarities into latent representations and hash codes. In the second step, adaptive margin matrices are embedded into the hash codes, and enlarge the gaps between positive and negative bits, which improves the discrimination and robustness of hash functions. On this basis, AMSH generates similarity-preserving hash codes and robust hash functions without strict one-to-one data correspondence requirement. Experiments are conducted on several benchmark datasets to demonstrate the superiority and flexibility of AMSH over some state-of-the-art CMR methods.