Miracle-dz/UniFAR-FTUs huggingface dataset:
Ousiometrics and Telegnomics: The essence of meaning conforms to a two-dimensional powerful-weak and dangerous-safe framework with diverse corpora presenting a safety bias

We define `ousiometrics' to be the study of essential meaning in whatever context that meaningful signals are communicated, and `telegnomics' as the study of remotely sensed knowledge. From work emerging through the middle of the 20th century, the essence of meaning has become generally accepted as being well captured by the three orthogonal dimensions of evaluation, potency, and activation (EPA). By re-examining first types and then tokens for the English language, and through the use of automatically annotated histograms -- `ousiograms' -- we find here that: 1. The essence of meaning conveyed by words is instead best described by a compass-like power-danger (PD) framework, and 2. Analysis of a disparate collection of large-scale English language corpora -- literature, news, Wikipedia, talk radio, and social media -- shows that natural language exhibits a systematic bias toward safe, low danger words -- a reinterpretation of the Pollyanna principle's positivity bias for written expression. To help justify our choice of dimension names and to help address the problems with representing observed ousiometric dimensions by bipolar adjective pairs, we introduce and explore `synousionyms' and `antousionyms' -- ousiometric counterparts of synonyms and antonyms. We further show that the PD framework revises the circumplex model of affect as a more general model of state of mind. Finally, we use our findings to construct and test a prototype `ousiometer', a telegnomic instrument that measures ousiometric time series for temporal corpora. We contend that our power-danger ousiometric framework provides a complement for entropy-based measurements, and may be of value for the study of a wide variety of communication across biological and artificial life.
Communication-Efficient Triangle Counting under Local Differential Privacy

Triangle counting in networks under LDP (Local Differential Privacy) is a fundamental task for analyzing connection patterns or calculating a clustering coefficient while strongly protecting sensitive friendships from a central server. In particular, a recent study proposes an algorithm for this task that uses two rounds of interaction between users and the server to significantly reduce estimation error. However, this algorithm suffers from a prohibitively high communication cost due to a large noisy graph each user needs to download. In this work, we propose triangle counting algorithms under LDP with a small estimation error and communication cost. We first propose two-rounds algorithms consisting of edge sampling and carefully selecting edges each user downloads so that the estimation error is small. Then we propose a double clipping technique, which clips the number of edges and then the number of noisy triangles, to significantly reduce the sensitivity of each user's query. Through comprehensive evaluation, we show that our algorithms dramatically reduce the communication cost of the existing algorithm, e.g., from 6 hours to 8 seconds or less at a 20 Mbps download rate, while keeping a small estimation error.
An Introduction to Automatic Diﬀerentiation for Machine Learning

Machine learning, and neural network models in particular have been improving the state of the art performance on many artiﬁcial intelligence related tasks. Neural network models are typically implemented using frameworks that perform gradient based optimization methods to ﬁt a model to a dataset. These frameworks use a technique of calculating derivatives called automatic diﬀerentiation (AD) which removes the burden of performing derivative calculations from the model designer. In this report we describe AD, its motivations, and diﬀerent implementation approaches. We brieﬂy describe dataﬂow programming as it relates to AD. Lastly, we present example programs that are implemented with the Tensorﬂow and PyTorch, which are two commonly used AD frameworks.
Learning to Select Historical News Articles for Interaction based Neural News Recommendation

The key to personalized news recommendation is to match the user's interests with the candidate news precisely and efficiently. Most existing approaches embed user interests into a representation vector then recommend by comparing it with the candidate news vector. In such a workflow, fine-grained matching signals may be lost. Recent studies try to cover that by modeling fine-grained interactions between the candidate news and each browsed news article of the user. Despite the effectiveness improvement, these models suffer from much higher computation costs online. Consequently, it remains a tough issue to take advantage of effective interactions in an efficient way. To address this problem, we proposed an end-to-end Selective Fine-grained Interaction framework (SFI) with a learning-to-select mechanism. Instead of feeding all historical news into interaction, SFI can quickly select informative historical news w.r. t. the candidate and exclude others from following computations. We empower the selection to be both sparse and automatic, which guarantees efficiency and effectiveness respectively. Extensive experiments on the publicly available dataset MIND validates the superiority of SFI over the state-of-the-art methods: with only five historical news selected, it can significantly improve the AUC by 2.17% over the state-of-the-art interaction-based models; at the same time, it is four times faster.
Voice-Assisted Image Labeling for Endoscopic Ultrasound Classification Using Neural Networks

Ultrasound imaging is a commonly used technology for visualising patient anatomy in real-time during diagnostic and therapeutic procedures. High operator dependency and low reproducibility make ultrasound imaging and interpretation challenging with a steep learning curve. Automatic image classification using deep learning has the potential to overcome some of these challenges by supporting ultrasound training in novices, as well as aiding ultrasound image interpretation in patient with complex pathology for more experienced practitioners. However, the use of deep learning methods requires a large amount of data in order to provide accurate results. Labelling large ultrasound datasets is a challenging task because labels are retrospectively assigned to 2D images without the 3D spatial context available in vivo or that would be inferred while visually tracking structures between frames during the procedure. In this work, we propose a multi-modal convolutional neural network (CNN) architecture that labels endoscopic ultrasound (EUS) images from raw verbal comments provided by a clinician during the procedure. We use a CNN composed of two branches, one for voice data and another for image data, which are joined to predict image labels from the spoken names of anatomical landmarks. The network was trained using recorded verbal comments from expert operators. Our results show a prediction accuracy of 76% at image level on a dataset with 5 different labels. We conclude that the addition of spoken commentaries can increase the performance of ultrasound image classification, and eliminate the burden of manually labelling large EUS datasets necessary for deep learning applications.
Dictionary Learning with Convex Update (ROMD)

Dictionary learning aims to find a dictionary under which the training data can be sparsely represented, and it is usually achieved by iteratively applying two stages: sparse coding and dictionary update. Typical methods for dictionary update focuses on refining both dictionary atoms and their corresponding sparse coefficients by using the sparsity patterns obtained from sparse coding stage, and hence it is a nonconvex bilinear inverse problem. In this paper, we propose a Rank-One Matrix Decomposition (ROMD) algorithm to recast this challenge into a convex problem by resolving these two variables into a set of rank-one matrices. Different from methods in the literature, ROMD updates the whole dictionary at a time using convex programming. The advantages hence include both convergence guarantees for dictionary update and faster convergence of the whole dictionary learning. The performance of ROMD is compared with other benchmark dictionary learning algorithms. The results show the improvement of ROMD in recovery accuracy, especially in the cases of high sparsity level and fewer observation data.
Exact Operator Fractionalization and Edge Physics in the Lowest-Landau-Level

Arkadiusz Bochniak, Zohar Nussinov, Alexander Seidel, and Gerardo Ortiz Institute of Theoretical Physics, Jagiellonian University, Lojasiewicza 11, 30-348 Kraków, Poland Department of Physics, Washington University in St. Louis, MO 63160, USA Department of Physics, Indiana University, Bloomington, IN 47405, USA Department of Mathematics, Indiana University, Bloomington, IN 47405, USA (Dated: October 14, 2021)
Maximizing Efficiency of Language Model Pre-training for Learning Representation

Pre-trained language models in the past years have shown exponential growth in model parameters and compute time. ELECTRA is a novel approach for improving the compute efficiency of pre-trained language models (e.g. BERT) based on masked language modeling (MLM) by addressing the sample inefficiency problem with the replaced token detection (RTD) task. Our work proposes adaptive early exit strategy to maximize the efficiency of the pre-training process by relieving the model's subsequent layers of the need to process latent features by leveraging earlier layer representations. Moreover, we evaluate an initial approach to the problem that has not succeeded in maintaining the accuracy of the model while showing a promising compute efficiency by thoroughly investigating the necessity of the generator module of ELECTRA.
ActiveEA: Active Learning for Neural Entity Alignment

Entity Alignment (EA) aims to match equivalent entities across different Knowledge Graphs (KGs) and is an essential step of KG fusion. Current mainstream methods – neural EA models – rely on training with seed alignment, i.e., a set of pre-aligned entity pairs which are very costly to annotate. In this paper, we devise a novel Active Learning (AL) framework for neural EA, aiming to create highly informative seed alignment to obtain more effective EA models with less annotation cost. Our framework tackles two main challenges encountered when applying AL to EA: (1) How to exploit dependencies between entities within the AL strategy. Most AL strategies assume that the data instances to sample are independent and identically distributed. However, entities in KGs are related. To address this challenge, we propose a structure-aware uncertainty sampling strategy that can measure the uncertainty of each entity as well as its impact on its neighbour entities in the KG. (2) How to recognise entities that appear in one KG but not in the other KG (i.e., bachelors). Identifying bachelors would likely save annotation budget. To address this challenge, we devise a bachelor recognizer paying attention to alleviate the effect of sampling bias. Empirical results show that our proposed AL strategy can significantly improve sampling quality with good generality across different datasets, EA models and amount of bachelors.
A Deep Learning Approach to Location- and Orientation-Aided 3D Beam Selection for mmWave Communications

Position-aided beam selection methods have been shown to be an effective approach to achieve high beamforming gain while limiting the overhead and latency of initial access in millimeter wave (mmWave) communications. Most research in the area, however, has focused on vehicular applications, where the orientation of the user terminal (UT) is mostly fixed at each position of the environment. This paper proposes a location- and orientation-based beam selection method to enable context information (CI)-based beam alignment in applications where the UT can take arbitrary orientation at each location. We propose three different network structures, with different amounts of trainable parameters that can be used with different training dataset sizes. A professional 3-dimensional ray tracing tool is used to generate datasets for an IEEE standard indoor scenario. Numerical results show the proposed networks outperform a CI-aided benchmark such as the generalized inverse fingerprinting (GIFP) method as well as hierarchical beam search as a non-CI-based approach. Moreover, compared to the GIFP method, the proposed deep learning-based beam selection shows higher robustness to different line-of-sight blockage probability in the training and test datasets and lower sensitivity to inaccuracies in the position and orientation information.
SAR-Net: A Scenario-Aware Ranking Network for Personalized Fair Recommendation in Hundreds of Travel Scenarios

The travel marketing platform of Alibaba serves an indispensable role for hundreds of different travel scenarios from Fliggy, Taobao, Alipay apps, etc. To provide personalized recommendation service for users visiting different scenarios, there are two critical issues to be carefully addressed. First, since the traffic characteristics of different scenarios, e.g., individual data scale or representative topic, are significantly different, it is very challenging to train a unified model to serve all. Second, during the promotion period, the exposure of some specific items will be re-weighted due to manual intervention, resulting in biased logs, which will degrade the ranking model trained using these biased data. In this paper, we propose a novel Scenario-Aware Ranking Network (SAR-Net) to address these issues. SAR-Net harvests the abundant data from different scenarios by learning users' cross-scenario interests via two specific attention modules, which leverage the scenario features and item features to modulate the user behavior features, respectively. Then, taking the encoded features of previous module as input, a scenario-specific linear transformation layer is adopted to further extract scenario-specific features, followed by two groups of debias expert networks, i.e., scenario-specific experts and scenario-shared experts. They output intermediate results independently, which are further fused into the final result by a multi-scenario gating module. In addition, to mitigate the data fairness issue caused by manual intervention, we propose the concept of Fairness Coefficient (FC) to measures the importance of individual sample and use it to reweigh the prediction in the debias expert networks. Experiments on an offline dataset covering over 80 million users and 1.55 million travel items and an online A/B test demonstrate the effectiveness of our SAR-Net and its superiority over state-of-the-art methods. SAR-Net has also been deployed in the online travel marketing platform of Alibaba and is serving hundreds of travel scenarios.
U SER E XPERIENCES O RIENTED S IGHTSEEING S POT R ECOMMENDATION

POI recommendation is a key task in tourism information systems. However, in contrast to conventional point of interest (POI) recommender systems, the available data is extremely sparse; most tourist visit a few sightseeing spots once and most of these spots have no check-in data from new tourists. Most conventional systems rank sightseeing spots based on their popularity, reputations, and category-based similarities with users’ preferences. They do not clarify what users can experience in these spots, which makes it difﬁcult to meet diverse tourism needs. To this end, in this work, we propose a mechanism to recommend POIs to tourists. Our mechanism include two components: one is a probabilistic model that reveals the user behaviors in tourism; the other is a pseudo rating mechanism to handle the cold-start issue in POIs recommendations. We carried out extensive experiments with two datasets collected from Flickr. The experimental results demonstrate that our methods are superior to the state-of-the-art methods in both the recommendation performances (precision, recall and F-measure) and fairness. The experimental results also validate the robustness of the proposed methods, i.e., our methods can handle well the issue of data sparsity.
Exploiting Sensor Multiplexing for Covert Channels and Application Fingerprinting on Mobile Devices

Mobile devices often distribute measurements from a single physical sensor to multiple applications using software-based multiplexing. On Android devices, the highest requested sampling frequency is returned to all applications even if other applications request measurements at lower frequencies. In this paper, we demonstrate that this design choice exposes practically exploitable side-channels based on frequency-key shifting. By carefully modulating sensor sampling frequencies in software, we show that unprivileged malicious applications can construct reliable spectral covert channels that bypass existing security mechanisms, e.g. Android’s permissions framework. Moreover, we present a variant of this technique that allows an unprivileged malicious observer app to ﬁngerprint other device applications and user interactions at a coarse-grained level. Both techniques do not impose any assumptions beyond application installation and accessing standard mobile services via the Android Sensors SDK. As such, they open a powerful attack vector that exploits subtle yet insecure design choices in mobile sensor stacks.
Joint Functional Gaussian Graphical Models

Functional graphical models explore dependence relationships of random processes. This is achieved through estimating the precision matrix of the coefficients from the Karhunen-Loeve expansion. This paper deals with the problem of estimating functional graphs that consist of the same random processes and share some of the dependence structure. By estimating a single graph we would be shrouding the uniqueness of different sub groups within the data. By estimating a different graph for each sub group we would be dividing our sample size. Instead, we propose a method that allows joint estimation of the graphs 1 ar X iv :2 11 0. 06 65 3v 1 [ st at . M E ] 1 3 O ct 2 02 1 while taking into account the intrinsic differences of each sub group. This is achieved by a hierarchical penalty that first penalizes on a common level and then on an individual level. We develop a computation method for our estimator that deals with the non-convex nature of the objective function. We compare the performance of our method with existing ones on a number of different simulated scenarios. We apply our method to an EEG data set that consists of an alcoholic and a non-alcoholic group, to construct brain networks.
As Easy as ABC: Adaptive Binning Coincidence Test for Uniformity Testing

We consider the problem of uniformity testing of Lipschitz continuous distributions with bounded support. The alternative hypothesis is a composite set of Lipschitz continuous distributions that are at least $\varepsilon$ away in $\ell_1$ distance from the uniform distribution. We propose a sequential test that adapts to the unknown distribution under the alternative hypothesis. Referred to as the Adaptive Binning Coincidence (ABC) test, the proposed strategy adapts in two ways. First, it partitions the set of alternative distributions into layers based on their distances to the uniform distribution. It then sequentially eliminates the alternative distributions layer by layer in decreasing distance to the uniform, and subsequently takes advantage of favorable situations of a distant alternative by exiting early. Second, it adapts, across layers of the alternative distributions, the resolution level of the discretization for computing the coincidence statistic. The farther away the layer is from the uniform, the coarser the discretization is needed for eliminating/exiting this layer. It thus exits both early in the detection process and quickly by using a lower resolution to take advantage of favorable alternative distributions. The ABC test builds on a novel sequential coincidence test for discrete distributions, which is of independent interest. We establish the sample complexity of the proposed tests as well as a lower bound.
DeepVecFont

Automatic font generation based on deep learning has aroused a lot of interest in the last decade. However, only a few recently-reported approaches are capable of directly generating vector glyphs and their results are still far from satisfactory. In this paper, we propose a novel method, DeepVecFont, to effectively resolve this problem. Using our method, for the first time, visually-pleasing vector glyphs whose quality and compactness are both comparable to human-designed ones can be automatically generated. The key idea of our DeepVecFont is to adopt the techniques of image synthesis, sequence modeling and differentiable rasterization to exhaustively exploit the dual-modality information (i.e., raster images and vector outlines) of vector fonts. The highlights of this paper are threefold. First, we design a dual-modality learning strategy which utilizes both image-aspect and sequence-aspect features of fonts to synthesize vector glyphs. Second, we provide a new generative paradigm to handle unstructured data (e.g., vector glyphs) by randomly sampling plausible synthesis results to get the optimal one which is further refined under the guidance of generated structured data (e.g., glyph images). Finally, qualitative and quantitative experiments conducted on a publicly-available dataset demonstrate that our method obtains high-quality synthesis results in the applications of vector font generation and interpolation, significantly outperforming the state of the art.
Online network change point detection with missing values and temporal dependence

In this paper we study online change point detection in dynamic networks with time heterogeneous missing pattern within networks and dependence across the time course. The missingness probabilities, the entrywise sparsity of networks, the rank of networks and the jump size in terms of the Frobenius norm, are all allowed to vary as functions of the pre-change sample size. On top of a thorough handling of all the model parameters, we notably allow the edges and missingness to be dependent. To the best of our knowledge, such general framework has not been rigorously nor systematically studied before in the literature. We propose a polynomial time change point detection algorithm, with a version of soft-impute algorithm (e.g. Mazumder et al., 2010; Klopp, 2015) as the imputation sub-routine. Piecing up these standard sub-routines algorithms, we are able to solve a brand new problem with sharp detection delay subject to an overall Type-I error control. Extensive numerical experiments are conducted demonstrating the outstanding performances of our proposed method in practice.