OpenRaiser/Intern-Atlas huggingface dataset:
Identifying Diversity, Equity, Inclusion, and Accessibility (DEIA) Indicators for Transportation Systems using Social Media Data: The Case of New York City during Covid-19 Pandemic
Venue: arXiv cs.SI 2022
Year: 2022

The adoption of transportation policies that prioritized highway expansion over public transportation has disproportionately impacted minorities and low-income people by restricting their access to social and economic opportunities and thus resulting in residential segregation. Policymakers, transportation researchers, planners, and practitioners have started acknowledging the need to build a diverse, equitable, inclusive, and accessible (DEIA) transportation system. Traditionally, this has been done through survey-based approaches that are time-consuming and expensive. While there is recent attention on leveraging social media data in transportation, the literature is inconclusive regarding the use of social media data as a viable alternative to traditional sources to identify the latent DEIA indicators based on public reactions and perspectives on social media. This study utilized large-scale Twitter data covering eight counties around the New York City (NYC) area during the initial phase of the Covid-19 lockdown to address this research gap. Natural language processing techniques were used to identify transportation-related major DEIA issues for residents living around NYC by analyzing their relevant tweet conversations. The study revealed that citizens, who had negative sentiments toward the DEIA of their local transportation system, broadly discussed racism, income, unemployment, gender, ride dependency, transportation modes, and dependent groups. Analyzing the socio-demographic information based on census tracts, the study also observed that areas with a higher percentage of low-income, female, Hispanic, and Latino populations share more concerns about transportation DEIA on Twitter.
On the Role of Risk Perceptions in Cyber Insurance Contracts
Venue: arXiv cs.CR 2022
Year: 2022

Risk perceptions are essential in cyber insurance contracts. With the recent surge of information, human risk perceptions are exposed to the influences from both beneficial knowledge and fake news. In this paper, we study the role of the risk perceptions of the insurer and the user in cyber insurance contracts. We formulate the cyber insurance problem into a principal-agent problem where the insurer designs the contract containing a premium payment and a coverage plan. The risk perceptions of the insurer and the user are captured by coherent risk measures. Our framework extends the cyber insurance problem containing a risk-neutral insurer and a possibly risk-averse user, which is often considered in the literature. The explicit characterizations of both the insurer's and the user's risk perceptions allow us to show that cyber insurance has the potential to incentivize the user to invest more on system protection. This possibility to increase cyber security relies on the facts that the insurer is more risk-averse than the user (in a minimization setting) and that the insurer's risk perception is more sensitive to the changes in the user's actions than the user himself. We investigate the properties of feasible contracts in a case study on the insurance of a computer system against ransomware.
Counting Perfect Matchings in Dense Graphs Is Hard
Venue: arXiv cs.DS 2022
Year: 2022

We show that the problem of counting perfect matchings remains #P-complete even if we restrict the input to very dense graphs, proving the conjecture in [5]. Here "dense graphs" refer to bipartite graphs of bipartite independence number $\leq 2$, or general graphs of independence number $\leq 2$. Our proof is by reduction from counting perfect matchings in bipartite graphs, via elementary linear algebra tricks and graph constructions.
A novel information gain-based approach for classification and dimensionality reduction of hyperspectral images
Venue: arXiv cs.CV 2022
Year: 2022

Recently, the hyperspectral sensors have improved our ability to monitor the earth surface with high spectral resolution. However, the high dimensionality of spectral data brings challenges for the image processing. Consequently, the dimensionality reduction is a necessary step in order to reduce the computational complexity and increase the classification accuracy. In this paper, we propose a new filter approach based on information gain for dimensionality reduction and classification of hyperspectral images. A special strategy based on hyperspectral bands selection is adopted to pick the most informative bands and discard the irrelevant and noisy ones. The algorithm evaluates the relevancy of the bands based on the information gain function with the support vector machine classifier. The proposed method is compared using two benchmark hyperspectral datasets (Indiana, Pavia) with three competing methods. The comparison results showed that the information gain filter approach outperforms the other methods on the tested datasets and could significantly reduce the computation cost while improving the classification accuracy. Keywords: Hyperspectral images; dimensionality reduction; information gain; classification accuracy. Keywords: Hyperspectral images; dimensionality reduction; information gain; classification accuracy.
DyREx: Dynamic Query Representation for Extractive Question Answering
Venue: arXiv cs.CL 2022
Year: 2022

Extractive question answering (ExQA) is an essential task for Natural Language Processing. The dominant approach to ExQA is one that represents the input sequence tokens (question and passage) with a pre-trained transformer, then uses two learned query vectors to compute distributions over the start and end answer span positions. These query vectors lack the context of the inputs, which can be a bottleneck for the model performance. To address this problem, we propose \textit{DyREx}, a generalization of the \textit{vanilla} approach where we dynamically compute query vectors given the input, using an attention mechanism through transformer layers. Empirical observations demonstrate that our approach consistently improves the performance over the standard one. The code and accompanying files for running the experiments are available at \url{https://github.com/urchade/DyReX}.
Masked Vision-Language Transformer in Fashion
Venue: arXiv cs.CV 2022
Year: 2022

We present a masked vision-language transformer (MVLT) for fashion-specific multi-modal representation. Technically, we simply utilize vision transformer architecture for replacing the BERT in the pre-training model, making MVLT the first end-to-end framework for the fashion domain. Besides, we designed masked image reconstruction (MIR) for a fine-grained understanding of fashion. MVLT is an extensible and convenient architecture that admits raw multi-modal inputs without extra pre-processing models (e.g., ResNet), implicitly modeling the vision-language alignments. More importantly, MVLT can easily generalize to various matching and generative tasks. Experimental results show obvious improvements in retrieval (rank@5: 17%) and recognition (accuracy: 3%) tasks over the Fashion-Gen 2018 winner Kaleido-BERT. Code is made available at https://github.com/GewelsJI/MVLT.
MEET: Mobility-Enhanced Edge inTelligence for Smart and Green 6G Networks
Venue: arXiv cs.NI 2022
Year: 2022

Edge intelligence is an emerging paradigm for real-time training and inference at the wireless edge, thus enabling mission-critical applications. Accordingly, base stations (BSs) and edge servers (ESs) need to be densely deployed, leading to huge deployment and operation costs, in particular the energy costs. In this article, we propose a new framework called Mobility-Enhanced Edge inTelligence (MEET), which exploits the sensing, communication, computing, and self-powering capabilities of intelligent connected vehicles for the smart and green 6G networks. Specifically, the operators can incorporate infrastructural vehicles as movable BSs or ESs, and schedule them in a more flexible way to align with the communication and computation traffic fluctuations. Meanwhile, the remaining compute resources of opportunistic vehicles are exploited for edge training and inference, where mobility can further enhance edge intelligence by bringing more compute resources, communication opportunities, and diverse data. In this way, the deployment and operation costs are spread over the vastly available vehicles, so that the edge intelligence is realized cost-effectively and sustainably. Furthermore, these vehicles can be either powered by renewable energy to reduce carbon emissions, or charged more flexibly during off-peak hours to cut electricity bills.
A Unified Blockchain-Semantic Framework for Wireless Edge Intelligence Enabled Web 3.0
Venue: arXiv cs.CR 2022
Year: 2022

Web 3.0 enables user-generated contents and user-selected authorities. With decentralized wireless edge computing architectures, Web 3.0 allows users to read, write, and own contents. A core technology that enables Web 3.0 goals is blockchain, which provides security services by recording content in a decentralized and transparent manner. However, the explosion of on-chain recorded contents and the fast-growing number of users cause increasingly unaffordable computing and storage resource consumption. A promising paradigm is to analyze the semantic information of contents that can convey precisely the desired meanings without consuming many resources. In this article, we propose a unified blockchain-semantic ecosystems framework for wireless edge intelligence-enabled Web 3.0. Our framework consists of six key components to exchange semantic demands. We then introduce an Oracle-based proof of semantic mechanism to implement on-chain and off-chain interactions of Web 3.0 ecosystems on semantic verification algorithms while maintaining service security. An adaptive Deep Reinforcement Learning-based sharding mechanism on Oracle is designed to improve interaction efficiency, which can facilitate Web 3.0 ecosystems to deal with varied semantic demands. Finally, a case study is presented to show that the proposed framework can dynamically adjust Oracle settings according to varied semantic demands.
Towards High-Quality Neural TTS for Low-Resource Languages by Learning Compact Speech Representations
Venue: arXiv cs.SD 2022
Year: 2022

This paper aims to enhance low-resource TTS by reducing training data requirements using compact speech representations. A Multi-Stage Multi-Codebook (MSMC) VQ-GAN is trained to learn the representation, MSMCR, and decode it to waveforms. Subsequently, we train the multi-stage predictor to predict MSMCRs from the text for TTS synthesis. Moreover, we optimize the training strategy by leveraging more audio to learn MSMCRs better for low-resource languages. It selects audio from other languages using speaker similarity metric to augment the training set, and applies transfer learning to improve training quality. In MOS tests, the proposed system significantly outperforms FastSpeech and VITS in standard and low-resource scenarios, showing lower data requirements. The proposed training strategy effectively enhances MSMCRs on waveform reconstruction. It improves TTS performance further, which wins 77% votes in the preference test for the low-resource TTS with only 15 minutes of paired data.
AutoAttention: Automatic Field Pair Selection for Attention in User Behavior Modeling
Venue: arXiv cs.IR 2022
Year: 2022

In Click-through rate (CTR) prediction models, a user's interest is usually represented as a fixed-length vector based on her history behaviors. Recently, several methods are proposed to learn an attentive weight for each user behavior and conduct weighted sum pooling. However, these methods only manually select several fields from the target item side as the query to interact with the behaviors, neglecting the other target item fields, as well as user and context fields. Directly including all these fields in the attention may introduce noise and deteriorate the performance. In this paper, we propose a novel model named AutoAttention, which includes all item/user/context side fields as the query, and assigns a learnable weight for each field pair between behavior fields and query fields. Pruning on these field pairs via these learnable weights lead to automatic field pair selection, so as to identify and remove noisy field pairs. Though including more fields, the computation cost of AutoAttention is still low due to using a simple attention function and field pair selection. Extensive experiments on the public dataset and Tencent's production dataset demonstrate the effectiveness of the proposed approach.
Global-to-local Expression-aware Embeddings for Facial Action Unit Detection
Venue: arXiv cs.CV 2022
Year: 2022

Expressions and facial action units (AUs) are two levels of facial behavior descriptors. Expression auxiliary information has been widely used to improve the AU detection performance. However, most existing expression representations can only describe pre-determined discrete categories (e.g., Angry, Disgust, Happy, Sad, etc.) and cannot capture subtle expression transformations like AUs. In this paper, we propose a novel fine-grained \textsl{Global Expression representation Encoder} to capture subtle and continuous facial movements, to promote AU detection. To obtain such a global expression representation, we propose to train an expression embedding model on a large-scale expression dataset according to global expression similarity. Moreover, considering the local definition of AUs, it is essential to extract local AU features. Therefore, we design a \textsl{Local AU Features Module} to generate local facial features for each AU. Specifically, it consists of an AU feature map extractor and a corresponding AU mask extractor. First, the two extractors transform the global expression representation into AU feature maps and masks, respectively. Then, AU feature maps and their corresponding AU masks are multiplied to generate AU masked features focusing on local facial region. Finally, the AU masked features are fed into an AU classifier for judging the AU occurrence. Extensive experiment results demonstrate the superiority of our proposed method. Our method validly outperforms previous works and achieves state-of-the-art performances on widely-used face datasets, including BP4D, DISFA, and BP4D+.
Improved Projection Learning for Lower Dimensional Feature Maps
Venue: arXiv cs.LG 2022
Year: 2022

The requirement to repeatedly move large feature maps off- and on-chip during inference with convolutional neural networks (CNNs) imposes high costs in terms of both energy and time. In this work we explore an improved method for compressing all feature maps of pre-trained CNNs to below a specified limit. This is done by means of learned projections trained via end-to-end finetuning, which can then be folded and fused into the pre-trained network. We also introduce a new `ceiling compression' framework in which evaluate such techniques in view of the future goal of performing inference fully on-chip.
Partially Oblivious Neural Network Inference
Venue: arXiv cs.CR 2022
Year: 2022

Oblivious inference is the task of outsourcing a ML model, like neural-networks, without disclosing critical and sensitive information, like the model's parameters. One of the most prominent solutions for secure oblivious inference is based on a powerful cryptographic tools, like Homomorphic Encryption (HE) and/or multi-party computation (MPC). Even though the implementation of oblivious inference systems schemes has impressively improved the last decade, there are still significant limitations on the ML models that they can practically implement. Especially when both the ML model and the input data's confidentiality must be protected. In this paper, we introduce the notion of partially oblivious inference. We empirically show that for neural network models, like CNNs, some information leakage can be acceptable. We therefore propose a novel trade-off between security and efficiency. In our research, we investigate the impact on security and inference runtime performance from the CNN model's weights partial leakage. We experimentally demonstrate that in a CIFAR-10 network we can leak up to $80\%$ of the model's weights with practically no security impact, while the necessary HE-mutliplications are performed four times faster.
Trust and Believe -- Should We? Evaluating the Trustworthiness of Twitter Users
Venue: arXiv cs.SI 2022
Year: 2022

Social networking and micro-blogging services, such as Twitter, play an important role in sharing digital information. Despite the popularity and usefulness of social media, they are regularly abused by corrupt users. One of these nefarious activities is so-called fake news -- a "virus" that has been spreading rapidly thanks to the hospitable environment provided by social media platforms. The extensive spread of fake news is now becoming a major problem with far-reaching negative repercussions on both individuals and society. Hence, the identification of fake news on social media is a problem of utmost importance that has attracted the interest not only of the research community but most of the big players on both sides - such as Facebook, on the industry side, and political parties on the societal one. In this work, we create a model through which we hope to be able to offer a solution that will instill trust in social network communities. Our model analyses the behaviour of 50,000 politicians on Twitter and assigns an influence score for each evaluated user based on several collected and analysed features and attributes. Next, we classify political Twitter users as either trustworthy or untrustworthy using random forest and support vector machine classifiers. An active learning model has been used to classify any unlabeled ambiguous records from our dataset. Finally, to measure the performance of the proposed model, we used accuracy as the main evaluation metric.
RePAST: A ReRAM-based PIM Accelerator for Second-order Training of DNN
Venue: arXiv cs.AR 2022
Year: 2022

The second-order training methods can converge much faster than first-order optimizers in DNN training. This is because the second-order training utilizes the inversion of the second-order information (SOI) matrix to find a more accurate descent direction and step size. However, the huge SOI matrices bring significant computational and memory overheads in the traditional architectures like GPU and CPU. On the other side, the ReRAM-based process-in-memory (PIM) technology is suitable for the second-order training because of the following three reasons: First, PIM's computation happens in memory, which reduces data movement overheads; Second, ReRAM crossbars can compute SOI's inversion in $O\left(1\right)$ time; Third, if architected properly, ReRAM crossbars can perform matrix inversion and vector-matrix multiplications which are important to the second-order training algorithms. Nevertheless, current ReRAM-based PIM techniques still face a key challenge for accelerating the second-order training. The existing ReRAM-based matrix inversion circuitry can only support 8-bit accuracy matrix inversion and the computational precision is not sufficient for the second-order training that needs at least 16-bit accurate matrix inversion. In this work, we propose a method to achieve high-precision matrix inversion based on a proven 8-bit matrix inversion (INV) circuitry and vector-matrix multiplication (VMM) circuitry. We design \archname{}, a ReRAM-based PIM accelerator architecture for the second-order training. Moreover, we propose a software mapping scheme for \archname{} to further optimize the performance by fusing VMM and INV crossbar. Experiment shows that \archname{} can achieve an average of 115.8$\times$/11.4$\times$ speedup and 41.9$\times$/12.8$\times$energy saving compared to a GPU counterpart and PipeLayer on large-scale DNNs.
Leveraging Computer Vision Application in Visual Arts: A Case Study on the Use of Residual Neural Network to Classify and Analyze Baroque Paintings
Venue: arXiv cs.MM 2022
Year: 2022

With the increasing availability of large digitized fine art collections, automated analysis and classification of paintings is becoming an interesting area of research. However, due to domain specificity, implicit subjectivity, and pervasive nuances that vaguely separate art movements, analyzing art using machine learning techniques poses significant challenges. Residual networks, or variants thereof, are one the most popular tools for image classification tasks, which can extract relevant features for well-defined classes. In this case study, we focus on the classification of a selected painting 'Portrait of the Painter Charles Bruni' by Johann Kupetzky and the analysis of the performance of the proposed classifier. We show that the features extracted during residual network training can be useful for image retrieval within search systems in online art collections.
Noise in the Clouds: Influence of Network Performance Variability on Application Scalability
Venue: arXiv cs.DC 2022
Year: 2022

Cloud computing represents an appealing opportunity for cost-effective deployment of HPC workloads on the best-fitting hardware. However, although cloud and on-premise HPC systems offer similar computational resources, their network architecture and performance may differ significantly. For example, these systems use fundamentally different network transport and routing protocols, which may introduce network noise that can eventually limit the application scaling. This work analyzes network performance, scalability, and cost of running HPC workloads on cloud systems. First, we consider latency, bandwidth, and collective communication patterns in detailed small-scale measurements, and then we simulate network performance at a larger scale. We validate our approach on four popular cloud providers and three on-premise HPC systems, showing that network (and also OS) noise can significantly impact performance and cost both at small and large scale.
Learning Discrete Directed Acyclic Graphs via Backpropagation
Venue: arXiv cs.LG 2022
Year: 2022

Recently continuous relaxations have been proposed in order to learn Directed Acyclic Graphs (DAGs) from data by backpropagation, instead of using combinatorial optimization. However, a number of techniques for fully discrete backpropagation could instead be applied. In this paper, we explore that direction and propose DAG-DB, a framework for learning DAGs by Discrete Backpropagation. Based on the architecture of Implicit Maximum Likelihood Estimation [I-MLE, arXiv:2106.01798], DAG-DB adopts a probabilistic approach to the problem, sampling binary adjacency matrices from an implicit probability distribution. DAG-DB learns a parameter for the distribution from the loss incurred by each sample, performing competitively using either of two fully discrete backpropagation techniques, namely I-MLE and Straight-Through Estimation.