OpenRaiser/Intern-Atlas huggingface dataset:
Social Network Mining (SNM): A Definition of Relation between the Resources and SNA
Venue: arXiv cs.SI 2022
Year: 2022

Social Network Mining (SNM) has become one of the main themes in big data agenda. As a resultant network, we can extract social network from different sources of information, but the information sources were growing dynamically require a flexible approach. To determine the appropriate approach needs the data engineering in order to get the behavior associated with the data. Each social network has the resources and the information source, but the relationship between resources and information sources requires explanation. This paper aimed to address the behavior of the resource as a part of social network analysis (SNA) in the growth of social networks by using the statistical calculations to explain the evolutionary mechanisms. To represent the analysis unit of the SNA, this paper only considers the degree of a vertex, where it is the core of all the analysis in the SNA and it is basic for defining the relation between resources and SNA in SNM. There is a strong effect on the growth of the resources of social networks. In total, the behavior of resources has positive effects. Thus, different information sources behave similarly and have relations with SNA.
Self-Stabilizing Clock Synchronization in Dynamic Networks
Venue: arXiv cs.DC 2022
Year: 2022

We consider the fundamental problem of clock synchronization in a synchronous multi-agent system. Each agent holds a clock with an arbitrary initial value, and clocks must eventually indicate the same value. Previous algorithms worked in static networks with drastic connectivity properties and assumed that global information is available at each agent. In this paper, we propose different solutions for time-varying topologies that require neither strong connectivity nor any global knowledge on the network. First, we study the case of unbounded clocks, and propose a self-stabilizing $MinMax$ algorithm that works if, in each sufficiently long but bounded period of time, there is an agent, called a root, that can send messages, possibly indirectly, to all other agents. Such networks are highly dynamic in the sense that roots may change arbitrarily over time. Moreover, the bound on the time required for achieving this rootedness property is unknown to the agents. Then we present a finite-state algorithm that synchronizes periodic clocks in dynamic networks that are strongly connected over bounded period of time. Here also, the bound on the time for achieving strong connectivity exists, but is not supposed to be known. Interestingly, our algorithm unifies several seemingly different algorithms proposed previously for static networks. Next, we show that strong connectivity is actually not required: our algorithm still works when the network is just rooted over bounded period of time with a set of roots that becomes stable. Finally, we study the time and space complexities of our algorithms, and discuss how initial timing information allows for more efficient solutions.
Towards Knowledge-based Mining of Mental Disorder Patterns from Textual Data
Venue: arXiv cs.IR 2022
Year: 2022

Mental health disorders may cause severe consequences on all the countries' economies and health. For example, the impacts of the COVID-19 pandemic, such as isolation and travel ban, can make us feel depressed. Identifying early signs of mental health disorders is vital. For example, depression may increase an individual's risk of suicide. The state-of-the-art research in identifying mental disorder patterns from textual data, uses hand-labelled training sets, especially when a domain expert's knowledge is required to analyse various symptoms. This task could be time-consuming and expensive. To address this challenge, in this paper, we study and analyse the various clinical and non-clinical approaches to identifying mental health disorders. We leverage the domain knowledge and expertise in cognitive science to build a domain-specific Knowledge Base (KB) for the mental health disorder concepts and patterns. We present a weaker form of supervision by facilitating the generating of training data from a domain-specific Knowledge Base (KB). We adopt a typical scenario for analysing social media to identify major depressive disorder symptoms from the textual content generated by social users. We use this scenario to evaluate how our knowledge-based approach significantly improves the quality of results.
Task Agnostic Representation Consolidation: a Self-supervised based Continual Learning Approach
Venue: arXiv cs.LG 2022
Year: 2022

Continual learning (CL) over non-stationary data streams remains one of the long-standing challenges in deep neural networks (DNNs) as they are prone to catastrophic forgetting. CL models can benefit from self-supervised pre-training as it enables learning more generalizable task-agnostic features. However, the effect of self-supervised pre-training diminishes as the length of task sequences increases. Furthermore, the domain shift between pre-training data distribution and the task distribution reduces the generalizability of the learned representations. To address these limitations, we propose Task Agnostic Representation Consolidation (TARC), a two-stage training paradigm for CL that intertwines task-agnostic and task-specific learning whereby self-supervised training is followed by supervised learning for each task. To further restrict the deviation from the learned representations in the self-supervised stage, we employ a task-agnostic auxiliary loss during the supervised stage. We show that our training paradigm can be easily added to memory- or regularization-based approaches and provides consistent performance gain across more challenging CL settings. We further show that it leads to more robust and well-calibrated models.
Secure Linear MDS Coded Matrix Inversion
Venue: arXiv cs.IT 2022
Year: 2022

A cumbersome operation in many scientific fields, is inverting large full-rank matrices. In this paper, we propose a coded computing approach for recovering matrix inverse approximations. We first present an approximate matrix inversion algorithm which does not require a matrix factorization, but uses a black-box least squares optimization solver as a subroutine, to give an estimate of the inverse of a real full-rank matrix. We then present a distributed framework for which our algorithm can be implemented, and show how we can leverage sparsest-balanced MDS generator matrices to devise matrix inversion coded computing schemes. We focus on balanced Reed-Solomon codes, which are optimal in terms of computational load; and communication from the workers to the master server. We also discuss how our algorithms can be used to compute the pseudoinverse of a full-rank matrix, and how the communication is secured from eavesdroppers.
Social media sharing by political elites: An asymmetric American exceptionalism
Venue: arXiv cs.CY 2022
Year: 2022

Increased sharing of untrustworthy information on social media platforms is one of the main challenges of our modern information society. Because information disseminated by political elites is known to shape citizen and media discourse, it is particularly important to examine the quality of information shared by politicians. Here we show that from 2016 onward, members of the Republican party in the U.S. Congress have been increasingly sharing links to untrustworthy sources. The proportion of untrustworthy information posted by Republicans versus Democrats is diverging at an accelerating rate, and this divergence has worsened since president Biden was elected. This divergence between parties seems to be unique to the U.S. as it cannot be observed in other western democracies such as Germany and the United Kingdom, where left-right disparities are smaller and have remained largely constant.
N-Grammer: Augmenting Transformers with latent n-grams
Venue: arXiv cs.CL 2022
Year: 2022

Transformer models have recently emerged as one of the foundational models in natural language processing, and as a byproduct, there is significant recent interest and investment in scaling these models. However, the training and inference costs of these large Transformer language models are prohibitive, thus necessitating more research in identifying more efficient variants. In this work, we propose a simple yet effective modification to the Transformer architecture inspired by the literature in statistical language modeling, by augmenting the model with n-grams that are constructed from a discrete latent representation of the text sequence. We evaluate our model, the N-Grammer on language modeling on the C4 data-set as well as text classification on the SuperGLUE data-set, and find that it outperforms several strong baselines such as the Transformer and the Primer. We open-source our model for reproducibility purposes in Jax.
SmartPubSub: Content-based Pub-Sub on IPFS
Venue: arXiv cs.NI 2022
Year: 2022

The InterPlanetary File System (IPFS) is a hypermedia distribution protocol enabling the creation of completely distributed applications. One of the most efficient and effective ways to distribute information is through notifications, with a producer of content (publisher) sharing content with other interested parts (subscribers). IPFS already implements topic-based publish-subscribe systems under an experimental flag. The goal of this work is to advance on that, by developing a content-based pub-sub system (with subscriptions as predicates about event content) to disseminate information on top of IPFS in an efficient and decentralized way, leveraging its infrastructure. We design two protocols: ScoutSubs that is completely decentralized; FastDelivery that is centered in the publisher. With these two approaches, we show the different advantages of having each of these protocols simultaneously by comparing ScoutSubs full decentralization, and FastDelivery centralization at data sources.
MDEAW: A Multimodal Dataset for Emotion Analysis through EDA and PPG signals from wireless wearable low-cost off-the-shelf Devices
Venue: arXiv cs.HC 2022
Year: 2022

We present MDEAW, a multimodal database consisting of Electrodermal Activity (EDA) and Photoplethysmography (PPG) signals recorded during the exams for the course taught by the teacher at Eurecat Academy, Sabadell, Barcelona in order to elicit the emotional reactions to the students in a classroom scenario. Signals from 10 students were recorded along with the students' self-assessment of their affective state after each stimulus, in terms of 6 basic emotion states. All the signals were captured using portable, wearable, wireless, low-cost, and off-the-shelf equipment that has the potential to allow the use of affective computing methods in everyday applications. A baseline for student-wise affect recognition using EDA and PPG-based features, as well as their fusion, was established through ReMECS, Fed-ReMECS, and Fed-ReMECS-U. These results indicate the prospects of using low-cost devices for affective state recognition applications. The proposed database will be made publicly available in order to allow researchers to achieve a more thorough evaluation of the suitability of these capturing devices for emotion state recognition applications.
The Free Energy Principle for Perception and Action: A Deep Learning Perspective
Venue: arXiv cs.LG 2022
Year: 2022

The free energy principle, and its corollary active inference, constitute a bio-inspired theory that assumes biological agents act to remain in a restricted set of preferred states of the world, i.e., they minimize their free energy. Under this principle, biological agents learn a generative model of the world and plan actions in the future that will maintain the agent in an homeostatic state that satisfies its preferences. This framework lends itself to being realized in silico, as it comprehends important aspects that make it computationally affordable, such as variational inference and amortized planning. In this work, we investigate the tool of deep learning to design and realize artificial agents based on active inference, presenting a deep-learning oriented presentation of the free energy principle, surveying works that are relevant in both machine learning and active inference areas, and discussing the design choices that are involved in the implementation process. This manuscript probes newer perspectives for the active inference framework, grounding its theoretical aspects into more pragmatic affairs, offering a practical guide to active inference newcomers and a starting point for deep learning practitioners that would like to investigate implementations of the free energy principle.
Deep Learning Discovery of Demographic Biomarkers in Echocardiography
Venue: arXiv cs.LG 2022
Year: 2022

Deep learning has been shown to accurately assess 'hidden' phenotypes and predict biomarkers from medical imaging beyond traditional clinician interpretation of medical imaging. Given the black box nature of artificial intelligence (AI) models, caution should be exercised in applying models to healthcare as prediction tasks might be short-cut by differences in demographics across disease and patient populations. Using large echocardiography datasets from two healthcare systems, we test whether it is possible to predict age, race, and sex from cardiac ultrasound images using deep learning algorithms and assess the impact of varying confounding variables. We trained video-based convolutional neural networks to predict age, sex, and race. We found that deep learning models were able to identify age and sex, while unable to reliably predict race. Without considering confounding differences between categories, the AI model predicted sex with an AUC of 0.85 (95% CI 0.84 - 0.86), age with a mean absolute error of 9.12 years (95% CI 9.00 - 9.25), and race with AUCs ranging from 0.63 - 0.71. When predicting race, we show that tuning the proportion of a confounding variable (sex) in the training data significantly impacts model AUC (ranging from 0.57 to 0.84), while in training a sex prediction model, tuning a confounder (race) did not substantially change AUC (0.81 - 0.83). This suggests a significant proportion of the model's performance on predicting race could come from confounding features being detected by AI. Further work remains to identify the particular imaging features that associate with demographic information and to better understand the risks of demographic identification in medical AI as it pertains to potentially perpetuating bias and disparities.
Gridless Channel Estimation for MmWave Hybrid Massive MIMO Systems with Low-Resolution ADCs
Venue: arXiv cs.IT 2022
Year: 2022

This paper proposes the Newtonized fully corrective forward greedy selection-cross validation-based (NFCFGS-CV-based) channel estimator for millimeter (mmWave) hybrid massive multiple-input multiple-output (MIMO) systems with low-resolution analog-to-digital converters (ADCs). The proposed NFCFGS algorithm is a gridless compressed sensing (CS) technique that combines the FCFGS and Newtonized orthogonal matching pursuit (NOMP) algorithms. In particular, NFCFGS performs single path estimation over the continuum at each iteration based on the previously estimated paths. The CV technique is adopted as an indicator of termination in the absence of the prior knowledge on the number of paths, which is a model validation technique that prevents overfitting.
A Simple Adaptive Procedure Converging to Forgiving Correlated Equilibria
Venue: arXiv cs.GT 2022
Year: 2022

Simple adaptive procedures that converge to correlated equilibria are known to exist for normal form games (Hart and Mas-Colell 2000), but no such analogue exists for extensive-form games. Leveraging inspiration from Zinkevich et al. (2008), we show that any internal regret minimization procedure designed for normal-form games can be efficiently extended to finite extensive-form games of perfect recall. Our procedure converges to the set of forgiving correlated equilibria, a refinement of various other proposed extensions of the correlated equilibrium solution concept to extensive-form games (Forges 1986a; Forges 1986b; von Stengel and Forges 2008). In a forgiving correlated equilibrium, players receive move recommendations only upon reaching the relevant information set instead of all at once at the beginning of the game. Assuming all other players follow their recommendations, each player is incentivized to follow her recommendations regardless of whether she has done so at previous infosets. The resulting procedure is completely decentralized: players need neither knowledge of their opponents' actions nor even a complete understanding of the game itself beyond their own payoffs and strategies.
Near-Optimal Bounds for Testing Histogram Distributions
Venue: arXiv cs.DS 2022
Year: 2022

We investigate the problem of testing whether a discrete probability distribution over an ordered domain is a histogram on a specified number of bins. One of the most common tools for the succinct approximation of data, $k$-histograms over $[n]$, are probability distributions that are piecewise constant over a set of $k$ intervals. The histogram testing problem is the following: Given samples from an unknown distribution $\mathbf{p}$ on $[n]$, we want to distinguish between the cases that $\mathbf{p}$ is a $k$-histogram versus $\varepsilon$-far from any $k$-histogram, in total variation distance. Our main result is a sample near-optimal and computationally efficient algorithm for this testing problem, and a nearly-matching (within logarithmic factors) sample complexity lower bound. Specifically, we show that the histogram testing problem has sample complexity $\widetilde Θ(\sqrt{nk} / \varepsilon + k / \varepsilon^2 + \sqrt{n} / \varepsilon^2)$.
Rethinking Super-Resolution as Text-Guided Details Generation
Venue: arXiv cs.CV 2022
Year: 2022

Deep neural networks have greatly promoted the performance of single image super-resolution (SISR). Conventional methods still resort to restoring the single high-resolution (HR) solution only based on the input of image modality. However, the image-level information is insufficient to predict adequate details and photo-realistic visual quality facing large upscaling factors (x8, x16). In this paper, we propose a new perspective that regards the SISR as a semantic image detail enhancement problem to generate semantically reasonable HR image that are faithful to the ground truth. To enhance the semantic accuracy and the visual quality of the reconstructed image, we explore the multi-modal fusion learning in SISR by proposing a Text-Guided Super-Resolution (TGSR) framework, which can effectively utilize the information from the text and image modalities. Different from existing methods, the proposed TGSR could generate HR image details that match the text descriptions through a coarse-to-fine process. Extensive experiments and ablation studies demonstrate the effect of the TGSR, which exploits the text reference to recover realistic images.
Deepfake Video Detection with Spatiotemporal Dropout Transformer
Venue: arXiv cs.CV 2022
Year: 2022

While the abuse of deepfake technology has caused serious concerns recently, how to detect deepfake videos is still a challenge due to the high photo-realistic synthesis of each frame. Existing image-level approaches often focus on single frame and ignore the spatiotemporal cues hidden in deepfake videos, resulting in poor generalization and robustness. The key of a video-level detector is to fully exploit the spatiotemporal inconsistency distributed in local facial regions across different frames in deepfake videos. Inspired by that, this paper proposes a simple yet effective patch-level approach to facilitate deepfake video detection via spatiotemporal dropout transformer. The approach reorganizes each input video into bag of patches that is then fed into a vision transformer to achieve robust representation. Specifically, a spatiotemporal dropout operation is proposed to fully explore patch-level spatiotemporal cues and serve as effective data augmentation to further enhance model's robustness and generalization ability. The operation is flexible and can be easily plugged into existing vision transformers. Extensive experiments demonstrate the effectiveness of our approach against 25 state-of-the-arts with impressive robustness, generalizability, and representation ability.
Source-Free Domain Adaptation for Real-world Image Dehazing
Venue: arXiv cs.CV 2022
Year: 2022

Deep learning-based source dehazing methods trained on synthetic datasets have achieved remarkable performance but suffer from dramatic performance degradation on real hazy images due to domain shift. Although certain Domain Adaptation (DA) dehazing methods have been presented, they inevitably require access to the source dataset to reduce the gap between the source synthetic and target real domains. To address these issues, we present a novel Source-Free Unsupervised Domain Adaptation (SFUDA) image dehazing paradigm, in which only a well-trained source model and an unlabeled target real hazy dataset are available. Specifically, we devise the Domain Representation Normalization (DRN) module to make the representation of real hazy domain features match that of the synthetic domain to bridge the gaps. With our plug-and-play DRN module, unlabeled real hazy images can adapt existing well-trained source networks. Besides, the unsupervised losses are applied to guide the learning of the DRN module, which consists of frequency losses and physical prior losses. Frequency losses provide structure and style constraints, while the prior loss explores the inherent statistic property of haze-free images. Equipped with our DRN module and unsupervised loss, existing source dehazing models are able to dehaze unlabeled real hazy images. Extensive experiments on multiple baselines demonstrate the validity and superiority of our method visually and quantitatively.
PIAT: Physics Informed Adversarial Training for Solving Partial Differential Equations
Venue: arXiv cs.LG 2022
Year: 2022

In this paper, we propose the physics informed adversarial training (PIAT) of neural networks for solving nonlinear differential equations (NDE). It is well-known that the standard training of neural networks results in non-smooth functions. Adversarial training (AT) is an established defense mechanism against adversarial attacks, which could also help in making the solution smooth. AT include augmenting the training mini-batch with a perturbation that makes the network output mismatch the desired output adversarially. Unlike formal AT, which relies only on the training data, here we encode the governing physical laws in the form of nonlinear differential equations using automatic differentiation in the adversarial network architecture. We compare PIAT with PINN to indicate the effectiveness of our method in solving NDEs for up to 10 dimensions. Moreover, we propose weight decay and Gaussian smoothing to demonstrate the PIAT advantages. The code repository is available at https://github.com/rohban-lab/PIAT.