Miracle-dz/UniFAR-FTUs huggingface dataset:
The role of optimization geometry in single neuron learning

Recent numerical experiments have demonstrated that the choice of optimization geometry used during training can impact generalization performance when learning expressive nonlinear model classes such as deep neural networks. These observations have important implications for modern deep learning, but remain poorly understood due to the difficulty of the associated nonconvex optimization. Towards an understanding of this phenomenon, we analyze a family of pseudogradient methods for learning generalized linear models under the square loss – a sim-plified problem containing both nonlinearity in the model parameters and nonconvexity of the optimization which admits a single neuron as a special case. We prove non-asymptotic bounds on the generalization error that sharply characterize how the interplay between the optimization geometry and the feature space geometry sets the out-of-sample performance of the learned model. Experimentally, selecting the optimization geometry as suggested by our theory leads to improved performance in generalized linear model estimation problems such as nonlinear and nonconvex variants of sparse vector recovery and low-rank matrix sensing.
Beyond Data Samples: Aligning Differential Networks Estimation with Scientific Knowledge

Learning the differential statistical dependency network between two contexts is essential for many real-life applications, mostly in the high dimensional low sample regime. In this paper, we propose a novel differential network estimator that allows integrating various sources of knowledge beyond data samples. The proposed estimator is scalable to a large number of variables and achieves a sharp asymptotic convergence rate. Empirical experiments on extensive simulated data and four real-world applications (one on neuroimaging and three from functional genomics) show that our approach achieves improved differential network estimation and provides better supports to downstream tasks like classification. Our results highlight significant benefits of integrating group, spatial and anatomic knowledge during differential genetic network identification and brain connectome change discovery.
"I Can't Keep It Up." A Dataset from the Defunct Voat.co News Aggregator

Voat.co was a news aggregator website that shut down on December 25, 2020. The site had a troubled history and was known for hosting various banned subreddits. This paper presents a dataset with over 2.3M submissions and 16.2M comments posted from 113K users in 7.1K subverses (the equivalent of subreddit for Voat). Our dataset covers the whole lifetime of Voat, from its developing period starting on November 8, 2013, the day it was founded, April 2014, up until the day it shut down (December 25, 2020). This work presents the largest and most complete publicly available Voat dataset, to the best of our knowledge. Along with the release of this dataset, we present a preliminary analysis covering posting activity and daily user and subverse registration on the platform so that researchers interested in our dataset can know what to expect. Our data may prove helpful to false news dissemination studies as we analyze the links users share on the platform, finding that many communities rely on alternative news press, like Breitbart and GatewayPundit, for their daily discussions. In addition, we perform network analysis on user interactions finding that many users prefer not to interact with subverses outside their narrative interests, which could be helpful to researchers focusing on polarization and echo chambers. Also, since Voat was one of the platforms banned Reddit communities migrated to, we are confident our dataset will motivate and assist researchers studying deplatforming. Finally, many hateful and conspiratorial communities were very popular on Voat, which makes our work valuable for researchers focusing on toxicity, conspiracy theories, cross-platform studies of social networks, and natural language processing.
Modeling the Multiband Light Curves of the Afterglows of Three Gamma-Ray Bursts and their Associated Supernovae

Some dozen supernovae (SNe) associated with long gamma-ray bursts (GRBs) have been confirmed. Most of the previous studies derive the physical properties of the GRB-SNe by fitting the constructed (pseudo-)bolometric light curves. However, many GRB-SNe only have a few filter data, for which the (pseudo-)bolometric light curves are very difficult to construct. Additionally, constructing (pseudo-)bolometric light curves rely on some assumptions. In this paper, we use the multiband broken power-law plus 56Ni model to fit the multiband light curves of the afterglows and the SNe (SN 2001ke, SN 2013dx, and SN 2016jca) associated with three GRBs (GRB 011121, GRB 130702A, and GRB 161219B). We find our model can account for the multiband light curves of the three GRB-SNe (except for the late-time z-band light curve of two events), indicating that the model is a reliable model. The 56Ni masses we derive are higher than those in the literature. This might be due to the fact that the 56Ni masses in the literature are usually obtained by fitting the pseudo-bolometric light curves whose luminosities are usually (significantly) underestimated. We suggest that the multiband model can not only be used to fit the multiband light curves of GRB-SNe that have many filter observations, but also fit those having sparse data.
Reinforcement Learning in Practice: Opportunities and Challenges

This article is a gentle discussion about the ﬁeld of reinforcement learning in practice, about opportunities and challenges, touching a broad range of topics, with perspectives and without technical details. The article is based on both historical and recent research papers, surveys, tutorials, talks, blogs, books, (panel) discussions, and workshops/conferences. Various groups of readers, like researchers, engineers, students, managers, investors, ofﬁcers, and people wanting to know more about the ﬁeld, may ﬁnd the article interesting. In this article, we ﬁrst give a brief introduction to reinforcement learning (RL), and its relationship with deep learning, machine learning and AI. Then we discuss opportunities of RL, in particular, products and services, games, bandits, recommender systems, robotics, transportation, ﬁnance and economics, healthcare, education, combinatorial optimization, computer systems, and science and engineering. Then we discuss challenges, in particular, 1) foundation, 2) representation, 3) reward, 4) exploration, 5) model, simulation, planning, and benchmarks, 6) off-policy/ofﬂine learning, 7) learning to learn a.k. a. meta-learning, 8) explainability and interpretability, 9) constraints, 10) software development and deployment, 11) business perspectives, and 12) more challenges. We conclude with a discussion, attempting to answer: “Why has RL not been widely adopted in practice yet? ” and “When is RL helpful? ”. for discussing transfer domain randomization, knowledge distillation, imitation learning, meta-learning, robust RL.
Enhanced Corrosion Detection in Airframe Structures Using Pulsed Eddy Current and Advanced Processing

The capabilities of the pulsed eddy current technique to extract depth related information have been successfully demonstrated for hidden corrosion detection. In the present work, the drive pulse and the transient response are analyzed in terms of the Fourier transform to understand the depth of material penetration with pulsed eddy current. A multifrequency analysis of the transient response that allows information to be selectively chosen from different layers is described. An observation time interval and a sampling rate are the components that define high- and low-frequency resolution of a pulsed eddy current system. An example of "in-depth slicing" of a multilayer structure is provided. The use of solid state sensors to acquire transient response is discussed. Examples of pulsed eddy current imaging with arrays of giant magnetoresistors are given. Recently developed two-dimensional arrays enable instant imaging of the structure under the probe without mechanical scanning. They also provide the fastest surface coverage for large area testing when used with 2D scanners. Experiments have been performed on aircraft aluminum skin with a portable prototype system. The advantages and limitations of the technique are discussed.
Comparative Analysis of Efficiency in the Economic Sectors of Lima Stock Exchange

Lima Stock Exchange is considered one of the smallest capital markets in Latin America, despite its favorable growth in the last five years. The performance of listed companies in the stock market may be compromised with national macroeconomic gaps and impact on the development of the economic sector, which is why this study aimed to estimate the financial efficiency of companies listed on the Lima Stock Exchange to know their performance by economic sector during the period 2015-2020. The non-parametric technique of Data Envelopment Analysis was used in a set of 76 companies belonging to the Agrarian, Industrial, Public Services, and Mining sectors; finally, the change in performance was estimated through the Malmquist Productivity Index. The results indicated that 2016 was the most efficient year for companies and 2018 the least efficient year. The most efficient sector was Mining with an efficiency of 0.56, Agrarian sector was the least efficient and with the highest volatility. Likewise, productivity results concluded that technological change does not contribute to productivity, while efficiency change contributed positively to all sectors. In addition, a trend of annual growth and stability of the Mining sector was evidenced, which, in the face of the economic crisis, only had a slight drop of -1.7% in its productivity, unlike the other sectors that were notably affected. Results of this study reflected that the macroeconomic indicators of the country often don't affect the performance of the economic sector, to know the performance of the companies it is necessary to analyze the characteristic factors of each sector. It is recommended to use the results of this study as a complementary instrument for making investment decisions in Lima Stock Exchange-listed companies.
Off-policy Learning over Heterogeneous Information for Recommendation

Reinforcement learning has recently become an active topic in recommender system research, where the logged data that records interactions between items and users feedback is used to discover the policy. Much off-policy learning, referring to the procedure of policy optimization with access only to logged feedback data, has been a popular research topic in reinforcement learning. However, the log entries are biased in that the logs over-represent actions favored by the recommender system, as the user feedback contains only partial information limited to the particular items exposed to the user. As a result, the policy learned from such off-line logged data tends to be biased from the true behaviour policy. In this paper, we are the first to propose a novel off-policy learning augmented by meta-paths for the recommendation. We argue that the Heterogeneous information network (HIN), which provides rich contextual information of items and user aspects, could scale the logged data contribution for unbiased target policy learning. Towards this end, we develop a new HIN augmented target policy model (HINpolicy), which explicitly leverages contextual information to scale the generated reward for target policy. In addition, being equipped with the HINpolicy model, our solution adaptively receives HIN-augmented corrections for counterfactual risk minimization, and ultimately yields an effective policy to maximize the long run rewards for the recommendation. Finally, we extensively evaluate our method through a series of simulations and large-scale real-world datasets, obtaining favorable results compared with state-of-the-art methods.
Winning Tracker: A New Model for Real-time Winning Prediction in MOBA Games

With an increasing popularity, Multiplayer Online Battle Arena (MOBA) games where two opposing teams compete against each other, have played a major role in E-sports tournaments. Among game analysis, real-time winning prediction is an important but challenging problem, which is mainly due to the complicated coupling of the overall Confrontation1, the excessive noise of the player’s Movement, and unclear optimization goals. Existing research is difficult to solve this problem in a dynamic, comprehensive and systematic way. In this study, we design a unified framework, namely Winning Tracker (WT), for solving this problem. Specifically, offense and defense extractors are developed to extract the Confrontation of both sides. A well-designed trajectory representation algorithm is applied to extracting individual’s Movement information. Moreover, we design a hierarchical attention mechanism to capture team-level strategies and facilitate the interpretability of the framework. To optimize accurately, we adopt a multi-task learning method to design short-term and long-term goals, which are used to represent immediate state and make end-state prediction respectively. Intensive experiments on a real-world data set demonstrate that our proposed method WT outperforms state-of-the-art algorithms. Furthermore, our work has been practically deployed in real MOBA games, and provided case studies reflecting its outstanding commercial value.
Emotion Bubbles: Emotional Composition of Online Discourse Before and After the COVID-19 Outbreak

The COVID-19 pandemic has been the single most important global agenda in the past two years. In addition to its health and economic impacts, it has affected people’s psychological states, including a rise in depression and domestic violence. We traced how the overall emotional states of individual Twitter users changed before and after the pandemic. Our data, including more than 9 million tweets posted by 9,493 users, suggest that the threat posed by the virus did not upset the emotional equilibrium of social media. In early 2020, COVID-related tweets skyrocketed in number and were filled with negative emotions; however, this emotional outburst was short-lived. We found that users who had expressed positive emotions in the pre-COVID period remained positive after the initial outbreak, while the opposite was true for those who regularly expressed negative emotions. Individuals achieved such emotional consistency by selectively focusing on emotion-reinforcing topics. The implications are discussed in light of an emotionally motivated confirmation bias, which we conceptualize as emotion bubbles that demonstrate the public’s resilience to a global health risk.
Causal Representation Learning for Out-of-Distribution Recommendation

Modern recommender systems learn user representations from historical interactions, which suffer from the problem of user feature shifts, such as an income increase. Historical interactions will inject out-of-date information into the representation in conflict with the latest user feature, leading to improper recommendations. In this work, we consider the Out-Of-Distribution (OOD) recommendation problem in an OOD environment with user feature shifts. To pursue high fidelity, we set additional objectives for representation learning as: 1) strong OOD generalization and 2) fast OOD adaptation. This work formulates and solves the problem from a causal view. We formulate the user feature shift as an intervention and OOD recommendation as post-intervention inference of the interaction probability. Towards the learning objectives, we embrace causal modeling of the generation procedure from user features to interactions. However, the unobserved user features cannot be ignored, which make the estimation of the interaction probability intractable. We thus devise a new Variational Auto-Encoder for causal modeling by incorporating an encoder to infer unobserved user features from historical interactions. We further perform counterfactual inference to mitigate the effect of out-of-date interactions. Moreover, a decoder is used to model the interaction generation procedure and perform post-intervention inference. Fast adaptation is inherent owing to the reuse of partial user representations. Lastly, we devise an extension to encode fine-grained causal relationships from user features to preference. Empirical results on three datasets validate the strong OOD generalization and fast adaptation abilities of the proposed method.
Modality Matches Modality: Pretraining Modality-Disentangled Item Representations for Recommendation

Recent works have shown the effectiveness of incorporating textual and visual information to tackle the sparsity problem in recommendation scenarios. To fuse these useful heterogeneous modality information, an essential prerequisite is to align these information for modality-robust features learning and semantic understanding. Unfortunately, existing works mainly focus on tackling the learning of common knowledge across modalities, while the specific characteristics of each modality is discarded, which may inevitably degrade the recommendation performance. To this end, we propose a pretraining framework PAMD, which stands for PretrAining Modality-Disentangled Representations Model. Specifically, PAMD utilizes pretrained VGG19 and Glove to embed items’ both visual and textual modalities into the continuous embedding space. Based on these primitive heterogeneous representations, a disentangled encoder is devised to automatically extract their modality-common characteristics while preserving their modality-specific characteristics. After this, a contrastive learning is further designed to guarantee the consistence and gaps between modality-disentangled representations. To the best of our knowledge, this is the first pretraining framework to learn modality-disentangled representations in recommendation scenarios. Extensive experiments on three public real-world datasets demonstrate the effectiveness of our pretraining solution against a series of state-of-the-art alternatives, which results in the significant performance gain of 4.70%-17.44%.
AmpSum: Adaptive Multiple-Product Summarization towards Improving Recommendation Captions

In e-commerce websites, multiple related product recommendations are usually organized into “widgets”, each given a name, as a recommendation caption, to describe the products within. These recommendation captions are usually manually crafted and generic in nature, making it difficult to attach meaningful and informative names at scale. As a result, the captions are inadequate in helping customers to better understand the connection between the multiple recommendations and make faster product discovery. We propose an Adaptive Multiple-Product Summarization framework (AmpSum) that automatically and adaptively generates widget captions based on different recommended products. The multiplicity of products to be summarized in a widget caption is particularly novel. The lack of well-developed labels motivates us to design a weakly supervised learning approach with distant supervision to bootstrap the model learning from pseudo labels, and then fine-tune the model with a small amount of manual labels. To validate the efficacy of this method, we conduct extensive experiments on several product categories of Amazon data. The results demonstrate that our proposed framework consistently outperforms state-of-the-art baselines over 9.47-29.14% on ROUGE and 27.31% on METEOR. With case studies, we illustrate how AmpSum could adaptively generate summarization based on different product recommendations.
Lie to Me: Abusing the Mobile Content Sharing Service for Fun and Profit

Online content sharing is a widely used feature in Android apps. In this paper, we observe a new Fake-Share attack that adversaries can abuse existing content sharing services to manipulate the displayed source of shared content to bypass the content review of targeted Online Social Apps (OSAs) and induce users to click on the shared fraudulent content. We show that seven popular content-sharing services (including WeChat, AliPay, and KakaoTalk) are vulnerable to such an attack. To detect this kind of attack and explore whether adversaries have leveraged it in the wild, we propose DeFash, a multi-granularity detection tool including static analysis and dynamic verification. The extensive in-the-lab and in-the-wild experiments demonstrate that DeFash is effective in detecting such attacks. We have identified 51 real-world apps involved in Fake-Share attacks. We have further harvested over 24K Sharing Identification Information (SIIs) that can be abused by attackers. It is hence urgent for our community to take actions to detect and mitigate this kind of attack.
Neuro-Symbolic Interpretable Collaborative Filtering for Attribute-based Recommendation

Recommender System (RS) is ubiquitous on today’s Internet to provide multifaceted personalized information services. While an enormous success has been made in pushing forward high-accuracy recommendations, the other side of the coin — the recommendation explainability — needs to be better handled for pursuing persuasiveness, especially for the era of deep learning based recommendation. A few research efforts investigate interpretable recommendation from the feature and result levels. Compared with them, model-level explanation, which unfolds the reasoning process of recommendation through transparent models, still remains underexplored and deserves more attention. In this paper, we propose a model-based explainable recommendation approach, i.e., NS-ICF, which stands for Neuro-Symbolic Interpretable Collaborative Filtering. Thanks to the recent advance on neuro-symbolic computation for automatic rule learning, NS-ICF learns interpretable recommendation rules (consisting of user and item attributes) based on neural networks with two innovations: (1) a three-tower architecture tailored for the user and item sides in the RS domain; (2) fusing the powerful personalized representations of users and items to achieve adaptive rule weights and without sacrificing interpretability. Comprehensive experiments on public datasets demonstrate NS-ICF is comparable to state-of-the-art deep recommendation models and is transparent for its unique neuro-symbolic architecture.
Cross Pairwise Ranking for Unbiased Item Recommendation

Most recommender systems optimize the model on observed interaction data, which is affected by the previous exposure mechanism and exhibits many biases like popularity bias. The loss functions, such as the mostly used pointwise Binary Cross-Entropy and pairwise Bayesian Personalized Ranking, are not designed to consider the biases in observed data. As a result, the model optimized on the loss would inherit the data biases, or even worse, amplify the biases. For example, a few popular items take up more and more exposure opportunities, severely hurting the recommendation quality on niche items — known as the notorious Mathew effect. In this work, we develop a new learning paradigm named Cross Pairwise Ranking (CPR) that achieves unbiased recommendation without knowing the exposure mechanism. Distinct from inverse propensity scoring (IPS), we change the loss term of a sample — we innovatively sample multiple observed interactions once and form the loss as the combination of their predictions. We prove in theory that this way offsets the influence of user/item propensity on the learning, removing the influence of data biases caused by the exposure mechanism. Advantageous to IPS, our proposed CPR ensures unbiased learning for each training instance without the need of setting the propensity scores. Experimental results demonstrate the superiority of CPR over state-of-the-art debiasing solutions in both model generalization and training efficiency. The codes are available at https://github.com/Qcactus/CPR.