dataesr/scientific-paragraphs-categorization huggingface dataset:
Finding a set of users of interest serves several applications in behavioral analytics. Often times, identifying users requires to explore the data and gradually choose potential targets. This is a special case of Exploratory Data Analysis (EDA), an iterative and tedious process. In this paper, we formalize and solve the problem of guided exploration of user groups whose purpose is to find target users. We model exploration as an iterative decision-making process, where an agent is shown a set of groups, chooses users from those groups, and selects the best action to move to the next step. To solve our problem, we apply reinforcement learning to discover an efficient exploration strategy from a simulated agent experience, and propose to use the learned strategy to recommend an exploration policy that can be applied to the same task for any dataset.
g. , SQL) to formulate queries, as well as a global understanding of the underlying database schema. Secondly, the analyst usually lacks information about the distribution of attribute values that characterize users or items on which users performed some action. Thus, a query that selects users based on user or item attributes may either return too few or too many results. For instance, a query that separates users into groups of males and females can surprisingly return very small output groups when most users do not have gender information. Thirdly, an analyst may want to refine her queries as she discovers more users. For instance, a PC chair usually has a partial idea of who to look for, such as a mix of senior and junior male and female researchers around the world. Hence, the PC chair may want to navigate in the space of potential PC members to iteratively choose who to include in the final PC depending on a mix of gender, coverage of topics, seniority, and countries obtained so far.
Due to the iterative nature of the task and its inherent difficulty, user group analysis can be viewed as an instance of Exploratory Data Analysis (EDA). In this setting, the exploratory analysis of user groups is an iterative decisionmaking process, whereby an analyst is shown a set of user groups labeled with user and item attribute values (e. g. , groups of researchers who published in SIGMOD), chooses target users from those groups, and selects the best exploration action to move to the next iteration (e. g. , select a group whose label is [prolific, female, publish in VLDB] that has researchers with well distributed geographical location and gender, and apply an explore-around action to return k diverse groups that overlap with the selected group). Although there exists a large body of work on the recommendation of data exploration actions, carefully reviewed in Section 2, these works have important shortcomings.
First, most work recommend (SQL/OLAP) queries, which is not adapted to analysts with no IT expertise, while other works focus on specific types of high-level actions that are too limited for user group analysis. Second, existing solutions either rely on tedious manual exploration  [28] , or on the collection of a large log of exploration sessions gathered from multiple users on the same dataset  [25] , which is not an acceptable assumption in our case since many analysis tasks are repeated only a few times or performed on different datasets. Last, very recent work on automating the exploration process does not generate interpretable policies  [13]. In this paper, we propose the first framework for user group exploration that learns an exploration policy without requiring prior collection of exploration log.
The learned policy is used to recommend an end-to-end exploration session on a given dataset. The recommended sessions can be easily interpreted by an analyst with no IT expertises. Our framework supports exploration sessions consisting of a sequence of high-level exploration actions of various types and can provide recommendations for the exploration of new datasets, provided they are similar (with respect to domainspecific features) to previously analyzed datasets. Additionally, our framework learns an exploration policy offline from a simulated agent experience in such a way that no human intervention is needed from an analyst during the online analysis task. It thus solves a fully guided EDA for user group analysis when the task is to find target users. Our first contribution is to formalize the guided EDA problem applied to user groups as a Markov decision process where a state displays several user groups, a transition is the application of an exploration action to a chosen group, and the reward is a function of whether or not some target users are found when applying the action to the group.
An exploration session is hence a sequence of exploration actions, and an exploration policy is a function that maps a state to an action. Our framework integrates a comprehensive set of high-level exploration actions for user groups that were introduced separately by previous work: explore-around  [28] , explore-within  [28] , by-facet  [37] , by-distribution  [2] , and bytopic  [9]. Those actions were never considered all together before. We define a unified semantics for all these actions to make them composable and take advantage of their expressivity at each exploration step. We then formalize the problem of guided EDA on user groups as finding a policy that maximizes utility, i. e. , that finds as many target users as possible regardless of the dataset, the user groups and the seed groups at every step.
Our second contribution is the of use Reinforcement Learning (RL) to learn an efficient policy from a simulated agent experience (i. e. , an RL agent that acts as an analyst who chooses exploration actions), and recommend a policy. A notable advantage of RL methods is that, unlike supervised learning methods, they do not require to gather labeled datasets. Instead, the agent learns from rewards computed by the environment during the interaction  [34]. This naturally fits our context since we can use, for instance, the PC of WebDB 2017 (or any previous PC of the same or different venue) to learn an exploration policy offline, and apply it online to build the PC of WebDB 2018. Additionally, the use of RL for EDA enables to model our problem as an interactive decision-making process. Our work bears similarity with recent proposals on RL-based EDA  [25, 33]  but differs in several ways.
First, we address a well-defined user data analysis task (gathering target users) and the exploration actions are chosen accordingly. Additionally, in case of interactive exploration, the analysis task is often unique and needs to be accomplished only once by an analyst. Running a large number of manual explorations for each new dataset and new analysis task is hence impractical. In  [13] , the authors propose to use a deep RL approach for EDA. However, our focus on producing interpretable exploration sessions for a human analyst warrants the use of classical RL methods instead of deep RL or contextual bandits, a special case of RL where the agent's action does not determine the next state of the environment  [21, 23, 31]. Our reliance on classical RL methods is a first step towards a deeper and comprehensive understanding of how to learn data exploration policies for user data analysis, including thorough experiments on real datasets.
Our third contribution is an extensive set of experiments that validate the use of RL to solve the guided EDA problem for user groups. We study various cases where target users to be found are scattered in groups. We examine the utility of learned policies on real user datasets for different variants where policies are learned on the same conference or transferred from one conference to another. Our experiments show that the simulated agent succeeds to learn an interpretable strategy for several realistic use cases of interactive exploration. It also validates that our exploration actions and offline policy learning naturally capture what a human analyst does manually (as in  [28] ) and can hence be used to automate a guided EDA. The rest of the paper is organized as follows. First, we review related work in Section 2.
User group exploration frameworks User group exploration is a recent and dynamic research area in data management  [27]. The most traditional way of exploring user groups is based on previous exploration frameworks  [5, 6]. Most notably, by-query can be used to select user groups that satisfy some predicates. However, to formulate a query, the analyst needs to know the database schema, the query language, and the underlying data distributions. We discard this type of exploration in our model because we want to target analysts that have no programming expertise. Higher-level exploration actions not requiring much expertise have been introduced in recent work. by-facet exploration returns groups that have the same value for some attribute, e. g. gender,  [17, 37]. by-example exploration  [26, 28]  finds groups that are similar to/different from an input group.
In  [28] , by-example-around returns k diverse groups that overlap with an input group, and by-example-within returns a set of k subgroups that maximize the coverage of the input group. by-analytics selects groups based on statistics. Some actions admit as input a set of distributions and find groups with similar rating distributions  [2]. Others are a variant of by-example and look for groups similar to, or different from, a given group in terms of their distributions  [16]. by-text exploration actions  [9]  leverage textual information such as tags and reviews to find groups that exhibit similar/dissimilar tags or reviews. Our framework accommodates the above actions and is sufficiently generic to capture other actions. There exist other types of actions that we did not consider. For instance, by-evolution actions  [19, 20, 35, 38, 39, 40]  are designed for timestamped datasets.
Recommendation of exploration actions A number of works recommend queries for the interactive exploration of databases  [7, 11]. One approach based on collaborative filtering  [1, 12, 18, 24]  uses previously collected query logs a dataset (SQL queries in  [12, 18] , and OLAP queries in  [1, 24] ) to recommend queries on the same dataset. This approach is not applicable to our problem because we cannot rely on a large number of previous exploration sessions for the same data analysis task, and we assume that the same analysis task can be performed on different datasets. Finally, these works focus on the recommendation of SQL/OLAP queries while we are interested in more abstract operations that do not require IT expertise. Another mode of interactive exploration, sometimes called data-driven approach  [7, 4, 10, 15, 29, 32] , recommends a single type of exploration action whose result is expected to optimize a measure of "interestingness" with respect to the current analysis context of a user on a given dataset.
For instance,  [15]  suggests different drill-down operations on a given table (a case of by-facet), each producing a different set of tuples. With a data-driven approach, the notion of context is predefined (e. g. , user profile  [4] , sequence of actions within the same session  [15, 32] , or data returned by a query  [10, 29] ). These works do not apply to our problem firstly because they only consider one type of exploration action. Secondly, the proposed measures of "interestingness" are predefined and cannot be adapted to different analysis tasks. The closest work to ours is the recommendation of various types of high-level exploration actions (e. g. , filter, roll-up, cluster-by) for interactive data analysis over different datasets  [25]. In the same spirit as the data-driven approach, an exploration action is recommended based on different user's analysis contexts modeled as trees in which previous actions are edges and their output dataset, called "display", are nodes.
User datasets and user groups We model user data as a set of users U, a set of items I, and a set D = { u, i, s, x } where u ∈ U, i ∈ I, s is an integer score, and x is a text. A tuple u, i, s, x represents the action of user u (e. g. , publishing, reviewing) on item i with an optional score s (e. g. , recency of research publications, product rating) and an optional text x (e. g. , publication titles/abstracts, product reviews). Each user u ∈ U is described with a set of attributevalue pairs u = { a, v }, where a ∈ AU is a demographic attribute (e. g. , age, gender, occupation), and v is a value in a's domain, i. e. , v ∈ dom(a). Similarly, each item i ∈ I is described with a set of attribute-value pairs i = { a, v } where a ∈ AI is an item attribute (e. g. , publication topic, hotel nightly price).
Martin identifies Lucian Popa, An-Hai Doan, Sihem Amer-Yahia and Michael Benedikt in that group, and applies another explore-around exploration to explore relevant groups. Among the 3 returned groups, he examines one containing 119 highly senior researchers, and requests explore-within to delve into that large group. He finds the group labeled [highly senior, very productive, VLDB, ICDE] containing 26 senior researchers out of which Francesco Bonchi, Kaushik Chakrabarti, Piero Fraternali and Felix Naumann are of interest. Martin then applies a by-topic exploration to find other groups whose research areas are similar to those 26 researchers. by-topic expands the explored topics which allows him to identify a new group of 38 researchers whose research covers "stream processing" and "data integration".
The latter is the main topic of WebDB in 2014. Martin decides to apply a by-facet on that group to separate males and females and balance his PC. At this stage and after 5 steps only, Martin has already covered 80% of the WebDB 2014 PC. Example 2 illustrates the need for an iterative process to find users of interest, since the analyst does not necessarily have a complete specification of the set of target users at the beginning, and a full understanding of the underlying data distributions. The example also illustrates several key aspects of the exploration: an effective end-to-end exploration depends on the seed group (the set of users with which the analyst starts the exploration) as well as the series of exploration actions and their utility in helping the analyst to locate target users.
Guided EDA problem We are now ready to state our problem: Given the task of finding a set of target users Ut, the guided EDA problem is formulated as finding a policy π * with the highest utility. More formally, π * = argmaxπp utility(π, s1, Ut), ∀s1 = g1, G k1 The guided EDA problem poses two major challenges: (i) how to simulate a human agent in such a way that we learn a policy that is applicable to any dataset and any input groups? (ii) how to characterize exploration states in such a way that they are independent from the underlying data, and that the decision of which action to apply next maximizes overall utility?.
Architecture of the RL-based approach The general architecture for our RL-based approach is depicted in Figure  1. The offline phase addresses our first challenge: simulate a human experience in such a way that we learn a policy that is applicable to any dataset and any initial group. During the offline phase, an agent simulating a human analyst is trained to learn a policy that maximizes utility, e. g. , for the PC of WebDB 2017. The policy is updated as the agent interacts with user groups via exploration actions. The outcome is final exploration policy that can be leveraged in an online phase to find any set of target users. For instance, the policy will apply explore-within at a given step i in case a group at step i -1 includes some target users but is too large and hence requires further splitting.
Exploration model We model EDA as a Markov Decision Process (MDP) comprising a quadruple (S, E, P, R) where: • S is a set of states of the process; • E is a set of exploration actions that change the process state; • P (si+1|si, ei) are probabilities that action ei will change state si ∈ S to state si+1 ∈ S; • R(si+1|si, ei) are rewards for transitioning from state si ∈ S to state si+1 ∈ S by applying exploration action ei. Each state si+1 in S is a tuple gi+1, G ki+1 obtained by applying an exploration action ei to a previous state si = gi, G ki , and for which an action ei+1 should be selected to continue the exploration. The probabilities P (si+1|si, ei) characterize the behavior of exploration actions, i. e. , they represent what will be displayed next if ei is selected. These probabilities are not known in advance and depend on the quality and relevance functions of the exploration actions and on properties of the dataset.
RL framework Reinforcement learning (RL) is a set of methods to find an optimal decision policy for an MDP when transition probabilities are not given, i. e. , the input to an RL model is (S, E, P, R) \ P. RL fits our context, because in EDA, the transition probabilities between exploration actions are unknown and depend on the dataset. To learn an optimal policy, we simulate interactions between an analyst and groups in the offline phase. An RL agent interacts with different states and gathers several simulated exploration sessions. At each state si = gi, G ki , the agent decides which exploration action ei to select. This decision is based on the action-value function Q(si, ei), which the agent learns. State-action features f (s, e). Our goal is to learn meaningful EDA policies that generate interpretable exploration sessions for human analysts.
Given a state-action feature vector f (s, e), we define a linear approximation of Q(s, e): Q(w, s, e) = w T f (s, e) (6) Algorithm 1: Policy learning Input: E, G, k, Ut, G c t , g0, , α, γ, w0 Output: agent 1 agent. set( , α, γ, w0, E) 2 while not end of learning do This approximation is essential as it still makes our representation data-driven but not strictly data-dependent. It blends together states with the same features in the same manner as grouping similar contexts is done in database exploration  [25]. Learning procedure. To learn the approximation of the action-value function, we apply a common method, a semigradient method  [34]  based on stochastic gradient descent (SGD) minimization of mean squared error: err (w) = s∈S,e∈E P (s, e)(Q(s, e) -Q(w, s, e)) 2 (7) where P (s, e) denotes probabilities of the state-action pairs in the agent-environment interaction.
Datasets We test our system on DM-Authors, a dataset we built from DBLP. 2  DM-Authors includes 1, 860 researchers with a total of 200, 000 publications in data management venues between 2000 and 2018: WWW, VLDB, SIGMOD, ICDE, SIGMOD, RecSys, EDBT, DEXA, WebDB, and HILDA. We crawled profiles of researchers from DBLP and added demographic attributes. Following our data model (Section 3), we describe the quadruple u, i, s, x in DM-Authors as follows: u ∈ U is a researcher, i ∈ I is a publication by a researcher, s ∈  [1, 5]  is a normalized value for publication recency, and x is a bag of words from the publication title. The score s is assigned based on the year of publication:  [2017,   User groups. To build the set of groups G, we rely on LCM, an implementation of the Apriori algorithm for closed frequent pattern mining  [36].
One can notice similar patterns in the policies: when increasing the number of targets discovered so far, the likelihood of using explore-around decreases (Figure  8 ). As it becomes less probable to find new targets in the groups overlapping with an input group (via other actions than explorearound), it is better to jump to new groups to increase the likelihood of finding target users. One can also see some differences: the policy trained for SIGMOD strongly prefers by-distribution for smaller groups. A possible explanation is the larger ratio of target users to all users for the SIGMOD PC, as by-distributioncan return more users with recent papers regardless of their research topics. The agent sees more active researchers from various areas which increases the likelihood of discovering SIGMOD PC members that make almost 20% of the dataset.
CONCLUSION In this paper, we examined the applicability of machine learning techniques to EDA on user data where the task is to find a set of users. We first proposed a unified formalization of the guided EDA problem that leverages a wide   range of user data exploration actions. We developed a solution based on a Markov Decision Process and reinforcement learning to learn an exploration policy. We conducted experiments on real datasets from DBLP where the purpose was to build the program committees of WebDB and SIGMOD. Our results validate the utility of different learning variants to find an exploration policy for guided PC formation. Our work opens many new questions at the intersection of data management and machine learning. A deeper experimental validation of several choices we made is warranted.
confirmed (22+) active Publication rate Average number of publications per year. Values are chosen to equalize the number of researchers in each category. (0. 18 to 1. 47), very active (1. 48 to 2. 48), productive (2. 49 to 3. 71), very productive (3. 72 to 6. 0), prolific (6. 1+) North America, UK/Ireland, Location Extracted from researchers' affiliations in DBLP profiles. South America, Europe, East/South Asia, Australia, Middle East, other Extracted by matching Gender researcher's first name to a database of more male, female than 40,000 names. 4.
SBS and MiChroM can reproduce important physical phenomena such as the dynamics of chromatin condensation leading to phase separation; however, no methods for calling specific many-body chromatin interactions based on these models have been reported yet. Several computational methods have been developed to detect specific pairwise chromatin interactions present within Hi-C datasets  [20]. These include the negative binomial model of Jin et al. [35] , the non-parametric spline approach of Fit-Hi-C  [36] , the binomial model of GOTHiC  [37] , the local neighborhood loop-calling approach of HiCCUPS  [9] , and the hidden Markov random field model of Xu et al. [38]. These methods rely on the empirical Hi-C for estimation of a background model that is then used to assess the significance of each pairwise chromatin contact; hence, these approaches may contain intrinsic bias as the observed Hi-C data is being used for construction of its own null hypothesis test.
Our predictions are overall concordant with SPRITE, with a majority of modeled genomic loci exhibiting significantly elevated median coverages for specific vs. non-specific many-body interactions. The landscape of many-body interactions emerging from our analysis of 39 active genomic loci showed super-enhancers (SE) as enriched in specific many-body principal loop participation compared to non-SE regions (p = 2. 24 × 10 -129 , Additional file 1: Figure  S8 ), with   S2 ), primarily from ENCODE reference epigenome for GM12878 (ENCSR447YYN)  [53, 54] , were used as predictive inputs to a random forest  [55, 56]  machine learning classifier. Illustrative signals shown are from the UCSC genome browser  [76, 77]  for locus chr 12: 11,690,000-12,210,000. b Cartoon illustration of enriched versus not enriched regions.
Concordance with sPRITE We compared our 3-body and maximal many-body principal loop predictions with publicly available SPRITE data for GM12878  [18]. To adjust for genomic distance bias, we stratified principal loops according to base pair span and computed the SPRITE coverage fraction, i. e. , proportion of SPRITE clusters that contained each principal loop complex. Specifically, we computed the median SPRITE coverage fraction at each 5-KB genomic distance span for both specific and non-specific principal loops (Additional file 1: Figure  S5 ). At each of the 39 modeled loci, we assessed the significance of the proportion of specific medians exceeding the corresponding non-specific medians by permutation testing: we randomly permuted the specific and non-specific labels assigned to each principal loop and re-computed the proportion of specific medians exceeding non-specific medians for 1000 total replicates.