SlowGuess/ACL_2020_OCR huggingface dataset:
Rui Wang† Xuemeng Hu† Deyu Zhou†* Yulan He§
Yuxuan Xiong† Chenchen Ye† Haiyang Xu‡
†School of Computer Science and Engineering, Key Laboratory of Computer Network and Information Integration, Ministry of Education, Southeast University, China
$\S$ Department of Computer Science, University of Warwick, UK
$\ddagger$ AI Labs - Didi Chuxing Co., Ltd. - Beijing, China
{rui_wang, xuemenghu, d.zhou, yuxuanxiong, chencheny} @seu.edu.cn,
yulan.he@warwick.ac.uk, xuhaiyangsnow@didiglobal.com
Recent years have witnessed a surge of interests of using neural topic models for automatic topic extraction from text, since they avoid the complicated mathematical derivations for model inference as in traditional topic models such as Latent Dirichlet Allocation (LDA). However, these models either typically assume improper prior (e.g. Gaussian or Logistic Normal) over latent topic space or could not infer topic distribution for a given document. To address these limitations, we propose a neural topic modeling approach, called Bidirectional Adversarial Topic (BAT) model, which represents the first attempt of applying bidirectional adversarial training for neural topic modeling. The proposed BAT builds a two-way projection between the document-topic distribution and the document-word distribution. It uses a generator to capture the semantic patterns from texts and an encoder for topic inference. Furthermore, to incorporate word relatedness information, the Bidirectional Adversarial Topic model with Gaussian (Gaussian-BAT) is extended from BAT. To verify the effectiveness of BAT and Gaussian-BAT, three benchmark corpora are used in our experiments. The experimental results show that BAT and Gaussian-BAT obtain more coherent topics, outperforming several competitive baselines. Moreover, when performing text clustering based on the extracted topics, our models outperform all the baselines, with more significant improvements achieved by Gaussian-BAT where an increase of near  $6\%$  is observed in accuracy.
Topic models have been extensively explored in the Natural Language Processing (NLP) community for unsupervised knowledge discovery. Latent Dirichlet Allocation (LDA) (Blei et al., 2003), the
Logistic-Normal
Dirichlet
Figure 1: Illustrated probability simplex with Logistic-Normal distribution and Dirichlet distribution.
most popular topic model, has been extended (Lin and He, 2009; Zhou et al., 2014; Cheng et al., 2014) for various extraction tasks. Due to the difficulty of exact inference, most LDA variants require approximate inference methods, such as mean-field methods and collapsed Gibbs sampling. However, these approximate approaches have the drawback that small changes to the modeling assumptions result in a re-derivation of the inference algorithm, which can be mathematically arduous.
One possible way in addressing this limitation is through neural topic models which employ blackbox inference mechanism with neural networks. Inspired by variational autoencoder (VAE) (Kingma and Welling, 2013), Srivastava and Sutton (2017) used the Logistic-Normal prior to mimic the simplex in latent topic space and proposed the Neural Variational LDA (NVLDA). Moreover, they replaced the word-level mixture in NVLDA with a weighted product of experts and proposed the ProdLDA (Srivastava and Sutton, 2017) to further enhance the topic quality.
Although Srivastava and Sutton (2017) used the Logistic-Normal distribution to approximate the Dirichlet distribution, they are not exactly the same. An illustration of these two distributions is shown in Figure 1 in which the Logistic-Normal distribution does not exhibit multiple peaks at the vertices of the simplex as that in the Dirichlet distribution and as such, it is less capable to capture
*corresponding author
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 340-350
the multi-modality which is crucial in topic modeling (Wallach et al., 2009). To deal with the limitation, Wang et al. (2019a) proposed the Adversarial-neural Topic Model (ATM) based on adversarial training, it uses a generator network to capture the semantic patterns lying behind the documents. However, given a document, ATM is not able to infer the document-topic distribution which is useful for downstream applications, such as text clustering. Moreover, ATM take the bag-of-words assumption and do not utilize any word relatedness information captured in word embeddings which have been proved to be crucial for better performance in many NLP tasks (Liu et al., 2018; Lei et al., 2018).
To address these limitations, we model topics with Dirichlet prior and propose a novel Bidirectional Adversarial Topic model (BAT) based on bidirectional adversarial training. The proposed BAT employs a generator network to learn the projection function from randomly-sampled document-topic distribution to document-word distribution. Moreover, an encoder network is used to learn the inverse projection, transforming a document-word distribution into a document-topic distribution. Different from traditional models that often resort to analytic approximations, BAT employs a discriminator which aims to discriminate between real distribution pair and fake distribution pair, thereby helps the networks (generator and encoder) to learn the two-way projections better. During the adversarial training phase, the supervision signal provided by the discriminator will guide the generator to construct a more realistic document and thus better capture the semantic patterns in text. Meanwhile, the encoder network is also guided to generate a more reasonable topic distribution conditioned on specific document-word distributions. Finally, to incorporate the word relatedness information captured by word embeddings, we extend the BAT by modeling each topic with a multivariate Gaussian in the generator and propose the Bidirectional Adversarial Topic model with Gaussian (Gaussian-BAT).
The main contributions of the paper are:
- We propose a novel Bidirectional Adversarial Topic (BAT) model, which is, to our best knowledge, the first attempt of using bidirectional adversarial training in neural topic modeling;
- We extend BAT to incorporate the word re
latedness information into the modeling process and propose the Bidirectional Adversarial Topic model with Gaussian (Gaussian-BAT);
- Experimental results on three public datasets show that BAT and Gaussian-BAT outperform the state-of-the-art approaches in terms of topic coherence measures. The effectiveness of BAT and Gaussian-BAT is further verified in text clustering.
Our work is related to two lines of research, which are adversarial training and neural topic modeling.
Adversarial training, first employed in Generative Adversarial Network (GAN) (Goodfellow et al., 2014), has been extensively studied from both theoretical and practical perspectives.
Theoretically, Arjovsky (2017) and Gulrajani (2017) proposed the Wasserstein GAN which employed the Wasserstein distance between data distribution and generated distribution as the training objective. To address the limitation that most GANs (Goodfellow et al., 2014; Radford et al., 2015) could not project data into a latent space, Bidirectional Generative Adversarial Nets (BiGAN) (Donahue et al., 2016) and Adversarily Learned Inference (ALI) (Dumoulin et al., 2016) were proposed.
Adversarial training has also been extensively used for text generation. For example, SeqGAN (Yu et al., 2017) incorporated a policy gradient strategy for text generation. RankGAN (Lin et al., 2017) ranked a collection of human-written sentences to capture the language structure for improving the quality of text generation. To avoid mode collapse when dealing with discrete data, MaskGAN (Fedus et al., 2018) used an actor-critic conditional GAN to fill in missing text conditioned on the context.
2.2 Neural Topic Modeling
To overcome the challenging exact inference of topic models based on directed graph, a replicated softmax model (RSM), based on the Restricted Boltzmann Machines was proposed in (Hinton and Salakhutdinov, 2009). Inspired by VAE, Miao et al. (2016) used the multivariate Gaussian as the prior distribution of latent space and proposed the
Figure 2: The framework of the Bidirectional Adversarial Topic (BAT) model.
Neural Variational Document Model (NVDM) for text modeling. To model topic properly, the Gaussian Softmax Model (GSM) (Miao et al., 2017) which constructs the topic distribution using a Gaussian distribution followed by a softmax transformation was proposed based on the NVDM. Likewise, to deal with the inappropriate Gaussian prior of NVDM, Srivastava and Sutton (2017) proposed the NVLDA which approximates the Dirichlet prior using a Logistic-Normal distribution. Recently, the Adversarial-neural Topic Model (ATM) (Wang et al., 2019a) is proposed based on adversarial training, it models topics with Dirichlet prior which is able to capture the multi-modality compared with logistic-normal prior and obtains better topics. Besides, the Adversarial-neural Event (AEM) (Wang et al., 2019b) model is also proposed for open event extraction by representing each event as an entity distribution, a location distribution, a keyword distribution and a date distribution.
Despite the extensive exploration of this research field, scarce work has been done to incorporate Dirichlet prior, word embeddings and bidirectional adversarial training into neural topic modeling. In this paper, we propose two novel topic modeling approaches, called BAT and Gaussian-BAT, which are different from existing approaches in the following aspects: (1) Unlike NVDM, GSM, NVLDA and ProdLDA which model latent topic with Gaussian or logistic-normal prior, BAT and Gaussian-BAT explicitly employ Dirichlet prior to
model topics; (2) Unlike ATM which could not infer topic distribution of a given document, BAT and Gaussian-BAT uses an encoder to generate the topic distribution corresponding to the document; (3) Unlike neural topic models that only utilize word co-occurrence information, Gaussian-BAT models topic with multivariate Gaussian and incorporates the word relatedness into modeling process.
Our proposed neural topic models are based on bidirectional adversarial training (Donahue et al., 2016) and aim to learn the two-way non-linear projection between two high-dimensional distributions. In this section, we first introduce the Bidirectional Adversarial Topic (BAT) model that only employs the word co-occurrence information. Then, built on BAT, we model topics with multivariate Gaussian in the generator of BAT and propose the Bidirectional Adversarial Topic model with Gaussian (Gaussian-BAT), which naturally incorporates word relatedness information captured in word embeddings into modeling process.
3.1 Bidirectional Adversarial Topic model
As depicted in Figure 2, the proposed BAT consists of three components: (1) The Encoder  $E$  takes the  $V$ -dimensional document representation  $\vec{d_r}$  sampled from text corpus  $C$  as input and transforms it into the corresponding  $K$ -dimensional topic distribution  $\vec{\theta_r}$ ; (2) The Generator  $G$  takes a random
topic distribution  $\vec{\theta}_f$  drawn from a Dirichlet prior as input and generates a  $V$ -dimensional fake word distribution  $\vec{d}_f$ ; (3) The Discriminator  $D$  takes the real distribution pair  $\vec{p_r} = [\vec{\theta_r};\vec{d_r}]$  and fake distribution pair  $\vec{p_f} = [\vec{\theta_f};\vec{d_f}]$  as input and discriminates the real distribution pairs from the fake ones. The outputs of the discriminator are used as supervision signals to learn  $E$ ,  $G$  and  $D$  during adversarial training. In what follows, we describe each component in more details.
3.1.1 Encoder Network
The encoder learns a mapping function to transform document-word distribution to document-topic distribution. As shown in the top-left panel of Figure 2, it contains a  $V$ -dimensional document-word distribution layer, an  $S$ -dimensional representation layer and a  $K$ -dimensional document-topic distribution layer, where  $V$  and  $K$  denote vocabulary size and topic number respectively.
More concretely, for each document  $d$  in text corpus,  $E$  takes the document representation  $\vec{d_r}$  as input, where  $\vec{d_r}$  is the representation weighted by TF-IDF, and it is calculated by:
$$
t f _ {i, d} = \frac {n _ {i , d}}{\sum_ {v} n _ {v , d}}, \qquad i d f _ {i} = \log \frac {| C |}{| C _ {i} |}
$$
$$
t f - i d f _ {i, d} = t f _ {i, d} * i d f _ {i}, \qquad d _ {r} ^ {i} = \frac {t f - i d f _ {i , d}}{\sum_ {v} t f - i d f _ {v , d}}
$$
where  $n_{i,d}$  denotes the number of  $i$ -th word appeared in document  $d$ ,  $|C|$  represents the number of documents in the corpus, and  $|C_i|$  means the number of documents that contain  $i$ -th word in the corpus. Thus, each document could be represented as a  $V$ -dimensional multinomial distribution and the  $i$ -th dimension denotes the semantic consistency between  $i$ -th word and the document.
With  $\vec{d_r}$  as input,  $E$  firstly projects it into an  $S$ -dimensional semantic space through the representation layer as follows:
$$
\vec {h} _ {s} ^ {e} = \mathrm {B N} \left(W _ {s} ^ {e} \vec {d} _ {r} + \vec {b} _ {s} ^ {e}\right) \tag {1}
$$
$$
\vec {\sigma} _ {s} ^ {e} = \max (\vec {h} _ {s} ^ {e}, l e a k * \vec {h} _ {s} ^ {e}) \qquad (2)
$$
where  $W_{s}^{e}\in \mathbb{R}^{S\times V}$  and  $\vec{b}_s^e$  are weight matrix and bias term of the representation layer,  $\vec{h}_s^e$  is the state vector normalized by batch normalization  $\mathrm{BN}(\cdot)$ , leak denotes the parameter of LeakyReLU activation and  $\vec{\sigma}_{s}^{e}$  represents the output of representation layer.
Then, the encoder transforms  $\bar{\sigma}_s^e$  into a  $K$ -dimensional topic space based on the equation below:
$$
\vec {\theta} _ {r} = \operatorname {s o f t m a x} \left(W _ {t} ^ {e} \vec {\sigma} _ {s} ^ {e} + \vec {b} _ {t} ^ {e}\right) \tag {3}
$$
where  $W_{t}^{e}\in \mathbb{R}^{K\times S}$  is the weight matrix of topic distribution layer,  $\vec{b}_t^e$  represents the bias term,  $\vec{\theta}_r$  denotes the corresponding topic distribution of the input  $\vec{d}_r$  and the  $k$ -th ( $k\in \{1,2,\dots,K\}$ ) dimension  $\theta_r^k$  represents the proportion of  $k$ -th topic in document  $d$ .
3.1.2 Generator network
The generator  $G$  is shown in the bottom-left panel of Figure 2. Contrary to encoder, it provides an inverse projection from document-topic distribution to document-word distribution and contains a  $K$ -dimensional document-topic layer, an  $S$ -dimensional representation layer and a  $V$ -dimensional document-word distribution layer.
As pointed out in (Wallach et al., 2009), the choice of Dirichlet prior over topic distribution is important to obtain interpretable topics. Thus, BAT employs the Dirichlet prior parameterized with  $\vec{\alpha}$  to mimic the multi-variate simplex over topic distribution  $\vec{\theta}_f$ . It can be drawn randomly based on the equation below:
$$
p \left(\vec {\theta} _ {f} \mid \vec {\alpha}\right) = D i r \left(\vec {\theta} _ {f} \mid \vec {\alpha}\right) \triangleq \frac {1}{\Delta (\vec {\alpha})} \prod_ {k = 1} ^ {K} \left[ \theta_ {f} ^ {k} \right] ^ {\alpha_ {k} - 1} \tag {4}
$$
where  $\vec{\alpha}$  is the  $K$ -dimensional hyper-parameter of Dirichlet prior,  $K$  is the topic number that should be set in BAT,  $\theta_f^k \in [0,1]$ , follows the constrain that  $\sum_{k=1}^{K} \theta_f^k = 1$ , represents the proportion of the  $k$ -th topic in the document, and normalization term  $\Delta(\vec{\alpha})$  is defined as  $\frac{\prod_{k=1}^{K} \Gamma(\alpha_k)}{\Gamma(\sum_{k=1}^{K} \alpha_k)}$ .
To learn the transformation from document-topic distribution to document-word distribution,  $G$  firstly projects  $\vec{\theta}_f$  into an  $S$ -dimensional representation space based on equations:
$$
\vec {h} _ {s} ^ {g} = \mathrm {B N} \left(W _ {s} ^ {g} \vec {\theta} _ {f} + \vec {b} _ {s} ^ {g}\right) \tag {5}
$$
$$
\vec {\sigma} _ {s} ^ {g} = \max  \left(\vec {h} _ {s} ^ {g}, l e a k * \vec {h} _ {s} ^ {g}\right) \tag {6}
$$
where  $W_{s}^{g}\in \mathbb{R}^{S\times K}$  is weight matrix of the representation layer,  $\vec{b}_s^g$  represents bias term,  $\vec{h}_s^g$  is the state vector normalized by batch normalization, Eq. 6 represents the LeakyReLU activation parameterized with leak, and  $\vec{o}_s^g$  is the output of the representation layer.
Then, to project  $\vec{\sigma}_s^g$  into word distribution  $\vec{d_f}$ , a subnet contains a linear layer and a softmax layer is used and the transformation follows:
$$
\vec {d} _ {f} = \operatorname {s o f t m a x} \left(W _ {w} ^ {g} \vec {\sigma} _ {s} ^ {g} + \vec {b} _ {w} ^ {g}\right) \tag {7}
$$
where  $W_w^g \in \mathbb{R}^{V \times S}$  and  $\vec{b}_w^g$  are weight matrix and bias of word distribution layer,  $\vec{d}_f$  is the word distribution correspond to  $\vec{\theta}_f$ . For each  $v \in \{1, 2, \dots, V\}$ , the  $v$ -th dimension  $d_f^v$  is the probability of the  $v$ -th word in fake document  $\vec{d}_f$ .
3.1.3 Discriminator network
The discriminator  $D$  is constituted by three layers (a  $V + K$ -dimensional joint distribution layer, an  $S$ -dimensional representation layer and an output layer) as shown in the right panel of Figure 2. It employs real distribution pair  $\vec{p_r}$  and fake distribution pair  $\vec{p_f}$  as input and then outputs  $D_{out}$  to identify the input sources (fake or real). Concretely, a higher value of  $D_{out}$  represents that  $D$  is more prone to predict the input as real and vice versa.
3.2 BAT with Gaussian (Gaussian-BAT)
In BAT, the generator models topics based on the bag-of-words assumption as in most other neural topic models. To incorporate the word relatedness information captured in word embeddings (Mikolov et al., 2013a,b; Pennington et al., 2014; Joulin et al., 2017; Athiwaratkun et al., 2018) into the inference process, we modify the generator of BAT and propose Gaussian-BAT, in which  $G$  models each topic with a multivariate Gaussian as shown in Figure 3.
Figure 3: The generator of Gaussian-BAT.
Concretely, Gaussian-BAT employs the multivariate Gaussian  $\mathcal{N}(\vec{\mu}_k,\Sigma_k)$  to model the  $k$ -th topic. Here,  $\vec{\mu}_k$  and  $\Sigma_{k}$  are trainable parameters, they represent mean and covariance matrix respectively. Following its probability density, for each word  $v\in \{1,2,\dots,V\}$ , the probability in the  $k$ -th
topic  $\phi_{k,v}$  is calculated by:
$$
\begin{array}{l} p (\vec {e} _ {v} | t o p i c = k) = \mathcal {N} (\vec {e} _ {v}; \vec {\mu} _ {k}, \Sigma_ {k}) \\ = \frac {\exp \left(- \frac {1}{2} \left(\vec {e} _ {v} - \vec {\mu} _ {k}\right) ^ {\mathrm {T}} \Sigma_ {k} ^ {- 1} \left(\vec {e} _ {v} - \vec {\mu} _ {k}\right)\right)}{\sqrt {(2 \pi) ^ {D _ {e}} \left| \Sigma_ {k} \right|}} \tag {8} \\ \end{array}
$$
$$
\phi_ {k, v} = \frac {p (\vec {e} _ {v} | t o p i c = k)}{\sum_ {v = 1} ^ {V} p (\vec {e} _ {v} | t o p i c = k)} \tag {9}
$$
where  $\vec{e}_v$  means the word embedding of  $v$ -th word,  $V$  is the vocabulary size,  $|\Sigma_k| = \operatorname*{det}\Sigma_k$  is the determinant of covariance matrix  $\Sigma_{k}$ ,  $D_{e}$  is the dimension of word embeddings,  $p(\vec{e}_v|topic = k)$  is the probability calculated by density, and  $\vec{\phi}_k$  is the normalized word distribution of  $k$ -th topic. With randomly sampled topic distribution  $\vec{\theta}_f$  and the calculated topic-word distributions  $\{\vec{\phi}_1,\vec{\phi}_2,\dots,\vec{\phi}_K\}$ , the fake word distribution  $\vec{d}_f$  corresponding to  $\vec{\theta}_f$  can be obtained by:
$$
\vec {d} _ {f} = \sum_ {k = 1} ^ {K} \vec {\phi} _ {k} * \theta_ {k} \tag {10}
$$
where  $\theta_{k}$  is the topic proportion of the  $k$ -th topic. Then,  $\vec{\theta}_{f}$  and  $\vec{d}_{f}$  are concatenated to form the fake distribution pair  $\vec{p}_{f}$  as shown in Figure 3. And encoder and discriminator of Gaussian-BAT are same as BAT, shown as Figure 2. In our experiments, the pre-trained 300-dimensional Glove (Pennington et al., 2014) embedding is used.
3.3 Objective and Training Procedure
In Figure 2, the real distribution pair  $\vec{p_r} = [\vec{\theta}_r;\vec{d_r} ]$  and the fake distribution pair  $\vec{p_f} = [\vec{\theta_f};\vec{d_f} ]$  can be viewed as random samples drawn from two  $(K + V)$  -dimensional joint distributions  $\mathbb{P}_r$  and  $\mathbb{P}_f$  , each of them comprising of a  $K$  -dimensional Dirichlet distribution and a  $V$  -dimensional Dirichlet distribution. The training objective of BAT and Gaussian-BAT is to make the generated joint distribution  $\mathbb{P}_f$  close to the real joint distribution  $\mathbb{P}_r$  as much as possible. In this way, a two-way projection between document-topic distribution and document-word distribution could be built by the learned encoder and generator.
To measure the distance between  $\mathbb{P}_r$  and  $\mathbb{P}_f$ , we use the Wasserstein-distance as the optimization objective, since it was shown to be more effective compared to Jensen-Shannon divergence (Arjovsky et al., 2017):
$$
L o s s = \mathbb {E} _ {\vec {p} _ {f} \sim \mathbb {P} _ {f}} [ D (\vec {p} _ {f}) ] - \mathbb {E} _ {\vec {p} _ {r} \sim \mathbb {P} _ {r}} [ D (\vec {p} _ {r}) ] \tag {11}
$$
where  $D(\cdot)$  represents the output signal of the discriminator. A higher value denotes that the discriminator is more prone to consider the input as a real distribution pair and vice versa. In addition, we use weight clipping which was proposed to ensure the Lipschitz continuity (Arjovsky et al., 2017) of  $D$ .
The training procedure of BAT and Gaussian-BAT is given in Algorithm 1. Here,  $c$  is the clipping parameter,  $n_d$  represents the number of discriminator iterations per generator iteration,  $m$  is the batch size,  $\alpha_{1}$  is the learning rate,  $\beta_{1}$  and  $\beta_{2}$  are hyper-parameters of Adam (Kingma and Ba, 2014), and  $p_a$  represents  $\{\alpha_1,\beta_1,\beta_2\}$ . In our experiments, we set the  $n_d = 5$ ,  $m = 64$ ,  $\alpha_{1} = 1e - 4$ ,  $c = 0.01$ ,  $\beta_{1} = 0.5$  and  $\beta_{2} = 0.999$ .
3.4 Topic Generation and Cluster Inference
After model training, learned  $G$  and  $E$  will build a two-way projection between document-topic distribution and document-word distribution. Thus,  $G$  and  $E$  could be used for topic generation and cluster inference.
To generate the word distribution of each topic, we use  $\vec{t} s_{(k)}$ , a  $K$ -dimensional vector, as the one-hot encoding of the  $k$ -th topic. For example,  $\vec{t} s_2 = [0,1,0,0,0,0]^{\mathrm{T}}$  in a six topic setting. And the word
distribution of the  $k$ -th topic is obtained by:
$$
\vec {\phi} _ {k} = G (\vec {t} s _ {(k)}) \tag {12}
$$
Likewise, given the document representation  $\vec{d}_r$ , topic distribution  $\vec{\theta}_r$  obtained by BAT/Gaussian-BAT could be used for cluster inference based on:
$$
\vec {\theta} _ {r} = E \left(\vec {d} _ {r}\right); \quad c _ {r} = \arg \max  \vec {\theta} _ {r} \tag {13}
$$
where  $c_{r}$  denotes the inferred cluster of  $\vec{d}_r$ .
In this section, we first present the experimental setup which includes the datasets used and the baselines, followed by the experimental results.
We evaluate BAT and Gaussian-BAT on three datasets for topic extraction and text clustering, 20Newsgroups $^{1}$ , Grolier $^{2}$  and NYTimes $^{3}$ . Details are summarized below:
20Newsgroups (Lang, 1995) is a collection of approximately 20,000 newsgroup articles, partitioned evenly across 20 different newsgroups.
Grolier is built from Grolier Multimedia Encyclopedia, which covers almost all the fields in the world. NYTimes is a collection of news articles published between 1987 and 2007, and contains a wide range of topics, such as sports, politics, education, etc.
We use the full datasets of 20Newsgroups<sup>1</sup> and Grolier<sup>2</sup>. For the NYTimes dataset, we randomly select 100,000 articles and remove the low frequency words. The final statistics are shown in Table 1:
Dataset	#Doc (Train)	#Doc (Test)	#Words
20Newsgroups	11,259	7,488	1,995
Grolier	29,762	-	15,276
NYtimes	99,992	-	12,604
Table 1: The statistics of datasets.