## 1 Introduction  
Deep learning has been successfully applied to various tasks of image and speech technologies in recent decades. Their success has influenced the research community to make use of these techniques in speaker recognition tasks [1; 2; 3; 4; 5]. Deep learning has been applied to extracting bottle neck features (BNF) and then compute Gaussian Mixture Models (GMM) posterior probabilities in a hybrid Deep Neural Network-Hidden Markov Model (DNN-HMM) model [6; 7]. At the front end, deep learning is capable of learning deep features from acoustic features, which are used in several speaker recognition tasks [5; 8; 9; 10]. Deep learning has also been applied to learning a vector representation of a speaker for speaker verification, such as in References [5; 12; 13; 14]. There are some interesting works that address the performance loss on degraded speech condition and acoustic mismatch between enrollment and test phases of speaker recognition systems [15; 16]. Also, there are several recent approaches to obtaining fast training, for example, the Extreme Learning Machine (ELM), which has been extremely efficient in representational learning and several other learning tasks [17; 18; 19]. Unsupervised deep learning architectures like Restricted Boltzmann Machines (RBMs), Deep Belief Networks (DBNs) and Deep Autoencoders have the ability of representational learning power. A first attempt to use RBMs at the backend in a speaker verification task was made in Reference [20]. The authors put their efforts into the front end of a speaker verification system, in order to learn a compact and fixed dimensional speaker representation in the form of a speaker vector by means of RBM adaptation [21; 22; 23; 24]. They also make use of DBNs in the i-vector/PLDA (Probabilistic Linear Discriminant Analysis) framework for speaker verification at the backend [25]. The vector representation of speakers in Reference [22], was referred to as an RBM vector. It has been shown that the RBM vectors can extract speaker specific information that can be competitive as compared to i-vector based speaker verification. This has led us to apply this kind of vector representation of speakers to the tasks of speaker clustering and speaker tracking. Speaker clustering refers to the task of grouping speech segments in order to have segments from the same speaker in the same group. Ideally each group or cluster must contain speech segments that belong to the same speaker. On the other hand, utterances from the same speakers must not be distributed among multiple clusters. Several approaches to speaker clustering tasks exist, for example, cost optimization, sequential and Agglomerative Hierarchical Clustering (AHC) [26; 27; 28; 29]. Some approaches rely on commonly used statistical speaker modeling like Gaussian Mixture Models (GMMs), while others use features extracted using Deep Neural Networks (DNNs). For example, in Reference [9], BN features extracted from different DNNs are used for speaker clustering using an AHC approach. In certain applications, a person of interest (target) is tracked in an audio file, by using his voice characteristics. To identify _when the target speaker speaks_ in the audio, is a speaker tracking task [30]. The main stages in speaker tracking are to determine the positions in the audio where the speaker changes occur, that is, speaker segmentation, and to then identify the speaker, that is, speaker identification. Based on these two stages, there can be a joint or a separate approach to a speaker tracking task. In the past, several approaches to speaker tracking were proposed based on different segmentation and speaker modeling strategies. In order to detect speaker changes, a fixed length window is slid over the audio and a distance metric is computed between consecutive windows. This distance is set to a threshold to decide whether there exists a speaker change. In Reference [31], speaker change detection is performed using a Generalized Likelihood Ratio [32; 33].