Original: d-DeVIS: A Gray Box Interpretable Visual Debugging Approach for Deep Sequence Learning Model
Revised: d-DeVIS: A Gray Box Interpretable Visual Debugging Approach for Deep Sequence Learning Model

Original: Abstract
Revised: Abstract

Original: The widespread usability of Deep Learning algorithms to solve various machinelearning problems demands transparent decision making policy.
Revised: The widespread usability of Deep Learning algorithms to solve various machinelearning problems demands transparent decision making policy.

Original: Deep Learningalgorithms are considered as  black box type learning model and they are toocomplex to understand.
Revised: Deep Learningalgorithms are considered as a black box type learning model and they are toocomplex to understand.

Original: Moreover, the learning models trained on sequential data,such as audio data, has intricate reasoning process due to their complex distributionof features.
Revised: Moreover, the learning models trained on sequential data,such as audio data, has intricate reasoning process due to their complex distributionof features.

Original: Thus, a visual simulator might be helpful to trace the internal decisionmaking mechanisms in response of adversarial input data, and thus, helps to debugand design appropriate deep learning models.
Revised: Thus, a visual simulator might be helpful to trace the internal decisionmaking mechanisms in response of adversarial input data, and thus, helps to debugand design appropriate deep learning models.

Original: However, interpreting the reasoningof the hidden layer is not well studied in the literature.
Revised: However, interpreting the reasoningof the hidden layer is not well studied in the literature.

Original: Hence, in this work, wehave developed a visual interactive web application, namely d-DeVIS, which helpsto visualize the internal reasoning of a learning model which is trained on the audiodata.
Revised: Hence, in this work, wehave developed a visual interactive web application, namely d-DeVIS, which helpsto visualize the internal reasoning of a learning model which is trained on the audiodata.

Original: d-DeVIS allows to perceive the behavior as well as to debug the model byinteractively generating adversarial audio data point.
Revised: d-DeVIS allows to perceive the behavior as well as to debug the model byinteractively generating adversarial audio data point.

Original: {+A demo video description ofd-DeVIS can be found here: https://youtu.be/cHuNhppYrf
+}
Revised: {+A demo video description ofd-DeVIS can be found here: https://youtu.be/cHuNhppYrf
+}

Original:  Introduction
Revised:  Introduction

Original: Machine Learning(ML) have been pouring the blessings in a form of solving Artiﬁcial Intelligence(AI)problems, such as classiﬁcation, clustering, genomics data visualization.
Revised: Machine Learning(ML) have been pouring the blessings in a form of solving Artiﬁcial Intelligence(AI)problems, such as classiﬁcation, clustering, genomics data visualization.

Original: Deep Learning(DL), aninﬂuential extension of ML, which is evolving rapidly in recent years and successfully applied insolving various real-world problems, for instance machine translation, speech recognition, imageclassiﬁcation, etc.
Revised: Deep Learning(DL), aninﬂuential extension of ML, which is evolving rapidly in recent years and successfully applied insolving various real-world problems, for instance machine translation, speech recognition, imageclassiﬁcation etc.

Original: While previously ML models requires the external domain knowledge, DL ismostly characterized for efﬁcient learning of the non-linear complex feature representation withoutdomain expert.
Revised: While previously ML models requires the external domain knowledge, DL ismostly characterized for efﬁcient learning of the non-linear complex feature representation withoutdomain expert.

Original: Due to the complex feature representation, it is tough to visualize a deep learningmodel and thus people tends to use the DL model as a black-box learning approach.
Revised: Due to the complex feature representation, it is tough to visualize a deep learningmodel and thus people tends to use the DL model as a black-box learning approach.

Original: As a result, theinterpretability and transparency of DL models are reduced signiﬁcantly [5].
Revised: As a result, theinterpretability and transparency of DL models are reduced signiﬁcantly [5].

Original: With the increasing use of the DL methodologies in real-world system, such as self driving car andmedical imaging, it becomes a prime concern to have publicly understandable systems explaining theunderlying reasoning.
Revised: With the increasing use of the DL methodologies in real-world system, such as self driving car andmedical imaging, it becomes a prime concern to have publicly understandable systems explaining theunderlying reasoning.

Original: Although the linear systems can be easily demonstrated with simple exampleshaving mathematical proofs, non-linear systems like Deep Neural Network(DNN) is complex tounderstand and visualize.
Revised: Although the linear systems can be easily demonstrated with simple exampleshaving mathematical proofs, non-linear systems like Deep Neural Network(DNN) is complex tounderstand and visualize.

Original: Nonetheless, the general users as well as researchers need to understandthe mechanism of the algorithms for debugging and model selection purposes.
Revised: Nonetheless, the general users as well as researchers need to understandthe mechanism of the algorithms for debugging and model selection purposes.

Original: In addition, theteachers and the learners are interested to visualize the algorithms to develop the basic intuition ofthe algorithm.
Revised: In addition, theteachers and the learners are interested to visualize the algorithms to develop the basic intuition ofthe algorithm.

Original: As it is proven in the literature that people can develop a mental model of an algorithm when they are taught using visualization approaches [8][3], the researchers have been working toutilize the visualization approaches to teach the ML algorithms [10].
Revised: As it is proven in the literature that people can develop a mental model of an algorithmwhen they are taught using visualization approaches [8][3], the researchers have been working toutilize the visualization approaches to teach the ML algorithms [10].

Original: Visualizing Neural Network(NN) methodologies is not a new talk in the research of visual learning[2].
Revised: Visualizing Neural Network(NN) methodologies is not a new talk in the research of visual learning[2].

Original: Previously most of the approaches utilize the static and Black Box visualization approaches tovisualize the hidden layer features representations and the relationships between input and output ofthe learning model.
Revised: Previously most of the approaches utilize the static and Black Box visualization approaches tovisualize the hidden layer features representations and the relationships between input and output ofthe learning model.

Original: However, it is paramount important to visualize the DL techniques using gray boxapproach so that user can understand the global behavior and the granular features representations toimprove the explain-ability of DL models.
Revised: However, it is paramount important to visualize the DL techniques using gray boxapproach so that user can understand the global behavior and the granular features representations toimprove the explain-ability of DL models.

Original: Previously, in the rule-based learning models it was easy to understand the underlying reasoning andvisualize the model behavior.
Revised: Previously, in the rule-based learning models it was easy to understand the underlying reasoning andvisualize the model behavior.

Original: Inspiring from this system, we designed d-DeVIS : Deep Sequence
Revised: Inspiring from this system, we designed d-DeVIS : Deep Sequence

Original: Learning Model’s Debugger and Visual Interactive Simulator.
Revised: Learning Model’s Debugger and Visual Interactive Simulator.

Original: In this work, we are focusing on greybox concept, where the outcome of internal architecture blocks are transparent to the users.
Revised: In this work, we are focusing on greybox concept, where the outcome of internal architecture blocks are transparent to the users.

Original: Here,we are interested to visualize the internal features representation of a Convolutional Neural Network(CNN) model in the response of multimodal audio sequence data.
Revised: Here,we are interested to visualize the internal features representation of a Convolutional Neural Network(CNN) model in the response of multimodal audio sequence data.

Original: The layer wise visualization ofhidden feature representation and the hyper-parameters assists to understand and the mathematicalinterpretations of the DL models.
Revised: The layer wise visualization ofhidden feature representation and the hyper-parameters assists to understand and the mathematicalinterpretations of the DL models.

Original: The main contributions of the d-DeVIS are as:
Revised: The main contributions of the d-DeVIS are as:

Original: • A web-based application,d-DeVIS , to visualize the hidden layers features representationsand the behavior of of a CNN model in response to the adversarial audio sequence data.
Revised: • A web-based application,d-DeVIS , to visualize the hidden layers features representationsand the behavior of of a CNN model in response to the adversarial audio sequence data.

Original: • d-DeVIS allows user to interactively change the audio features, such as pitch, amplitude etc,
Revised: • d-DeVIS allows user to interactively change the audio features, such as pitch, amplitude etc,

Original: and interpret the the behavior of the learning model based on the modiﬁed data.
Revised: and interpret the the behavior of the learning model based on the modiﬁed data.

Original: • We designed a visually transparent debugging UI, which demonstrates layer-wise featuresrepresentations and model hyper parameters.
Revised: • We designed a visually transparent debugging UI, which demonstrates layer-wise featuresrepresentations and model hyper parameters.

Original: Thus, it guides DL model’s debugging.
Revised: Thus, it guides DL model’s debugging.

Original: • d-DeVIS enables users to hear and visualize the intermediary hidden layer results in order
Revised: • d-DeVIS enables users to hear and visualize the intermediary hidden layer results in order

Original: to interpret the ﬁnal prediction.
Revised: to interpret the ﬁnal prediction.

Original: It also allows to compare the performance of the learning
Revised: It also allows to compare the performance of the learning

Original: model in response to different adversarial audio input.
Revised: model in response to different adversarial audio input.

Original: The rest of the works are designed as follows.
Revised: The rest of the works are designed as follows.

Original: In Section 2, we have discussed the related works.
Revised: In Section 2, we have discussed the related works.

Original: Thereafter, Section ??
Revised: Thereafter, Section ??

Original: is designed with the goals and features of the proposed system.
Revised: is designed with the goals and features of the proposed system.

Original: Section 3 and
Revised: Section 3 and

Original: Section 4 describe the detailed proposed system and use case, respectively.
Revised: Section 4 describe the detailed proposed system and use case, respectively.

Original: Finally, Section 5 hasconcluded the work with future plans.
Revised: Finally, Section 5 hasconcluded the work with future plans.

Original:  Related Work
Revised:  Related Work

Original: The recent widespread use of deep learning models in various artiﬁcial intelligence task attracts boththe visualization and the deep learning communities to tackle the new challenge of improving theinterpretability and explainability of these models [2].
Revised: The recent widespread use of deep learning models in various artiﬁcial intelligence task attracts boththe visualization and the deep learning communities to tackle the new challenge of improving theinterpretability and explainability of these models [2].

Original: However, visualizing the Neural Network(NN)models is not a new research areas, it has been studied well before the recent surge of deep learningmodels.
Revised: However, visualizing the Neural Network(NN)models is not a new research areas, it has been studied well before the recent surge of deep learningmodels.

Original: For instance, N 2 Vis [10] visualizes the attributes of NN, such as hidden layers weights,weights’ volatility, network structure and nodal activation levels.
Revised: For instance, N 2 Vis [10] visualizes the attributes of NN, such as hidden layers weights,weights’ volatility, network structure and nodal activation levels.

Original: Nonetheless, most of the previousapproaches utilize the static graphical visualization to describe hidden reasoning of the learningmodels.
Revised: Nonetheless, most of the previousapproaches utilize the static graphical visualization to describe hidden reasoning of the learningmodels.

Original: In recent years, a few works address explainability and transparency issue of the DL models anddesigned interactive DL models visualization models to depict underline reasoning.
Revised: In recent years, a few works address explainability and transparency issue of the DL models anddesigned interactive DL models visualization models to depict underline reasoning.

Original: For example,
Revised: For example,

Original: Tensorﬂow Playground [9] designed an interactive interface, where user can change the parametersand structure of the NN models and examine their effect.
Revised: Tensorﬂow Playground [9] designed an interactive interface, where user can change the parametersand structure of the NN models and examine their effect.

Original: Moreover, ShapeShop [4], enables user tointeractively change input image and visualize the behavior of the DL models and it’s internal featuresrepresentations.
Revised: Moreover, ShapeShop [4], enables user tointeractively change input image and visualize the behavior of the DL models and it’s internal featuresrepresentations.

Original: Similarly, in [1], author designed an application, which allows user to examine thebehavior of a DL based image classiﬁcation models.
Revised: Similarly, in [1], author designed an application, which allows user to examine thebehavior of a DL based image classiﬁcation models.

Original: Apart from these black-box application wise visualization approaches, a few works visualize thebehavior of deep learning models.
Revised: Apart from these black-box application wise visualization approaches, a few works visualize thebehavior of deep learning models.

Original: For instance, in [6] author presents a static visualization of hiddenstate representation and the prediction model behavior of Long-Short-Term-Memory(LSTM) basedlanguage model.
Revised: For instance, in [6] author presents a static visualization of hiddenstate representation and the prediction model behavior of Long-Short-Term-Memory(LSTM) basedlanguage model.

Original: Similar to the previous work, LSTMVis [12], designed an interactive visualizationapproach to visualize the hidden state representations of recurrent neural network and allows user toexamine the internal behavior of LSTM model on different application scenarios.
Revised: Similar to the previous work, LSTMVis [12], designed an interactive visualizationapproach to visualize the hidden state representations of recurrent neural network and allows user toexamine the internal behavior of LSTM model on different application scenarios.

Original: Additionally, in [7]and [13], author visualize the Convolutional Neural Network(CNN) and provide visually explainablereasoning of internal feature representation.
Revised: Additionally, in [7]and [13], author visualize the Convolutional Neural Network(CNN) and provide visually explainablereasoning of internal feature representation.

Original: Furthermore, Seq2Seq[11] designed a visual debuggingtools for the sequence-
Revised: Furthermore, Seq2Seq[11] designed a visual debuggingtools for the sequence-to

Original: [Table]
Revised: [Table]

Original: d-DeVIS shows how deep learning models work and how they extract features in the hidden layers which adds transparency for the users as they can easily interpret the functionality of the training model.
Revised: d-DeVIS shows how deep learning models work and how they extract features in the hidden layers which adds transparency for the users as they can easily interpret the functionality of the training model.

Original: [Table]
Revised: [Table]

Original: A good grasp of the feature extraction method of deep neural networks is required for any DL enthusiast and d-DeVIS will be able to give them the means for a
Revised: A good grasp of the feature extraction method of deep neural networks is required for any DL enthusiast and d-DeVIS will be able to give them the means for a

Original: ﬂuid
Revised: ﬂuid

Original: gray box debugging experience as they can see how 
Revised: gray box debugging experience as they can see how much

Original: much hidden layers
Revised: hidden layers

Original: affect the training
Revised: affect the training.

Original: [Table]
Revised: [Table]

Original: : Interactively Examining
Revised: : Interactively Examining

Original: [Table]
Revised: [Table]

Original: We provide an interactive web tool for users to change audio features and manipulate different parameters of the waveform.
Revised: We provide an interactive web tool for users to change audio features and manipulate different parameters of the waveform.

Original: Simple clicks are enough to examine various behaviors of audio sequence data, so it provides a fast learning experience
Revised: Simple clicks are enough to examine various behaviors of audio sequence data, so it provides a fast learning experience

Original: [Table]
Revised: [Table]

Original: Comparison and exposure of what
Revised: Comparison and exposure of what

Original: features CNN
Revised: features CNN

Original: [Table]
Revised: [Table]

Original: Our Interactive web tool will let users listen to the extracted audio data from different ﬁlters of each layers of Convolutional Neural
Revised: Our Interactive web tool will let users listen to the extracted audio data from different ﬁlters of each layers of Convolutional Neural

Original: Network and the users will be able to get a grasp of the audio features that the hidden layers extract.
Revised: Network and the users will be able to get a grasp of the audio features that the hidden layers extract.

Original: Inspired from the previous works [4][12][11], we designed an interactive visualization system, dDeVIS :
Revised: Inspired from the previous works [4][12][11], we designed an interactive visualization system, dDeVIS :

Original: Deep Sequence Learning Model’s Debugger and Visually Interactive Simulator.
Revised: Deep Sequence Learning Model’s Debugger and Visually Interactive Simulator.

Original: Mostof the previous works utilize the black box visualization approaches to help developing the basicintuition of the deep learning models.
Revised: Mostof the previous works utilize the black box visualization approaches to help developing the basicintuition of the deep learning models.

Original: Surprisingly, visualizing the deep learning model behavior andfeature representation of the multimodal data, such as audio and visual, is neglected in the literature.
Revised: Surprisingly, visualizing the deep learning model behavior andfeature representation of the multimodal data, such as audio and visual, is neglected in the literature.

Original: Moreover, visualizing the correlation between the hidden layer features representations and the modelbehavior is not properly studied for sequence based models.
Revised: Moreover, visualizing the correlation between the hidden layer features representations and the modelbehavior is not properly studied for sequence based models.

Original: d-DeVIS allows user to interactivelychange the multimodal audio data and enables users to examine the deep learning model behavior tovisualize the feature representations.
Revised: d-DeVIS allows user to interactivelychange the multimodal audio data and enables users to examine the deep learning model behavior tovisualize the feature representations.

Original:  Proposed System
Revised:  Proposed System

Original: In this section, we present the key components of our visualizer.
Revised: In this section, we present the key components of our visualizer.

Original: In order for the interactivity of theusers and ﬂexibility of the system, we made a web application which shows the gray box debuggingmethod for deep neural network of sequence data.
Revised: In order for the interactivity of theusers and ﬂexibility of the system, we made a web application which shows the gray box debuggingmethod for deep neural network of sequence data.

Original: The complete system is composed of threeimportant steps:
Revised: The complete system is composed of threeimportant steps:

Original: • Audio Feature Manipulation
Revised: • Audio Feature Manipulation

Original: • Model Visualization.
Revised: • Model Visualization.

Original: • Extracted Feature Comparison
Revised: • Extracted Feature Comparison

Original: Before discussing our proposed system, we ﬁrst illustrate the goals and features of d-DeVIS -
Revised: Before discussing our proposed system, we ﬁrst illustrate the goals and features of d-DeVIS -

Original: 3.1 Goals
Revised: 3.1 Goals

Original: The prime goal of designing d-DeVIS is to make the learning experience of DL model easier andlearn structures of DNN and feature representations of hidden layers with the help of an interactiveuser interface.
Revised: The prime goal of designing d-DeVIS is to make the learning experience of DL model easier andlearn structures of DNN and feature representations of hidden layers with the help of an interactiveuser interface.

Original: All the goals are described in the Table 

Revised: All the goals are described in the Table

Original: 3.2 Features
Revised: 3.2 Features

Original: We designed d-DeVIS applications by considering the above design goals.
Revised: We designed d-DeVIS applications by considering the above design goals.

Original: In Table 2, we discuss allthe features of d-DeVIS and how the features meet the design goals.
Revised: In Table 2, we discuss allthe features of d-DeVIS and how the features meet the design goals.

Original: [Table]
Revised: [Table]

Original: Visualizing the Hidden Extracted Features of Convolutional neural networks: d-DeVIS provides visualization of the extracted features in each layers as image data and shows the various features of different ﬁlters of the deep convolutional network.
Revised: Visualizing the Hidden Extracted Features of Convolutional neural networks: d-DeVIS provides visualization of the extracted features in each layers as image data and shows the various features of different ﬁlters of the deep convolutional network.

Original: Users can also hear the audio representation of the hidden extracted features
Revised: Users can also hear the audio representation of the hidden extracted features

Original: G1 & G & G
Revised: G1 & G & G

Original: Interactive User Experience:
Revised: Interactive User Experience:

Original: For a ﬂuid user experience,we provide an interactive platform for the users so that they will be able to focus on the productivity of the system without any unnecessary hassles.
Revised: For a ﬂuid user experience,we provide an interactive platform for the users so that they will be able to focus on the productivity of the system without any unnecessary hassles.

Original: G3 &
Revised: G3 &

Original: [Table]
Revised: [Table]

Original: Visualizing the Audio Features as well as Modifying the Waveforms: Due to the complex structure of audio data, our system let’s users modify various aspects of the sound property and visualize the updated waveform to provide a keen knowledge on audio data representation.
Revised: Visualizing the Audio Features as well as Modifying the Waveforms: Due to the complex structure of audio data, our system let’s users modify various aspects of the sound property and visualize the updated waveform to provide a keen knowledge on audio data representation.

Original: G
Revised: G

Original: [Table]
Revised: [Table]

Original: Custom Audio Input for Testing and Feature Distribution Visualization: User can not only upload a default audio data but also they can record custom speech to test the trained model.
Revised: Custom Audio Input for Testing and Feature Distribution Visualization: User can not only upload a default audio data but also they can record custom speech to test the trained model.

Original: Proper distribution of the weights is also visualized.
Revised: Proper distribution of the weights is also visualized.

Original: G1 &
Revised: G1 &

Original: [Table]
Revised: [Table]

Original: Comparing different audio inputs and their hidden features: d-DeVIS also enables users to measure the differences of different audio inputs and check their extracted layer features.
Revised: Comparing different audio inputs and their hidden features: d-DeVIS also enables users to measure the differences of different audio inputs and check their extracted layer features.

Original: G
Revised: G

Original: 3.3 Implementation of d-DeVIS
Revised: 3.3 Implementation of d-DeVIS

Original: We have developed a web based deep learning model visualizer where users can change the audio features and examine the behavior of the model.
Revised: We have developed a web based deep learning model visualizer where users can change the audio features and examine the behavior of the model.

Original: In the following section we present the implementationdetails of d-DeVIS .
Revised: In the following section we present the implementationdetails of d-DeVIS .

Original: 3.3.1 Front-end and Back-end of d-DeVIS
Revised: 3.3.1 Front-end and Back-end of d-DeVIS

Original: Audio ﬁles are manipulated such as Slicing the audio, Changing in Loudness, Cross-fading, Repeatingthe Sound, Invert and Fade using Numpy, Pydub and Scipy.
Revised: Audio ﬁles are manipulated such as Slicing the audio, Changing in Loudness, Cross-fading, Repeatingthe Sound, Invert and Fade using Numpy, Pydub and Scipy.

Original: Matplotlib is utilized to visualize theaudio features.
Revised: Matplotlib is utilized to visualize theaudio features.

Original: After the training the model, we have saved the data using Pickle python module.
Revised: After the training the model, we have saved the data using Pickle python module.

Original: We have used
Revised: We have used

Original: HTML5, Bootstrap, Javascript for designing the front-end of the application.
Revised: HTML5, Bootstrap, Javascript for designing the front-end of the application.

Original: We have also usedpython ﬂask web framework that helped to maintain the interaction between front-end and back-endcommunication.
Revised: We have also usedpython ﬂask web framework that helped to maintain the interaction between front-end and back-endcommunication.

Original: 2 Trained CNN Model
Revised: 2 Trained CNN Model

Original: We trained a CNN model on Speech Commands dataset 1 , which is used to visualize the behavior ofthe model.
Revised: We trained a CNN model on Speech Commands dataset 1 , which is used to visualize the behavior ofthe model.

Original: We have used Keras deep learning framework which is a wrapper library of Tensorﬂow totrain our models.
Revised: We have used Keras deep learning framework which is a wrapper library of Tensorﬂow totrain our models.

Original: We have also utilized the computation system of Google Colaboratory platform.
Revised: We have also utilized the computation system of Google Colaboratory platform.

Original: The dataset consists of almost 30 speech classes but for the sake of the simplicity and runtime weused 10 classes which are English audio recordings of zero to nine digits.
Revised: The dataset consists of almost 30 speech classes but for the sake of the simplicity and runtime weused 10 classes which are English audio recordings of zero to nine digits.

Original: All the clips were 2 secondslong.
Revised: All the clips were 2 secondslong.

Original: And we calculated logarithmic spectrogram as features of the audio (.wav) data to feed intothe training model.
Revised: And we calculated logarithmic spectrogram as features of the audio (.wav) data to feed intothe training model.

Original: For learning algorithm, a three layer Convolutional Neural Network (CNN) isused as the spectrogram feature matrix can represent an image and CNNs have proven to be decentat image classiﬁcation.
Revised: For learning algorithm, a three layer Convolutional Neural Network (CNN) isused as the spectrogram feature matrix can represent an image and CNNs have proven to be decentat image classiﬁcation.

Original: Our baseline model reached 95% validation accuracy with a minimal hyperparameter tuning.
Revised: Our baseline model reached 95% validation accuracy with a minimal hyperparameter tuning.

Original: 3.3.3 Extracted Hidden Layer Features
Revised: 3.3.3 Extracted Hidden Layer Features

Original: During any forward propagation step, the spectrogram feature data of the audio ﬁles are traversed intothe hidden layer of the CNN.
Revised: During any forward propagation step, the spectrogram feature data of the audio ﬁles are traversed intothe hidden layer of the CNN.

Original: At each layer, the convolution matrix tries to extract signiﬁcant hiddenfeatures from the input matrix and optimizes itself during backward propagation in order to give the best training prediction.
Revised: At each layer, the convolution matrix tries to extract signiﬁcant hiddenfeatures from the input matrix and optimizes itself during backward propagation in order to give the best training prediction.

Original: In our system, the users will be able to upload an audio ﬁle of English digitsas well as record an audio of their own.
Revised: In our system, the users will be able to upload an audio ﬁle of English digitsas well as record an audio of their own.

Original: After the input of the audio ﬁle, our system will calculate thelogarithmic spectrogram and feed it to the trained model to give the prediction.
Revised: After the input of the audio ﬁle, our system will calculate thelogarithmic spectrogram and feed it to the trained model to give the prediction.

Original: At each convolutionlayer there are predeﬁned tunned ﬁlters.
Revised: At each convolutionlayer there are predeﬁned tunned ﬁlters.

Original: The types of features it extracts from the input data is relatedto the ﬁlters chosen by user.
Revised: The types of features it extracts from the input data is relatedto the ﬁlters chosen by user.

Original: In our model, the ﬁrst layer and second layers have 16 ﬁlters, the thirdlayer has 32 ﬁlters, each.
Revised: In our model, the ﬁrst layer and second layers have 16 ﬁlters, the thirdlayer has 32 ﬁlters, each.

Original: So, our system visualizes the features corresponding to the ﬁlters and alsothe distributions of the trained weights.
Revised: So, our system visualizes the features corresponding to the ﬁlters and alsothe distributions of the trained weights.

Original: A particular feature extracted by the 13th ﬁlter of ﬁrst layer is visualized in Fig 1.
Revised: A particular feature extracted by the 13th ﬁlter of ﬁrst layer is visualized in Fig 1.

Original: The predictionof the input audio digit is 0.
Revised: The predictionof the input audio digit is 0.

Original: The red number 1 marks the extracted feature image of given input bythe 13th ﬁlter.
Revised: The red number 1 marks the extracted feature image of given input bythe 13th ﬁlter.

Original: When a user clicks on the image it zooms in to show the spectrogram clearly and theheader marked by number 2 tells which speciﬁc ﬁlter is applied for the feature extraction.
Revised: When a user clicks on the image it zooms in to show the spectrogram clearly and theheader marked by number 2 tells which speciﬁc ﬁlter is applied for the feature extraction.

Original: Users canalso listen to the hidden extracted feature by clicking on the button indicated by number 3.
Revised: Users canalso listen to the hidden extracted feature by clicking on the button indicated by number 3.

Original: It allowsthem to get insight on the extraction method of the hidden neurons for sequence audio data.
Revised: It allowsthem to get insight on the extraction method of the hidden neurons for sequence audio data.

Original: 3.4 Model Visualization
Revised: 3.4 Model Visualization

Original: The primary purpose of our work is to show how the deep learning models learn.
Revised: The primary purpose of our work is to show how the deep learning models learn.

Original: In addition, itextracts the features from audio sequence data and visualizes the features extracted in each hiddenlayer during forward propagation.
Revised: In addition, itextracts the features from audio sequence data and visualizes the features extracted in each hiddenlayer during forward propagation.

Original: For that, we have built a baseline model which satisﬁes ourpurposes and showed the weight distributions as well as the features retrieved after each layers asimages. 
Revised: For that, we have built a baseline model which satisﬁes ourpurposes and showed the weight distributions as well as the features retrieved after each layers asimages. [Figure]

Revised: Manipulation

Original: [Figure]
Revised: [Figure]

Original: Our system enables the users to not only test a sample ﬁle or recording of their voice but also itprovides them the means to edit.
Revised: Our system enables the users to not only test a sample ﬁle or recording of their voice but also itprovides them the means to edit.

Original: They can manipulate features and parameters like pitch of the audio,cropping some portion of the audio, changing the trained weights and choosing the number of ﬁlters
Revised: They can manipulate features and parameters like pitch of the audio,cropping some portion of the audio, changing the trained weights and choosing the number of ﬁlters

Original: [Figure]
Revised: [Figure]

Original: to use to retrain the model.
Revised: to use to retrain the model.

Original: For every action, we show the results of how the prediction is changingand what type of new features the model is extracting.
Revised: For every action, we show the results of how the prediction is changingand what type of new features the model is extracting.

Original: 3.5.1 Audio Feature Manipulation
Revised: 3.5.1 Audio Feature Manipulation

Original: d-DeVIS allows users to manipulate different parameters of the waveform of a given input audio.
Revised: d-DeVIS allows users to manipulate different parameters of the waveform of a given input audio.

Original: They are given various options to change the audio in a way such that users will be able to observethe changes in extracted features and relate to how they are affected by the changes in the waveforms.
Revised: They are given various options to change the audio in a way such that users will be able to observethe changes in extracted features and relate to how they are affected by the changes in the waveforms.

Original: The various edit options of the system are mentioned below:
Revised: The various edit options of the system are mentioned below:

Original: Slicing the audio: Users can slice an audio.
Revised: Slicing the audio: Users can slice an audio.

Original: This option slices the audio and make an one
Revised: This option slices the audio and make an one

Original: second slice.
Revised: second slice.

Original: Cross-fading: This option changes the amplitude of the sound waves.
Revised: Cross-fading: This option changes the amplitude of the sound waves.

Original: Changing the loudness: This option will make the beginning louder and the ending quieter.
Revised: Changing the loudness: This option will make the beginning louder and the ending quieter.

Original: Repeating the sound: This option repeats the sound twice.
Revised: Repeating the sound: This option repeats the sound twice.

Original: Invert:
Revised: Invert:

Original: This option inverts the sound wave.
Revised: This option inverts the sound wave.

Original: If the sound is played, then it would startplaying from the ending.6.
Revised: If the sound is played, then it would start playing from the ending.6.

Original: Fade: This option does a fade in for one second and then a fade out for the other second.
Revised: Fade: This option does a fade in for one second and then a fade out for the other second.

Original: 3.6 Extracted Feature Comparison
Revised: 3.6 Extracted Feature Comparison

Original: d-DeVIS allows user to not only visualize the features extracted by the hidden layer ﬁlters, but also itenables them to listen to the audio representation of the extracted features of the input audio by the
Revised: d-DeVIS allows user to not only visualize the features extracted by the hidden layer ﬁlters, but also itenables them to listen to the audio representation of the extracted features of the input audio by the

Original: CNN.
Revised: CNN.

Original: So users can get a grasp of what type of features are extracted by different ﬁlters.
Revised: So users can get a grasp of what type of features are extracted by different ﬁlters.

Original: Users canalso input different audio ﬁles, see their predictions and measure the similarity and dissimilarity ofthe features extracted by the same ﬁlters.
Revised: Users canalso input different audio ﬁles, see their predictions and measure the similarity and dissimilarity ofthe features extracted by the same ﬁlters.

Original: It is illustrated in Fig 4, The red numbers 1 and 2 mark thetwo spectrogram representation of the two audio inputs with their predictions by our trained model.
Revised: It is illustrated in Fig 4, The red numbers 1 and 2 mark thetwo spectrogram representation of the two audio inputs with their predictions by our trained model.

Original: The button marked by 3 enables users to listen to the audio.
Revised: The button marked by 3 enables users to listen to the audio.

Original: There are three layers of the CNN andusers can see any layer features by clicking on the layer buttons marked by 4.
Revised: There are three layers of the CNN andusers can see any layer features by clicking on the layer buttons marked by 4.

Original: There are differentspectrogram images of the extracted features by every ﬁlters marked by 5 and users can click on themto listen to the audio representation.
Revised: There are differentspectrogram images of the extracted features by every ﬁlters marked by 5 and users can click on themto listen to the audio representation.

Original: Finally users can also see the weight distribution of each layersby clicking on the button marked by 6.
Revised: Finally users can also see the weight distribution of each layersby clicking on the button marked by 6.

Original:  Use Cases
Revised:  Use Cases

Original: While experimenting with the system, we have applied several modiﬁcations to both the CNN trainedmodel and the input audio ﬁle.
Revised: While experimenting with the system, we have applied several modiﬁcations to both the CNN trainedmodel and the input audio ﬁle.

Original: Thus, we have analyzed the results that we got due to the tunning withthe system.
Revised: Thus, we have analyzed the results that we got due to the tunning withthe system.

Original: In this section, we present important use cases that demonstrate the general applicabilityof our system.
Revised: In this section, we present important use cases that demonstrate the general applicabilityof our system.

Original:  Visualizing 
Revised: .1 Visualizing the Audio Features

Original: the Audio Features:

Original: Speech is a sequence data which is hard to grasp just by looking at the amplitude vs time representation.
Revised: Speech is a sequence data which is hard to grasp just by looking at the amplitude vs time representation.

Original: In our system, a user can upload or record a custom audio ﬁle and tune with various aspects of thewaveform.
Revised: In our system, a user can upload or record a custom audio ﬁle and tune with various aspects of thewaveform.

Original: So predictions of the audio will change with accordance to the change in the waveformsand users can easily see the changed results.
Revised: So predictions of the audio will change with accordance to the change in the waveformsand users can easily see the changed results.

Original: 4.2 Learning Medium for the Academia:
Revised: 4.2 Learning Medium for the Academia:

Original: Our system provides an interactive web application with which learners will be able to test varioustypes of aspects of audio data and the deep learning model.
Revised: Our system provides an interactive web application with which learners will be able to test varioustypes of aspects of audio data and the deep learning model.

Original: After the model training completion, theproper visualizations are given.
Revised: After the model training completion, theproper visualizations are given.

Original: The academia will be able to use it for teaching purpose.
Revised: The academia will be able to use it for teaching purpose.

Original: 4.3 Experimenting Platform for AI Enthusiasts
Revised: 4.3 Experimenting Platform for AI Enthusiasts:

Original: Due to our feature manipulation and interactivity of the system, the deep learning enthusiasts andengineers can do various experiments on it.
Revised: Due to our feature manipulation and interactivity of the system, the deep learning enthusiasts andengineers can do various experiments on it.

Original: We provide a platform for easy training and proper resultsof the feature extractions which are shown as a form of images.
Revised: We provide a platform for easy training and proper resultsof the feature extractions which are shown as a form of images.

Original: Users can test their own custom inputand experiment on various types of hyper-parameters from which it can get the maximum accuracy.
Revised: Users can test their own custom inputand experiment on various types of hyper-parameters from which it can get the maximum accuracy.

Original:  Conclusion and Future Work
Revised:  Conclusion and Future Work

Original: In this work, we developed a web-based interactive visualization of CNN model, d-DeVIS , whichenables user to examine it’s behavior in response to the audio sequence data.
Revised: In this work, we developed a web-based interactive visualization of CNN model, d-DeVIS , whichenables user to examine it’s behavior in response to the audio sequence data.

Original: d-DeVIS also allowsusers to interactively manipulate the audio sequence data and simultaneously visualizes the CNNmodel’s internal features representations, hyper-parameters and input and learned output representation relationships.
Revised: d-DeVIS also allowsusers to interactively manipulate the audio sequence data and simultaneously visualizes the CNNmodel’s internal features representations, hyper-parameters and input and learned output representation relationships.

Original: Hence, it helps users to develop a better intuition of the underlying reasoning of a
Revised: Hence, it helps users to develop a better intuition of the underlying reasoning of a

Original: DL model.
Revised: DL model.

Original: In future extension of d-DeVIS , we have the plan to visualize the other deep learning models behaviorallowing users to manipulate the input data representation interactively.
Revised: In future extension of d-DeVIS , we have the plan to visualize the other deep learning models behaviorallowing users to manipulate the input data representation interactively.

Original: Moreover, visualizing thehidden layer complex feature representations for multi-modal sequence data, is a great avenue forfuture research work.
Revised: Moreover, visualizing thehidden layer complex feature representations for multi-modal sequence data, is a great avenue forfuture research work.