Original: d-DeVIS: A Gray Box Interpretable Visual Debugging Approach for Deep Sequence Learning Model
Revised: d-DeVIS: A Gray Box Interpretable Visual Debugging Approach for Deep Sequence Learning Model

Original: Abstract
Revised: Abstract

Original: The widespread usability of Deep Learning algorithms to solve various machinelearning problems demands transparent decision making policy.
Revised: The widespread usability of Deep Learning algorithms to solve various machinelearning problems demands transparent decision making policy.

Original: Deep Learningalgorithms are considered as a black box type learning model and they are toocomplex to understand.
Revised: Deep Learningalgorithms are considered as a black box type learning model and they are toocomplex to understand.

Original: Moreover, the learning models trained on sequential data,such as audio data, has intricate reasoning process due to their complex distributionof features.
Revised: Moreover, the learning models trained on sequential data,such as audio data, has intricate reasoning process due to their complex distributionof features.

Original: Thus, a visual simulator might be helpful to trace the internal decisionmaking mechanisms in response of adversarial input data, and thus, helps to debugand design appropriate deep learning models.
Revised: Thus, a visual simulator might be helpful to trace the internal decisionmaking mechanisms in response of adversarial input data, and thus, helps to debugand design appropriate deep learning models.

Original: However, interpreting the reasoningof the hidden layer is not well studied in the literature.
Revised: However, interpreting the reasoningof the hidden layer is not well studied in the literature.

Original: Hence, in this work, wehave developed a visual interactive web application, namely d-DeVIS, which helpsto visualize the internal reasoning of a learning model which is trained on the audiodata.
Revised: Hence, in this work, wehave developed a visual interactive web application, namely d-DeVIS, which helpsto visualize the internal reasoning of a learning model which is trained on the audiodata.

Original: d-DeVIS allows to perceive the behavior as well as to debug the model byinteractively generating adversarial audio data point.
Revised: d-DeVIS allows to perceive the behavior as well as to debug the model byinteractively generating adversarial audio data point.

Original: A demo video description ofd-DeVIS can be found here: [-https://youtu.be/cHuNhppYrf
-]
Revised: A demo video description ofd-DeVIS can be found here: [-https://youtu.be/cHuNhppYrf
-]https://youtu.be/v8lgR-ses_Y

Original:  Introduction
Revised:  Introduction

Original: Machine Learning(ML) have been pouring the blessings in a form of solving Artiﬁcial Intelligence(AI)problems, such as classiﬁcation, clustering, genomics data visualization.
Revised: Machine Learning(ML) have been pouring the blessings in a form of solving Artiﬁcial Intelligence(AI)problems, such as classiﬁcation, clustering, genomics data visualization.

Original: Deep Learning(DL), aninﬂuential extension of ML, which is evolving rapidly in recent years and successfully applied insolving various real-world problems, for instance machine translation, speech recognition, imageclassiﬁcation etc.
Revised: Deep Learning(DL), aninﬂuential extension of ML, which is evolving rapidly in recent years and successfully applied insolving various real-world problems, for instance machine translation, speech recognition, imageclassiﬁcation etc.

Original: While previously ML models requires the external domain knowledge, DL ismostly characterized for efﬁcient learning of the non-linear complex features representation withoutdomain expert.
Revised: While previously ML models requires the external domain knowledge, DL ismostly characterized for efﬁcient learning of the non-linear complex features representation withoutdomain expert.

Original: Due to the complex feature representation, it is tough to visualize a deep learningmodel and thus people tends to use the DL model as a black-box learning approach.
Revised: Due to the complex feature representation, it is tough to visualize a deep learningmodel and thus people tends to use the DL model as a black-box learning approach.

Original: As a result,the interpretability and transparency of DL models are reduced signiﬁcantly [5].
Revised: As a result,the interpretability and transparency of DL models are reduced signiﬁcantly [5].

Original: Although, DLapproaches have been used widely, in the literature a few works address the interpretability issue ofdeep learning models.
Revised: Although, DLapproaches have been used widely, in the literature a few works address the interpretability issue ofdeep learning models.

Original: With the increasing use of the DL methodologies in real-world system, such as self driving car andmedical imaging, it becomes a prime concern to have publicly understandable systems explaining theunderlying reasoning.
Revised: With the increasing use of the DL methodologies in real-world system, such as self driving car andmedical imaging, it becomes a prime concern to have publicly understandable systems explaining theunderlying reasoning.

Original: Although the linear systems can be easily demonstrated with simple exampleshaving mathematical proofs, non-linear systems like Deep Neural Network(DNN) is complex tounderstand and visualize.
Revised: Although the linear systems can be easily demonstrated with simple exampleshaving mathematical proofs, non-linear systems like Deep Neural Network(DNN) is complex tounderstand and visualize.

Original: Nonetheless, the general users as well as researchers need to understandthe mechanism of the algorithms for debugging and model selection purposes.
Revised: Nonetheless, the general users as well as researchers need to understandthe mechanism of the algorithms for debugging and model selection purposes.

Original: In addition, theteachers and the learners are interested to visualize the algorithms to develop the basic intuition ofthe algorithm.
Revised: In addition, theteachers and the learners are interested to visualize the algorithms to develop the basic intuition ofthe algorithm.

Original: As it is proven in the literature that people can develop a mental model of an algorithm when they are taught using visualization approaches [8][3], the researchers have been working toutilize the visualization approaches to teach the ML algorithms [10].
Revised: As it is proven in the literature that people can develop a mental model of an algorithm when they are taught using visualization approaches [8][3], the researchers have been working toutilize the visualization approaches to teach the ML algorithms [10].

Original: Visualizing Neural Network(NN) methodologies is not a new talk in the research of visual learning[2].
Revised: Visualizing Neural Network(NN) methodologies is not a new talk in the research of visual learning[2].

Original: Previously most of the approaches utilize the static and Black Box visualization approaches tovisualize the hidden layer features representations and the relationships between input and output ofthe learning model.
Revised: Previously most of the approaches utilize the static and Black Box visualization approaches tovisualize the hidden layer features representations and the relationships between input and output ofthe learning model.

Original: However, it is paramount important to visualize the DL techniques using gray boxapproach so that user can understand the global behavior and the granular features representations toimprove the explain-ability of DL models.
Revised: However, it is paramount important to visualize the DL techniques using gray boxapproach so that user can understand the global behavior and the granular features representations toimprove the explain-ability of DL models.

Original: Previously, in the rule-based learning models it was easy to understand the underlying reasoning andvisualize the model behavior.
Revised: Previously, in the rule-based learning models it was easy to understand the underlying reasoning andvisualize the model behavior.

Original: Inspiring from this system, we designed d-DeVIS : Deep Sequence
Revised: Inspiring from this system, we designed d-DeVIS : Deep Sequence

Original: Learning Model’s Debugger and Visual Interactive Simulator.
Revised: Learning Model’s Debugger and Visual Interactive Simulator.

Original: In this work, we are focusing on greybox concept, where the outcome of internal architecture blocks are transparent to the users.
Revised: In this work, we are focusing on graybox concept, where the outcome of internal architecture blocks are transparent to the users.

Original: Here,we are interested to visualize the internal features representation of a Convolutional Neural Network(CNN) model in the response of multimodal audio sequence data.
Revised: Here,we are interested to visualize the internal features representation of a Convolutional Neural Network(CNN) model in the response of multimodal audio sequence data.

Original: The layer wise visualization ofhidden feature representation and the hyper-parameters assists to understand and the mathematicalinterpretations of the DL models.
Revised: The layer wise visualization ofhidden feature representation and the hyper-parameters assists to understand and the mathematicalinterpretations of the DL models.

Original: The main contributions of the d-DeVIS are as:
Revised: The main contributions of the d-DeVIS are as:

Original: • A web-based application,d-DeVIS , to visualize the hidden layers features representations
Revised: • A web-based application,d-DeVIS , to visualize the hidden layers features representations

Original: and the behavior of of a CNN model in response to the adversarial audio sequence data.
Revised: and the behavior of of a CNN model in response to the adversarial audio sequence data.

Original: • d-DeVIS allows user to interactively change the audio features, such as pitch, amplitude etc,
Revised: • d-DeVIS allows user to interactively change the audio features, such as pitch, amplitude etc,

Original: and interpret the the behavior of the learning model based on the modiﬁed data.
Revised: and interpret the the behavior of the learning model based on the modiﬁed data.

Original: • We designed a visually transparent debugging UI, which demonstrates layer-wise features
Revised: • We designed a visually transparent debugging UI, which demonstrates layer-wise features

Original: representations and model hyper parameters.
Revised: representations and model hyper parameters.

Original: Thus, it guides DL model’s debugging.
Revised: Thus, it guides DL model’s debugging.

Original: • d-DeVIS enables users to hear and visualize the intermediary hidden layer results in order
Revised: • d-DeVIS enables users to hear and visualize the intermediary hidden layer results in order

Original: to interpret the ﬁnal prediction.
Revised: to interpret the ﬁnal prediction.

Original: It also allows to compare the performance of the learning
Revised: It also allows to compare the performance of the learning

Original: model in response to different adversarial audio input.
Revised: model in response to different adversarial audio input.

Original: The rest of the works are designed as follows.
Revised: The rest of the works are designed as follows.

Original: In Section 2, we have discussed the related works.
Revised: In Section 2, we have discussed the related works.

Original: Thereafter, Section 3 is designed with the goals and features of the proposed system.
Revised: Thereafter, Section 3 is designed with the goals and features of the proposed system.

Original: Section 4describe the detailed proposed system and use case, respectively.
Revised: Section 4describe the detailed proposed system and use case, respectively.

Original: Finally, Section 5 has concludedthe work with future plans.
Revised: Finally, Section 5 has concludedthe work with future plans.

Original:  Related Work
Revised:  Related Work

Original: The recent widespread use of deep learning models in various artiﬁcial intelligence task attracts boththe visualization and the deep learning communities to tackle the new challenge of improving theinterpretability and explainability of these models [2].
Revised: The recent widespread use of deep learning models in various artiﬁcial intelligence task attracts boththe visualization and the deep learning communities to tackle the new challenge of improving theinterpretability and explainability of these models [2].

Original: However, visualizing the Neural Network(NN)models is not a new research areas, it has been studied well before the recent surge of deep learningmodels.
Revised: However, visualizing the Neural Network(NN)models is not a new research areas, it has been studied well before the recent surge of deep learningmodels.

Original: For instance, N 2 Vis [10] visualizes the attributes of NN, such as hidden layers weights,weights’ volatility, network structure and nodal activation levels.
Revised: For instance, N 2 Vis [10] visualizes the attributes of NN, such as hidden layers weights,weights’ volatility, network structure and nodal activation levels.

Original: Nonetheless, most of the previousapproaches utilize the static graphical visualization to describe hidden reasoning of the learningmodels.
Revised: Nonetheless, most of the previousapproaches utilize the static graphical visualization to describe hidden reasoning of the learningmodels.

Original: In recent years, a few works address explainability and transparency issue of the DL models anddesigned interactive visualization models to depict underline reasoning.
Revised: In recent years, a few works address explainability and transparency issue of the DL models anddesigned interactive visualization models to depict underline reasoning.

Original: For example, Tensorﬂow
Revised: For example, Tensorﬂow

Original: Playground [9] designed an interactive interface, where user can change the parameters and structureof the NN models and examine their effect.
Revised: Playground [9] designed an interactive interface, where user can change the parameters and structureof the NN models and examine their effect.

Original: Moreover, ShapeShop [4], enables user to interactivelychange input image and visualize the behavior of the DL models and it’s internal features representations.
Revised: Moreover, ShapeShop [4], enables user to interactivelychange input image and visualize the behavior of the DL models and it’s internal features representations.

Original: Similarly, in [1], author designed an application, which allows user to examine the behavior ofa DL based image classiﬁcation models.
Revised: Similarly, in [1], author designed an application, which allows user to examine the behavior ofa DL based image classiﬁcation models.

Original: Apart from these black-box visualization approaches, a few works visualize the behavior of deeplearning models.
Revised: Apart from these black-box visualization approaches, a few works visualize the behavior of deeplearning models.

Original: For instance, in [6] author presents a static visualization of hidden state representation and the prediction model behavior of Long-Short-Term-Memory(LSTM) based languagemodel.
Revised: For instance, in [6] author presents a static visualization of hidden state representation and the prediction model behavior of Long-Short-Term-Memory(LSTM) based languagemodel.

Original: Similar to the previous work, LSTMVis [12], designed an interactive visualization approachto visualize the hidden state representations of recurrent neural network and allows user to examinethe internal behavior of LSTM model on different application scenarios.
Revised: Similar to the previous work, LSTMVis [12], designed an interactive visualization approachto visualize the hidden state representations of recurrent neural network and allows user to examinethe internal behavior of LSTM model on different application scenarios.

Original: Additionally, in [7] and [13],author visualize the Convolutional Neural Network(CNN and visually explainable reasoning of internal feature representation.
Revised: Additionally, in [7] and [13],author visualize the Convolutional Neural Network(CNN and visually explainable reasoning of internal feature representation.

Original: Furthermore, Seq2Seq[11] designed a visual debugging tools forthe sequence-to-sequence learning model and enables users to interact with the model to develop aninsight about the model.
Revised: Furthermore, Seq2Seq[11] designed a visual debugging tools forthe sequence-to-sequence learning model and enables users to interact with the model to develop aninsight about the model.

Original: Inspired from the previous works [4][12][11], we designed an interactive visual DL models debuggingsystem, d-DeVIS : Deep Sequence Learning Model’s Debugger and Visually Interactive Simulator.
Revised: Inspired from the previous works [4][12][11], we designed an interactive visual DL models debuggingsystem, d-DeVIS : Deep Sequence Learning Model’s Debugger and Visually Interactive Simulator.

Original: Most of the previous works utilize the black box visualization approaches to help developing the basicintuition of the deep learning models.
Revised: Most of the previous works utilize the black box visualization approaches to help developing the basicintuition of the deep learning models.

Original: Surprisingly, visualizing the deep learning model behavior andfeature representation of the multimodal data, such as audio or visual, is neglected in the literature.
Revised: Surprisingly, visualizing the deep learning model behavior andfeature representation of the multimodal data, such as audio or visual, is neglected in the literature.

Original: Moreover, visualizing the correlation between the hidden layer features representations and the modelbehavior is not properly studied for sequence based models.
Revised: Moreover, visualizing the correlation between the hidden layer features representations and the modelbehavior is not properly studied for sequence based models.

Original: d-DeVIS allows user to interactivelychange the multimodal audio data to generate adversarial data examples and enables users to examinethe deep learning model behavior to visualize the feature representations.
Revised: d-DeVIS allows user to interactivelychange the multimodal audio data to generate adversarial data examples and enables users to examinethe deep learning model behavior to visualize the feature representations.

Original:  Proposed System: d-DeVIS
Revised:  Proposed System: d-DeVIS

Original: In this section, we present the key components and goals in designing our proposed interactiveapplication to visualize DL model in response to the adversarial data input.
Revised: In this section, we present the key components and goals in designing our proposed interactiveapplication to visualize DL model in response to the adversarial data input.

Original: In order for the interactivityof the users and ﬂexibility of the system, we made a web application which shows the gray boxdebugging method for deep neural network of sequence data.
Revised: In order for the interactivityof the users and ﬂexibility of the system, we made a web application which shows the gray boxdebugging method for deep neural network of sequence data.

Original: The prime goal of designing d-DeVISis to make sure the learning and debugging DL model easier and visualize the internal reasoning ofdeep sequence model and feature representations of hidden layers with the help of an interactiveuser interface.
Revised: The prime goal of designing d-DeVISis to make sure the learning and debugging DL model easier and visualize the internal reasoning ofdeep sequence model and feature representations of hidden layers with the help of an interactiveuser interface.

Original: We listed a couple of crucial design goals in designing an interpretable deep audiosequence learning model in the Table 1. 
Revised: We listed a couple of crucial design goals in designing an interpretable deep audiosequence learning model in the Table 1. [Table] and

Original: [Table]
Revised: [Table]

Original: An interpretable system of DL models depicts how deep sequence learning models work and how the hidden layer features can help to easily interpret the functionality of the learning model
Revised: An interpretable system of DL models depicts how deep sequence learning models work and how the hidden layer features can help to easily interpret the functionality of the learning model

Original: [Table]
Revised: [Table]

Original: A good grasp of the feature extraction method of deep neural networks is required for DL enthusiast and d-DeVIS
Revised: A good grasp of the feature extraction method of deep neural networks is required for DL enthusiast and d-DeVIS

Original: provides a ﬂuid gray box debugging experience  enables  users to understand how  hidden layersfeatures affect the training.
Revised: provides a ﬂuid gray box debugging experience which enables the users to understand how the features of the hidden layers affect the training.

Original: G3:
Revised: G3:

Original: Interactively Examining
Revised: Interactively Examining

Original: the Deep Sequence Model Behavior
Revised: the Deep Sequence Model Behavior

Original: An interactive tool is required, where user can manipulate audio features(such as slicing, cross-fading, loudness, repetition etc) to generate adversarial example data.
Revised: An interactive tool is required, where user can manipulate audio features(such as slicing, cross-fading, loudness, repetition etc) to generate adversarial example data.

Original: Moreover, it allows user to examine the internal reasoning in response of the modiﬁed adversarial data.
Revised: Moreover, it allows user to examine the internal reasoning in response of the modiﬁed adversarial data.

Original: G4:
Revised: G4:

Original: Comparison and exposure of the extracted features
Revised: Comparison and exposure of the extracted features

Original: fromaudio data
Revised: from

Revised: audio data

Original: The proposed system must enable users to listen the extracted audio data from different layer after applying the ﬁlters of ConvolutionalNeural Network.
Revised: The proposed system must enable users to listen the extracted audio data from different layer after applying the ﬁlters of Convolutional

Revised: Neural Network.

Original: Hence, users able to grasp the audio extracted hidden layer audio features.
Revised: Hence users able to grasp the audio extracted hidden layer audio features.

Original: 3.1 Features of d-DeVIS
Revised: 3.1 Features of d-DeVIS

Original: We designed d-DeVIS as an interactive web applications by considering the design goals, which arelisted in Table 1.
Revised: We designed d-DeVIS as an interactive web applications by considering the design goals, which arelisted in Table 1.

Original: The primary goal of our proposed is to ease the interpretation of the intermediatereasoning and the deep audio sequence learning model.
Revised: The primary goal of our proposed is to ease the interpretation of the intermediatereasoning and the deep audio sequence learning model.

Original: We divide the proposed d-DeVIS model intothe following three major components.
Revised: We divide the proposed d-DeVIS model intothe following three major components.

Original: • Model Visualization.
Revised: • Model Visualization.

Original: • Audio FeatureManipulation
Revised: • Audio Feature

Original: • Adversarial Feature Comparison
Revised: • Adversarial Feature Comparison

Original: 3.1.1 Model Visualization
Revised: 3.1.1 Model Visualization

Original: The primary purpose of our work is to interpret the internal reasoning of the deep sequence learningmodel in response of adversarial audio example data.
Revised: The primary purpose of our work is to interpret the internal reasoning of the deep sequence learningmodel in response of adversarial audio example data.

Original: For this reason, d-DeVIS provides an interactiveweb application interface, which depict the intermediate layer wise visual features representationin the form of audio spectrogram.
Revised: For this reason, d-DeVIS provides an interactiveweb application interface, which depict the intermediate layer wise visual features representationin the form of audio spectrogram.

Original: Moreover, we employed the inverse Fourier transformation toextract the audio feature from the intermediate layer spectrogram.
Revised: Moreover, we employed the inverse Fourier transformation toextract the audio feature from the intermediate layer spectrogram.

Original: d-DeVIS allows user to not onlyvisualize the features extracted by the hidden layer ﬁlters, but also it enables them to listen to theaudio representation of the extracted features of the input audio by the CNN.
Revised: d-DeVIS allows user to not onlyvisualize the features extracted by the hidden layer ﬁlters, but also it enables them to listen to theaudio representation of the extracted features of the input audio by the CNN.

Original: The web interfaceto visualize the layer wise feature is depicted in Fig 1.
Revised: The web interfaceto visualize the layer wise feature is depicted in Fig 1.

Original: Furthermore, d-DeVIS allows the users toexamine the weight distributions of the internal hidden layer.
Revised: Furthermore, d-DeVIS allows the users toexamine the weight distributions of the internal hidden layer.

Original: To extract the intermediate features,we trained a baseline CNN model on audio sequence data.
Revised: To extract the intermediate features,we trained a baseline CNN model on audio sequence data.

Original: The details of the trained model and thebacked system of d-DeVIS is presented in Section 3.2.
Revised: The details of the trained model and thebacked system of d-DeVIS is presented in Section 3.2.

Original: During any forward propagation step, the spectrogram feature data of the audio ﬁles are traversedthrough the hidden layer of the CNN.
Revised: During any forward propagation step, the spectrogram feature data of the audio ﬁles are traversedthrough the hidden layer of the CNN.

Original: At each layer, the convolution ﬁlter tries to extract signiﬁcanthidden features from the audio data input and optimizes itself during backward propagation in orderto minimize the training loss.
Revised: At each layer, the convolution ﬁlter tries to extract signiﬁcanthidden features from the audio data input and optimizes itself during backward propagation in orderto minimize the training loss.

Original: In our system, the users will be able to upload an audio ﬁle or recordan audio of their own.
Revised: In our system, the users will be able to upload an audio ﬁle or recordan audio of their own.

Original: After the processing the input, our system will calculate the logarithmicspectrogram and feed into the trained model to produce the prediction.
Revised: After the processing the input, our system will calculate the logarithmicspectrogram and feed into the trained model to produce the prediction.

Original: At each convolution layerthere are predeﬁned tunned ﬁlters.
Revised: At each convolution layerthere are predeﬁned tunned ﬁlters.

Original: The types of features it extracts from the input data dependson the ﬁlters.
Revised: The types of features it extracts from the input data dependson the ﬁlters.

Original: In our trained model, the ﬁrst layer and second layers have 16 ﬁlters, the third layerhas 32 ﬁlters, each.
Revised: In our trained model, the ﬁrst layer and second layers have 16 ﬁlters, the third layerhas 32 ﬁlters, each.

Original: So, our system visualizes the features corresponding to the ﬁlters and also thedistributions of the trained weights.
Revised: So, our system visualizes the features corresponding to the ﬁlters and also thedistributions of the trained weights.

Original: A particular feature extracted by the 13th ﬁlter of ﬁrst layer is visualized in Fig 1(a).
Revised: A particular feature extracted by the 13th ﬁlter of ﬁrst layer is visualized in Fig 1(a).

Original: When a userclicks on the image it zooms in to show the spectrogram clearly.
Revised: When a userclicks on the image it zooms in to show the spectrogram clearly.

Original: Users can also listen to the hiddenextracted feature by clicking on the play button, which is depicted in Fig.
Revised: Users can also listen to the hiddenextracted feature by clicking on the play button, which is depicted in Fig.

Original: 1(c).
Revised: 1(c).

Original: 3.1.2 Adversarial Feature Comparison
Revised: 3.1.2 Adversarial Feature Comparison

Original: d-DeVIS allows users to interpret the DL model behavior by examine the intermediate featuresrepresentation based on the different audio data input.
Revised: d-DeVIS allows users to interpret the DL model behavior by examine the intermediate featuresrepresentation based on the different audio data input.

Original: The adversarial behavior comparison isillustrated in Fig 2 and the different module of this feature is presented in red numbers.
Revised: The adversarial behavior comparison isillustrated in Fig 2 and the different module of this feature is presented in red numbers.

Original: The module aand b are the two spectrogram representation of the two audio inputs with their predictions by thetrained deep sequence learning model.
Revised: The module aand b are the two spectrogram representation of the two audio inputs with their predictions by thetrained deep sequence learning model.

Original: Users can observe the feature representation of different layerin module d and e. There are different spectrogram images of the extracted features and users canclick on them to listen to the audio representation.
Revised: Users can observe the feature representation of different layerin module d and e. There are different spectrogram images of the extracted features and users canclick on them to listen to the audio representation.

Original: Finally users can also see the weight distributionof each layers by clicking on the button marked by f.
Revised: Finally users can also see the weight distributionof each layers by clicking on the button marked by f.

Original: 3.1.3 Audio Feature Manipulation
Revised: 3.1.3 Audio Feature Manipulation

Original: Our proposed interactive system, d-DeVIS , enables the users to not only examine the behavior ofthe learning model in response to a sample ﬁle or recording of their voice but also it allows users tomanipulate the different properties of audio example data and thus enables to generate adversarialexample data.
Revised: Our proposed interactive system, d-DeVIS , enables the users to not only examine the behavior ofthe learning model in response to a sample ﬁle or recording of their voice but also it allows users tomanipulate the different properties of audio example data and thus enables to generate adversarialexample data.

Original: Among the various characteristics, which allows to change the audio data, d-DeVISallows to manipulate the following audio features.
Revised: Among the various characteristics, which allows to change the audio data, d-DeVISallows to manipulate the following audio features.

Original: • Slicing allows users to slice an audio.
Revised: • Slicing allows users to slice an audio.

Original: • Cross-fading changes the amplitude of the sound waves.
Revised: • Cross-fading changes the amplitude of the sound waves.

Original: • Changing the loudness option will make the beginning louder and the ending quieter.
Revised: • Changing the loudness option will make the beginning louder and the ending quieter.

Original: • Repeating option repeats the sound twice.
Revised: • Repeating option repeats the sound twice.

Original: • Invert: allows to invert the sound wave,
Revised: • Invert: allows to invert the sound wave,

Original: i
Revised: i

Original: e. inverted sound will be played from the ending.
Revised: e. inverted sound will be played from the ending.

Original: • Fade: option fade in for one second and then a fade out for the other second.
Revised: • Fade: option fade in for one second and then a fade out for the other second.

Original: A pictorial modiﬁcation of audio feature is presented in Fig.
Revised: A pictorial modiﬁcation of audio feature is presented in Fig.

Original: After manipulating and generating theaudio example data, d-DeVIS allows users to examine the behavior of audio deep sequence learningmodel by observing behavior change in response of original and adversarial audio data input.
Revised: After manipulating and generating theaudio example data, d-DeVIS allows users to examine the behavior of audio deep sequence learningmodel by observing behavior change in response of original and adversarial audio data input.

Original: In Table 2, we discuss all the features of d-DeVIS and how the features meet the design goals.
Revised: In Table 2, we discuss all the features of d-DeVIS and how the features meet the design goals.

Original: 3.2 Implementation of d-DeVIS
Revised: 3.2 Implementation of d-DeVIS

Original: d-DeVIS is developed as a web application so that users can seamlessly interact with the systemto interpret the behavior of deep learning model by generating adversarial audio input data.
Revised: d-DeVIS is developed as a web application so that users can seamlessly interact with the systemto interpret the behavior of deep learning model by generating adversarial audio input data.

Original: In thefollowing section we present the implementation details of d-DeVIS .
Revised: In thefollowing section we present the implementation details of d-DeVIS .

Original: 3.2.1 Trained Deep Learning Model with Audio Data
Revised: 3.2.1 Trained Deep Learning Model with Audio Data

Original: We trained a CNN model on Speech Commands dataset 1 , which is used to visualize the behavior ofthe model.
Revised: We trained a CNN model on Speech Commands dataset 1 , which is used to visualize the behavior ofthe model.

Original: The dataset consists of almost 30 speech classes but for the sake of the simplicity andreduce the training time we used 10 classes, which are the audio recordings of zero to nine digitsin English language.
Revised: The dataset consists of almost 30 speech classes but for the sake of the simplicity andreduce the training time we used 10 classes, which are the audio recordings of zero to nine digitsin English language.

Original: All the clips are two seconds long.
Revised: All the clips are two seconds long.

Original: We calculated logarithmic spectrogram asfeatures of the audio (.wav) data to feed into the training model.
Revised: We calculated logarithmic spectrogram asfeatures of the audio (.wav) data to feed into the training model.

Original: A three layer Convolutional Neural
Revised: A three layer Convolutional Neural

Original: Network (CNN) is used as the spectrogram feature matrix can represent an image and CNNs have
Revised: Network (CNN) is used as the spectrogram feature matrix can represent an image and CNNs have

Original: [Figure]
Revised: [Figure]

Original: Feature
Revised: Feature

Original: Goal
Revised: Goal

Original: Visualizing the Hidden Extracted Features of Convolutional neural networks: d-DeVIS provides visualization of the extracted features in each layers as image data and shows the various features of different ﬁlters of the deep convolutional network.
Revised: Visualizing the Hidden Extracted Features of Convolutional neural networks: d-DeVIS provides visualization of the extracted features in each layers as image data and shows the various features of different ﬁlters of the deep convolutional network.

Original: Users can also hear the audio representation of the hidden extracted features
Revised: Users can also hear the audio representation of the hidden extracted features

Original: G1 & G & G
Revised: G1 & G & G

Original: Interactive User Experience:
Revised: Interactive User Experience:

Original: For a ﬂuid user experience,we provide an interactive platform for the users so that they will be able to focus on the productivity of the system without any unnecessary hassles.
Revised: For a ﬂuid user experience,we provide an interactive platform for the users so that they will be able to focus on the productivity of the system without any unnecessary hassles.

Original: G3 & G
Revised: G3 & G

Original: Visualizing the Audio Features as well as Modifying the Waveforms: Due to the complex structure of audio data, our system let’s users modify various aspects of the sound property and visualize the updated waveform to provide a keen knowledge on audio data representation.
Revised: Visualizing the Audio Features as well as Modifying the Waveforms: Due to the complex structure of audio data, our system let’s users modify various aspects of the sound property and visualize the updated waveform to provide a keen knowledge on audio data representation.

Original: G2 & G
Revised: G2 & G

Original: Custom Audio Input for Testing and Feature Distribution Visualization: User can not only upload a default audio data but also they can record custom speech to test the trained model.
Revised: Custom Audio Input for Testing and Feature Distribution Visualization: User can not only upload a default audio data but also they can record custom speech to test the trained model.

Original: Proper distribution of the weights is also visualized.
Revised: Proper distribution of the weights is also visualized.

Original: G1 & G
Revised: G1 & G

Original: Comparing different audio inputs and their hidden features: d-DeVIS also enables users to measure the differences of different audio inputs and check their extracted layer features.
Revised: Comparing different audio inputs and their hidden features: d-DeVIS also enables users to measure the differences of different audio inputs and check their extracted layer features.

Original: G proven to be decent at image classiﬁcation.
Revised: G proven to be decent at image classiﬁcation.

Original: Our baseline model reached 95% validation accuracywith a minimal hyper parameter tuning.
Revised: Our baseline model reached 95% validation accuracywith a minimal hyper parameter tuning.

Original: We have used Keras deep learning framework which is awrapper library of Tensorﬂow to train our deep learning model.
Revised: We have used Keras deep learning framework which is awrapper library of Tensorﬂow to train our deep learning model.

Original: We have utilized the computationsystem of Google Colaboratory platform for the training purpose.
Revised: We have utilized the computationsystem of Google Colaboratory platform for the training purpose.

Original: 3.2.2 Front-end and Back-end of d-DeVIS
Revised: 3.2.2 Front-end and Back-end of d-DeVIS

Original: Audio ﬁles are manipulated such as Slicing the audio, Changing in Loudness, Cross-fading, Repeatingthe Sound, Invert and Fade using Numpy, Pydub and Scipy libraries.
Revised: Audio ﬁles are manipulated such as Slicing the audio, Changing in Loudness, Cross-fading, Repeatingthe Sound, Invert and Fade using Numpy, Pydub and Scipy libraries.

Original: Matplotlib is utilized to visualizethe audio features.
Revised: Matplotlib is utilized to visualizethe audio features.

Original: After the training the model, we have saved the data using Pickle python module.
Revised: After the training the model, we have saved the data using Pickle python module.

Original: We have used HTML5, Bootstrap, Javascript for designing the front-end of the application.
Revised: We have used HTML5, Bootstrap, Javascript for designing the front-end of the application.

Original: We havealso used python ﬂask web framework that helped to maintain the interaction between front-end andback-end communication.
Revised: We havealso used python ﬂask web framework that helped to maintain the interaction between front-end andback-end communication.

Original:  Use Cases
Revised:  Use Cases

Original: While experimenting with the system, we have applied several modiﬁcations to both the CNN trainedmodel and the input audio ﬁle.
Revised: While experimenting with the system, we have applied several modiﬁcations to both the CNN trainedmodel and the input audio ﬁle.

Original: Thus, we have analyzed the results that we got due to the tunning withthe system.
Revised: Thus, we have analyzed the results that we got due to the tunning withthe system.

Original: In this section, we present important use cases that demonstrate the general applicabilityof our system.
Revised: In this section, we present important use cases that demonstrate the general applicabilityof our system.

Original:  Speech is a sequence data which is hard to grasp just by looking at the amplitude vs time representation.
Revised: • Visualizing the Audio Features: Speech is a sequence data which is hard to grasp just bylooking at the amplitude vs time representation.

Original: In our system, a user can upload or record a custom audio ﬁle and tune with various aspects of thewaveform.
Revised: In our system, a user can upload or recorda custom audio ﬁle and tune with various aspects of the waveform.

Original: So predictions of the audio will change with accordance to the change in the waveformsand users can easily see the changed results. 4.2 Learning Medium for the Academia:
Revised: So predictions of theaudio will change with accordance to the change in the waveforms and users can easily seethe changed results.• Learning Medium for the Academia:

Original: Our system provides an interactive web application with which learners will be able to test varioustypes of aspects of audio data and the deep learning model.
Revised: Our system provides an interactive web applicationwith which learners will be able to test various types of aspects of audio data and the deeplearning model.

Original: After the model training completion, theproper visualizations are given.
Revised: After the model training completion, the proper visualizations are given.

Original: The academia will be able to use it for teaching purpose.
Revised: The academia will be able to use it for teaching purpose.•

Original: Due to our feature manipulation and interactivity of the system, the deep learning enthusiasts andengineers can do various experiments on it. We provide a platform for easy training and proper resultsof the feature extractions which are shown as a form of images.
Revised: Experimenting Platform for AI Enthusiasts: We provide a platform for easy training andproper results of the feature extractions which are shown as a form of images.

Original: Users can test their own custom inputand experiment on various types of hyper-parameters from which it can get the maximum accuracy.
Revised: Users can testtheir own custom input and experiment on various types of hyper-parameters from whichit can get the maximum accuracy.

Revised: Thus, the feature manipulation and interactivity of thesystem will inspire the deep learning enthusiasts and engineers to do various experiments onit.

Original:  Conclusion and Future Work
Revised:  Conclusion and Future Work

Original: In this work, we developed a web-based interactive visualization of CNN model, d-DeVIS , whichenables user to examine it’s behavior in response to the audio sequence data. 
Revised: d-DeVIS allows the user to visualize how CNN recognizes digits from audio sequence data. It collectsinput from user and allows them to interactively manipulate it.

Original: d-DeVIS also allowsusers to interactively manipulate the audio sequence data and simultaneously visualizes the CNNmodel’s internal features representations, hyper-parameters and input and learned output representation relationships.
Revised: The tool easily allows for comparingthe given input with other adverserial examples.

Original: Hence, it helps users to develop a better intuition of the underlying reasoning of a DL model.
Revised: Overall, this helps users to develop a better intuitionof the underlying reasoning of the model which will allow them to make more learned decisionsregarding learning model development in the future.

Original: In future extension of d-DeVIS , we have the plan to visualize the other deep learning models behaviorallowing users to manipulate the input data representation interactively.
Revised: In future extension of d-DeVIS , we have the plan to visualize the other deep learning models behaviorallowing users to manipulate the input data representation interactively.

Original: Moreover, visualizing thehidden layer complex feature representations for multi-modal sequence data, is a great avenue forfuture research work.
Revised: Moreover, visualizing thehidden layer complex feature representations for multi-modal sequence data, is a great avenue forfuture research work.