Original: Online Collective Animal Movement Activity Recognition
Revised: Online Collective Animal Movement Activity Recognition

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: Learning the activities of animals is important for the purpose of monitoring theirwelfare vis a vis their behaviour with respect to their environment and conspeciﬁcs.
Revised: Learning the activities of animals is important for the purpose of monitoring their welfare vis a vis their behaviour with respect to their environment and conspeciﬁcs.

Original: While previous works have largely focused on activity recognition in a singleanimal, little or no work has been done in learning the collective behaviour ofanimals.
Revised: While previous works have largely focused on activity recognition in a single animal, little or no work has been done in learning the collective behaviour of animals.

Original: In this work, we address the problem of recognising the collectivemovement activities of a group of sheep in a ﬂock.
Revised: In this work, we address the problem of recognising the collective movement activities of a group of sheep in a ﬂock.

Original: We present a discriminativeframework that learns to track the positions and velocities of all the animals in theﬂock in an online manner whilst estimating their collective activity.
Revised: We present a discriminative framework that learns to track the positions and velocities of all the animals in the ﬂock in an online manner whilst estimating their collective activity.

Original: We investigatethe performance of two simple deep network architectures and show that we canlearn the collective activities with good accuracy even when the distribution of theactivities is skewed.
Revised: We investigate the performance of two simple deep network architectures and show that we can learn the collective activities with good accuracy even when the distribution of the activities is skewed.

Original:  Introduction
Revised:  Introduction

Original: Recognising the collective movement activities of animals is important for several real world applications such as monitoring their welfare vis a vis their behaviour with respect to their environmentand conspeciﬁcs, predicting the onset of an epidemic or attacks from predators most especially foranimals who live in cooperative societies.
Revised: Recognising the collective movement activities of animals is important for several real world applications such as monitoring their welfare vis a vis their behaviour with respect to their environment and conspeciﬁcs, predicting the onset of an epidemic or attacks from predators most especially for animals who live in cooperative societies.

Original: While several works have been carried out in recognising collective activities with respective to human interactions [ 2], previous works in the animalbehaviour community however have either tried to recognize collective behaviour from differentspecies of animal where a model is fed the entire collective behaviour input and asked to identifythe specie generating such behaviour [3] or learn the activity of just a single animal [4, 5].
Revised: While several works have been carried out in recognising collective activities with respective to human interactions [1, 2], previous works in the animal behaviour community however have either tried to recognize collective behaviour from different species of animal where a model is fed the entire collective behaviour input and asked to identify the specie generating such behaviour [3] or learn the activity of just a single animal [4, 5].

Original: There arehowever problems with these approaches.
Revised: There are however problems with these approaches.

Original: Learning collective activity in an ofﬂine manner doesn’tscale into real world applications where the collective activity is required real time for example inmonitoring poaching activities.
Revised: Learning collective activity in an ofﬂine manner doesn’t scale into real world applications where the collective activity is required real time for example in monitoring poaching activities.

Original: In addition, learning the individual activity and aggregating them iscomputationally intensive and may not be informative compared to the collective context.
Revised: In addition, learning the individual activity and aggregating them is computationally intensive and may not be informative compared to the collective context.

Original: With the recent developments in deep learning research however, there has been a rise in developingmodels that can overcome some of these limitations.
Revised: With the recent developments in deep learning research however there has been a rise in developing models that can overcome some of these limitations.

Original: Notable among these techniques are the
Revised: Notable among these techniques are the

Original: Convolution Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) for modelling spatialand temporal dependencies respectively.
Revised: Convolution Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) for modelling spatial and temporal dependencies respectively.

Original: In this work, we focus on the task of recognising thecollective movement activities of a group of sheep in a ﬂock.
Revised: In this work, we focus on the task of recognising the collective movement activities of a group of sheep in a ﬂock.

Original: We gather movement data of 36 sheepand investigate the use of several deep learning models and two features (spatial orientations andvelocities) of the sheep to capture their collective activities.
Revised: We gather movement data of 36 sheep and investigate the use of several deep learning models and two features (spatial orientations and velocities) of the sheep to capture their collective activities.

Original: Our approach is one that learns in anonline fashion the collective activity of the ﬂock using a deep recurrent and convolutional neuralnetwork.
Revised: Our approach is one that learns in an online fashion the collective activity of the ﬂock using a deep recurrent and convolutional neural network.

Original: The main contributions of this paper are: (i)
Revised: The main contributions of this paper are: (i)

Original: We extend the idea of activity recognition in animalbehaviour to the collective settings and propose a model that can learn the collective movement behaviour in an online fashion (ii)
Revised: We extend the idea of activity recognition in animal behaviour to the collective settings and propose a model that can learn the collective movement behaviour in an online fashion (ii)

Original: We demonstrate that our approach can learn to classify activitieseven when some activities are underrepresented.
Revised: We demonstrate that our approach can learn to classify activities even when some activities are underrepresented.

Original: X 1 V
Revised: X 1 V

Original: X 2 V
Revised: X 2 V

Original: X m-1 V mX m V m .
Revised: X m-1 V mX m V m .

Original: …… LSTM y k
Revised: …… LSTM y k

Original: (Collective activity)
Revised: (Collective activity)

Original: LSTM LSTM LSTM
Revised: LSTM LSTM LSTM

Original: Dropout
Revised: Dropout

Original: Dense
Revised: Dense

Original: X 1 V
Revised: X 1 V

Original: X 2 V
Revised: X 2 V

Original: X m-1 V mX m V m .
Revised: X m-1 V mX m V m .

Original: …… CNN y
Revised: …… CNN y

Original: k
Revised: k

Original: (Collective activity)
Revised: (Collective activity)

Original: CNN CNN CNN
Revised: CNN CNN CNN

Original: Dropout
Revised: Dropout

Original: Dense
Revised: Dense

Original: LSTM LSTM LSTM .
Revised: LSTM LSTM LSTM .

Original: LSTM
Revised: LSTM

Original:  Problem Formulation
Revised:  Problem Formulation

Original: We brieﬂy describe the formulation of the collective movement activity recognition problem here.
Revised: We brieﬂy describe the formulation of the collective movement activity recognition problem here.

Original: Assume D tr = { ( X n , V n ) , y k } represents the training dataset of N s samples and D te = { ( X n , V n ) }denotes the test set of N d samples, where N d may or may not be equal to N s , and X n , V n representsspatial and velocity features of all the animals of interest respectively, while the corresponding labelspace representing the collective activities are Y
Revised: Assume D tr = { ( X n , V n ) , y k } represents the training dataset of N s samples and D te = { ( X n , V n ) } denotes the test set of N d samples, where N d may or may not be equal to N s , and X n , V n represents spatial and velocity features of all the animals of interest respectively, while the corresponding label space representing the collective activities are Y

Original: = { y 1 , y 2 , ...., y k } with k being the number ofunique collective activities.
Revised: = { y 1 , y 2 , ...., y k } with k being the number of unique collective activities.

Original: Now the problem is as follows, given a history of observations over atime window t described by { X i , V i | i
Revised: Now the problem is as follows, given a history of observations over a time window t described by { X i , V i | i

Original: = 1 , ..., t } , the goal is to predict y k ∈ Y for each datapointin D te .
Revised: = 1 , ..., t } , the goal is to predict y k ∈ Y for each datapoint in D te .

Original: That is, we aim to learn p ( y tk | ( X t , V t ) , ( X t − 1 , V t − 1 ) , ....., ( X t − m , V t − m )) where m is thetime window of feature observations used for the prediction task.
Revised: That is, we aim to learn p ( y tk | ( X t , V t ) , ( X t − 1 , V t − 1 ) , ....., ( X t − m , V t − m )) where m is the time window of feature observations used for the prediction task.

Original:  Model Description
Revised:  Model Description

Original: Temporal Dynamics Modelling:
Revised: Temporal Dynamics Modelling:

Original: We use the recurrent neural network (RNN) to model temporaldependencies.
Revised: We use the recurrent neural network (RNN) to model temporal dependencies.

Original: In particular, we use the LSTM (Long Short-Term Memory)
Revised: In particular, we use the LSTM (Long Short-Term Memory)

Original: [6] a variant of the
Revised: [6] a variant of the

Original: RNN to model long term dependencies and has been used in previous studies [7, 8] for handwritingsynthesis and language translation.
Revised: RNN to model long term dependencies and has been used in previous studies [7, 8] for handwriting synthesis and language translation.

Original: The LSTM can be described by the following equations:
Revised: The LSTM can be described by the following equations:

Original: [Equation]
Revised: [Equation]

Original: where i t , f t , and o t are input, forget, and output gate respectively.
Revised: where i t , f t , and o t are input, forget, and output gate respectively.

Original: W is the weight matrix, x t is thecurrent input data, h t − 1 is previous hidden output, C t is the cell state and σ is the logistic sigmoidfunction.
Revised: W is the weight matrix, x t is the current input data, h t − 1 is previous hidden output, C t is the cell state and σ is the logistic sigmoid function.

Original: Temporal and Spatial Dynamics Modelling:
Revised: Temporal and Spatial Dynamics Modelling:

Original: We use the recurrent convolutional network modelof [9] where the input is passed through a CNN to produce a reduced representation of the inputwhich is further passed into a RNN (Figure 1).
Revised: We use the recurrent convolutional network model of [9] where the input is passed through a CNN to produce a reduced representation of the input which is further passed into a RNN (Figure 1).

Original:  Data Acquisition and Pre-processing
Revised:  Data Acquisition and Pre-processing

Original: Data collection: We present here our data collection system.
Revised: Data collection: We present here our data collection system.

Original: All experiments involved with theanimals complied with the Australian ethics laws guiding the handling of animals.
Revised: All experiments involved with the animals complied with the Australian ethics laws guiding the handling of animals.

Original: We collectedmovement data of 36 sheep in a ﬁeld with the aid of a GPS device designed in house for two daysand attached to the back of the sheep.
Revised: We collected movement data of 36 sheep in a ﬁeld with the aid of a GPS device designed in house for two days and attached to the back of the sheep.

Original: Previous work has shown that such harness equipment carriedby sheep does not affect their locomotion [10].
Revised: Previous work has shown that such harness equipment carried by sheep does not affect their locomotion [10].

Original: Data was collected at a sampling rate of 1 sample/s toensure all forms of interesting movement patterns by the sheep are captured.
Revised: Data was collected at a sampling rate of 1 sample/s to ensure all forms of interesting movement patterns by the sheep are captured.

Original: Each dataset contains aphase in which loggers were attached to each sheep in a holding pen followed by a phase in whichthe sheep were herded into the ﬁeld and then a phase in which the sheep were left to roam across theﬁeld and a ﬁnal phase in which the sheep were herded back to the holding pen to have the loggingdevice removed and re-charged.
Revised: Each dataset contains a phase in which loggers were attached to each sheep in a holding pen followed by a phase in which the sheep were herded into the ﬁeld and then a phase in which the sheep were left to roam across the ﬁeld and a ﬁnal phase in which the sheep were herded back to the holding pen to have the logging device removed and re-charged.

Original: We extracted only portions of the dataset where all the loggers wereworking.
Revised: We extracted only portions of the dataset where all the loggers were working.

Original: All missing data were interpolated between the last and next seen co-ordinates using theexpectation-maximization algorithm [11].
Revised: All missing data were interpolated between the last and next seen co-ordinates using the expectation-maximization algorithm [11].

Original: Data labelling: With the aid of a viewer designed in house, we label the collective activities withrespect to the instances where they occur in the dataset.
Revised: Data labelling: With the aid of a viewer designed in house, we label the collective activities with respect to the instances where they occur in the dataset.

Original: Movement data of all the sheep in the ﬂockwere labelled with respective to the collective behaviours described in (Table 1).
Revised: Movement data of all the sheep in the ﬂock were labelled with respective to the collective behaviours described in (Table 1).

Original: Due diligence wasensured to make sure the labelling was of high quality.
Revised: Due diligence was ensured to make sure the labelling was of high quality.

Original: [Table]
Revised: [Table]

Original: Where the animals are gathered together in close proximity with little or no movement activities.
Revised: Where the animals are gathered together in close proximity with little or no movement activities.

Original: Correspond to instances where the animals are resting or sleeping.
Revised: Correspond to instances where the animals are resting or sleeping.

Original: [Table]
Revised: [Table]

Original: The animals are moving and scattered in their habitat with diverse movement activities.
Revised: The animals are moving and scattered in their habitat with diverse movement activities.

Original: [Table]
Revised: [Table]

Original: The animals are being herded at a high velocity over a narrow space.
Revised: The animals are being herded at a high velocity over a narrow space.

Original: [Table]
Revised: [Table]

Original:  Experiments & Results
Revised:  Experiments & Results

Original: Experiments: We used dataset of activities for one day for training and another day for testing see(Table 2).
Revised: Experiments: We used dataset of activities for one day for training and another day for testing see (Table 2).

Original: Models were trained with the following parameters: Adam optimizer [12] at a learning rate= 0.001, look-back & recurrent cells = 30, batch-size=10 over 50 epochs, dropout = 0.2, categoricalcrossentropy for all losses and a softmax for all classiﬁcation tasks, activation = tanh.
Revised: Models were trained with the following parameters: Adam optimizer [12] at a learning rate = 0.001, look-back & recurrent cells = 30, batch-size=10 over 50 epochs, dropout = 0.2, categorical crossentropy for all losses and a softmax for all classiﬁcation tasks, activation = tanh.

Original: For the CNNsegment of the CNN+LSTM model, we used a 1D CNN with a 2x2 ﬁlter, stride =1 and relu activation.
Revised: For the CNN segment of the CNN+LSTM model, we used a 1D CNN with a 2x2 ﬁlter, stride =1 and relu activation.

Original: All models were basic with only one layer and trained on a 2.3 GHz Intel Core i5 PC.
Revised: All models were basic with only one layer and trained on a 2.3 GHz Intel Core i5 PC.

Original: We investigatethe use of two features, the velocities of the animals and the distance of each sheep to their centroidin an ablation manner with respect to the two models.
Revised: We investigate the use of two features, the velocities of the animals and the distance of each sheep to their centroid in an ablation manner with respect to the two models.

Original: The collective activities were one hot encoded.
Revised: The collective activities were one hot encoded.

Original: All evaluations were carried out with respect to the classiﬁcation accuracy and confusion matrix.
Revised: All evaluations were carried out with respect to the classiﬁcation accuracy and confusion matrix.

Original: Results:
Revised: Results:

Original: From the results in (Table 3), the LSTM (velocities & distance to centroid) architectureoutperform others although marginally relative to the LSTM (velocities) and CNN+LSTM (velocities& distance to centroid) architectures.
Revised: From the results in (Table 3), the LSTM (velocities & distance to centroid) architecture outperform others although marginally relative to the LSTM (velocities) and CNN+LSTM (velocities & distance to centroid) architectures.

Original: The distance of each sheep to the entire sheep centroid seemsto be the least informative of the two features but produces better results when combined with thevelocities of the sheep using LSTM.
Revised: The distance of each sheep to the entire sheep centroid seems to be the least informative of the two features but produces better results when combined with the velocities of the sheep using LSTM.

Original: On the average the CNN+LSTM architectures perform slightly better than the LSTM architectures over all feature combinations.
Revised: On the average the CNN+LSTM architectures perform slightly better than the LSTM architectures over all feature combinations.

Original: More impressive is the fact that ourmodel is able to classify very well the underrepresented activity.
Revised: More impressive is the fact that our model is able to classify very well the underrepresented activity.

Original: As seen in the confusion matrices(Figure 2), only the models including spatial features were able to learn the Herd movement activitywith the CNN+LSTM performing better than the LSTM while the remaining two models 1 entirelymisclassiﬁed this activity.
Revised: As seen in the confusion matrices (Figure 2), only the models including spatial features were able to learn the Herd movement activity with the CNN+LSTM performing better than the LSTM while the remaining two models 1 entirely misclassiﬁed this activity.

Original: This suggests that a fusion of spatial and movement features is essentialto disentangle some of these complex collective activities especially in very rough and challengingterrains.
Revised: This suggests that a fusion of spatial and movement features is essential to disentangle some of these complex collective activities especially in very rough and challenging terrains.

Original: [Table]
Revised: [Table]

Original:  Conclusions & Future Work
Revised:  Conclusions & Future Work

Original: In this paper, we have shown how to learn the collective movement activities of sheep using deepneural network.
Revised: In this paper [Table] we have shown how to learn the collective movement activities of sheep using deep neural network.

Original: Our approach leveraged the fusion of spatio-temporal features from all animals ofinterest to learn collective activities even when their distribution is skewed.
Revised: Our approach leveraged the fusion of spatio-temporal features from all animals of interest to learn collective activities even when their distribution is skewed.

Original: This work has implicationsfor example in building automatic systems that can help farmers better understand the health of theﬂock as a whole for further applications in epidemic management as well as help conservationist learncollective behaviours of animals that are indicative of poaching activities.
Revised: This work has implications for example in building automatic systems that can help farmers better understand the health of the ﬂock as a whole for further applications in epidemic management as well as help conservationist learn collective behaviours of animals that are indicative of poaching activities.

Original: It is however not clear howour approach will perform when faced with a ﬂock of different size.
Revised: It is however not clear how our approach will perform when faced with a ﬂock of different size.

Original: One potential solution is to usean embedding to project behaviours (features) into a low dimensional space similar to [13].
Revised: One potential solution is to use an embedding to project behaviours (features) into a low dimensional space similar to [13].

Original: While wehave used animals who live in co-operative societies, the method used here may not scale into otheranimal societies without any structure.
Revised: While we have used animals who live in co-operative societies, the method used here may not scale into other animal societies without any structure.

Original: In the future, we aim to explore the use of an hierarchicalmodel to improve the classiﬁcation accuracy most especially when the distribution of activities isskewed.
Revised: In the future, we aim to explore the use of an hierarchical model to improve the classiﬁcation accuracy most especially when the distribution of activities is skewed.

Original: While we have used very simple neural network architectures, in the future, it is our desireto investigate other complex deep learning architectures to improve the accuracy.
Revised: While we have used very simple neural network architectures, in the future, it is our desire to investigate other complex deep learning architectures to improve the accuracy.

Original: Herd M. Active Not Active
Revised: Herd M. Active Not Active

Original: Herd M.
Revised: Herd M.

Original: Active 0.
Revised: Active 0.

Original: Not Active 0.
Revised: Not Active 0.

Original: [Figure]
Revised: [Figure]

Original: Herd [Figure] Active Not Active
Revised: Herd M. Active Not Active

Original: Herd M.
Revised: Herd M.

Original: Active 0.
Revised: Active 0.

Original: Not Active 0.
Revised: Not Active 0.

Original: [Figure]
Revised: [Figure]

Original: Herd [Figure] Not Active
Revised: Herd M. Active Not Active

Original: Herd M.
Revised: Herd M.

Original: Active 0.
Revised: Active 0.

Original: Not Active 0.
Revised: Not Active 0.

Original: [Figure]
Revised: [Figure]

Original: Herd M [Figure] Active Not Active
Revised: Herd M. Active Not Active

Original: Herd M.
Revised: Herd M.

Original: Active 0.
Revised: Active 0.

Original: Not Active 0.
Revised: Not Active 0.

Original: [Figure]
Revised: [Figure]