Original:  Adaptive Bio-Inspired Fish Simulation with Deep Reinforcement LearningAnonymous Author(s) Affiliation Address email
Revised: DeepFoids: Adaptive Bio-Inspired Fish Simulation with Deep Reinforcement Learning

Original: Abstract
Revised: Abstract

Original: Our goal is to synthesize realistic underwater scenes with various fish speciesin different fish cages, which can be utilized to train computer vision models toautomate fish counting and sizing tasks.
Revised: Our goal is to synthesize realistic underwater scenes with various fish species in different fish cages, which can be utilized to train computer vision models to automate fish counting task.

Original: It is a challenging problem to prepare asufficiently diverse labeled dataset of images from aquatic environments.
Revised: It is a challenging problem to prepare a sufficiently diverse labeled dataset of images from aquatic environments.

Original: We solvethis challenge by introducing an adaptive bio-inspired fish simulation.
Revised: We solve this challenge by introducing an adaptive bio-inspired fish simulation.

Original: The behaviorof caged fish changes based on the species, size and number of fish and the sizeand shape of the cage, among other variables.
Revised: The behavior of caged fish changes based on the species, size and number of fish, and the size and shape of the cage, among other variables.

Original: In this paper, we propose a methodfor achieving schooling behavior for any given combination of variables, usingmulti-agent deep reinforcement learning (DRL) in various fish cages in arbitraryenvironments.
Revised: In this paper, we propose a method for achieving schooling behavior for any given combination of variables, using multi-agent deep reinforcement learning (DRL) in various fish cages in arbitrary environments.

Original: Furthermore, to visually reproduce the underwater scene in differentlocations and seasons, we incorporate a physically-based underwater simulation.
Revised: Furthermore, to visually reproduce the underwater scene in different locations and seasons, we incorporate a physically-based underwater simulation.

Original:  Introduction
Revised:  Introduction

Original: The sustainability of the food supply and the protection of marine resources are two of the mostimportant global issues, and are part of the agenda set by the United Nations to be achieved by2030 1 .
Revised: The sustainability of the food supply and the protection of marine resources are two of the most important global issues, and are part of the agenda set by the United Nations to be achieved by 2030 § .

Original: Fish farming is one of the keys to a sustainable seafood supply to support the globalpopulation, as seafood consumption is growing rapidly.
Revised: Fish farming is one of the keys to a sustainable seafood supply to support the global population, as seafood consumption is growing rapidly.

Original: One of the current major problems in thefish farming industry is managing feeding.
Revised: One of the current major problems in the fish farming industry is managing feeding.

Original: Under-feeding slows down the growth of the fish due tomalnutrition.
Revised: Under-feeding slows down the growth of the fish due to malnutrition.

Original: Over-feeding is even worse as it can kill the fish and the residual food contaminates themarine environment.
Revised: Over-feeding is even worse as it can kill the fish and the residual food contaminates the marine environment.

Original: Therefore, we attempt to automate fish countingand sizing using inexpensivecommercially available RGB cameras and computer vision to optimize the amount and timing offeeding.
Revised: Therefore, we attempt to automate fish counting using inexpensive commercially available RGB cameras and computer vision to optimize the amount and timing of feeding.

Original: Deep learning has achieved great success in the field of computer vision.
Revised: Deep learning has achieved great success in the field of computer vision.

Original: The quality of the datasetdetermines the accuracy of deep learning models, but it is difficult to obtain data to train a fish countingmodel.
Revised: The quality of the dataset determines the accuracy of deep learning models but it is difficult to obtain data to train a fish counting model.

Original: Ishiwaka et al.
Revised: Ishiwaka et al.

Original: [1] proposed a fish schooling simulation called Foids, and demonstratedthe efficacy of using a CG synthetic dataset for training.
Revised: [1] proposed a fish schooling simulation called Foids and demonstrated the efficacy of using a CG synthetic dataset for training.

Original: Foids adds rules based on fish biologyto the Boids [2] algorithm.
Revised: Foids adds rules based on fish biology to the Boids [2] algorithm.

Original: However, both Boids and Foids require manually setting a number ofparameters for a given fish species, which must be adjusted again should any conditions change.
Revised: However, both Boids and Foids require manually setting a number of parameters for a given fish species, which must be adjusted again should any conditions change.

Original: Tosolve this problem, we propose a method to autonomously generate schooling behavior in fish usingmulti-agent deep reinforcement learning.
Revised: To solve this problem, we propose a method to autonomously generate schooling behavior in fish using multi-agent deep reinforcement learning.

Original: Fish behavior in a fish farming cage depends not only onnatural factors like temperature and light intensity, but also the size and shape of the fish cage, andeven the species, number, and size of the fish themselves.
Revised: Fish behavior in a fish farming cage depends not only on natural factors like temperature and light intensity, but also the size and shape of the fish cage, and even the species, number, and size of the fish themselves.

Original: By taking into account biological data suchas the preferred light intensity, temperature, and inter-fish distance of each fish, our method achievedseveral distinct patterns of collective behavior depending on population density through autonomouslearning.
Revised: By taking into account biological data such as the preferred light intensity, temperature, and inter-fish distance of each fish, our method achieved several distinct patterns of collective behavior depending on population density through autonomous learning.

Original: Furthermore, we developed a physically-based underwater environment simulation.
Revised: Furthermore, we developed a physically-based underwater environment simulation.

Original: This simulation is capable of accurately reproducing the conditions of underwater scenes of arbitrarylocations and seasons.
Revised: This simulation is capable of accurately reproducing the conditions of underwater scenes of arbitrary locations and seasons.

Original: The bio-inspired fish simulation and physically-based environment simulationallow for the creation of a high-quality synthetic dataset, with which we successfully trained a deeplearning model to count fish of various species in fish cages.
Revised: The bio-inspired fish simulation and physically-based environment simulation allow for the creation of a high-quality synthetic dataset, with which we successfully trained a deep learning model to count fish of various species in fish cages.

Original: In this paper, we explore related work in Sec.
Revised: In this paper, we explore related work in Sec.

Original: We then describe the implementation details of thesimulation, data synthesis, and the computer vision models in Sec. 3 and 4.
Revised: We then describe the implementation details of the simulation, data synthesis, and the computer vision models in Sec. 3 and 4.

Original: In Sec. 5, we show theexperimental results and analysis followed by a conclusion in Sec. 6.
Revised: In Sec. 5, we show the experimental results and analysis followed by a conclusion in Sec. 6.

Original:  Related Work
Revised:  Related Work

Original: It is well known that mammals use reinforcement learning [3].
Revised: It is well known that mammals use reinforcement learning [3].

Original: Experiments in neuroscience havedemonstrated that dopamine neurons play a role in rewards in reinforcement learning [4, 5, 6].
Revised: Experiments in neuroscience have demonstrated that dopamine neurons play a role in rewards in reinforcement learning [4, 5, 6].

Original: Thesedopamine neurons are present not only in mammals but in flies and fish as well [7].
Revised: These dopamine neurons are present not only in mammals but in flies and fish as well [7].

Original: It has been shownthat dopamine neurons are used in cooperative behavior [8] and learning [9] in fish.
Revised: It has been shown that dopamine neurons are used in cooperative behavior [8] and learning [9] in fish.

Original: Therefore, weadopted reinforcement learning for training cooperative schooling behavior.
Revised: Therefore, we adopted reinforcement learning for training cooperative schooling behavior.

Original: In the field of fish swimming simulations, DRL has been used to produce efficient swimming andverify adaptation behavior [10, 11, 12, 13].
Revised: In the field of fish swimming simulations, DRL has been used to produce efficient swimming and verify adaptation behavior [10, 11, 12, 13].

Original: These studies focused on simulations of a few fish orsoft-bodied animals, but they did not cover large numbers of fish or differences between species.
Revised: These studies focused on simulations of a few fish or soft-bodied animals, but they did not cover large numbers of fish or differences between species.

Original: Tu and Terzopoulos [14] proposed a fish simulation with perception, cognition and muscle basedlocomotion.
Revised: Tu and Terzopoulos [14] proposed a fish simulation with perception, cognition and muscle based locomotion.

Original: Lindsey [15] defined 12 types of locomotion in fish, and Satoi et al.[16] incorporatedthis definition and proposed a controller which automatically selects a type of locomotion based on adesired target behavior.
Revised: Lindsey [15] defined 12 types of locomotion in fish, and Satoi et al.[16] incorporated this definition and proposed a controller which automatically selects a type of locomotion based on a desired target behavior.

Original: Although those methods worked well at simulating individual fish behaviors,they were not directly useful for our objective of simulating tens of thousands of fish, while formingschooling behavior organically through interactions among the fish.
Revised: Although those methods worked well at simulating individual fish behaviors, they were not directly useful for our objective of simulating tens of thousands of fish, while forming schooling behavior organically through interactions among the fish.

Original: DRL-based approaches have also been studied for flocking control tasks [17, 18, 19] and crowdsimulation [20, 21].
Revised: DRL-based approaches have also been studied for flocking control tasks [17, 18, 19] and crowd simulation [20, 21].

Original: These models do not take biological specifics and their influences on flockingbehaviors of simulated objects into consideration, while the adaptation of these species-dependentcharacteristics constitute a crucial part of our fish control system.
Revised: These models do not take biological specifics and their influences on flocking behaviors of simulated objects into consideration, while the adaptation of these species-dependent characteristics constitute a crucial part of our fish control system.

Original: There are relatively few publicly available computer vision datasets of underwater images.
Revised: There are relatively few publicly available computer vision datasets of underwater images.

Original: Threemajor such datasets are OzFish [22], Deepfish [23], and a dataset from Ditria et al.
Revised: Three major such datasets are OzFish [22], Deepfish [23], and a dataset from Ditria et al.

Original: Thesedatasets lack the species and environments seen in fish farms, making them unsuitable for our goals.
Revised: These datasets lack the species and environments seen in fish farms, making them unsuitable for our goals.

Original: The biological properties we use in our simulation are aggregation, cohesion and separation rules[25,26];fish’s preferred temperature and light intensity [27, 28, 29]; personal space [30, 31]; sense ofdepth due to hydrostatic pressure [32, 33]; response to obstacles [34, 35]; and decision makinginterval [36], as well as the social ranking of fish.
Revised: The biological properties we use in our simulation are aggregation, cohesion and separation rules[25, 26]; fish’s preferred temperature and light intensity [27, 28, 29]; personal space [30, 31]; sense of depth due to hydrostatic pressure [32, 33]; response to obstacles [34, 35]; and decision making interval [36], as well as the social ranking of fish.

Original: Fish often form dominance hierarchies, where thedominant fish display aggressive behavior by chasing the subordinate individuals from time to time[37, 38].
Revised: Fish often form dominance hierarchies, where the dominant fish display aggressive behavior by chasing the subordinate individuals from time to time [37, 38].

Original: The details of the biological background are described in the Appendix.
Revised: The details of the biological background are described in the Appendix.

Original: In this paper, weset rewards and penalties for DRL based on these properties.
Revised: In this paper, we set rewards and penalties for DRL based on these properties.

Original:  Fish Simulation
Revised:  Fish Simulation

Original: In this section, we introduce a bio-inspired fish simulation trained with DRL.
Revised: In this section, we introduce a bio-inspired fish simulation trained with DRL.

Original: The bold letters v , dand u are used to denote directional 3D vectors, and the uppercase calligraphic letter I denotes ahigher-order tensor.
Revised: The bold letters v , d and u are used to denote directional 3D vectors, and the uppercase calligraphic letter I denotes a higher-order tensor.

Original: We structure the behavioral model of fish as a multi-agent reinforcement learningproblem, where fish agents need to navigate around a constrained 3D space that resembles a fishfarm environment while adapting to multiple factors that simultaneously influence their behavior.
Revised: We structure the behavioral model of fish as a multi-agent reinforcement learning problem, where fish agents need to navigate around a constrained 3D space that resembles a fish farm environment while adapting to multiple factors that simultaneously influence their behavior.

Original: During the course of interacting with the environment and other fish, each agent acts based on its ownobservations and is responsible for learning a general policy that maximizes a reward.
Revised: During the course of interacting with the environment and other fish, each agent acts based on its own observations and is responsible for learning a general policy that maximizes a reward.

Original: We model thepolicies using neural networks and optimize them using Proximal Policy Optimization (PPO) [39]with Generalized Advantage Estimation (GAE) [40].
Revised: We model the policies using neural networks and optimize them using Proximal Policy Optimization (PPO) [39] with Generalized Advantage Estimation (GAE) [40].

Original: A schematic illustration of our fish simulation pipeline is provided in Fig.1.
Revised: A schematic illustration of our fish simulation pipeline is provided in Fig.1.

Original: The parameters controllingbiological and environmental factors in the fish simulation are set based on the literature presentedin the Appendix and field data collected from fish farms at Onmaehama and Nishiki, Japan.
Revised: The parameters controlling biological and environmental factors in the fish simulation are set based on the literature presented in the Appendix and field data collected from fish farms at Onmaehama and Nishiki, Japan.

Original: Fishanimation is generated by the finite state machine based on the velocity of the locomotion 
Revised: Fish animation is generated by the finite state machine based on the velocity of the locomotion and the fish’s behavior state, which is explained in more details in the Appendix.

Original: Biological and Environmental Factors:
Revised: Biological and Environmental Factors:

Original: We incorporate important biological and environmentaldrivers of fish swimming patterns into the simulation framework.
Revised: We incorporate important biological and environmental drivers of fish swimming patterns into the simulation framework.

Original: A fish stays within a comfortabledistance of its neighboring fish and aligns its direction with that of its nearest neighbor in the forwarddirection [2, 41].
Revised: A fish stays within a comfortable distance of its neighboring fish and aligns its direction with that of its nearest neighbor in the forward direction [2, 41].

Original: Its activity space is constrained by the cage boundaries and water surface, whichreproduce the commercial cage environment and facilitate the formation of schooling.
Revised: Its activity space is constrained by the cage boundaries and water surface, which reproduce the commercial cage environment and facilitate the formation of schooling.

Original: Additionally,caged fish are divided into dominant and subordinate groups, where the dominant individuals mayinitiate antagonistic acts by aggressively approaching the subordinate members.
Revised: Additionally, caged fish are divided into dominant and subordinate groups, where the dominant individuals may initiate antagonistic acts by aggressively approaching the subordinate members.

Original: These factors arecarefully integrated into the process of learning control policies that generate the delta velocity offish ∆ v ft at each time step t through the use of the PPO algorithm.
Revised: These factors are carefully integrated into the process of learning control policies that generate the delta velocity of fish ∆ v ft at each time step t through the use of the PPO algorithm.

Original: There are effects of underwater light intensity and temperature on the fish’s vertical distribution,which we represent as ∆ v light and ∆ v temp .
Revised: There are effects of underwater light intensity and temperature on the fish’s vertical distribution, which we represent as ∆ v light and ∆ v temp .

Original: A fish has a preferred range of light intensity and watertemperature, and it changes its vertical position to stay in the comfortable zone.
Revised: A fish has a preferred range of light intensity and water temperature, and it changes its vertical position to stay in the comfortable zone.

Original: The details of thecomputation are explained in the Appendix.
Revised: The details of the computation are explained in the Appendix.

Original: In addition to the above-mentioned components, we integrate the fish decision making interval intothe framework to emulate the latency of fish’s responses to environmental changes.
Revised: In addition to the above-mentioned components, we integrate the fish decision making interval into the framework to emulate the latency of fish’s responses to environmental changes.

Original: The duration (inunits of simulation steps) of the decision making interval, ∆ t res , for simulated species is predefinedin accordance with the literature studies described in Sec.2.
Revised: The duration (in units of simulation steps) of the decision making interval, ∆ t res , for simulated species is predefined in accordance with the literature studies described in Sec.

Original: At the time of simulation, given thetime interval of simulation steps ( ∆ t sim ), a fish updates its observation of the environment every⌊ ∆ t res / ∆ t sim ⌋ steps and takes actions between updates.
Revised: At the time of simulation, given the time interval of simulation steps ( ∆ t sim ), a fish updates its observation of the environment every ⌊ ∆ t res / ∆ t sim ⌋ steps and takes actions between updates.

Original: The delta accumulated velocity to apply ateach simulation step ( ∆ v at ) can then be derived as follows:
Revised: The delta accumulated velocity to apply at each simulation step ( ∆ v at ) can then be derived as follows:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: States and Actions: Each state s t ∈ state space S encodes the information a fish agent observes inthe environment at time step t .
Revised: States and Actions: Each state s t ∈ state space S encodes the information a fish agent observes in the environment at time step t .

Original: It can be represented by a tuple ( u t , d t , I t ) , where u t is the differencebetween forward directions of the agent and its nearest neighboring fish in front of it at current step t ,d t is the depth of the agent with respect to the water surface, and I t is a visual observation tensor.
Revised: It can be represented by a tuple ( u t , d t , I t ) , where u t is the difference between forward directions of the agent and its nearest neighboring fish in front of it at current step t , d t is the depth of the agent with respect to the water surface, and I t is a visual observation tensor.

Original: During the time of exploration, a fish agent collects visual observations with a spatial grid sensor thatimitates the sensing area of real fish [42].
Revised: During the time of exploration, a fish agent collects visual observations with a spatial grid sensor that imitates the sensing area of real fish [42].

Original: The visual observation is stored as a third-order tensor,whose dimensions are grid width, grid height, and number of channels. 
Revised: The visual observation is stored as a third-order tensor, whose dimensions are grid width, grid height, and number of channels. The width and height are defined by the grid resolution, which is set to be 34 × 20.

Original: The channels contain thedistance information and multi-hot values of detectable objects (e.g. other agents and cage walls)within the fish’s sensing range d sense .
Revised: There are 6 channels encoding a scalar value of the normalized distance from the closest detected object within the fish’s sensing range d sense to the agent and a one-hot encoding of the object type (i.e. fish, boundary or obstacle).

Original: d sense is valued at 2 body lengths (BL) for yellowtail amberjack(yellowtail) [43] and 3 BL for coho salmon and red seabream [42].
Revised: d sense is valued at 2 body lengths (BL) for yellowtail amberjack (yellowtail) [43] and 3 BL for coho salmon and red seabream [42].

Original: We stack three visual observationstogether to infer movement before passing them to the networks.
Revised: We stack three visual observations together to infer movement before passing them to the networks.

Original: All the state components arecomputed in the local coordinate system of the agent, with the origin located at the body center andthe z-axis parallel to the fish’s facing direction.
Revised: All the state components are computed in the local coordinate system of the agent, with the origin located at the body center and the z-axis parallel to the fish’s facing direction.

Original: Note that a simulated fish is not capable of preciselyobserving its speed and rotation since a real fish can only sense its relative rotation through the use ofthe lateral line system [44, 45].
Revised: Note that a simulated fish is not capable of precisely observing its speed and rotation since a real fish can only sense its relative rotation through the use of the lateral line system [44, 45].

Original: The action a ∈ action space A determined by the policy specifies the delta speed ( ∆ v t ), as well asdelta rotation about the x-axis ( ∆ θ xt ) and the y-axis ( ∆ θ yt ) in degrees.
Revised: The action a ∈ action space A determined by the policy specifies the delta speed ( ∆ v t ), as well as delta rotation about the x-axis ( ∆ θ xt ) and the y-axis ( ∆ θ yt ) in degrees.

Original: The rotation angle about thez-axis is clamped by a small angle θ zt to avoid unnatural rolling behavior.
Revised: The rotation angle about the z-axis is clamped blueto a small angle θ zt to avoid unnatural rolling behavior.

Original: ∆ v t is also clamped by the maximum delta speed ∆ v max allowed in the cage environment.
Revised: ∆ v t is also clamped by the maximum delta speed ∆ v max allowed in the cage environment.

Original: The three action components arethen used to compute the delta velocity of fish ∆ v ft at the current time step and thus drive the motionof the fish agent as depicted in Eq.
Revised: The three action components are then used to compute the delta velocity of fish ∆ v ft at the current time step and thus drive the motion of the fish agent as depicted in Eq.

Original: Reward: The reward r t at each time step is defined to encourage schooling behavior while avoidingboundary collisions and to be consistent with the biological studies described in Sec. 2:
Revised: Reward: The reward r t at each time step is defined to encourage schooling behavior while avoiding boundary collisions and to be consistent with the biological studies described in Sec. 2:

Original: [Equation]
Revised: [Equation]

Original: The reward r BCt represents the penalty of colliding with the spatial boundaries, which include thecage walls and water surface.
Revised: The reward r BCt represents the penalty of colliding with the spatial boundaries, which include the cage walls and water surface.

Original: It has a fixed value of − 300 if a boundary collision occurs or 0otherwise.
Revised: It has a fixed value of − 300 if a boundary collision occurs orotherwise.

Original: r NCt penalizes the collision with neighboring fish detected by box colliders using anassociated weight w NC and accumulates with the number of colliding agents N hit :
Revised: r NCt penalizes the collision with neighboring fish detected by box colliders using an associated weight w NC and accumulates with the number of colliding agents N hit :

Original: [Equation]
Revised: [Equation]

Original: The boundary avoidance reward r BDt encourages the fish to keep a distance from a detected spatialboundary.
Revised: The boundary avoidance reward r BDt encourages the fish to keep a distance from a detected spatial boundary.

Original: Its value depends on the agent’s sensing range d sense , the number of detected boundaries
Revised: Its value depends on the agent’s sensing range d sense , the number of detected boundaries

Original: N bnd , the distance d i to the boundary
Revised: N bnd , the distance d i to the boundary

Original: i and the boundary avoidance weight w BD :
Revised: i and the boundary avoidance weight w BD :

Original: [Equation]
Revised: [Equation]

Original: The neighbor interaction reward r NDt encourages the fish to stay close to its neighbors within thesensing range and to align its direction with those of its neighbors.
Revised: The neighbor interaction reward r NDt encourages the fish to stay close to its neighbors within the sensing range and to align its direction with those of its neighbors.

Original: The angle ∆ θ mov i in degreesbetween the directions of the agent and each of its N nei neighbors is calculated and a neighborinteraction weight w ND is used for the computation:
Revised: The angle ∆ θ mov i in degrees between the directions of the agent and each of its N nei neighbors is calculated and a neighbor interaction weight w ND is used for the computation:

Original: [Equation]
Revised: [Equation]

Original: On the other hand, r Et penalizes energy consumption of the fish while rotating its body or adjustingspeed.
Revised: On the other hand, r Et penalizes energy consumption of the fish while rotating its body or adjusting speed.

Original: It is computed from a rotation penalty weight w r , a speed penalty weight w s together with thedelta angle ∆ θ t of the body rotation and the delta accumulated speed ∆ v at at the current time step:
Revised: It is computed from a rotation penalty weight w r , a speed penalty weight w s together with the delta angle ∆ θ t of the body rotation and the delta accumulated speed ∆ v at at the current time step:

Original: [Equation]
Revised: [Equation]

Original: The movement reward r Mt encourages the fish to swim faster than a minimum speed and penalizessudden changes in depth caused by aggressive pitch motion (around the local x-axis).
Revised: The movement reward r Mt encourages the fish to swim faster than a minimum speed and penalizes sudden changes in depth caused by aggressive pitch motion (around the local x-axis).

Original: In the expressionbelow, the variable θ rt denotes the pitch angle threshold, v st is the speed threshold, θ xt is the currentpitch angle, and v at represents the current accumulated speed.
Revised: In the expression below, the variable θ rt denotes the pitch angle threshold, v st is the speed threshold, θ xt is the current pitch angle, and v at represents the current accumulated speed:

Original: The reward r Mt has a value of − 10when θ rt ≤ θ xt ≤ (360 − θ rt ) , a value of 2 when − θ rt < θ xt ≤ θ rt and v at ≥ v st , and a value of 0otherwise.
Revised: [Equation]

Original: Lastly, r C t denotes the chase reward which encourages aggression or escape behavior based on thesocial rank of the fish.
Revised: Lastly, r Ct denotes the chase reward which encourages aggression or escape behavior based on the social rank of the fish.

Original: Specifically, a dominant fish (aggressor) randomly starts a chase mode with asmall probability p a and initiates an attack on its nearest subordinate neighbor (target).
Revised: Specifically, a dominant fish (aggressor) randomly starts a chase mode with a small probability p a and initiates an attack on its nearest subordinate neighbor (target).

Original: This triggersthe subordinate being chased to start its chase mode and swim away from the aggressor.
Revised: This triggers the subordinate being chased to start its escape mode and swim away from the aggressor.

Original: We rewardthe aggressor by a fixed large value if it collides with its target.
Revised: We reward the aggressor by a fixed large value if it collides with its target.

Original: This process can be expressed usingthe aggressor’s accumulated velocity v at , a normalized vector d spanning from the aggressor to thetarget, a chase reward weight w agg for the aggressor, and a chase penalty weight w tar for the target asfollows: the reward r
Revised: This process can be expressed using the aggressor’s accumulated velocity v at , a normalized vector d spanning from the aggressor to the target, a chase reward weight w agg for the aggressor, and an escape penalty weight w tar for the target as follows: [Equation] fish is an aggressor in chase mode and chases after its target [Equation]

Original: Ct has a value of w agg ( d · v at ) if a fish is an aggressor in chase mode and chasesafter its target, a value of 100 ifa fish is an aggressor in chase mode and collides with its target, avalue of − w tar ( d · v at ) ifa fish is a target in chase mode, and a value of 0 otherwise.
Revised: if fish is an aggressor in chase mode and collides with its target. [Equation] tar ( d · v at ) if fish is a target in escape mode. otherwise.

Original:  The chasemode ends for both fish  when they collide, the target escapes to the back of the aggressor, or after ashort period of time as the dominants prefer not wasting energy on a prolonged chase.
Revised: [Figure] The chase and escape modes end for the two fish either when they collide, the target escapes to the back of the aggressor, or after a short period of time as the dominants prefer not wasting energy on a prolonged chase.

Original: A detailedexplanation of parameters used in action definition and reward function is available in Appendix.
Revised: A detailed explanation of parameters used in action definition and reward function is available in Appendix.

Original: Network: The simulation of realistic fish schooling behaviors can be broken down into a seriesof tasks where fish agents need to choose optimal actions from continuous action space.
Revised: Network: The simulation of realistic fish schooling behaviors can be broken down into a series of tasks where fish agents need to choose optimal actions from continuous action space.

Original: In the
Revised: In the

Original: Actor-Critic framework used by PPO, a policy (actor) π is modeled as a neural network (Fig. 2) [Figure] [Figure]
Revised: Actor-Critic framework used by PPO, a policy (actor) π is modeled as a neural network (Fig. 2) that maps a state s t to a Gaussian distribution over action π ( a t | s t ) .

Original: [Figure]

Original: that maps a state s t to a Gaussian distribution over action π ( a t

Original: | s t ) .

Original: The agent’s visual observation
Revised: The agent’s visual observation

Original: I t , which is formed as a 3D tensor, is processed by a visual encoder comprising two convolutionallayers and a dense layer before being passed to a fully-connected network with other observationcomponents.
Revised: I t , which is formed as a 3D tensor, is processed by a visual encoder comprising two convolutional layers and a dense layer before being passed to a fully-connected network with other observation components.

Original: The first and the second convolutional layers contain 16 8 × 8 filters and 32 4 × 4 filtersrespectively and use Leaky ReLU activation.
Revised: The first and the second convolutional layers contain 16 8 × 8 filters and 32 4 × 4 filters respectively and use Leaky ReLU activation.

Original: The fully-connected layer in the visual encoder contains128 units and also uses Leaky ReLU as the activation function.
Revised: The fully-connected layer in the visual encoder contains 128 units and also uses Leaky ReLU as the activation function.

Original: Each of the other four fully-connectedlayers consists of 128 units and uses the Swish activation function.
Revised: Each of the other four fully-connected layers consists of 128 units and uses the Swish activation function.

Original: The output layer is just a linearlayer with the output size equal to the size of action space.
Revised: The output layer is just a linear layer with the output size equal to the size of action space.

Original: We train the policies using PPO with aclipped surrogate objective [39].
Revised: We train the policies using PPO with a clipped surrogate objective [39].

Original: An in-depth review of this algorithm can be found in the Appendix.
Revised: An in-depth review of this algorithm can be found in the Appendix.

Original: The advantage for the policy gradient is calculated by GAE [40].
Revised: The advantage for the policy gradient is calculated by GAE [40].

Original: The value function (critic) V ( s t )is represented as another deep neural network with similar architecture except the output layer is asingle unit.
Revised: The value function (critic) V ( s t ) is represented as another deep neural network with similar architecture except the output layer is a single unit.

Original: The networks are built on the open source library PyTorch.
Revised: The networks are built on the open source library PyTorch.

Original:  Synthetic Dataset and Fish Counting
Revised:  Synthetic Dataset and Fish Counting

Original: Our simulation is developed on Unity Engine 2 , and Crest Ocean System HDRP asset 3 is utilized tocreate the effect of sun and ocean waves.
Revised: Our simulation is developed on Unity Engine ¶ with ML-Agents toolkit [46], and Crest Ocean System HDRP asset || is utilized to create the effect of sun and ocean waves.

Original: The simulation is controlled by parameters including, butnot limited to, fish biological details such as fish species, count, size, and speed; temperature and lightpreferences; latitude, longitude, date and time; wave size, sediment and chlorophyll concentrations;sunlight color and intensity; and light attenuation and scattering.
Revised: The simulation is controlled by parameters including, but not limited to, fish biological details such as fish species, count, size, and speed; temperature and light preferences; latitude, longitude, date and time; wave size, sediment and chlorophyll concentrations; sunlight color and intensity; and light attenuation and scattering.

Original: A major drawback of prior fishsimulation systems [31, 1] is the need for manual parameter tuning when applying the simulationto new species or new environments.
Revised: A major drawback of prior fish simulation systems [31, 1] is the need for manual parameter tuning when applying the simulation to new species or new environments.

Original: However, in this work we leverage the power of DRL to trainthe fish to adapt to changing environmental conditions given their biological data, and apply thespecies-specific trained models to emulate the corresponding swimming patterns.
Revised: However, in this work we leverage the power of DRL to train the fish to adapt to changing environmental conditions given their biological data, and apply the species-specific trained models to emulate the corresponding swimming patterns.

Original: We also use thephysically-based environment simulation described in the Appendix to control the scene with a smallnumber of hyperparameters instead of extensive adjustment with programmable shaders.
Revised: We also use the physically-based environment simulation described in the Appendix to control the scene with a small number of hyperparameters instead of extensive adjustment with programmable shaders.

Original: Theseimprovements significantly automate the simulation process and allow us to generate an arbitrarilylarge and varied dataset by randomly setting the above mentioned parameters within a physicallyvalid range.
Revised: These improvements significantly automate the simulation process and allow us to generate an arbitrarily large and varied dataset by randomly setting the above mentioned parameters within a physically valid range.

Original: Fig.
Revised: Fig.

Original: 3 displays an example of a synthesized dataset.
Revised: 3 displays an example of a synthesized dataset.

Original: The fish counting system comprises three modules: an image pre-processing module, which convertsthe input video to a sequence of images and applies denoising; a fish detection module, which is atrained network based on YOLOv4 [46] with the synthetic dataset; and a fish counting module.
Revised: The fish counting system comprises three modules: an image pre-processing module, which converts the input video to a sequence of images and applies denoising; a fish detection module, which is a trained network based on YOLOv4 [47] with the synthetic dataset; and a fish counting module.

Original: Thesystem takes as input a fish cage video, and outputs an estimate of the total number of fish shown inthe video.
Revised: The system takes as input a fish cage video, and outputs an estimate of the total number of fish shown in the video.

Original: The details of each component can be found in the Appendix.
Revised: The details of each component can be found in the Appendix.

Original:   Experiments and Results
Revised: [Figure]  Experiments and Results

Original: We describe the training process, experimental results of the fish behavioral model with diversebiological and cage settings, underwater simulation, and fish counting.
Revised: We describe the training process, experimental results of the fish behavioral model with diverse biological and cage settings, underwater simulation, and fish counting.

Original: Training: We trained the behavioral models of coho salmon, yellowtail and red seabream.
Revised: Training: We trained the behavioral models of coho salmon, yellowtail and red seabream.

Original: Atwo-step training scheme was employed to facilitate the adaption for agents in eleven environmentswith the same action space, observation space and types of detectable objects, but different fishspecies, fish sizes, fish numbers, cage sizes and cage shapes.
Revised: A two-step training scheme was employed to facilitate the adaption for agents in eleven environments with the same action space, observation space and types of detectable objects, but different fish species, fish sizes, fish numbers, cage sizes and cage shapes.

Original: We pretrained the three species intheir corresponding default scenes, which contain 1000 coho salmon, 45 yellowtail and 10 redseabream, respectively, with the cage configurations set to be the same as the fish farm environmentswe collected field data from.
Revised: We pretrained the three species in their corresponding default scenes, which contain 1000 coho salmon, 45 yellowtail and 10 red seabream, respectively, with the cage configurations set to be the same as the fish farm environments we collected field data from.

Original: The pretraining results were usually sufficient to enable the agents tonavigate and avoid cage boundaries to some extent, but collisions between fish and with boundariesstill occur and unnatural movements persist.
Revised: The pretraining results were usually sufficient to enable the agents to navigate and avoid cage boundaries to some extent, but collisions between fish and with boundaries still occur and unnatural movements persist.

Original: We then performed transfer learning based on thepretrained policies for each species in its corresponding environment-specific experiments, includingthe default scene that is used to reproduce the real scene.
Revised: We then performed transfer learning based on the pretrained policies for each species in its corresponding environment-specific experiments, including the default scene that is used to reproduce the real scene.

Original: We found the use of such a two-step trainingscheme led to faster convergence and overall performance improvements compared to training fromscratch for each environment.
Revised: We found the use of such a two-step training scheme led to faster convergence and overall performance improvements compared to training from scratch for each environment.

Original: We enabled early termination of an episode if the agent collides witheither a cage wall or the water surface.
Revised: We enabled early termination of an episode if the agent collides with either a cage wall or the water surface.

Original: The agent would subsequently restart a new episode from arandom position within the cage with a random valid rotation and an initial speed v 0 .
Revised: The agent would subsequently restart a new episode from a random position within the cage with a random valid rotation and an initial speed v 0 .

Original: This strategydiscourages boundary collision and prevents the agents from collecting data that is irrelevant toboundary avoidance.
Revised: This strategy discourages boundary collision and prevents the agents from collecting data that is irrelevant to boundary avoidance.

Original: The values of hyperparameters in the pretraining and transfer learning phasesare similar if not identical.
Revised: The values of hyperparameters in the pretraining and transfer learning phases are similar if not identical.

Original: For yellowtail and red seabream, we ran a total number of two milliontime steps and used a linearly decaying learning rate of 0.001 in both phases.
Revised: For yellowtail and red seabream, we ran a total number of two million time steps and used a linearly decaying learning rate of 0.001 in both phases.

Original: For coho salmon, weadopted the same settings, but increased the pretraining and transfer learning steps to four millionand one million time steps respectively, because the 1000 coho salmon in the default scene is muchmore than the other two species.
Revised: For coho salmon, we adopted the same settings, but increased the pretraining and transfer learning steps to four million and one million time steps respectively, because the 1000 coho salmon in the default scene is much more than the other two species.

Original: The total wall-clock training times for four million and five millionsimulation steps were around 5500 seconds and 6800 seconds, respectively.
Revised: The total wall-clock training times for four million and five million simulation steps were around 5500 seconds and 6800 seconds, respectively.

Original: A detailed explanation ofthe eleven simulation environments, hyperparameter settings, and the learning curves of two trainingstages are available in the Appendix.
Revised: A detailed explanation of the eleven simulation environments, hyperparameter settings, and the learning curves of two training stages are available in the Appendix.

Original: We also demonstrate the training progress of yellowtail ina recorded demo that is part of our supplementary video.
Revised: We also demonstrate the training progress of yellowtail in a recorded demo that is part of our supplementary video.

Original: The training took place on an NVIDIA
Revised: The training took place on an NVIDIA

Original: GeForce RTX 3080 Laptop GPU installed on a Windows 10 machine with a 3.3 GHz AMD Ryzen 95900HX CPU.
Revised: GeForce RTX 3080 Laptop GPU installed on a Windows 10 machine with a 3.3 GHz AMD Ryzen5900HX CPU.

Original: Fish Behavior: We simulated swimming scenes of the above-mentioned three species in order toprovide a variety of synthetic data to train the computer vision system that detects and tracks fish.
Revised: Fish Behavior: We simulated swimming scenes of the above-mentioned three species in order to provide a variety of synthetic data to train the computer vision system that detects and tracks fish.

Original: Fish agents were initialized at random positions in the cage with random valid rotations, and wouldstart to form circling patterns by following the trained policies.
Revised: Fish agents were initialized at random positions in the cage with random valid rotations, and would start to form circling patterns by following the trained policies.

Original: A qualitative comparison between theimage sequences from the simulation and the videos captured underwater is displayed in Fig.
Revised: A qualitative comparison between the image sequences from the simulation and the videos captured underwater is displayed in Fig.

Original: Notethat the swimming patterns of the three fish species are distinct in the captured video.
Revised: Note that the swimming patterns of the three fish species are distinct in the captured video.

Original: Our simulationsuccessfully reproduced such behavioral differences as well as the environmental variation such aslighting, water color and turbidity.
Revised: Our simulation successfully reproduced such behavioral differences as well as the environmental variation such as lighting, water color and turbidity.

Original: We also showcase the simulation results of five example environments with different species, fishsize, fish number, cage size and shape settings in Fig.
Revised: We also showcase the simulation results of five example environments with different species, fish size, fish number, cage size and shape settings in Fig.

Original: A body scale of 1.0 refers to the default bodylengths (BL) of coho salmon, yellowtail and red seabream, which were set to be 0.49 meters, 0.52meters, and 0.34 meters respectively according to the collected field data.
Revised: A body scale of 1.0 refers to the default body lengths (BL) of coho salmon, yellowtail and red seabream, which were set to be 0.49 meters, 0. meters, and 0.34 meters respectively according to the collected field data.

Original: We additionally applied amild repulsive force acting as an object-to-object colliding force to push any penetrating fish awayfrom each other to further facilitate collision avoidance among agents.
Revised: We additionally applied a mild repulsive force acting as an object-to-object colliding force to push any penetrating fish away from each other to further facilitate collision avoidance among agents.

Original: The fish swimming patternsvary with the scene configuration since they result from a combination of factors discussed in Sec 3.
Revised: The fish swimming patterns vary with the scene configuration since they result from a combination of factors discussed in Sec 3.

Original: Notably, the 10 red seabream in Fig.
Revised: Notably, the 10 red seabream in Fig.

Original: 5(a: left 3) slowly circle around the cage center given the smallcage size, whereas the same 10 fish in Fig.
Revised: 5(a: left 3) slowly circle around the cage center given the small cage size, whereas the same 10 fish in Fig.

Original: 5(a: right 3) learn to swim rapidly and away from thecenter since the cage size is much larger.
Revised: 5(a: right 3) learn to swim rapidly and away from the center since the cage size is much larger.

Original: On the other hand, the red seabream in Fig. 5(b) form amore compact school and slowly circle around the entire small cage because of their larger quantity.
Revised: On the other hand, the red seabream in Fig. 5(b) form a more compact school and slowly circle around the entire small cage because of their larger quantity.

Original: In addition, we simulated two fictional scenes where fish with large body size disparity (Fig. 5(c))and multiple species (Fig. 5(d)) are mixed together.
Revised: In addition, we simulated two fictional scenes where fish with large body size disparity (Fig. 5(c)) and multiple species (Fig. 5(d)) are mixed together.

Original: With associated biological measurements andtrained policies, fish in each of the size and species groups exhibit distinct swimming patterns whileinfluencing those of other groups in the same fish cage.
Revised: With associated biological measurements and trained policies, fish in each of the size and species groups exhibit distinct swimming patterns while influencing those of other groups in the same fish cage.

Original: All fish have random body scales within the interval [0.9, 1.1] by default unless specified.

Original: Simulation vs Reality: We focus on the relationship between fish density in a cage and the resultingfish school states, and compare simulation results with data obtained in the field.
Revised: Simulation vs Reality: We focus on the relationship between fish density in a cage and the resulting fish school states, and compare simulation results with data obtained in the field.

Original: We use two kindsof schools in our simulations, sparse (0.59 fish / m 3 ) and dense (14.7 fish / m 3 ), where the densitiesare set to be consistent with the field work.
Revised: We use two kinds of schools in our simulations, sparse (0.59 fish / m 3 ) and dense (14.7 fish / m 3 ), where the densities are set to be consistent with the field work.

Original: In the real fish farms, the sparse cage is 6 .
Revised: In the real fish farms, the sparse cage is 6 .

Original: 5m × 6 .
Revised: 5m × 6 .

Original: 5msquare with a depth of 6m and contains 272 fish.
Revised: 5m square with a depth of 6m and contains 272 fish.

Original: The dense cage is an octagon with 6 . 5m edgesand a depth of 10m and contains about 30 000 fish.
Revised: The dense cage is an octagon with 6 . 5m edges and a depth of 10m and contains about 30000 fish.

Original: We quantify states of schools, and focus onmilling, characterized by regular circulation around an axis, and swarming, in which fish aggregatewith a weak polarization [47, 48].
Revised: We quantify states of schools, and focus on milling, characterized by regular circulation around an axis, and swarming, in which fish aggregate with a weak polarization [48, 49].

Original: Fig.6 shows sampled trajectories, a polar order parameter P givenby the norm of the average of direction vectors for each fish in a school, and a normalized angularmomentum M representing how regular the circulation is [48] for two schools with different densities.
Revised: Fig.6 shows sampled trajectories, a polar order parameter P given by the norm of the average of direction vectors for each fish in a school, and a normalized angular momentum M representing how regular the circulation is [49] for two schools with different densities.

Original: Details can be found in the Appendix.
Revised: Details can be found in the Appendix.

Original: The simulation result demonstrates that the collective behaviorof the school transitions from swarming to milling as density increases, and this tendency is in goodagreement with the real data. 
Revised: The simulation result demonstrates that the collective behavior of the school transitions from swarming to milling as density increases, and this tendency is in good agreement with the real data. Several trajectories of individual fish in real video in both sparse and dense schools are exhibited in Fig. 7, and they resemble the ones in simulation results drawn by the bold trajectories in the right panels.

Original: Moreover, we show the influence of the dominance hierarchy on fish behavior in Fig.
Revised: Moreover, we show the influence of the dominance hierarchy on fish behavior in Fig.

Original: We initializedthe scene with 20% of fish being dominant and the rest being subordinate based on the reporteddistribution of social hierarchies among fish in [38].
Revised: We initialized the scene with 20% of fish being dominant and the rest being subordinate based on the reported distribution of social hierarchies among fish in [38].

Original: The dominant individuals would occasionallychase their subordinate neighbor until colliding with or losing sight of the latter, which matches ourobservation of the fish behavior in underwater videos captured in aquaculture sites.
Revised: The dominant individuals would occasionally chase their subordinate neighbor until colliding with or losing sight of the latter, which matches our observation of the fish behavior in underwater videos captured in aquaculture sites.

Original: A demonstrationof the simulation scenes is also included in our supplementary video.
Revised: A demonstration of the simulation scenes is also included in our supplementary video.

Original: PPO vs SAC: We compare the results of simulations trained by PPO and SAC [49], in the defaultenvironment of coho salmon, in which 1000 coho salmon swim in a large octagonal cage with edgesof 3 meters and a height of 4.6 meters.
Revised: PPO vs SAC: We compare the results of simulations trained by PPO and SAC [50], in the default environment of coho salmon, in which 1000 coho salmon swim in a large octagonal cage with edges of 3 meters and a height of 4.6 meters.

Original: Both PPO and SAC eventually produced agents formingswimming patterns that resembled those of real coho salmon in reference videos, while the formeryielded more stable outcomes.
Revised: Both PPO and SAC eventually produced agents forming swimming patterns that resembled those of real coho salmon in reference videos, while the former yielded more stable outcomes.

Original: The same reward parameter setting used for PPO was initially appliedto SAC in the pretraining stage.
Revised: The same reward parameter setting used for PPO was initially applied to SAC in the pretraining stage.

Original: Although the SAC agents collected a higher average reward over[Figure] [Figure] episodes from an early stage of training than their PPO counterparts, they became over-trained inthe later phase of training and exhibited a circling pattern where all salmon tended to cruise in the same direction and align their body rotations, which would be unnatural in a real-world farmingcage setting.
Revised: Although the SAC agents collected a higher average reward over episodes from an early stage of training than their PPO counterparts, they became over-trained in the later phase of training and exhibited a circling pattern where all salmon tended to cruise in the same direction and align their body rotations, which would be unnatural in a real-world farming cage setting.

Original: This result was jointly caused by a higher sample efficiency and entropy maximizationstrategy of the SAC algorithm that encourages exploration.
Revised: This result was jointly caused by a higher sample efficiency and entropy maximization strategy of the SAC algorithm that encourages exploration.

Original: After adjusting the reward parametersaccordingly, SAC agents were able to swim in equally natural patterns as PPO agents after thepretraining and fine tuning stages.
Revised: After adjusting the reward parameters accordingly, SAC agents were able to swim in equally natural patterns as PPO agents after the pretraining and fine tuning stages.

Original: However, no significant improvement in the cumulative averagereward was observed in the fine tuning stage of SAC and its resulting value was lower than that of
Revised: However, no significant improvement in the cumulative average reward was observed in the fine tuning stage of SAC and its resulting value was lower than that of

Original: PPO as shown in Fig.
Revised: PPO as shown in Fig.

Original: A possible explanation is that the aggregated penalties from constrainingfactors such as boundary and neighbor collisions overwhelmed SAC agents as their off-policy naturerequired them to collect experience from previous episodes into the replay buffer for future updates.
Revised: A possible explanation is that the aggregated penalties from constraining factors such as boundary and neighbor collisions overwhelmed SAC agents as their off-policy nature required them to collect experience from previous episodes into the replay buffer for future updates.

Original: Underwater Simulation: The results of the experiments with each component of the environmentsimulation are illustrated in Fig.
Revised: Underwater Simulation: The results of the experiments with each component of the environment simulation are illustrated in Fig.

Original: Fig.
Revised: Fig.

Original: 9(a) shows how the underwater scene becomes more reddishbrown with higher concentrations of chlorophyll resulting in an increasing attenuation of blue.
Revised: 10(a) shows how the underwater scene becomes more reddish brown with higher concentrations of chlorophyll resulting in an increasing attenuation of blue.

Original: Ahigher value of sediment concentration makes the water cloudier as shown in Fig. 9(b).
Revised: A higher value of sediment concentration makes the water cloudier as shown in Fig. 10(b).

Original: Fig.
Revised: Fig.

Original: 9(c)depicts a timelapse of the scene.
Revised: 10(c) depicts a timelapse of the scene.

Original: Fig. 9(d) demonstrates the change in brightness with increasingdepth, since more light is absorbed at greater depths. 
Revised: Fig. 10(d) demonstrates the change in brightness with increasing depth, since more light is absorbed at greater depths. Fig.

Original: Fig. 9(e) shows the influence of changingsmall particle concentration on the brightness of the scene.
Revised: 10(e) shows the influence of changing

Original: A higher concentration of small particles
Revised: All fish have random body scales within the interval [0.9, 1.1] by default unless specified

Original: [Figure]
Revised: [Figure]

Original:  increases the amount of light scattered, both towards and away from the camera, causing the lightaround the sun to brighten and everything away from the light source to darken.
Revised: small particle concentration on the brightness of the scene. A higher concentration of small particles increases the amount of light scattered, both towards and away from the camera, causing the light around the sun to brighten and everything away from the light source to darken.

Original: Fig.
Revised: Fig.

Original: 9(f) shows theinfluence of large particle concentration, which works the same way as small particle concentrationbut has a smaller effect on light scattering. 
Revised: 10(f) shows the influence of large particle concentration, which works the same way as small particle concentration but has a smaller effect on light scattering. [Figure] [Figure]

Original: Fish Counting:
Revised: Fish Counting:

Original: We applied our modified YOLOv4 model [46] 4 trained with the synthetic dataset toreal footage.
Revised: We applied our modified YOLOv4 model [47] ** trained with the synthetic dataset to real footage.

Original: Fig.
Revised: Fig.

Original: 10 shows some results with the videos of coho salmon, yellowtail and red seabream.
Revised: 11 shows some results with the videos of coho salmon, yellowtail and red seabream.

Original: The trained fish counting algorithm estimated the number of fish in each frame, and the total numberof fish was computed by taking the max of the estimated values while the network processes all thevideo frames.
Revised: The trained fish counting algorithm estimated the number of fish in each frame, and the total number of fish was computed by taking the max of the estimated values while the network processes all the video frames.

Original: The counting results for coho salmons, yellowtails and red seabreams are 69:61, 22:32,and 7:7 (left:our system, right: manually counting) respectively.
Revised: The counting results for coho salmons, yellowtails and red seabreams are 69:61, 22:32, and 7:7 (left:our system, right manually counting) respectively.

Original: When the density of the fish is highand many occlusions are observed, the accuracy of the counting system drops.
Revised: When the density of the fish is high and many occlusions are observed, the accuracy of the counting system drops.

Original:  Conclusions and Future Work
Revised:  Conclusions and Future Work

Original: [Figure] We presented a bio-inspired fish simulation system with DRL which adapts to various conditionsautonomously by applying transfer learning, and which enabled us to simulate a broad range of fishbehaviors in many scenarios.
Revised: We presented a bio-inspired fish simulation system with DRL which adapts to various conditions autonomously by applying transfer learning, and which enabled us to simulate a broad range of fish behaviors in many scenarios.

Original: In addition, we introduced a physically-based underwater simulationwhich enabled us to simulate various undersea conditions.
Revised: In addition, we introduced a physically-based underwater simulation which enabled us to simulate various undersea conditions.

Original: Using these, we were able to synthesize alarge dataset with accurate annotations which we could use for computer vision tasks.
Revised: Using these, we were able to synthesize a large dataset with accurate annotations which we could use for computer vision tasks.

Original: We presentedone application, fish counting, in this paper, but the potential of this synthetic dataset created withour novel simulations goes beyond fish counting.
Revised: We presented one application, fish counting, in this paper, but the potential of this synthetic dataset created with our novel simulations goes beyond fish counting.

Original: We will extend the application to 3D detection,pixel-wise segmentation and tracking tasks in order to make more useful tools for fish farms. 
Revised: We will extend the application to 3D detection, pixel-wise segmentation, size estimation and tracking tasks in order to make more useful tools for fish farms. A limitation of the current pipeline is the manual setting of weights for reward functions.

Original: In thispaper, we excluded the effect of tidal currents from our simulations, and  the proposed simulationmethod is not applicable to  the fish behavior in feeding situation yet.
Revised: We will explore techniques that can dynamically adjust weights based on the performance of the DRL controller in the training process. Moreover, we chose not to apply any behavior cloning method because it was difficult to collect and annotate enough samples of fish trajectory data from real video, which was the main challenge we aimed to solve with the proposed method for the preparation of the dataset for computer vision models. However, the fine-tuning of the trained DRL model using behavior cloning with a small sample of the real fish behavioral data is an interesting method to enhance the reality of the simulation that can be worth working on. [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: In this paper, we excluded the effect of tidal currents and the influence of fish body temperature on movement dynamics from our simulations, and the proposed simulation method is not applicable to special occasions such as fish behavior in feeding and disease conditions yet.

Original: We plan to incorporate them tomake our simulation more biologically accurate.
Revised: We plan to incorporate them to make our simulation more biologically accurate.

Original: Checklist
Revised: Acknowledgment

Revised: We are grateful to K. Suda, T. Yoshida, G. Yasui and others from SoftBank Corp., R. Mizutani, T.

Original: For all authors...
Revised: Tsurumi, S. Sone, and others from Nosan Corporation for field works. We also thank M L. Eastman and S. Kakazu from SoftBank Corp. for their supports on computer graphics pipeline and M A. N.

Original:  (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: Kemas from NeuralX Inc. for computer vision models. Checklist For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See section the fish counting part of 5 and 6.
Revised: [Yes] See section the fish counting part of 5 and 6.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: No] We could not find anything.
Revised: [No] We could not find anything.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results.. (b) Did you include complete proofs of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results? [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [No] The code and the data are proprietary.
Revised: [No] The code and the data are proprietary.

Original: The instructions of logic to implement the code is included.
Revised: The instructions of logic to implement the code is included.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)?
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [No] Our research target is to generate fish schools, so we don’t need to report error bars.
Revised: [No] Our research target is to generate fish schools, so we don’t need to report error bars.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See the training part of Section 5. 
Revised: [Yes] See the training part of Section 5. 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] See Section 4 and 5.
Revised: [Yes] See Section 4 and 5.

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [Yes] See Section 4 and 5.
Revised: [Yes] See Section 4 and 5.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes] See Section 4 and 5.
Revised: [Yes] See Section 4 and 5.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [No] We collected and generated the data by ourselves, so it is proprietary.
Revised: [No] We collected and generated the data by ourselves, so it is proprietary.

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [No] All of our data is fish.
Revised: [No] All of our data is fish.

Original: These measurement methods, while designed to avoid injuring the fish.
Revised: These measurement methods, while designed to avoid injuring the fish.

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects.

Original: (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? 
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and  the total amount
Revised: Fish species prefer different temperatures and will change their swimming patterns or vertical alignment based on the surrounding water temperature [1]. Generally, adult salmonids prefer water temperature within the 8-9 degrees Celsius range [2], whereas yellowtail amberjack and red seabream like higher temperatures [3, 4]. Caged fish also show distinct swimming patterns at daytime and night.

Original: spent on participant compensation?
Revised: They move very little and stay closer to water surface after sunset, but gradually become more active and swim deeper after sunrise as the amount of sunlight increases [5, 6]. Additionally, when fish in the cage reach a certain crowding density threshold, they attempt to maintain their optimal personal space while still being able to move around [7] by forming a circular schooling pattern, consistent with the Boids model [8]. Fish normally react to their neighbors within a distance of 2 to 3 body lengths (BLs) [9, 10] and can learn to avoid cage walls [11, 12]. The reaction appears regulated by visual stimuli [7], as well as the lateral line system, which allows the fish to feel the vibrations of their neighbors [ 13].

Revised: Fish align their orientation primarily to the fish in front of them, and adjust speed as necessary to avoid collision with neighbors [10].

Revised: Caged fish like salmonids often settle into two or three vertical layers [14] since they typically group with similarly sized fish [15], where larger fish tend to swim at lower depths and smaller fish stay at higher depths [16, 17].

Revised: Moreover, the decision making interval of 95% of fish species is controlled by the Mauthner cells (M-cells) that trigger a rapid escaping movement (C-start) for risk avoidance and command neurons active in decision-making [18, 19].

Revised: Miller et al.

Revised: [20] studied zebrafish social groups and found that action potentials for swimming, which is generated by the neural circuit, lasted for a duration of 50-200 ms.

Revised: Since salmonids have lower body temperature whereas yellowtail amberjack and red seabream have similar body temperatures as zebrafish, we set 200ms as the the decision-making interval for coho salmon and 100ms as that for yellowtail amberjack and red seabream.

Revised: B Environmental Background

Revised: Colors underwater are determined by the intrinsic colors of objects and the scattering and absorption of light.

Revised: The main factors that determine the absorption and scattering are the properties of the water itself and the particulates in the water, usually chlorophyll and sediments [21].

Revised: Water absorbs light of different wavelengths at different rates, characterized by attenuation coefficients, which can be changed by the presence of particulates [22, 23].

Revised: The exact relationship between transmitted light and the light absorbed is expressed by the Beer Lambert law [21].

Revised: Chlorophyll absorbs blue light the most, followed by red, then green.

Revised: Thus a higher concentration of chlorophyll shifts the coloring in a reddish green direction.

Revised: Sediments in the water scatter the light, altering the amount of light absorbed as well as changing the attenuation coefficients.

Revised: Sediments and suspended solids also create turbidity and occlusion underwater.

Revised: In our model, sediment concentration is the main driving force behind turbidity.

Revised: C Simulation Setting

Revised: We created eleven simulation environments to assess the performance of proposed fish simulation framework and generate synthetic dataset for computer vision tasks.

Revised: The configuration of each environment is available in Table 1.

Revised: The first three environments reproduce the underwater scenes in fish farms we collected field data from and are used as default scenes to pretrain each of the three species.

Revised: The other eight environments are used to examine the adaptiveness of the framework and transfer learning is applied.

Revised: Values in the cage size column are in units of meter.

Revised: The biological and environmental parameters used in these scenes are summarized in in Table 2.

Revised: D Effects of Light Intensity and Temperature on Fish Behavior

Revised: A fish prefers to stay within a certain area of the tank where its light intensity preference matches the local light intensity computed by the Beer-Lambert law:

Revised: [Equation]

Revised: where I and I 0 denote the light intensity at the local depth and on water surface, a is an attenuation coefficient and d is the depth of fish.

Original: {+I 
+}
Revised: {+I 
+}

Revised: and α are valued based on the field data collected from

Revised: [Table]

Revised: [Table]

Revised: aquaculture sites as described in Sec.

Revised: I.

Revised: Any deviation of I from the light intensity preference range results in a vertical delta velocity ( ∆ v light ) for the fish as shown in Eq.

Revised: [Equation]

Revised: where parameters I lsteep and I hsteep control the steepness of the reaction to light and are set in an identical manner to [9].

Revised: I lpref and I hpref denote the lower and upper bounds of the light preference interval and are valued based on the biological studies of each simulated species [24, 3, 4, 5].

Revised: Similarly, a fish likes to stay in the area where the local temperature satisfies its temperature preference.

Revised: We calculate the temperature induced vertical delta velocity ( ∆ v temp )

Revised: [Equation]

Revised: where the under-water ambient temperature ( T ) is obtained using a two-layer method based on the studies by [25] and [26].

Revised: The temperature response steepness parameters ( T lsteep and T hsteep ) are set to be the same as those in [9], and the bounds of the temperature preference interval ( T lpref and T hpref ) are determined according to [3], [4] and [2].

Revised: E Deep Reinforcement Learning

Revised: Proximal Policy Optimization Proximal Policy Optimization (PPO) algorithm

Revised: [27] improves upon vanilla policy gradient method by providing more stability and reliability during learning.

Revised: It is a variant of Trust Region Policy Optimization (TRPO) algorithm [28] with significantly simpler implementation by optimizing a surrogate loss instead of KL divergence constraints.

Revised: In our work, the clipped surrogate loss L CLIP ( θ ) with respect to the current policy parameters θ is defined as follows:

Original: [N/A]
Revised: [Equation]

Revised: π θ and π θ old are current and old policies.

Revised: l t ( θ ) is the likelihood ratio of the action probability under current policy over that of old policy.

Revised: As the new policy deviates from the old policy, l t ( θ ) will deviate from 1 and be constrained by the interval [1 − ϵ, 1 + ϵ ] .

Revised: This means whenever excessive policy update happens the gradient will be set to zero and therefore prevents the new policy from being too different from previous one.

Revised: ˆ A t represents the generalized advantage estimation [29].

Revised: A minimum value is then chosen to obtain a lower bound of loss.

Revised: Since the parameters between the policy and value function are shared in networks, we further integrate the squared-error loss of value function ( L VF t ( θ ) ) and an entropy term ( S [ π θ ]( s t ) ) ensuring sufficient exploration to obtain the final objective loss:

Revised: [Equation]

Revised: where value function loss coefficient c is valued at 0.5 and entropy coefficient β is valued at 0.0005.

Revised: Parameters in Action and Reward Definition

Revised: The values of parameters used in action definition and reward function for PPO are listed in Table 3.

Revised: The weight of each reward term is set according to the objective of each training.

Revised: For example, we set the boundary avoidance weight to be the highest when the DRL controller was trained for the very first time because the primary goal of the training for the fish was to avoid the collision to the fish cage.

Revised: When it is learnt, we increased the weights for the neighbor collision penalty and energy consumption penalty so that fish learn the smooth schooling in an environment as well as energy saving, which is an important benefit of being in a school [30].

Revised: PPO Training Configuration We present the training hyperparameters of PPO in Table 4.

Revised: Similar configurations were used for pretraining and transfer learning phases except the total number of steps when training coho salmon.

Revised: This is primarily due to the variation in environment configurations, which makes retraining necessary.

Revised: We also include an illustration of the learning curves of the three species in Fig.

Revised: 1 to showcase the performance improvements over the entire training process.

Revised: SAC Training Configuration

Revised: We report the training hyperparameters of Soft Actor-Critic (SAC) algorithm [31] that we employed to train the fish agents aside from PPO in Table 5.

Revised: Since we only compare the results of SAC and PPO in the default environment of coho salmon, the total numbers of time steps for SAC agents in pretraining and fine tuning phases are also four and one million.

Revised: [Table]

Revised: F Physically Based Environment Simulation

Revised: Light Attenuation The simulation colors the water by first marching rays in fixed distance steps for every pixel on screen.

Revised: The ray’s direction is determined based on the pixel’s location when transformed into world space.

Revised: The current color of the pixel is then treated as incoming light, and becomes the input for the attenuation calculation using the Beer-Lambert Law shown in Eq.

Revised: I 0 represents the incoming light (or in this case the current associated pixel color) of the ray, a (or in this case a r/g/b ) is the attenuation coefficient of red, green or blue color in the vector ac which is determined by three components that we explain next, and d is the distance the ray of light has traveled through the water [32].

Revised: Then the simulation calculates the ambient light at the current point the ray is at.

Revised: This is calculated by first summing up all the light that is heading towards the current point, and multiplying it by the dot product of the sunlight direction and ray direction.

Revised: Next, the output of the ambient calculation is multiplied by the output of the attenuation calculation and is then added to a running total.

Revised: Then the ray marches for a step in its set direction and the process repeats until it reaches its maximum distance.

Revised: Once that is done, the running total is returned as the output color for the pixel associated with the given ray.

Revised: [Table]

Revised: Notably followed by one million during transfer learning because of its larger quantity.

Revised: Nevertheless the other two species were trained for two million steps in both phases.

Revised: The attenuation coefficient vector consists of three parts: the attenuation of the water itself, the attenuation of chlorophyll, and the attenuation of the suspended sediments.

Revised: Each part is a 3 dimensional vector representing the attenuation coefficients for red, green, and blue.

Revised: The values of these coefficient vectors are determined based on data found in [32, 33, 34].

Revised: The coefficients of each are combined together, with the chlorophyll and sediments both multiplied by the concentration of their respective parameters using the model described in [32] and [35].

Revised: The formula is as follows:

Revised: [Equation]

Revised: where ac bw is the attenuation coefficient vector of the base water, co chl is the concentration of chlorophyll, ac chl is the attenuation coefficient vector of chlorophyll, co sed is the concentration of the sediments, and ac sed is the attenuation coefficient vector of sediments.

Revised: [Table]

Revised: Time horizon

Revised: Total number of steps (pretraining) 4 million Total number of steps (fine tuning) 1 million

Revised: The factors such as the concentrations of sediments and chlorophyll are set by the simulation to random values from within user defined ranges.

Revised: These ranges are based off values found in real world, like those laid out in [33] and [35].

Revised: The scene uses these factors to determine everything else about the environment.

Revised: For example, the turbidity of the water is determined by the concentration of suspended sediments.

Revised: Light Scattering A light scattering system computes the light refraction based on the medium’s properties such as the particles inside a fluid [36].

Revised: It uses a phase function, which can be obtained from a volume scattering function (VSF), to calculate the amount of light traveling through a medium and being scattered towards the viewer.

Revised: We use the Kopelevich model [37] as the VSF to determine how different wavelengths of light are scattered underwater using large and small suspended particles.

Revised: It can be formulated as the following equation [38]:

Revised: [Equation]

Revised: where β w , β s and β f denote the VSF of pure sea water, small and large particles respectively and their values can be selected from tabular values in [37] by performing visual inspection.

Revised: v s and v l are the concentrations of small and large particles (in units of parts per million).

Revised: ψ is the angle between the light and view directions, and λ is the wavelengths of red, green and blue.

Revised: The resulting VSF is stored in a vector that is used to calculate the scattering coefficients for the three colors via numerical integration using the trapezoidal rule.

Revised: Finally, the VSF vector is divided by the scattering coefficients to obtain the phase function for each of the colors, which is then used to multiply the output of the ray marching system to scatter the color based on the RGB value.

Revised: G Characteristics of fish school states

Revised: Data provided by the simulation are body length l BL , position x = ( x, y, z ) of fish body center, speed v and its direction vector d = ( d x , d y , d z ) ( | d | = 1 ) for each individual fish and each time slice.

Revised: The velocity vector is v = v d , and the deviation from the center of the school x c = 1 N (cid:80) Ni

Original: {+=
+}
Revised: {+=
+}

Revised: x

Revised: i is r i

Revised: = x i − x c , where N denotes the number of fish in the school.

Revised: Order parameters According to [39, 40] two parameters, a polar order parameter P and a normalized angular momentum M of the school, are used to characterize state of school.

Revised: The parameters P and M are given respectively as

Revised: [Equation]

Revised: in the same manner as in Ref.[40].

Revised: All fish move in the same direction in parallel when P ≃[39, 40], which cannot be observed in a fish cage.

Revised: A large M ≃ 1 and small P ≃ 0 brings about the milling state, in which fish in a school rotate around an axis regularly [39, 40].

Revised: The smaller M and P indicate that the state is swarming state rather than milling state [39].

Revised: Similarity between pictures of real and the simulation To see how the simulation results are consistent with the real data obtained from the field work, we compare the images obtained by both of them.

Revised: These images are fish-eye lens view, and black and white images are made with some threshold from them.

Revised: We compare the density of black pixels for each bin obtained by dividing each image into 16 annuli.

Revised: As a result exhibited in Fig.

Revised: 2, the simulation image seems to be consistent with the real one.

Revised: H Fish Counting

Revised: Image Pre-Processing The first module converts the input video to a sequence of images and applies denoising to produce the data format required by the fish detection module.

Revised: The denoising process applies median blur to the images to reduce the effect of noise, resulting from caustics and sediment and other particles, often seen in underwater photography.

Revised: [Figure]

Revised: Fish Detection Fig.

Revised: 3 shows the network architecture of the fish detection model, based on

Revised: YOLOv4 [41].

Revised: The input to the model is a sequence of images, and the output is a list of proposed bounding boxes ( x, y, w, h ) , where ( x, y ) is the coordinate of the top left corner of the bounding box, and w and h are the width and height of the bounding box, respectively, with an associated class label and confidence score.

Revised: We set the confidence score threshold to be 0.5 and any proposed regions below the threshold are discarded.

Revised: In order to solve the issue of false negatives in the case of smaller

Revised: , a is

Revised: The

Revised: [Equation]

Revised: where SI width and SI height are the width and height of the sliding window, and I width

Revised: and I height are the width and height of the original image, respectively.

Revised: s is a step-size term and is set to be half of the sliding window size.

Revised: This technique magnifies small fish in the original images.

Revised: The computation time increases by a factor of R when processed serially; however, it helps to reduce the number of false negatives which is a major problem with the original YOLOv4.

Revised: Note that we concatenate all the results from the cropped images into a single array, and apply Non-Maximum Suppression (NMS) to produce a single output, unlike the standard YOLO where the NMS is applied just after a single detection block.

Revised: Fish Counting After the completion of the detection module, the estimation of fish count is tabulated for each frame, and the maximum value is taken as the final estimate of the number of fish in the cage.

Revised: Despite its simplicity, we found the MAX method to give us the highest counting accuracy.

Revised: Training We trained the model for 2000 epochs on a single Nvidia GeForce GTX 1080 Ti with an initial learning rate of 0.001.

Revised: The training process completed in 5 hours.

Revised: We then manually annotated 5 frames from 5 videos recorded at a real fish farm–a total of 25 images–and used them to fine-tune the model.

Revised: I Field Data collection

Revised: We collected data on four species (two salmonids, yellowtail, red seabream) at marine fish cages.

Revised: The salmonids cages are in the north, and the yellowtail and red seabream are in the south.

Revised: Our goal was to measure the physical properties and dimensions of the environment and fish, to set the parameters of our simulation, and to record video of caged fish schooling behavior.

Revised: In this appendix, we will discuss that data collection process and show our results.

Revised: Location In March, May and July of 2021, we collected data from coho salmon and trout salmon cages on a northern fish farm (latitude 38.465, longitude 141.480).

Revised: Both fish cages were cubes with a side length of 6.5m (Fig. 4(a)).

Revised: Then, in June and November of 2021, we collected data from yellowtail and red seabream cages on a southern fish farm (latitude 34.215 and longitude 136.386).

Revised: Both cages were cubes with side lengths of 3m (Fig. 4(b)).

Revised: Equipment Fig.

Revised: 5 shows the equipment used.

Revised: Fig.

Revised: 5a is circuit board used to measure light intensity.

Revised: A Raspberry Pi is used to collect data from the TSL2571 light intensity sensor.

Revised: To help it withstand the water pressure, the circuit board was placed in a clear smartphone case with the sensor positioned to avoid being obstructed.

Revised: Additionally, the case was placed in a waterproof housing.

Revised: After waterproofing, the sensor was attached alongside cameras to PVC pipes and used to measure light intensity at various depths underwater.

Revised: Fig.

Revised: 5b shows the device used to measure water temperature, the SK SATO SK1260.

Revised: The sensor must be in direct contact with sea water, so it was lowered into the ocean.

Revised: To film video of the fish schooling behaviors, we used KODAK 4KVR360 cameras.

Revised: A camera placed at the center of the floor of the cage pointing up (Fig.5c) was fixed in place with four ropes.

Revised: The cameras placed at the sides of the cage facing the center (Fig.5d) were fixed to PVC pipes and lowered to the target depth.

Revised: When filming from multiple depths simultaneously, the cameras were fixed to long PVC pipes at 1m intervals, as in Fig.5f.

Revised: The PVC pipe fixtures are shown in Fig.5e.

Revised: By fixing the T-shaped part to the iron pipes of the cage, long video shoots are possible.

Revised: The cameras are placed at R, G, B, and Y as shown in Fig.

Revised: 4c.

Revised: Environment Data Fig.

Revised: 6a shows an example of the results of our light intensity measurements.

Revised: The measurements were taken at the southern fish farm in November 2021, starting at 11:10 AM.

Revised: The x-axis shows the number of minutes since the start, while the y-axis is the light intensity (Lux).

Revised: Light blue shows the readings at a depth of 5m (center bottom), red shows readings at 3m (side), yellow shows readings at 2m (side), and black shows readings at 1m (side).

Revised: It can be seen that light has lower intensity at greater depths.

Revised: Sudden noisy drops in brightness are likely the result of fish blocking light from reaching the sensor.

Revised: Fig.

Revised: 6b shows the temperature at each depth.

Revised: The top green line is data from the southern farm in June.

Revised: The rest are data from the north farm, recorded in (from bottom to top)

Revised: March, May, and July.

Revised: Although there are seasons without any temperature variation, at other times the temperature can differ by 3-4 degrees.

Revised: During the period from May to July, when

Revised: [Figure]

Revised: [Figure]

Revised: water temperature exceeds 10 degrees, the chlorophyll content of the water increases, lowering the clarity of the water.

Revised: Body

Revised: Measurements

Revised: and Fish

Revised: Count The number of fish in a fish farm is unclear.

Revised: Fish are purchased by weight as juveniles.

Revised: This weight is divided by an average weight to give an approximate count, but this can result in up to 50% error.

Revised: A fish’s value on the market is determined by weight.

Revised: Therefore, fish farmers try to monitor the state of their fish (length, height, width, and weight) in order to optimize their feeding, but the process is dependent on farmers’ intuition.

Revised: The only way to accurately measure a fish is by hand after removing it from the water 1 .

Revised: To do so, fish are scooped up by a net a few fish at a time; when the fish are released again, they are counted by eye.

Revised: When populations are small this process can be performed for all fish, but it is not realistic for more than 1000 fish.

Revised: For such cases, the process is performed for approximately 10% of the fish and the result is multiplied by 10, giving an inexact value.

Revised: Fig.

Revised: 7c shows the current fish counting process.

Revised: Fish body measurements are even more difficult.

Revised: To measure a fish, the fish is removed from the water and anesthetized to prevent injury, and measured by hand with a tape measure (see Fig. 7).

Revised: Fig.

Revised: 8 shows the body lengths of 10 randomly selected fish.

Revised: Fig.

Revised: 8a shows the results for salmon (coho salmon and trout salmon), and Fig.

Revised: 8b shows the results for yellowtail and red seabream.

Revised: The x-axis shows the month the measurement was performed.

Revised: The lines show the range of body lengths, and the symbol is the average length (coho salmon: gray, trout salmon: orange, yellowtail: yellow, red seabream: red).

Revised: Salmon farming begins in December, and they are shipped to market in July.

Revised: The graph shows that there exists large variation in salmon body length, which gets smaller as July approaches.

Revised: The shipping date for yellowtail and red seabream is not fixed; they are shipped once they reach a set size.

Revised: [Figure]

Revised: [Figure]

Revised: Video Recording Videos at the fish farm were filmed with KODAK 4KVR360 cameras.

Revised: One camera was placed at the bottom center of the cage, facing the ocean surface.

Revised: The others were attached to PVC pipes placed at the center of each side at 1m intervals.

Revised: The footage captured in

Revised: March, May, and July 2021 on the northern farm and June 2021 on the southern farm was filmed in the KODAK 4KVR360 235 degree dome mode.

Revised: The November 2021 footage on the southern farm was filmed in the KODAK 4KVR360 155 degree front mode.

Revised: The coho salmon, yellowtail, and red seabream footage is show in Fig.

Revised: The cameras are placed at R, G, B, Y in Fig.4c.

Revised: The salmon and yellowtail footage was taken at a depth of 3m, while the red seabream footage was taken at a depth of 2m. The footage at these depths contained the largest number of fish for each species, showing that the preferred depth differs per species.