# Continuous Autonomous Ship Learning Framework for Human Policies on Simulation
## 1 Introduction
### Challenges  
In this study, an autonomous vessel moving to its destination was considered, taking into account the topographical features of the marine environment; the vessel follows the navigation rules in a human-like manner. Each harbor has a different marine traffic environment, which is defined by humans, and there are rules with regard to other ships the vessel encounters. These rules are not provided as signs or traffic lights in marine environments or ports. The typical method is to observe the behavior of other ships, learn the rules, set sail with nearby ships, or wait until all the ships have arrived [15]. However, this process is time-consuming, and the simulation is expensive to implement if all these environments have been considered. In addition, it is difficult to build an environment by considering all the cases [16; 17; 18]. Furthermore, previous learning experiences can be forgotten depending on the order of learning (catastrophic forgetting), and new environments can be learned [19; 20]. The following is a summary of the issues to be considered relating to the learning environment, method, and the structure of the RL algorithm to learn for the application of RL algorithms to autonomous ships.  
The first is the data of the RL algorithm, which is insufficient to learn the rules defined by humans make decisions. The sailing environment to the destination considers the topographical features and the natural environment. Surrounding vessels, which have rules embedded in them, should respond to an autonomous learning ship. These vessels present a variety of vessel types and behaviors.  
Second, learning a human rule requires a systematic learning method. Random learning can provide a wide range of experiences by providing different states, but the rules are hard to learn. Therefore, the learning environment must have rules and should be consistent.  
The last issue is the complexity of decision-making for autonomous ships to operate. Autonomous ships shall plan a destination at the human level, decide the direction and speed by considering obstacle avoidance, sliding of the water surface, and control of the postures depending on the features of the vehicle itself to achieve the given missions. The more data provided, the larger the state space becomes. The decision-making space is also expanded depending on the number of actions. Substantial learning time may be required, or obscure results may be acquired because the learning results are not collected.  
To solve these problems, previous studies have presented an environment that includes various partner ships for ship avoidance and route planning [21; 22]. Simulation with other ships (including strategies) can make autonomous ships learn human-defined navigation rules in the environment. Through this research, we expect to not only learn a solution to solve the aforementioned problems but also learn human-defined navigation rules at the human level [23; 24; 25]. Autonomous cars and robots have developed separate training platforms for continuously acquiring data and training in a given environment to solve learning environment problems, or the implemented simulation using OpenAI's GYM based on open sources provides a relative vehicle environment [26, 27, 28].