# Algorithm for Base Action Set Generation Focusing on Undiscovered Sensor Values
## 2 Base Action Set Generation Algorithm Focusing on Undiscovered Sensor Values
### Random Motion Generation Algorithm for Comparison  
We define a random motion generation algorithm for comparison before explaining the proposed algorithm. The use of a simple random number as actuator output signals does not work for robot motion in most cases. Therefore, we use a Fourier series to generate random actuator output signals by determining the Fourier coefficients using uniform random numbers. A variety of waves are generated in this way. The \(j\)th actuator output signal is determined as Equation (14). \[a_{j}(t)=b^{(j)}+\sum_{k=1}^{n_{k}}\left(c_{k}^{(j)}cos(kt)+d_{k}^{(j)}sin(kt)\right) \tag{14}\]  
The operation to generate a random action fragment of length \(l\) using this method is denoted as \(f_{rnd}(l)\). Coefficients \(b^{(j)},c_{k}^{(j)},d_{k}^{(j)}\) are reset using uniform random numbers each time \(f_{rnd}(l)\) is used. The random motion generation algorithm for comparison is referred to as \"Algorithm Random\" in this paper, and Algorithm Random uses \(f_{rnd}(l)\) multiple times to generate actuator output signals of necessary length (Figure 1). Base Action Set Generation Algorithm to Extract Actions That Cause Changes Effectively in the External Environment and to Combine Those Actions  
We developed a base action set generation algorithm focusing on finding processes of undiscovered sensor values. First, when a robot finds undiscovered sensor values, we assume that some of the actions that effectively change the external environment are generated around that time. Then, the algorithm extracts a part of the actuator output signals before the undiscovered sensor values are found. These extracted parts are used to generate new actuator output signals by combining them. The newly generated signals should effectively change the external environment and enable finding undiscovered sensor values. A discovery of new sensor values is defined as follows. First, we divide the \(m\)-dimensional sensor space of a robot that has \(m\) sensors into several parts and assume each part as a bin of a histogram. This histogram is denoted as \(D_{m}\). Each bin of the m-dimensional histogram \(D_{m}\) is denoted as \(b(i_{1},i_{2},...,i_{m})\), and the number of data in each bin is denoted as \(n_{b}(i_{1},i_{2},...,i_{m})\). The \(m\)-dimensional histogram \(D_{m}\) at time \(t\) is denoted as \(D_{m}(t)\). The sensor value at time \(t\)\(s(t)\) is allocated to the corresponding bin. We assume that the corresponding bin of sensor value \(s(t)\) is \(b(i_{1},i_{2},...,i_{m})\). If \(n_{b}(i_{1},i_{2},...,i_{m})=0\) in \(D_{m}(t-1)\), the sensor value \(s(t)\) is an undiscovered sensor value. Next, we explain the operations to extract the part of the actuator output signals that have contributed to finding undiscovered sensor values. When a sensor value at time \(t\)\(s(t)\) is an undiscovered sensor value, this operation extracts a part of the actuator output signals from time \(t-l_{u}\) to time \(t\) as a contributed part to find new sensor values. This part is constructed as a sub action fragment of \(\mathbf{F}\) that records all actuator output signals from time \(t=0\). This sub action fragment \(l_{i}\) is defined as \(l_{i}=\mathbf{F}[t-l_{u}:t]\) (Figure 2a). \(l_{i}\) is added to extracted action fragment set \(U_{I}\). The part of the actuator output signals that contributed to finding undiscovered sensor values is extracted and maintained using these operations and used to generate new actuator output signals. This algorithm uses elements of extracted action fragment set \(U_{I}\) when an actuator output signal is newly generated. Now, we describe generating action fragment \(\mathbf{F}_{p}\) of length \(l\). \(\mathbf{F}_{p}\) is generated according to Equation (15)-(17), where \(n(U_{I})\) is the number of elements of extracted action fragment set \(U_{I}\) and \(l_{g}\) is a uniform random number in \([l_{min}:l_{max}]\). \[\mathbf{F}_{p}^{\prime}=\sum_{j}g(j) \tag{15}\]  
\[\mathbf{F}_{p}=\mathbf{F}_{p}^{\prime}[0:l] \tag{16}\]  
\[g(j)=