## 2 Problem Specification
### Robotic Platform  
For the development of this work, a robotic simulator that allows realistic simulations of multiple environments was implemented. Some ideas from the Argos simulator ([[https://www.argos-sim.info/](https://www.argos-sim.info/)]([https://www.argos-sim.info/](https://www.argos-sim.info/)) accessed on 22 March 2021) --a simulator oriented towards large swarms mainly based on realistic physical engines--were taken as a basis. Specifically, our simulator uses the realistic Chipmunk ([[https://chipmunk-physics.net/](https://chipmunk-physics.net/)]([https://chipmunk-physics.net/](https://chipmunk-physics.net/)) accessed on 22 March 2021) physical engine, which is one of the fastest on the market. It is designed to execute hundreds of instances of simulations to be able to parallelize the RL algorithms. This is a basic requirement to use a robotic swarm simulator for reinforcement learning tasks.  
A holonomic robot with two active wheels and one for support was modeled. The shape of the robot was approximated to a 10 cm radius circle with 650 grams of weight. A low-level controller was used to alter the robot speed to move each agent. On this basis, our simulator provided a differential controller that allowed the speeds of the wheels \((vl,vr)\) to be calculated given the required rotational and translational speeds.  
A differential drive consists of two drive wheels mounted on a common axis, where each wheel can be independently driven either forward or backward. With this configuration, a robot must rotate (at a rate of \(\omega\)) about a point that lies along its common left and right wheel axes, called the instantaneous center of curvature (\(R_{ICC}\)). By varying the velocities of the two wheels, we were able to vary the trajectories that each robot took. There are three interesting cases with these kinds of drives: if \(vl=vr\), then the robot develops a forward linear motion along a straight line. If \(vl=-vr\), then \({}_{R}=0\), and we have rotation about the midpoint of the wheel axis. If \(vl=0\), then we have rotation about the left wheel (which is symmetric with \(vr\)). At time \(t\), we can predict the next position of the robot \(t+\delta t\):  
\[\left[\begin{array}{c}R_{x}^{\prime}\\ R_{y}^{\prime}\\ R_{x}^{\prime}\end{array}\right]\quad=\quad\left[\begin{array}{ccc}\cos( \omega\delta t)&-\sin(\omega\delta t)&0\\ \sin(\omega\delta t)&\cos(\omega\delta t)&0\\ 0&0&1\end{array}\right]\left[\begin{array}{c}R_{x}-R_{ICCx}\\ R_{y}-R_{ICCy}\\ R_{a}\end{array}\right]\ +\ \left[\begin{array}{c}R_{ICCx}\\ R_{ICCy}\\ \omega\delta t\end{array}\right] \tag{1}\]We can describe the position of any robot capable of moving in a particular direction \(\alpha(t)\) at a given velocity \(V(t)\) as  
\[R_{x}(t) =\int_{0}^{t}V(t)\cos[\alpha(t)]dt \tag{2}\] \[R_{y}(t) =\int_{0}^{t}V(t)\sin[\alpha(t)]dt\] \[R_{\alpha}(t) =\int_{0}^{t}\omega(t)dt\]  
Unfortunately, a differential drive robot imposes nonholonomic constraints on the establishment of its position; thus, answering the question of how to control the robot to reach a given configuration is not trivial (we cannot simply specify a robot pose and find the velocities needed to move to this position). This consideration motivates the classical strategy of moving the robot along a straight line, rotating in place to perform a turn, and then moving straight again, repeating this process until the goal is reached.  
Finally, it is worth mentioning that all robots had a laser sensor based on the SLAMTC RPLIDAR A1 device, set to a detection range of 0 to 4 m of \(360^{\circ}\). We discretized this information to use 13 angular readings to develop this behavior.  
Both the actuator and the robot range sensor were realistically modeled, taking noise models into account. Specifically, the models used were based on the perceptual and actuator models presented in [9], adjusted to our physical robots.