# Simpler Learning of Robotic Manipulation of Clothing by Utilizing DIY Smart Textile Technology
## 5 Results  
As described in Section 3, the task consists of folding a rectangular textile piece by using a low-cost dual robotic arm. The arms are positioned in an opposing manner to avoid a collision. In order to avoid damaging the smart textile, we limited the movement of each robot arm to three degrees of freedom (DOF) instead of the available five DOF. Consequently, the movements of the robots are limited to one plane. In order to find the control policy for folding, we formulated the problem as a fitted Q-learning problem. The state-space \(S=\mathbb{R}^{6}\) is defined by the joint space of the dual-arm robot, with six motor angels in total. For each joint, we define two actions: \(\pm\Delta\) with \(\Delta\) equal to \(4^{\circ}\). The reward function \(R\) is sparse and defined by  
\[R= \begin{cases}100&\text{on success}\\ -1&\text{penalty for wandering}\end{cases} \]  
We approximate the Q-function with a simple MLP with six input neurons (one for each of the controlled joints), two hidden layers with 128 and 64 ReLu neurons, respectively, and 12 output (linear) neurons (one for each of the possible actions). In order to find a good set of parameters, we implemented a simple forward kinematic model of the robot arms in a reaching task setting. This allowed us to sweep across various hyperparameter settings. The optimizer we used is stochastic \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Samples** & **Precision** & **Recall** & **f1-Score** \\ \hline Unfolded & 342 & 0.98 & 0.98 & 0.98 \\ Folded & 169 & 0.96 & 0.95 & 0.96 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Test set results of the linear classifier that detects whether the cloth has been folded (511 samples). The high classification performance means that the smart textile is suitable to be used as a reward function. Figure 5: The average and standard deviation of the collected data from the 12 by 7 tactile cells show a higher voltage in the center column—where the fold is—when the cloth is in the folded state. These figures qualitatively indicate that the tactile cells in the textile react to different folding configurations. gradient descent with a learning rate of 0.01 and batch size of 64 samples. The target network is updated each 1000 steps. Exploration is done using an \(\epsilon\)-softmax policy with 5000 exploration steps. Thus, the agent will first execute fully random motor actions and gradually start choosing actions that maximize the expected q-values. After the selection of the hyperparameters, we trained our dual robotic arm setup from scratch in the real world. As shown in Figure 5(a), after approximately 60 episodes (17,000 steps, or 8 h), the robot successfully learned to complete the folding task. Figure 5(b) shows the number of steps the robot needs to fold the cloth per episode. The low dimensional state space of the agent and the learned reward function from tactile information using supervised learning allows the agent to learn to fold the cloth after relatively few training examples. An example trajectory of the learned policy is shown in Figure 7. The action probabilities show that at the start of the episode, the agent primarily moves the shoulder joints, as they lead to the largest movement of the end-effector. This behavior has the benefit that it minimizes the penalty associated with wandering or other suboptimal movements. As the episode progresses, the agent has a higher probability of actuating the elbow and end-effector motors for more refined movement. The end-effector trajectories, shown in yellow, are not smooth given that the policy is an action probability function calculated over the Q-values of the neural network. Because there are multiple reaching points in the plane to fold the cloth, the agent has multiple options to move during the first part of the episode, leading to jagged end-effector trajectories. As our experiment illustrates, it is possible to avoid the need for complex vision-based state estimation to learn difficult tasks such as the folding of clothing with low-cost robots.