## 2 Materials and Methods
### Procedure  
Using a combination of existing deep learning solutions and low-level computer vision algorithms, we developed the following set of procedures:  
1. The recorded games contained different camera perspectives as well as different sequences of the game; there were not only played rallies, but also replays, set breaks and breaks between rallies. To detect only the positions of the players in the played rallies, the first step was to identify the table. For this purpose, we first extracted a set of straight lines from each frame using a Hough transform. Due to the typical camera position looking down onto the table, orthogonally to the net, the table induced a distinct set of lines. The absence of this line configuration indicated that the frame did not contain an active rally and the frame was not processed further ([34] for image processing steps) (Figure 1). 2. Next, the table was detected in each of the remaining frames. The lines computed in the previous step were used again to find the potential pairs of lines that could indicate the table. In Figure 1, all of the horizontal and vertical lines are specified. Green lines were the likely options for the table, whereas the blue lines were ignored. For each pair of potential lines, we finally computed whether they surrounded an area that was mainly uniform in color (e.g., green/blue as a table tennis table). 3. From all valid frames, the system detected all of the people using the publicly available state-of-the-art library Detectron2, developed by Facebook Research [35]. As can be seen in Figure 1, the players, as well as the referee (or many viewers in other videos) around them, were detected (yellow circles). Simple heuristics and tracking were applied to select just the two players out of all of the detected people. 4. Using the position of the table in each frame, the 2D image plane position of the players could be transformed into a 2D top view plane. We created this by projecting the 2D hip position in the image onto the detected 3D table plane, with the assumption being that players' hips were roughly at table height. To define the positions of the players, a coordinate system was established which was relative to the table. Here, the zero point represented the exact center of the table tennis table. Since the camera was in constant motion in these videos, tracking the position of the table was crucial. 5. The entire video was then split into segments, i.e., single rallies that were used to create heatmaps, such as in the example below (Figure 2). The four frames represented a rally that was around five seconds long and resulted in the player positions depicted on the right:
6. To create the heatmap in Figure 2, the sequence of positions of each segment was converted into a probability map. For this purpose, we created an empty occurrence map of size \(3.5\times 4\) m. Each of the athletes' (x, y) positions then incremented the count of occurrences around an area of 4.5 cm2 around (x, y) by 1. The occurrence map was then normalized by dividing it by the overall sum of all the counts in the map, resulting in the probability map. 7. To combine the heatmaps from multiple rallies, time stamps were manually collected for the sets of each of the games. A set started when the ball was thrown up during the first serve and ended when the ball was terminated by losing or winning the last point of the set (e.g., bouncing on the floor, landing in the net). Variations in annotating the exact start and end frame of each set could be neglected to the millisecond because each heatmap consisted of positions over an average of 48 min per player. Each game of over 100 segments per set (interrupted by breaks, replays, viewpoint variations) was thus combined into the game sets for each player (5 sets for 1 player each). These heatmaps could be used to compare players with different playing styles. As dependent variables, we established the mean value of the x and y coordinates of a player per set, as well as the area in which the players moved.