# An Augmented Warning System for Pedestrians: User Interface Design and Algorithm Development
## 2 Materials and Methods
### Projection Algorithms Development  
To ensure that the interface provided instant warning information to the pedestrian based on his/her location, an algorithm was developed to determine the projection of the vehicle's path on the AR interface based on the instant pedestrian position. Global positioning system (GPS) data and inertial measurement unit (IMU) data from the vehicle were used to determine the vehicle pathway using the perspective projection method [29]. The GPS and the IMU from AR glasses can be used to track the pedestrian's instant location and their head rotation/facing direction, respectively. It is assumed that the pedestrian does not look to the side without rotating their head, thus the pedestrian's head rotation can be used to represent their viewing direction. This system would be activated after the potential conflict is detected and the conflict point is identified. The algorithms for conflict detection and pedestrian/vehicle positioning are not introduced here because they are beyond the scope of this study. The focus of this study was developing algorithms and interfaces to correctly and efficiently present relevant warning information on the AR display. Overall, the data used in the system included vehicle position and orientation, pedestrian's position and orientation, pedestrian's facing direction, predicted conflict position if no action was taken, and predicted turning position. Predicted conflict position and predicted turning position were assumed already known from conflict detecting algorithms. In detail, a two-step algorithm was developed to first convert and synchronize the vehicle position, the conflict position, as well as the vehicle turning position into the pedestrian's local coordinates and further project the localized information on the projection plane using perspective projection methods. The local coordinates used the pedestrian's position as the origin (0,0), the pedestrian's facing direction as +z, and the pedestrian's right-hand direction as +x. After the coordinates' transformation, the warning information needed to be projected on the projection plane. Different methods can be used to project the 3D world onto a 2D plane. However, many of these methods may lose the depth cue during the projection and confuse the audience when they read the projected information. For example, the orthographic projection method uses orthogonal projection lines to project 3D items onto a 2D plan [30], which means the projected image on the 2D plane is the same size no matter the distance of the 3D item from the 2D plane. However, the human visual system needs such a depth cue to understand the distance between objects. It can confuse the audience if the depth cue is lost during the projection. In this study, the depth cue was necessary for the projection not to lose the accuracy of the warning information. The weak perspective projection method uses a similar concept to the orthographic projection methods [31], thus it loses the depth cue when doing the projection as well and thus also could not be used in this study. Perspective projection, on the other hand, is a method that can project a 3D object onto a 2D plane and not lose the depth cue [29] because the projection line it uses is the line connecting the 3D item to human eyes rather than orthogonal projection lines. Thus, the perspective projection method was used in this study to do the projection. The basic perspective projection matrix is shown in Equation (1) [32]:\[\left[ \begin{array}{ccc}1&0&0&0\\ 0&1&0&0\\ 0&0&1&0\\ 0&0&\frac{1}{d}&0\end{array} \right]\times\left[ \begin{array}{c}\mathrm{x}\\ \mathrm{y}\\ \mathrm{z}\\ 1\end{array} \right]=\left[ \begin{array}{c}\mathrm{x}\\ \mathrm{y}\\ \mathrm{z}\\ \mathrm{z}\\ \frac{\mathrm{z}}{d}\end{array} \right] \tag{1}\]  
The \(\left[ \begin{array}{c}\mathrm{x}\\ \mathrm{y}\\ \mathrm{z}\end{array} \right]\) represents the 3D coordinates of a point needing to be projected, and d is the distance from the point to the projecting plane.