# A Computer-Vision Based Application for Student Behavior Monitoring in Classroom
## 2 Proposed System
### Position Estimation  
Two students may have two similar view-directions, but the objects being observed may be different. It depends on the position of the students. For example, two tabulates (left and right) look to the left, but one student can look at the board while the other is looking out the door. Therefore, determining the relative position of students directly affects the prediction of where students are observing. In our camera model, a scene view is formed by projecting 3D points into the image plane using a perspective transformation [48]. In order to convert coordinates of a projection point in pixels to its coordinates in the world coordinate system.  
We measure the angles x and z axes of camera coordinate system made with x, y, and z axes of the world coordinate system (normally, the original matched with the classroom corner) to compute extrinsic parameters R and \(t\). Since if \(\alpha\), \(\beta\), and \(\gamma\) are the angles a vector made with the x, y, and z axes, respectively, then \(cos(\alpha)i~{}+~{}cos(\beta)j~{}+~{}cos(\gamma)k\) is a unit vector in that direction. Thus, we can obtain the unit vectors \(i_{C}\) and \(k_{C}\) in the direction of the x and z axes of the camera coordinate system. The remaining unit vector \(j_{C}\) can be obtained by taking the cross product of \(i_{C}\) and \(k_{C}\) (\(j_{C}=-i_{C}\times k_{C}\)). After the transformation from the world coordinate system to the camera coordinate system, these basis vectors \(i_{C},~{}j_{C}\), and \(k_{C}\) respectively, have the new values of \((1,~{}0,~{}0),(0,~{}1,~{}0)\), and \((0,~{}0,~{}1)\). Solving the system of linear equations, we have \(R=\left(i_{C},~{}j_{C},~{}k_{C}\right)^{-1}\) and \(t=-R(x_{C},~{}y_{C},~{}z_{C})\), where \((x_{C},~{}y_{C},~{}z_{C})\) are the coordinates of the camera in the world coordinate system.  
Consider two upper points of the bounding boxes of two detected faces. We made an assumption that the distance between them has the value of \(\lambda\). We also assumed that the coordinates in the camera coordinate system of these two points are \(S_{1}=(x_{1},~{}y_{1},~{}z_{1})\) and \(S_{2}=(x_{2},~{}y_{2},~{}z_{2})\). It can be seen that the vector created by these two points is perpendicular to the normal vector \((0,~{}0,~{}1)\) of the camera lens. Or in other words, it is parallel to the plane z = 0 of the camera coordinate system, which means \(z_{1}\) must be equal to \(z_{2}\). Solving the system, we have the value of \(z_{1}\) and \(z_{2}\). Then, we can obtain the coordinates of \(S_{1}\) and \(S_{2}\) in the world coordinate system: \(S_{i_{w}}=R^{-1}((x_{i},~{}y_{i},~{}z_{i})-t),~{}i=1,~{}2\).  
From the values of those two points, we can approximate the location of a student by taking the coordinates of the midpoint of \(S_{1_{w}}\) and \(S_{2_{w}}\). In our problem, we assume that \(\lambda\) has a value of 14 cm. Figure 3 illustrates an output example of the algorithm. We have there, an image frame acquired from a lecture video and two diagrams alongside it represent the approximate locations of students in the sample image frame, as pairs of \(S_{1_{w}}\) and \(S_{2_{w}}\) points. Another approach is to consider two landmark points representing two eyes in a face. In this case, the vector created by these two points is perpendicular to the head-pose vector of the face (in this case, \(z_{1}\
eq z_{2}\)). We also assumed that the known distance between these points is fixed at \(\lambda^{\prime}\).  
Figure 3: (**A**) The frame acquired from the camera. (**B**) Estimated locations of each student in the sample. The image on the left side illustrates student points in the 3D coordinate system with the perspective from the board. Likewise, the right one represents perspective from top to bottom. The blue x’s represent four room corner points, which belong to the space containing the board. Pairs of \(S_{1_{w}}\) and \(S_{2_{w}}\) points are denoted in red and blue dots respectively.