Patent ID: 11908344
Assignee: CENTRAL CHINA NORMAL UNIVERSITY
Field: Computer technology (Electrical engineering)
Classification: CPC G  Y | IPC G

Claim 8:
9. The method of claim 5, wherein step (S5) comprises:
(S5-1) constructing the virtuality-reality fusion coordinate system, wherein an intersection point of the division wall, a wall and a floor of the classroom is configured as an origin of coordinate axes; each two intersecting lines is configured as a coordinate axis; and an upward direction is configured as a positive direction of a Z-axis; superposing the skeleton of the teacher with an object model of the virtual teaching resources presented on a screen of the electronic sand table using the virtuality-reality fusion coordinate system constructed by a left-handed coordinate system with reference to the internal and external parameters of the holographic projector, in combination of the coordinates of the skeletal joint points of the teacher and point-to-point postures;
(S5-2) determining a position of each of the skeletal joint points in a virtual space according to the 3D coordinate of each of the skeletal joint points obtained by the acquisition and processing module, in combination with the inverse perspective transformation equation; determine a position, a posture and a scaling parameter of each of the skeletal joint points in the virtuality-reality fusion coordinate system; and
(S5-3) identifying hand movements, body postures and facial expression of the teacher through matching tracked motion trajectory of hand, body and facial joint points of the teacher with a gesture data set, a skeleton data set and an expression data set trained by artificial intelligence (AI); and positioning the interaction commands based on changes in hand joints and elbow joints, and distance and component hierarchy relationship with a virtual teaching resource model, wherein the interaction commands comprise clicking, dragging, drop, hauling and pulling.