Patent Description:
During movement of a patient into a CT machine through a patient table, an operator observes, through the CT machine or a plurality of cameras in a scanning room, a status/action of the patient from all angles, which overcomes blind spots caused by the room layout and the patient orientation, thereby ensuring better patient care and patient safety during the entire CT examination.

In an existing method, a camera stream is displayed on a dedicated monitoring screen, and a video corresponding to each camera is played separately in a specific region of the monitoring screen. The operator needs to frequently switch the sight among the plurality of videos on the monitoring screen to capture a desired image, which easily leads to the loss of focus and visual fatigue.

<CIT> refers to an apparatus for monitoring a subject during CT-imaging and comprises an image providing unit providing a first monitoring image and a second monitoring image acquired at different support positions, a monitoring position providing unit providing a first monitoring position of a region of interest in the first monitoring image, a support position providing unit providing support position data of the support positions, a position map providing unit providing a position map mapping calibration support positions to calibration monitoring positions, and a region of interest position determination unit determining a position of the region of interest in the second monitoring image based on the first monitoring position, the support position data, and the position map. The apparatus allows to determine the position of the region of interest.

The present invention is intended to provide a monitoring video generation method, which can facilitate observation of a patient from a plurality of angles in a monitoring video.

The present invention is further intended to provide a monitoring video generation system, which can facilitate observation of a patient from a plurality of angles in a monitoring video.

The present invention provides a monitoring video generation method applicable to a CT machine. A monitored target is located on a patient table of the CT machine. The patient table of the CT machine is movable relative to a gantry of the CT machine in an extending direction. The monitoring video generation method includes: acquiring a plurality of primary videos including the monitored target, where fields of view of the plurality of primary videos are fixed relative to a position of the gantry of the CT machine; determining a keyframe according to a region in an initial frame of each of the primary videos surrounding a monitored target, and determining a bounding box surrounding the monitored target in the keyframe; determining, for each of the primary videos, a bounding box surrounding the monitored target in each subsequent frame; and selecting, for each frame moment from the frames of the plurality of primary videos, a frame in which the region of the monitored target within the bounding box has a largest area as the keyframe, and generating a monitoring video in real time according to a part of each selected keyframe located within the bounding box.

According to the monitoring video generation method provided in the present invention, a monitoring video may be generated in real time according to the primary videos of a plurality of cameras, which can facilitate observation of a patient from a plurality of angles in the monitoring video, thereby helping reduce the loss of focus and visual fatigue as a result of an operator frequently switching the sight among a plurality of videos on a monitoring screen.

In an exemplary implementation of the monitoring video generation method, the step of determining the keyframe according to the region in the initial frame of each of the primary videos surrounding the monitored target specifically includes: determining the region in the initial frame of each of the primary videos surrounding the monitored target through an automatic detection algorithm or manual selection, and selecting a frame in the initial frames in which the region surrounding the monitored target has a largest area as the keyframe.

In another exemplary implementation of the monitoring video generation method, the step of determining, for each of the primary videos, the bounding box surrounding the monitored target in each subsequent frame specifically includes: determining, for each of the primary videos, the bounding box surrounding the monitored target in each subsequent frame by using an automatic detection algorithm, a coordinate conversion algorithm, or an optical flow method.

In still another exemplary implementation of the monitoring video generation method, the step of determining the bounding box by using the coordinate conversion algorithm specifically includes: mapping, through a coordinate conversion algorithm, coordinates of a bounding box in a keyframe at a previous frame moment to a coordinate system where the gantry of the CT machine is located, calculating, according to a physical movement vector of the patient table of the CT machine relative to the gantry of the CT machine between two frames, a position of a bounding box in a frame at a next frame moment of the same primary video in the coordinate system where the gantry of the CT machine is located, then mapping the coordinates to a frame at a next frame moment of each of the primary videos according to coordinate correspondences between cameras that acquire the primary videos and between the cameras and the gantry, and determining a bounding box surrounding the monitored target after coordinate update in the frame at the next frame moment of each of the primary videos. Through the coordinate conversion, the coordinates of the bounding box in each subsequent frame may be calculated, which helps save calculation resources, thereby improving the real-time performance of the monitoring video.

In yet another exemplary implementation of the monitoring video generation method, the step of determining the bounding box by using the optical flow method specifically includes: calculating coordinates of a bounding box in a frame at a next frame moment of the same primary video through the optical flow method by using coordinates of a bounding box in a keyframe at a previous frame moment as a reference, then mapping, through coordinate conversion, the calculated coordinates of the bounding box in the frame at the next frame moment to a coordinate system where the gantry of the CT machine gantry is located, mapping the coordinates to a frame at a next frame moment of each of the primary videos according to coordinate correspondences between cameras that acquire the primary videos and between the cameras and the gantry, and determining a bounding box surrounding the monitored target after coordinate update in the frame at the next frame moment of each of the primary videos. In this way, the consumption of calculation resources is reduced.

The present invention further provides a monitoring video generation method. A monitored target is located on a patient table of a CT machine. The patient table of the CT machine is movable relative to a gantry of the CT machine in an extending direction. The monitoring video generation method includes: acquiring a plurality of primary videos including the monitored target, where fields of view of the plurality of primary videos are fixed relative to a position of the gantry of the CT machine; splicing and/or stacking the plurality of primary videos to form a secondary video composed of keyframes at all frame moments; determining a bounding box in each of the keyframes of the secondary video to surround the monitored target; and generating a monitoring video in real time according to a part of each of the keyframes of the secondary video located within the bounding box.

In an exemplary implementation of the monitoring video generation method, the step of determining the bounding box in each of the keyframes of the secondary video to surround the monitored target specifically includes: determining the bounding box in each of the keyframes of the secondary video by using an automatic detection algorithm to surround the monitored target. In this way, automatic operation can be achieved, thereby reducing the labor consumption.

In another exemplary implementation of the monitoring video generation method, the step of determining the bounding box in each of the keyframes of the secondary video to surround the monitored target specifically includes: determining a bounding box in an initial keyframe through an automatic detection algorithm or manual selection to surround the monitored target, and determining a bounding box in each subsequent keyframe by using a coordinate conversion algorithm according to the bounding box determined in the initial keyframe. The coordinate conversion algorithm includes: calculating a movement vector from coordinates of the bounding box in the keyframe at a previous frame moment to coordinates of the bounding box in the keyframe at a next frame moment according to a physical movement vector of the patient table of the CT machine relative to the gantry of the CT machine between two frame moments, and calculating the coordinates of the bounding box in the keyframe at the next frame moment according to the coordinates of the bounding box in the keyframe at the previous frame moment and the coordinate movement vector. Through the coordinate conversion, the coordinates of the bounding box in each subsequent keyframe may be calculated, which helps save calculation resources, thereby improving the real-time performance of the monitoring video.

In another exemplary implementation of the monitoring video generation method, the step of determining the bounding box in each of the keyframes of the secondary video to surround the monitored target specifically includes: determining a bounding box in an initial keyframe through an automatic detection algorithm or manual selection to surround the monitored target, and determining a bounding box in each subsequent keyframe by using an optical flow method according to the bounding box determined in the initial keyframe. In this way, the consumption of calculation resources is reduced.

The present invention further provides a monitoring video generation system, including a computer-readable storage medium and a processor. The computer-readable storage medium stores code. When the processor executes the code, the monitoring video generation system performs the above monitoring video generation method. According to the monitoring video generation system, a monitoring video may be generated in real time according to the primary videos of a plurality of cameras, which can facilitate observation of a patient from a plurality of angles in the monitoring video, thereby helping reduce the loss of focus and visual fatigue as a result of an operator frequently switching the sight among a plurality of videos on a monitoring screen.

In an exemplary implementation of the monitoring video generation system, the monitoring video generation system further includes a display. The display has a display window, and the display window is configured to display a monitoring video generated in real time. The display displays the monitoring video generated in real time in a minimap mode or a Picture-in-Picture mode.

In another exemplary implementation of the monitoring video generation system, in the minimap mode, the display window is composed of a main window and a minimap window. The minimap window is configured to display a keyframe at each frame moment. The main window is configured to display a part of each keyframe located within a bounding box. A rectangular box exists on each keyframe displayed in the minimap window to indicate a position of the monitored target in each keyframe. In this way, the region of real interest is zoomed in in the main window while retaining an original image in the minimap window to provide position information.

In still another exemplary implementation of the monitoring video generation system, the display window in the Picture-in-Picture mode is composed of a main window and a child window. One of the main window and the child window is configured to display the monitoring video generated in real time, and the other is configured to display a primary video including the monitored target from a camera. Contents displayed in the main window and the child window are switched by clicking/tapping the child window. In this way, two different videos may be viewed simultaneously.

The following drawings are merely exemplary descriptions and explanations and do not limit the scope of the present invention.

For clearer understanding of the technical features, objectives, and effects of the present invention, specific implementations of the present invention are described with reference to the drawings. The same labels in the figures represent components with the same structure or similar structures but the same function.

Herein, "exemplary" means "being used as an instance, example, or description", and any illustration or implementation described as "exemplary" herein should not be interpreted as a more preferred or advantageous technical solution.

<FIG> is a schematic flowchart of an exemplary implementation of a monitoring video generation method. The monitoring video generation method is applicable to a CT machine. The CT machine is also referred to as a computed X-ray tomographic unit. During movement of a patient into a CT machine through a patient table, an operator observes, through the CT machine or a plurality of cameras in a scanning room, a status/action of the patient from all angles, which overcomes blind spots caused by the room layout and the patient orientation, thereby ensuring better patient care and patient safety during the entire CT examination.

The CT machine includes a display with a display window. A monitored target is located on a patient table of the CT machine. The patient table of the CT machine is movable relative to a gantry of the CT machine in an extending direction. Referring to <FIG>, the monitoring video generation method includes the following steps:.

In an exemplary implementation, the keyframe may be determined through manual selection from the plurality of primary videos acquired in step S10, which specifically includes: comparing areas of the monitored target in the initial frames, selecting an initial frame in which the monitored target has a largest area as the keyframe, and drawing a bounding box on the keyframe, where the bounding box needs to surround the monitored target.

In another exemplary implementation, the keyframe may be determined through an automatic detection algorithm from the plurality of primary videos acquired in step S10, which specifically includes: inputting the initial frames of the acquired primary videos into the automatic detection algorithm, and executing the algorithm to automatically select an initial frame in which the monitored target has a largest area as the keyframe and to obtain coordinates of the bounding box on the keyframe. In this case, the bounding box is invisible.

As a frame of the monitoring video, the acquired keyframe is displayed in real time on a display window. As shown in <FIG>, the CT machine includes a display with a display window <NUM>. The display has an operating interface <NUM> configured to display CT scan results. The display window <NUM> configured to display the monitoring video generated in real time is arranged at a corner of the operating interface <NUM>. The monitoring video generated in real time is displayed in a minimap mode or a Picture-in-Picture mode, for example.

In an exemplary implementation, the monitoring video generated in real time may be displayed in the minimap mode, as shown in <FIG>. In the minimap mode, the display window <NUM> is composed of a main window <NUM> and a minimap window <NUM>. The minimap window <NUM> is located at a corner of the main window <NUM> and is configured to display the keyframe. The main window <NUM> is configured to display the monitored targets within the bounding box of the keyframe. A rectangular box <NUM> exists on the keyframe displayed in the minimap window <NUM> to indicate a position of the monitored target in the keyframe.

In another exemplary implementation, the monitoring video generated in real time may be displayed in the Picture-in-Picture mode, as shown in <FIG>. In the Picture-in-Picture mode, the display window <NUM> is composed of a main window <NUM> and a child window <NUM>. The child window <NUM> is located at a corner of the main window <NUM>. One of the main window <NUM> and the child window <NUM> is configured to display the monitoring video generated in real time, and the other is configured to display a primary video including the monitored target from a camera. Contents displayed in the main window <NUM> and the child window <NUM> are switched by clicking/tapping the child window <NUM>. In this way, two different videos may be viewed simultaneously.

Step S30: Determine, for each of the primary videos, a bounding box surrounding the monitored target in each subsequent frame.

The step of determining, for each of the primary videos, the bounding box surrounding the monitored target in each subsequent frame specifically includes: determining the bounding box surrounding the monitored target in each subsequent frame by using an automatic detection algorithm, a coordinate conversion algorithm, or an optical flow method.

In an exemplary implementation, the step of determining the bounding box surrounding the monitored target in each subsequent frame by using the automatic detection algorithm specifically includes: detecting a frame of each inputted primary video at each moment, and acquiring, for each frame, the bounding box surrounding the monitored target.

In another exemplary implementation of the monitoring video generation method, the step of determining the bounding box surrounding the monitored target in each subsequent frame by using the coordinate conversion algorithm specifically includes: mapping, through a coordinate conversion algorithm, coordinates of a bounding box in a keyframe at a previous frame moment to a coordinate system where the gantry of the CT machine is located, calculating, according to a physical movement vector of the patient table of the CT machine relative to the gantry of the CT machine between two frames, a position of a bounding box in a frame at a next frame moment of the same primary video in the coordinate system where the gantry of the CT machine is located, then mapping the coordinates to a frame at a next frame moment of each of the primary videos according to coordinate correspondences between cameras that acquire the primary videos and between the cameras and the gantry, and determining a bounding box surrounding the monitored target after coordinate update in the frame at the next frame moment of each of the primary videos. Through the coordinate conversion, the coordinates of the bounding box in each next frame may be calculated, which helps save calculation resources, thereby improving the real-time performance of the monitoring video. A person skilled in the art knows that conversion among coordinates in each frame, camera coordinates, and gantry coordinates may be performed after calibrating internal and external parameters.

In still another exemplary implementation of the monitoring video generation method, the step of determining the bounding box surrounding the monitored target in the frame at each next frame moment by using the optical flow method specifically includes: calculating coordinates of a bounding box in a frame at a next frame moment of the same primary video through the optical flow method by using coordinates of a bounding box in a keyframe at a previous frame moment as a reference, then mapping, through coordinate conversion, the calculated coordinates of the bounding box in the frame at the next frame moment to a coordinate system where the gantry of the CT machine gantry is located, mapping the coordinates to a frame at a next frame moment of each of the primary videos according to coordinate correspondences between cameras that acquire the primary videos and between the cameras and the gantry, and determining a bounding box surrounding the monitored target after coordinate update in the frame at the next frame moment of each of the primary videos. In this way, the consumption of calculation resources is reduced.

Step S40: Select, for each frame moment from the frames of the plurality of primary videos, a frame in which the region of the monitored target within the bounding box has a largest area as the keyframe, and generate a monitoring video in real time according to a part of each selected keyframe located within the bounding box. For example, the monitoring video is displayed in the display window <NUM>. The monitoring video generated in real time is composed of the keyframes at all frame moments. Therefore, a display mode of the monitoring video is the display mode of the keyframes. Details are not described.

In addition, by arranging the display window <NUM> on the operating interface <NUM> of the display of the CT machine and displaying the monitoring video on the display window <NUM>, no additional display screen is required, which can avoid the loss of focus and visual fatigue as a result of an operator switching the sight by a long distance between the operating interface <NUM> and the monitoring screen.

<FIG> is a schematic flowchart of another exemplary implementation of the monitoring video generation method. Referring to <FIG>, the monitoring video generation method includes the following steps:.

In an exemplary implementation, the monitored target is detected in each of the keyframes of the secondary video by using an automatic detection algorithm, and the bounding box is determined to surround the monitored target. In this way, automatic operation can be achieved, thereby reducing the labor consumption.

In another exemplary implementation, the monitored target is detected in an initial keyframe of the secondary video by using the automatic detection algorithm and the bounding box is determined to surround the monitored target, or the monitored target is manually selected and the bounding box is manually drawn to surround the monitored target. Then, the bounding box is determined in each subsequent frame according to the bounding box determined in the initial keyframe by using the coordinate conversion algorithm or the optical flow method.

The coordinate conversion algorithm specifically includes: calculating a movement vector from coordinates of the bounding box in the keyframe at a previous frame moment to coordinates of the bounding box in the keyframe at a next frame moment according to a physical movement vector of the patient table of the CT machine relative to the gantry of the CT machine between two frame moments, and calculating the coordinates of the bounding box in the keyframe at the next frame moment according to the coordinates of the bounding box in the keyframe at the previous frame moment and the coordinate movement vector. Through the coordinate conversion, the coordinates of the bounding box in each next keyframe may be calculated, which helps save calculation resources and generate the monitoring video in real time.

The optical flow specifically includes: calculating an optical flow between the keyframe at a previous frame moment and the keyframe at a next frame moment by using coordinates of the bounding box in the keyframe at the previous frame moment as a reference, and applying the optical flow to the coordinates of the bounding box in the keyframe at the previous frame moment to calculate coordinates of the bounding box in the keyframe at the next frame moment. In this way, there is no need to frequently read system information, which helps reduce the consumption of calculation resources.

Step S70: Generate and display a monitoring video in real time according to a part of each of the keyframes of the secondary video located within the bounding box. The display mode may be a minimap mode or a Picture-in-Picture mode.

The present invention further provides a monitoring video generation system for a CT machine, including a computer-readable storage medium, a processor, and a display. The computer-readable storage medium stores code. When the processor executes the code, the monitoring video generation system performs the above monitoring video generation method. The display has an operating interface <NUM>. A display window <NUM> is arranged on the operating interface <NUM>. The display window <NUM> is configured to display a monitoring video generated in real time. By displaying the monitoring video generated in real time through the display window <NUM>, no additional display screen is required.

The display displays the generated monitoring video in a minimap mode or a Picture-in-Picture mode in real time.

In the minimap mode, the display window <NUM> is composed of a main window <NUM> and a minimap window <NUM>. The minimap window <NUM> is located at a corner of the main window <NUM> and is configured to display the keyframe. The main window <NUM> is configured to display a part of the keyframe located within a bounding box. A rectangular box <NUM> exists on each keyframe displayed in the minimap window <NUM> to indicate a position of the monitored target in each keyframe. In this way, the region of real interest is zoomed in in the main window <NUM> while retaining an original image in the minimap window <NUM> to provide position information.

In the Picture-in-Picture mode, the display window <NUM> is composed of a main window <NUM> and a child window <NUM>. The child window <NUM> is located at a corner of the main window <NUM>. One of the main window <NUM> and the child window <NUM> is configured to display the monitoring video generated in real time, and the other is configured to display a primary video including the monitored target from a camera. Contents displayed in the main window <NUM> and the child window <NUM> are switched by clicking/tapping the child window <NUM>. In this way, two different videos may be viewed simultaneously.

It should be understood that although the description is illustrated according to each embodiment, each embodiment does not necessarily include an independent technical solution. The illustration of the specification is merely for clarity. A person skilled in the art should consider the description as a whole, and the technical solutions in the embodiments may be properly combined to form other implementations that a person skilled in the art can understand.

Claim 1:
A monitoring video generation method, applicable to a CT machine, wherein a monitored target is located on a patient table of the CT machine, and the patient table of the CT machine is movable relative to a gantry of the CT machine in an extending direction, the monitoring video generation method comprising:
Acquiring (S10) a plurality of primary videos comprising the monitored target, wherein fields of view of the plurality of primary videos are fixed relative to a position of the gantry of the CT machine;
characterized in that it further comprises:
Determining (S20) a keyframe according to a region in an initial frame of each of the primary videos surrounding the monitored target, and determining a bounding box surrounding the monitored target in the keyframe;
Determining (S30), for each of the primary videos, a bounding box surrounding the monitored target in each subsequent frame; and
Selecting (S40), for each frame moment from the frames of the plurality of primary videos, a frame in which the region of the monitored target within the bounding box has a largest area as the keyframe, and generating a monitoring video in real time according to a part of each selected keyframe located within the bounding box.