Patent Description:
A three-dimensional (<NUM>-D) scanner is a machine capable of obtaining and digitizing information on a shape of an object, that is, a scan target, and performs measurement on the object by projecting light onto the object. <NUM>-D scan data is used in fields, such as the fabrication of vehicles, the fabrication of figures, quality tests, medical fields, and customerization, and the range of use thereof tends to be gradually expanded.

For example, in general, in a dental clinic, etc., an intraoral tissue structure, such as teeth, is checked through impression taking for teeth of a patient. Treatment and a cure are performed based on the checking of the intraoral tissue structure. In order to obtain <NUM>-D information of the intraoral tissue structure, an oral scanner system for dental surgery which implements a <NUM>-D modeling image of the intraoral tissue structure by using a measurement light, such as a laser, is recently widely used.

In the case of a common scanning device for a <NUM>-D object, if a <NUM>-D object is scanned, in order to represent the complexity of the <NUM>-D object, <NUM>-D scan data is generated by generating a large amount of geometry information called polygons and performing an operation on the geometry information. When the scan task is terminated and the data is stored, the state in which the scanning has been completed may be checked, but there is a problem in that it is difficult to check the process of actually performing the scanning.

<CIT> describes a computing device implemented method for motion compensation in a three dimensional scan. The motion compensation includes receiving three-dimensional (3D) scans of a dentition, estimating a motion trajectory from one scan to another, and calculating a corrected scan by compensating for the motion trajectory. Estimating the motion trajectory can include estimating an amount of motion of a 3D scanner during the scan as a rigid body transformation based on input from a position tracking device.

<CIT> describes a method for scanning anatomical structures, such as the teeth or the jaw of patients, in order to create an optical, intraoral impression and to display and visualize the scanning results. An extraoral detection unit, such as a <NUM>-D camera, detects a spatial position of an intraoral scanner relative to an observer. A scanning result is generated based on the intraorally captured image of the anatomical structures and the detected spatial position of the intraoral scanner relative to the observer. During pauses in the scanning procedure, the position, orientation and scaling of the anatomical structures are estimated and, as a scanning result, an image of the anatomical structures corresponding to the estimation is generated and displayed.

Further, <CIT> discloses a method for dental modeling in which views of three-dimensional models are supplemented with still images of the modeled subject matter.

An object of the present invention is to provide a method of replaying a scanning process, which can simulate and display a scanning process of obtaining previously stored scan data from the scan data.

Technical objects of the present invention are not limited to the aforementioned technical object, and other technical objects not described above may be evidently understood by those skilled in the art from the following description.

To achieve the above object, the present invention provides a method in accordance with claim <NUM>.

A method of replaying a scanning process according to the present invention includes a scanning step of obtaining scan data by scanning a scan target by using a scanner, an information acquisition step of obtaining state information of the scanner obtained in the scanning step, a calculation step of calculating relative information between the scanner and the scan target based on the state information of the scanner obtained in the information acquisition step, and a display step of displaying, in a user interface, the scan data obtained by scanning the scan target.

Furthermore, the scanning step may include a two-dimensional (<NUM>-D) image acquisition step of obtaining at least one <NUM>-D image data by receiving light incident through an opening part formed at one end of the scanner, a three-dimensional (<NUM>-D) image generation step of converting, into <NUM>-D volume data, the at least one <NUM>-D image data obtained in the <NUM>-D image acquisition step, and an alignment step of aligning a plurality of the <NUM>-D volume data so that the <NUM>-D volume data is connected and aligned.

According to the invention, the information obtained in the information acquisition step includes position information and rotation information of a camera.

Furthermore, the position information of the camera may be obtained in the form of a <NUM>-D orthogonal Cartesian coordinate system represented as x, y, and z values.

Furthermore, the rotation information of the camera may be obtained in the form of a 3x3 rotation matrix.

Furthermore, the position information of the camera and the rotation information of the camera may be obtained together in the form of a 3x4 matrix.

According to the invention, in the information acquisition step, the position information and rotation information of a scanner tip are obtained based on the position information of the camera and the rotation information of the camera.

Furthermore, the position information and rotation information of the scanner tip may be formed to operate in conjunction with a scan time.

Furthermore, in the display step, a process of forming the scan data may be replayed in a way to be sequentially displayed over time.

Furthermore, in the display step, in replaying the process of forming the scan data, a replay speed may be adjustable through a replay speed control unit formed in the user interface.

Furthermore, in the display step, in replaying the process of forming the scan data, a replay position may be adjustable through a scan time indication unit formed in the user interface.

Furthermore, in the display step, in replaying the process of forming the scan data, one end of a shape of the scanner including a scanner tip may be displayed in the user interface.

Furthermore, the shape of the scanner may be semi-transparently displayed in the user interface, and the state and scan data of the scanner including the scanner tip may be simultaneously replayed.

Furthermore, in the display step, a moving path of the scanner tip may be additionally displayed.

Furthermore, the moving path of the scanner tip may include scan time information. When a part in the moving path of the scanner tip is selected, the scanner tip may move to a replay position corresponding to the part.

Furthermore, the scan data and the position information and rotation information of the scanner tip may be divided into a plurality of groups in a time sequence.

According to the method of replaying a scanning process according to an embodiment of the present invention, a process of performing a scan task can be simulated in a time sequence by implementing a scanning process of obtaining scan data in the form of a replay image.

Accordingly, information, including an environment in which a scan task performed to obtain scan data is performed, a scanning method and a problem occurring during scanning, can be checked along a flow in which the scan task is performed. The checked information may be used to evaluate the reliability of the scan data, or reference may be made to the checked information as a guide for improving efficiency when a next scan task is performed.

Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. In adding reference numerals to the elements of each drawing, it should be noted that the same elements have the same reference numerals as much as possible even if they are displayed in different drawings. Furthermore, in describing embodiments of the present invention, when it is determined that a detailed description of the related well-known configuration or function hinders understanding of an embodiment of the present invention, the detailed description thereof will be omitted.

Furthermore, in describing elements of an embodiment of the present invention, terms, such as a first, a second, A, B, (a), and (b), may be used. Such terms are used only to distinguish one component from the other component, and the essence, order, or sequence of a corresponding component is not limited by the terms. All terms used herein, including technical or scientific terms, have the same meanings as those commonly understood by a person having ordinary knowledge in the art to which an embodiment pertains, unless defined otherwise in the specification. Terms, such as those commonly used and defined in dictionaries, should be construed as having the same meanings as those in the context of a related technology, and are not construed as being ideal or excessively formal unless explicitly defined otherwise in the specification.

<FIG> is a schematic flowchart of a method of replaying a scanning process according to the present invention. <FIG> is a flowchart more specifically illustrating a scanning step in the method of replaying a scanning process according to the present invention.

Referring to <FIG>, the method of replaying a scanning process according to the present invention includes a scanning step S10 of obtaining scan data by scanning a scan target <NUM> by using a scanner. In the scanning step S10, light reflected by a surface of the scan target <NUM> is introduced into the scanner through an opening part formed at one end of the scanner. The light introduced into the scanner is received through a camera disposed within the scanner. The camera obtains the received light as digital scan data through an imaging sensor connected to the camera. In this case, the scan data may be at least one two-dimensional (<NUM>-D) image data (this step may correspond to step S11 of obtaining <NUM>-D image data in the scanning step S10). Meanwhile, the obtained <NUM>-D image data may be displayed in a scan execution unit <NUM> in real time or may be used as data for generating three-dimensional (<NUM>-D) volume data when the obtained <NUM>-D image data is generated as the <NUM>-D volume data by structural light radiated from an optical projector, additionally disposed within the scanner, toward the scan target <NUM>.

Meanwhile, after the <NUM>-D image data is obtained through the camera and the imaging sensor connected to the camera, the obtained <NUM>-D image data may be formed to be converted into <NUM>-D volume data by combining the <NUM>-D image data (a <NUM>-D image sensing step S12). The <NUM>-D volume data may be formed to include a voxel having graphic information in a <NUM>-D space. As a result, the scan data may be displayed in a user interface by a set of such <NUM>-D volume data. However, in order for the scan data to be displayed in the user interface, an alignment step S13 of aligning a plurality of the <NUM>-D volume data needs to be performed so that the plurality of <NUM>-D volume data is connected and aligned without fragmentarily displaying each of the <NUM>-D volume data. A connection between the <NUM>-D volume data generated through the alignment step S13, the alignment of coordinates, etc. are performed. Such an alignment process may be performed in various ways, but the alignment step S13 may be performed using an iterative closest point (ICP) algorithm for connecting an overlap portion of the other data with one data, preferably.

Furthermore, as described above, while the scanning step S10 is performed, state information of the scanner is obtained along with the scan data (an information acquisition step S20). In this case, the state information of the scanner may be variously obtained, but position information and rotation information of the camera disposed within the scanner are obtained. The number of cameras disposed within the scanner may be one or more. The number of cameras whose position information and rotation information are obtained may also be one or more. To obtain the position information and the rotation information through the camera is for finally obtaining distance information between the scanner and the scan target.

<FIG> is a schematic diagram of the scan target and the scanner for scanning the scan target in the method of replaying a scanning process according to the present invention.

Referring to <FIG>, the scan target <NUM> is scanned through the scanner. The scan target <NUM> may be a plaster cast obtained through impression taking or may be an actual mouth including teeth and gums of a patient. The scanner includes various types of scanners. However, it is preferred to adopt and use a handheld type <NUM>-D scanner when comprehensively considering characteristics, use convenience, etc. of the present invention. Light reflected by the scan target <NUM> enters the scanner through a scanner tip of the scanner. The light may be formed as <NUM>-D-modeled scan data after <NUM>-D and <NUM>-D image acquisition and alignment are performed on the light.

<FIG> is a diagram schematically describing a process of performing scanning while performing the scanning from T1 to T6 when the scanner scans the scan target in the method of replaying a scanning process according to the present invention. <FIG> is a diagram schematically illustrating a relation between the camera embedded in the scanner and the scan target when the scan target is scanned.

Referring to <FIG> and <FIG>, the scanner may perform scanning on the scan target <NUM> while moving from one end of the scan target <NUM> to the other end thereof. Meanwhile, when a user performs scanning through the scanner, the scanning is sequentially performed from a first scan point T1 to next scan points T2, T3, T4, T5, and T6. In the process of performing the scanning, the scan points T1, T2, T3, T4, T5, and T6 may have portions that overlap neighboring scan points. As the scanning is performed so that the overlap portions are present, data is aligned, and one model data for the scan target may be finally generated.

Meanwhile, when the scanner performs the scanning, position information of the camera obtained in the information acquisition step S20 may have a form of a <NUM>-D orthogonal Cartesian coordinate system represented as x, y, and z values. The position information of the camera may be indicated as a relative position based on a specific point. Preferably, coordinates when scanning is started may be set as a starting point (<NUM>, <NUM>, <NUM>), and relative coordinates of each scan points may be obtained.

Rotation information of the camera may be obtained in the form of three angles represented as (a, β, γ). In this case, α may mean an angle of the camera in an xy plane, β may mean an angle of the camera in a yz plane, and γ may mean an angle of the camera in a zx plane. Furthermore, the rotation information of the camera may be obtained in the form of a 3x3 matrix. In this case, the <NUM>×<NUM> matrix may include information on an angle in the xy plane, an angle in the yz plane, and an angle in the zx plane. The rotation information means information on a tilt angle with respect to a reference position not information on a rotation speed, etc. The rotation information of the camera may be converted from the form of the three angles, represented as (a, β, γ), to the form of the 3x3 matrix, if necessary, or may be converted from the form of the 3x3 matrix to the form of the three angles as a reverse case thereof.

Furthermore, as described above, the position information of the camera and the rotation information of the camera may be separately obtained in the forms of the <NUM>-D orthogonal Cartesian coordinate system and the <NUM>×<NUM> matrix, respectively. The position information and rotation information of the camera may be obtained in the form of the 3x4 matrix at a time.

When the position information and rotation information of the camera are obtained, position information and rotation information of the scanner tip are obtained from the obtained position information and rotation information of the camera by using position information and rotation information between the camera and the scanner tip. More specifically, the scanner tip is configured to have a given distance and given angle with respect to the camera in terms of a structural characteristic of the scanner based on position information and rotation information of the camera. Accordingly, if a distance between the camera and the scanner tip is added and an angle formed by the camera and the scanner tip is incorporated with respect to the position information and rotation information of the camera, the position information of the scanner tip and the rotation information of the scanner tip may be derived. In this case, the position information of the scanner tip may be variously indicated, but may use, as a reference, a portion corresponding to the center of a cross-sectional shape of the opening part of the scanner. Furthermore, the rotation information of the scanner tip may be an angle of a normal vector in a virtual plane including the cross section of the opening part of the scanner.

When the information acquisition step S20 is performed, a calculation step S30 of calculating relative information between the scanner and the scan target <NUM> based on the state information of the scanner is performed. In the calculation step S30, the distance between the scanner tip and the scan target <NUM>, which is obtained in the information acquisition step S20, is measured. The distance may be obtained as distance data by calculating a straight-line distance and an angle between scan points (e.g., the scan points may correspond to T1, T2, T3, T4, T5, and T6, but are not limited to the six scan points, and may correspond to a plurality of points scanned when the scanning step S10 is performed) and the scanner tip. Referring to <FIG>, while the camera moves (from C1 to C2 and from C2 to C3), position information and rotation information of the camera may be obtained. Position information and rotation information of the scanner tip are obtained based on the position information and rotation information of the camera. Furthermore, distance and angle information between the scan target <NUM> and the scanner tip are calculated. A moving path <NUM> of the scanner tip to be described later is generated based on the distance and angle information calculated in the calculation step S30.

Meanwhile, when the scanning step S10 is performed through the scanner, the position information and rotation information of the scanner tip may be changed depending on scan time. Accordingly, the position information and rotation information of the scanner tip may be formed to operate in conjunction with each other from the nature of data in a way to have information on a scan time when the position information and the rotation information are generated. Accordingly, the position information and rotation information of the scanner tip may be sequentially aligned in a scan time sequence. There is an advantage in that data can be aligned and replayed in a time sequence because position information and rotation information of the scanner tip are sequentially aligned in a scan time sequence.

<FIG> are diagrams illustrating a process of generating scan model data in a user interface in a time sequence in the method of replaying a scanning process according to the present invention.

Generally referring to <FIG>, the method of replaying a scanning process according to the present invention includes a display step S40 of displaying the scan data in the user interface in the form of a <NUM>-D model M by scanning the scan target <NUM>. In the display step S40, the model M completed through a scanning process may be displayed, but a process of forming the model until the model is completed may be displayed.

Referring to <FIG> and <FIG>, the user interface may include a scan display unit <NUM> on which scan data is displayed in the form of the <NUM>-D model M, a scan execution unit <NUM> in which a scanning process is displayed, and a manipulation interface unit <NUM> in which a manipulation is performed on the <NUM>-D model. The scan execution unit <NUM> may perform scanning in real time or may visually display <NUM>-D image data generated through the camera and the imaging sensor in a corresponding time when a scanning process is replayed. Meanwhile, the scan execution unit <NUM> and the scan display unit <NUM> may be formed to be displayed on the same screen, and may simultaneously replay a process of forming the <NUM>-D model M in accordance with the same scan timing and <NUM>-D image data obtained in a scanning process. Furthermore, the scan execution unit <NUM> may be isolated from the scan display unit <NUM>. In order for the <NUM>-D model M to be more importantly recognized, the scan execution unit <NUM> may be formed to occupy a smaller area than the scan display unit <NUM>.

Meanwhile, the manipulation interface unit <NUM> may perform various manipulations on the <NUM>-D model M. According to circumstances, if the results of scanning are different from those expected by a user, an option capable of deleting data of a specific portion, an option capable of checking data reliability of data of the <NUM>-D model M, an option capable of checking an operating state (waiting, scanning, etc.) of the scanner, etc. may be included in the manipulation interface unit <NUM>. In the present invention, however, the replay of a scanning process is primary, and an option button not related to the replay of a scanning process is omitted.

The manipulation interface unit <NUM> may include a replay manipulation unit <NUM>. In general, the scan display unit <NUM> displays only the final data displayed by forming, in the form of the <NUM>-D model M, data until scanning is terminated after the scanning is started, but a mode of the user interface may be changed into a replay mode by manipulating (e.g., clicking on) the replay manipulation unit <NUM>.

When the mode of the user interface is changed into the replay mode, portions <NUM> to <NUM> corresponding to the replay mode may be additionally displayed on the lower side of the scan display unit <NUM>. In the replay mode, a scan time indication unit <NUM> may temporally indicate the time when scanning was performed. In this case, the scan time may appear in a slider bar 32a in response to an operation of a search button 32b moving from one end to the other end. That is, when the search button 32b is positioned at one end (e.g., the end on the left of the slider bar), the scan time may correspond to a scan start time. When the search button 32b is positioned at the other end (e.g., the end on the right of the slider bar), the scan time may correspond to a scan end time. In the display step S40 according to the present invention, a process of forming scan data may be replayed in the replay mode in a way to be sequentially displayed over time. The process of forming scan data means that the scan data obtained in the scanning step S10 is displayed in a time sequence in which the scan data is obtained.

Meanwhile, the scan time indication unit <NUM> may move to scan timing corresponding to a corresponding point by clicking on a specific point in the slider bar 32a. A scanning process may be replayed from the scan timing corresponding to the corresponding point. Alternatively, the scan time indication unit <NUM> may move to scan timing desired by a user by clicking on and dragging & dropping the search button 32b.

Furthermore, the display step S40 may be configured to adjust a replay speed through a replay speed control unit <NUM> formed in the user interface in replaying a process of forming scan data. The replay speed control unit <NUM> includes a replay speed deceleration unit 33a and a replay speed acceleration unit 33b. The replay speed deceleration unit 33a may be commonly indicated as a mark of "-". The replay speed acceleration unit 33b may be commonly indicated as a mark of "+". A replay speed now applied when a replay process is performed may be displayed between the replay speed deceleration unit 33a and the replay speed acceleration unit 33b. The replay speed may be variously adjusted like ×<NUM>, ×<NUM>, ×<NUM>, ×<NUM>, ×<NUM>, ×<NUM>, etc. of a reference speed.

Meanwhile, referring to <FIG>, in the display step S40, in replaying a process of forming scan data, one end of a shape <NUM> of the scanner including the scanner tip may be displayed in the user interface (more specifically, the scan display unit <NUM>). As the shape <NUM> of the scanner is displayed on the scan display unit <NUM>, how scanning was performed where and through how much rotation (tilt angle) when a user scans the scan target <NUM> may be visually checked. A portion for which the scanning was insufficient may be fed back. Furthermore, the shape <NUM> of the scanner may be semi-transparently displayed in the user interface, so that a state of the scanner including the scanner tip and scan data may be simultaneously replayed. That is, when the shape <NUM> of the scanner is opaquely displayed, some of or the entire scan data in which scanner graphics are displayed in the form of the <NUM>-D model M may be covered. In this case, the scan data and a state of the scanner including the scanner tip may be checked by adjusting transparency of the shape <NUM> of the scanner.

In this case, the <NUM>-D model M and the shape <NUM> of the scanner including the scanner tip may be replayed so that the position information and rotation information of the scanner tip and the distance and angle information between the scanner tip and the scan target <NUM>, obtained in the information acquisition step S20 and the calculation step S30, are displayed. That is, when a scanning process is replayed in the display step S40, piece of information sequentially obtained over time may be visually displayed on the scan display unit <NUM>. According to the invention, in scanning the scan target <NUM>, if the scanner has rotated, the <NUM>-D model M is represented to rotate as the scanner rotates with respect to the scan target <NUM> when the scanner scans the scan target <NUM>.

Meanwhile, if only a process of forming scan data is to be checked by covering the scanner tip when a scanning process is replayed, the shape <NUM> of the scanner including the scanner tip may not be displayed by clicking on a scanner tip display/release button <NUM> formed on one side of the scan display unit <NUM>. As the shape <NUM> of the scanner is not displayed, a process of obtaining scan data that form the <NUM>-D model M can be more closely monitored.

Referring to <FIG>, a scan portion selection unit <NUM> may be formed at one end of the scan display unit <NUM>. For example, the scan portion selection unit <NUM> may be formed on the upper side of the scan display unit <NUM>. The scan portion selection unit <NUM> may include an upper jaw selection unit <NUM>, a lower jaw selection unit <NUM>, and an occlusion selection unit <NUM>. If the replay mode is entered by enabling (e.g., clicking on) the replay manipulation unit <NUM>, only a process of scanning the upper jaw may be replayed when the upper jaw selection unit <NUM> is selected, and only a process of scanning the lower jaw may be replayed when the lower jaw selection unit <NUM> is selected. Furthermore, when the occlusion selection unit <NUM> is selected in the replay mode, only a process of scanning an occlusion form of the upper and lower jaws in order to form the occlusion of the upper jaw and the lower jaw may be replayed. In this case, a portion whose occlusion information is obtained may be displayed so that the upper jaw and the lower jaw are distinguished from each other. The portion may be displayed using at least one visual method of color, a shade, and a pattern <NUM>. Meanwhile, the process of scanning the upper jaw, the process of scanning the lower jaw, and the process of scanning the occlusion may be divided in a time sequence. In order to separate and replay the scanning processes according to each scanning process, scan data and position information and rotation information of the scanner tip may be divided into a plurality of groups in a time sequence. There is an advantage in that a user can rapidly take only feedback for a specific scanning process although the user does not check the entire scanning process because a process of scanning the upper jaw, a process of scanning the lower jaw, and a process of scanning occlusion can be separated and viewed as described above.

Meanwhile, in the replay mode, a portion scanned at each piece of scan timing may be displayed in the <NUM>-D model M, displayed on the scan display unit <NUM>, in the form of a shade <NUM>. A user may receive feedback for a portion for which scanning was insufficient with reference to a moving path of the shade <NUM>, etc., and may use the feedback as reference data in subsequent scanning.

<FIG> is a diagram illustrating that a scan path is displayed in the user interface in the method of replaying a scanning process according to the present invention.

Referring to <FIG>, in the method of replaying a scanning process according to the present invention, the display step S40 may additionally display a moving path <NUM> of the scanner tip. The moving path <NUM> of the scanner tip is obtained by connecting pieces of position information of the scanner tip in a time sequence. Accordingly, each of points constituting the moving path <NUM> of the scanner tip may include scan time information (scan timing). When a part in the moving path <NUM> of the scanner tip is selected, the scanner tip may move to scan timing (replay position) corresponding to the corresponding point, so that replay may be performed at the scan timing. Accordingly, there are advantages in that a user can visually check the moving path <NUM> of the scanner tip and can rapidly receive feedback for a scanning process by taking the feedback based on the moving path <NUM> of the scanner tip.

The embodiments described in the present invention should not be construed as limiting the scope of the present invention, which is defined in the claims. The scope of the present invention is not restricted by the embodiments.

Claim 1:
A method of replaying a scanning process, comprising:
a scanning step (S10) of obtaining scan data by scanning a scan target (<NUM>) by using a scanner;
an information acquisition step (S20) of obtaining state information of the scanner obtained in the scanning step (S10), wherein the state information comprises position information and rotation information of a camera, and wherein, in the information acquisition step (S20), position information and rotation information of a scanner tip (<NUM>) are obtained based on the position information of the camera and the rotation information of the camera;
a calculation step (S30) of calculating relative information between the scanner and the scan target (<NUM>) based on the state information of the scanner obtained in the information acquisition step (S20), wherein the relative information includes distance and angle information between the scan target (<NUM>) and the scanner tip (<NUM>); and
a display step (S40) of displaying, in a user interface, the scan data obtained by scanning the scan target (<NUM>) to replay a process of forming the scan data, wherein the scan data is displayed in the form of a <NUM>-D model (M) together with the position information and rotation information of the scanner tip (<NUM>) and the distance and angle information between the scanner tip (<NUM>) and the scan target (<NUM>) in the display step, and wherein the <NUM>-D model (M) is represented to rotate in the user interface as the scanner rotates with respect to the scan target (<NUM>).