Wall visualisation from virtual point of view

A slope monitoring system for monitoring deformation of a wall or slope from a virtual point of view. A slope monitoring device directs signals at a slope or wall and compiles direction data, range data and amplitude data from signals reflected from the slope or wall. The direction data, range data and amplitude data comes from a first real position and a second real position. The data may be recorded sequentially or concurrently.

FIELD OF THE INVENTION

The present invention relates to the field of mine safety. More particularly, the invention relates to a method and apparatus for monitoring a wall or slope for movement. The invention finds particular application in underground mine wall or roof monitoring using slope stability lidar.

BACKGROUND TO THE INVENTION

The Applicant has previously described a method and apparatus for monitoring slope deformation using laser ranging in International Patent Publication number WO 2017/063033, the content of which is incorporated by reference. The Applicant has also described a slope monitoring system based on radar in International Patent Publication number WO 2002/046790, the content of which is also incorporated by reference. The systems described in these international patent publications have a number of operational similarities. For instance, both systems are described as operating from a fixed point and accumulating data over time. Any movement of a monitored slope or wall is detected as a change in the range from the lidar or radar. The systems have proven to be highly reliable with sub-millimeter accuracy.

It has been recognised that any movement of the radar or lidar will disrupt the monitoring and introduce errors that reduce the monitoring accuracy. Various techniques have been developed to handle short term errors, such as atmospheric variations or ground vibrations but the existing techniques do not account for significant data discontinuities that occur if equipment is removed and then returned to approximately the same position for continued monitoring. It will be appreciated that long-term monitoring produces more reliable slope deformation maps and thus better safety outcomes but it is sometimes necessary to move equipment which disrupts the monitoring continuity. The need to move equipment may be due to mine operations, such as blasting, or it may be due to economic considerations, such as equipment shared between multiple locations.

At present the best solution for relocating monitoring equipment to a previous location is to use precision surveying techniques, but these are not able to achieve sufficient accuracy for the sub-millimetre monitoring capability of the radar and lidar equipment described in the patent applications mentioned above.

SUMMARY OF THE INVENTION

In one form, although it need not be the only or indeed the broadest form, the invention resides in a slope monitoring system that monitors deformation of a wall or slope from a virtual point of view comprising:a slope monitoring device that directs signals at a slope or wall and compiles a point cloud of direction data, range data and amplitude data from signals reflected from the slope or wall; andat least one processor that performs the steps of:(a) acquiring first direction data and first range data received from a first point cloud compiled by the slope monitoring device when in a first position having a first real point of view;(b) acquiring second direction data and second range data received from a second point cloud compiled by the slope monitoring device when in a second position having a second real point of view;(c) transforming the first direction data and first range data from the first real point of view and the second direction data and second range data from the second real point of view to virtual direction data and virtual range data having a virtual point of view by:(i) rigid registration wherein the size and shape of the point cloud and distance between points in the point cloud are preserved; or(ii) non-rigid registration wherein the size and shape of the point cloud and distance between points in the point cloud are not preserved;(d) segmenting the transformed data into blocks of data defining a voxel; and(e) comparing voxel range values over time to identify movement.

The slope monitoring device may be a slope stability radar or slope stability lidar as described in International Patent Publication numbers WO2002/001570 and WO2017/063033 respectively.

The direction data and range data is contained in the point cloud recorded by the slope monitoring device. A point cloud is a set of data points in space representing the measurement of an external surface of objects from a measuring device like a 3D laser scanner. Individual points are represented by a x, y, z coordinate in both local or absolute coordinate systems. The distance between each point collectively gives the shape and dimensions (size) of the point cloud. Point cloud data are conventionally used without distorting the shape or dimensions of the cloud by altering the individual x, y, z attributes of each points. However, constant translation and or rotation of all the points in the point cloud without distorting the shape and dimensions may also be used as part of the registration process.

The virtual point of view may be the first real point of view from the first position or the second real point of view from the second position.

The step of transforming the data may be implemented using permutations of rigid registration and non-rigid registration. A transform matrix acts on the translation and rotation of the real position to transform to the virtual position within the 6 DOF (degrees of freedom).

DETAILED DESCRIPTION OF THE INVENTION

Embodiments of the present invention reside primarily in a slope monitoring system comprising a slope monitoring device and a processor for transforming measured data to a virtual point of view. Accordingly, the elements of the device and the method steps of the processor have been illustrated in concise schematic form in the drawings, showing only those specific details that are necessary for understanding the embodiments of the present invention, but so as not to obscure the disclosure with excessive detail that will be readily apparent to those of ordinary skill in the art having the benefit of the present description.

In this specification, adjectives such as first and second, left and right, and the like may be used solely to distinguish one element or action from another element or action without necessarily requiring or implying any actual such relationship or order. Words such as “comprises” or “includes” are intended to define a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed, including elements that are inherent to such a process, method, article, or apparatus.

Referring toFIG.1there is a shown a slope monitoring device which is a Slope Stability Lidar10that is viewing a scene11that may have regions which are stable and other regions which are moving. The following description is presented as applicable to Slope Stability Lidar but the invention is also applicable to Slope Stability Radar. In the embodiment ofFIG.1the scene11is part of a slope of an open cut mine12. For this embodiment the invention is applicable to a situation where a single slope monitoring device is used to monitor multiple locations. The slope monitoring device may monitor from a location for a period of time and then be moved to another location for a period of time, before being moved back to the original location.

As depicted inFIG.2, the slope monitoring device will be in a first position10ahaving a first point of view11awhen initially in the original location but when moved back to the original location will have a second position10bhaving a second point of view11b. Although effort may be made to return the slope monitoring device to the same position it will almost inevitably be slightly displaced and give rise to an error as depicted inFIG.2. Because slope monitoring occurs over long ranges compared to the accuracy of measurement, even a small displacement between first position10aand second position10bwill mean there is a discontinuity in the deformation data, as shown inFIG.3. In order to use all the available data for continuous slope monitoring the data is processed to remove the measurement error as described below.

FIG.2shows a horizontal displacement so the measurement error manifests as a parallax error. It will be appreciated that an error occurs with displacement or rotation of the slope monitoring device in any direction. The problem is therefore three-dimensional, not two-dimensional as depicted inFIG.2. In fact, there are six degrees of freedom (6DoF: x-translation; y-translation; z-translation; pitch; yaw; roll) that can all give rise to error.

Another situation in which the data must be corrected is shown inFIG.4.FIG.4represents an underground mine20in which a slope monitoring device is mounted on a mobile platform and conveyed through the mine. Typically this may be a lidar system mounted on a vehicle. As the lidar system traverses the mine it takes measurements of the mine wall from a number of locations such as21,22,23, etc. at a subsequent time, such as for example the next day, the process is repeated. A comparison of the data collected could be used to produce a deformation map as long as the data was repeatedly taken from the same position with the same point of view, but this is not the case. As explained below, the data taken from similar points is transformed from a first position having a first real point of view and a second position having a second real point of view to a virtual point of view. Conveniently the virtual point of view can provide the user with a consistent point of view irrespective of the actual first point of view, second point of view or subsequent points of view.

For the avoidance of confusion, it should be understood that the locations21,22,23etc inFIG.4are not the first position and second position referred to above. For each of the locations21,22,23there will be a first position corresponding to data collected on a first day, a second position corresponding to data collected on a second day, a third position etc.

One process for the data transformation is shown inFIG.5. In a first step50first data is acquired from a first position with a first real point of view. In subsequent step51second data is acquired from a second position with a second real point of view. The first data and the second data includes at least direction data and range data. The data is transformed at step52to a common virtual point of view.

The process of steps50,51and52may be repeated multiple times or may occur only once. In the first example mentioned above of monitoring an open pit the slope monitoring device may only be moved once so that the steps of acquiring data may be relatively long time periods with multiple measurements to produce deformation maps. In the second example of monitoring an underground mine each acquisition period is short and the steps are repeated multiple times to build a deformation map.

Once the data is aggregated the direction data and range data is segmented to form voxels. The range data for each voxel is averaged and compared to previous averaged range data from which a deformation map is produced and displayed. One approach to this step is described in International Patent Publication number WO 2017/063033, the content of which is incorporated by reference.

The process of the transformation can best be explained by reference toFIG.6. Let [Ap, Bp] be two point sets of point P from location A point of view and location B point of view respectively. The objective is to derive a virtual point transformation matrix to transform Ap such that the distance between Ap and Bp on all axes is minimised, hence creating a virtual point Ãp. This can be achieved by various image processing techniques such as rigid registration or non-rigid registration.

So, let T be the transform model
T(Ap)=Bp·Err

Where Erris the error function. So referring toFIG.6, SDA is the slope distance from point A to P and SDA is the transformed slope distanced from the new virtual point A to point P.
SDÃ=SDB·Err
Ã=B·Err
Ãp=Bp·Err

The distance Err can be derived using standard Euclidean distance for each pair of points or any other distance function. For example:

Subsequent point clouds captured from the Nthdifferent location [Np] can then be transformed to the nominated virtual point. Therefore, in this example:
T(Np)=Bp·Err
SDÑ=SDB·Err
Ñ=B·Err
Np=Bp·Err

An alternate approach to the process of wall visualization is depicted inFIG.7. In a first step70first data is acquired from a first position with a first real point of view. The first data includes at least direction data and range data. In a subsequent step71second data is acquired from a second position with a second real point of view. The second data includes at least direction data and range data. The data is transformed at step72to virtual data as from a common virtual point of view. The virtual data includes at least virtual direction data and virtual range data which is transformed to be data that has apparently been recorded at the virtual point of view. The virtual direction data and virtual range data is segmented to form voxels in step73. The change in direction and range of the voxels is used to produce a deformation map74.

A further alternate processing approach is depicted inFIG.8. The process ofFIG.8may be applicable if two slope monitoring devices have overlapping fields of view. That is to say, they both monitor the same section of slope or wall. Because each slope monitoring device is monitoring from a different point of view the data is not considered to be able to be combined. However, by using the process depicted inFIG.8a single deformation map of the overlapping area can be produced from the data from both slope monitoring devices. In a first step80first data is acquired from a first position with a first real point of view. The first data includes at least direction data and range data. The direction data and range data is segmented to form voxels in step81. In a simultaneous step82second data is acquired from a second position with a second real point of view. The second data includes at least direction data and range data. The direction data and range data is segmented to form voxels in step83. The data is transformed at step84to a common virtual point of view from which a deformation map85is produced and displayed.

FIG.9is a point cloud image of an extraction drive in an underground mine taken using a GML Lidar from a first real point of view. The image shows a tunnel going straight ahead and a draw point to the half right. The areas of interest for the users is the walls, the backs (roof) and the pillars. This scene represents a typical underground mine environment. This data is a non-voxelised raw data from the Lidar. The anomaly in the centre of the image is a sign.

FIG.10shows the non-voxelised raw point cloud image taken from a second real point of view of the same scene at a later time (next day).

FIG.11shows the data ofFIG.8voxelised based on the first real point of view. In this example, the projection is nominated to be the Common Point of View that subsequent scans will be transformed and compared with to derive deformation information.

FIG.12shows the data ofFIG.9voxelised based on the common point of view, which in this case is the same as the first real point of view plus occlusion errors. The dark occlusions on the corner of the image and adjacent to objects such as the sign are example errors where the data from the current scan does not exist in the common virtual point of view.

FIG.13shows a deformation heat map as if viewed from the common point of view. The magnitude of the deformation is displayed in a hot cold colour pallet heat map. In this example, the yellow to red colour map depicts the magnitude of the detected deformation movement from 2.0 mm to 5.0 mm respectively.