Patent Description:
This disclosure relates to monitoring walls. In particular, but not limited to, this disclosure relates to methods and systems including computer systems for monitoring walls.

<FIG> refers to application in mining and illustrates a simplified open-pit mine <NUM>. The mine <NUM> comprises an ore deposit <NUM>, two blasthole drills <NUM> and <NUM>, a shovel <NUM>, empty trucks <NUM> and <NUM> and loaded trucks <NUM>, <NUM> and <NUM>. The mine <NUM> further comprises a supply machine <NUM>, such as a fuel truck. The drill <NUM> drills blastholes, the material is blasted and then loaded onto truck <NUM>. The truck <NUM> then transports the material to a processing plant <NUM>. Similarly, drill <NUM> also drills blastholes for later blasting.

While some of the examples herein relate to the mining of iron ore, it is to be understood that the invention is also applicable to other mining and non-mining operations, such as extraction of coal, copper or gold or monitoring of deformations of natural or man-made slopes, excavations, constructions and mobile plants.

Mine <NUM> further comprises a control centre <NUM> connected to an antenna <NUM> and hosting on a computer <NUM> a mine control system. The mine control system <NUM> monitors operation and status data received from the mining machines wirelessly via antenna <NUM>. In one example, the control centre <NUM> is located in proximity to the mine site while in other examples, the control centre <NUM> is remote from the mine site, such as in the closest major city or in the headquarters of the resource company.

As a result of in situ stresses in the rock mass, sub-optimal mine design given the uncertain geological conditions, continuous blasting and extraction of material from mine <NUM> or a combination of the above, some areas of mine <NUM> may become unstable. For example, an area <NUM> is highlighted in <FIG>, which may become unstable. Unstable areas, and in particular, unstable walls, such as slopes, can fail and collapse, which poses a significant risk to the live of staff as well as to the commercial operation of the mine.

There exist monitoring systems that monitor potentially unstable areas, such as systems using laser range finders and radars. These existing systems generally monitor the line of sight movement. That is, these existing systems monitor whether the potentially unstable area moves towards or away from the laser and/or radar. A significant acceleration can indicate that the area is unstable and the system can raise an alarm. In many cases, however, the prediction of instability is inaccurate and in particular, areas have become unstable where the line of sight movement and/or acceleration did not indicate any problems.

<CIT> discloses a phase map calculated from radar images and video data to serve as a visual history of faults. While this offers a visual aid to the operator for monitoring line of sight movements, the outcome suffers from the above problem of insufficient inaccuracy especially when used for automatic monitoring without relying on a human operator.

<CIT> Pett discloses using a 2D interferometric synthetic aperture radar (InSAR) technique that provides a third resolving dimension. In particular, the third dimension is resolved by using a multi-baseline differential InSAR approach from two separate phase centres. While this solution is not limited to line of sight movement, it relies on multiple radar locations, which increases installation cost and complexity. Further, radar measurements have poor spatial resolution, which again leads to inaccurate results.

A method for monitoring a wall comprises:.

Since the data indicative of the change of the two-dimensional locations is determined based on the image data as well as the depth data, correlations between the image data and the depth data can be considered, which results in a more accurate determination of the change of the two-dimensional locations. As a result, the line of sight measurement of the deformation of the wall can be enriched by a perpendicular line of sight measurement, which allows more complete monitoring of the wall than existing methods that rely on only line of sight measurements.

Receiving depth data comprises receiving depth data indicative of an acceleration of the wall towards the depth sensor and determining the data indicative of the change of the two-dimensional locations is based on the acceleration of the wall.

Determining the data indicative of the change of the two-dimensional location is based on a positive correlation between the acceleration of the wall towards the depth sensor and an acceleration of the two-dimensional locations of the multiple points of the wall.

Determining the data indicative of the change of the two-dimensional locations comprises applying an adaptive filter to the first and second image data and the adaptive filter is parameterised based on the acceleration of the wall towards the depth sensor.

Applying an adaptive filter may comprise applying a Kalman filter and the Kalman filter may be parameterised based on the deformation of the wall towards the depth sensor.

The method may further comprise determining three dimensional movement of multiple blocks of a block model of the wall based on the data indicative of a change of the two-dimensional locations of the multiple points of the wall and the depth data.

The method may further comprise generating a graphical display indicative of the three dimensional movement of the multiple blocks of the block model of the wall.

Generating the graphical display may comprise generating graphical vectors indicating a three-dimensional velocity and three-dimensional direction of each block of the multiple blocks of the block model of the wall.

Generating the graphical display may comprise generating a graphical representation of each the multiple blocks of the block model of the wall with a colour indicative of the speed of movement of that block.

The method may further comprise generating an alarm based on the data indicative of a change of the two-dimensional locations of the multiple points of the wall.

Monitoring a wall may comprise monitoring a mining wall, receiving first image data may comprises receiving first image data representing two-dimensional locations of multiple points of the mining wall, receiving second image data may comprise receiving second image data representing the two-dimensional locations of the multiple points of the mining wall, receiving depth data may comprise receiving depth data indicative of a change in depth of the mining wall, and determining data indicative of a change may comprise determining data indicative of a change of the two-dimensional locations of the multiple points of the mining wall.

The mining wall may be a bench wall defined as the wall between two benches of the mine pit.

The method may further comprise adjusting an optical property of the image sensor such that the multiple points of the mining wall are multiple points of a desired area to be monitored.

wherein determining the data indicative of the change of the two-dimensional locations of the multiple points of the wall is based on the further image data and the further depth data.

The method may further comprise updating a graphical display each time further image data is received or each time further depth data is received.

Software, when installed on a computer, causes the computer to perform the method of any one of the preceding methods on a system defined below.

A computer system for monitoring a wall, which is not covered by the claimed subject-matter, comprises:.

A system for monitoring a wall comprises:.

The image sensor may be an optical camera and the depth sensor may be a radar or laser range finder.

A spatial resolution of the image sensor may be such that each of the multiple points of the wall covers less than <NUM> square metres on the wall and a spatial resolution of the depth sensor may be such that the depth data is indicative of the change in depth of more than <NUM> square metres on the wall. The spatial resolution of the image sensor may be higher than <NUM> megapixels.

The spatial resolution of the image sensor in relation to a spatial resolution of the depth sensor may be such that one value for change in depth relates to more than <NUM> two-dimensional locations.

The image sensor and the depth sensor may be located on respective mine benches of a mining pit opposite to the mining wall.

An open pit mine, which is outside the claimed invention, comprises:.

Optional features described of any aspect of method, system, computer system and software for monitoring a wall, where appropriate, similarly apply to the other aspects also described here.

For the purposes of the present invention, a wall includes any interface or surface between two media, where the first medium is often air and the second medium is often a solid, such as rock, brick concrete and other solids. The purpose of the wall is often to retain the solid against forces from within the solid and movement of the solid can be observed by observing the wall. This includes a slope, deformation or other structure having a surface of which one end or side is at a higher level than another (i.e. a rising or falling surface). The wall may form a barrier or enclose a space. The monitoring of the wall may relate to monitoring changing dimensions of the wall as it responds to forces acting upon it. The wall may comprise deformable or non-deformable material of variations sizes and compositions.

<FIG> illustrates an improved mine <NUM>, where a monitoring system <NUM> for monitoring mining wall <NUM> is installed on an opposite edge to the wall <NUM> to be monitored.

It is to be understood that mining wall <NUM> may be any inclined or vertical surface that is related to the extraction of a resource. The inclined surface may be rugged, broken and may comprise faults, defects and other discontinuities and rock mass structures. Mining walls may be man-made, such as walls separating benches created by excavation of a mining pit. Mining walls may also be nature-made, such as hill slopes or naturally occurring rock walls. The proposed methods and systems may apply to mining and non-mining operations, such as extraction of coal, copper or gold or monitoring of deformations of natural or man-made slopes, excavations, constructions and mobile plants.

The monitoring system <NUM> comprises an image sensor <NUM> and a depth sensor <NUM> both of which are connected to a computer system <NUM>. Generally, the image sensor <NUM> can be any sensor that acquires a multi-pixel two-dimensional representation that is perpendicular to a line of sight <NUM> from the image sensor <NUM> to the mining wall <NUM>. In one example, this may be a photographic camera, such as a consumer single lens reflex (SLR) camera, a hyperspectral camera, any other passive multi-pixel sensor or an active image sensor, such as an infrared camera with an infrared light source that does not require daylight to operate.

The depth sensor <NUM> may be a radar system, such as the IDS IBIS radar system or the GroundProbe SSR radar system, or a laser range finding system, such as the Site Monitor laser system by 3D laser mapping. It is noted, however, that current radar systems are more accurate than current laser range finding systems and therefore, radar systems are the preferred choice. The depth sensor <NUM> may capture a distance of one location of the wall <NUM> from the depth sensor <NUM> along a line of sight <NUM> from the depth sensor <NUM> to the wall <NUM>. In another example, depth sensor <NUM> captures the distance of multiple different locations of the wall <NUM> from the depth sensor <NUM>.

In one example, image sensor <NUM> and depth sensor <NUM> are located in close proximity to each other. As a result, when the following description makes reference to the line of sight, the line of sight <NUM> of the image sensor <NUM> and the line of sight <NUM> from the depth sensor <NUM> are considered to be identical.

Considering the depth sensor <NUM> is a radar system, depth sensor <NUM> generates a beam of electromagnetic waves and focusses the beam onto the mining wall. The beam width depends on the length of radar antenna or diameter of a dish-type antenna. For practical antenna lengths and dish sizes the beam width is relatively large compared to the field of view of each pixel of the image sensor <NUM>.

In the example of <FIG>, the radar beam is incident on a first area <NUM> and the image sensor <NUM> captures an image of a second area <NUM> with the individual pixels indicated as squares within second area <NUM>. While the first area <NUM> and the second area <NUM> are shown disjoint for clarity of illustration, in most examples, the first area <NUM> would overlap the second area <NUM>. Depth sensor <NUM> captures a single depth value at one time for the first area <NUM> while image sensor <NUM> captures multiple pixel values at one time for second area <NUM>. In one example, depth sensor <NUM> is rotatably mounted to allow the depth sensor <NUM> to scan the wall <NUM>, such that over time multiple depth values are captured covering the entire mining wall <NUM>.

Depth sensor <NUM> may continuously scan the mining wall <NUM> over time such that the first area <NUM> is scanned multiple times at subsequent points in time. This allows the depth sensor <NUM> to determine depth data that is indicative of a change in the depth of area <NUM> of the wall <NUM>, such as a change in millimetres per day.

As shown in <FIG>, the open pit mine <NUM> comprises five benches <NUM>, <NUM>, <NUM>, <NUM> and <NUM>, respectively. Between the benches are four bench walls or faces <NUM>, <NUM>, <NUM> and <NUM>, respectively. In the example of <FIG>, mine wall <NUM> that is to be monitored covers bench wall <NUM> and bench wall <NUM>. That is, monitoring system <NUM> monitors the wall between benches <NUM> and <NUM> and the wall between benches <NUM> and <NUM>. Image sensor <NUM> and depth sensor are located on top bench <NUM> and opposite to the mining wall <NUM>. Further, opposite mining wall <NUM> may also refer to a location on a line from mining wall <NUM> which is not exactly perpendicular to mining wall <NUM>. For example, location <NUM> may also be considered opposite the mining wall <NUM> as the direct line of sight from location <NUM> to mining wall <NUM> passes a substantial part of the mine pit and offers substantially unobstructed view of the mining wall <NUM>.

In one example, open pit mine further comprises a signalling system <NUM> to initiate a mitigation process in relation to the bench wall <NUM>. The signalling system may be a data transmission system that sends alerts to the mining machines to instruct the machines or the operators of those machines to leave the mine pit or to move away from wall <NUM>. Signalling system <NUM> may also be an acoustic system that generates an acoustic alarm, such as a siren, to alert staff of the unstable bench wall <NUM>.

<FIG> illustrates a side elevation of mining wall <NUM> and depth sensor <NUM>. In this example, mining wall <NUM> moves from a first position <NUM> to a second position <NUM> and deforms during this move. Depth sensor <NUM> captures the distance at each of four vertically aligned areas on mining wall <NUM> at two consecutive points in time. Depth sensor <NUM> then determines depth data indicative of a change in the depth of the wall <NUM>, which is indicated as four arrows in <FIG>. For example, the topmost arrow <NUM> indicates a relatively large change, while the bottom arrow <NUM> indicates a relatively small change. Note that these arrows represent deformation along the line of sight of the depth sensor <NUM> and may converge. As an example, the deformation of the wall <NUM> may be assessed as leading to likely collapse in the near future and shovel <NUM> should be moved to safety immediately to avoid risking machinery and personnel. The depth data may be presented as a graphical representation <NUM> of the mining wall where dark areas indicate areas of relatively great movement while light areas indicate areas of relatively small movement.

In one example, depth sensor <NUM> pre-processes the measurements such that the depth data is indicative of an acceleration of the mining wall <NUM> towards the depth sensor <NUM>.

It is noted that depth sensor <NUM> only detects movement of the mining wall <NUM> in the direction of line of sight <NUM> (LOS) and not in the direction perpendicular to the line of sight (PLOS).

<FIG> illustrate first image data <NUM> and second image data <NUM>, respectively, as captured by image sensor <NUM>. First image data <NUM> represents two-dimensional locations of multiple points of the mining wall. For example, a first object <NUM> can be considered as covering multiple points of the mining wall and the first image data <NUM> represents the two-dimensional location of the points of first object <NUM>. More particularly, the first image data <NUM> represents the two-dimensional location as a pixel coordinates of these points relative to the sensor. For example, the image sensor has a resolution of <NUM>,<NUM> x <NUM>,<NUM> pixels and as a result, the horizontal coordinate of the points on the mining wall ranges from <NUM> to <NUM>,<NUM> and the vertical coordinate ranges from <NUM> to <NUM>,<NUM>.

Similarly, image data <NUM> represents two-dimensional locations of further points or objects <NUM>, <NUM>, <NUM> and <NUM>. These objects may be rock formations, such as visible fault lines, ledges, boulders, rocks, stones, cobbles and pebbles as well as equipment, such as optical markers or light reflectors. Image sensor <NUM> may also be a hyperspectral camera and the objects may be defined by a continuous reflectance spectrum. Image sensor <NUM> may also comprise an infra-red image sensor that captures infra-red radiation emitted by an infrared light source and reflected off the wall <NUM>. The infra-red light source may illuminate the wall <NUM> from sideways to generate more prominent shadows that may facilitate the detection of the objects or the shadows may constitute the multiple points of the mining wall <NUM>.

First image data <NUM> is associated with a first point in time, such as the capture time of the image data <NUM>. For example, first image data <NUM> was captured at 1pm on <NUM> July <NUM>. The capture date and time may be stored as meta-data of the image data according to JPEG format or RAW formats.

Second image data <NUM> in <FIG> also represents two-dimensional locations of objects <NUM>, <NUM>, <NUM>, <NUM> and <NUM> as described with reference to the first image data <NUM> in <FIG>. However, the second image data <NUM> is associated with a second point in time being later than the first point in time associated with the first image data <NUM>. For example, the second point in time is one day after the first point in time, that is, 1pm on <NUM> July <NUM>.

As can be seen in <FIG>, the objects <NUM>, <NUM>, <NUM>, <NUM> and <NUM> have moved to different two-dimensional locations during that day as indicated by straight arrows in <FIG>. By matching the objects in <FIG> to the objects in <FIG>, computer system <NUM> can determine the two-dimensional movement of the mining wall perpendicularly to the line of sight <NUM>. However, in most practical applications, the noise on this determination leads to unacceptable inaccuracies. In order to improve the accuracy, computer system <NUM> determines data indicative of a change of the two-dimensional locations of the multiple points of the mining wall between the first point in time and the second point in time based on the first image data <NUM>, the second image data <NUM>, and the depth data <NUM> of <FIG>.

This fusion of depth data <NUM> and image data <NUM>/<NUM> is particularly useful where the depth data is highly accurate in terms of the detected movement and where the line of sight deformation of the mining wall <NUM> is correlated with the perpendicular to line of sight deformation of the mining wall <NUM>. For example, processor <NUM> may use a linear relationship between the line of sight deformation and the perpendicular line of sight deformation. The mathematical details of the determination of the change of locations are provided further below.

<FIG> illustrates computer system <NUM> for monitoring mining wall <NUM> in more detail. The computer system <NUM> comprises a processor <NUM> connected to a program memory <NUM>, a data memory <NUM>, a communication port <NUM> and a user port <NUM>. The program memory <NUM> is a non-transitory computer readable medium, such as a hard drive, a solid state disk or CD-ROM. Software, that is, an executable program stored on program memory <NUM> causes the processor <NUM> to perform the method in <FIG>, that is, processor <NUM> receives first and second image data and depth data and determines data indicative of a change of the mining wall.

The processor <NUM> may then store the data indicative of the change on data store <NUM>, such as on RAM or a processor register. Processor <NUM> may also send the determined data via communication port <NUM> to a server, such as mine control centre.

The processor <NUM> may receive data, such as image data and depth data, from data memory <NUM> as well as from the communications port <NUM> and the user port <NUM>, which is connected to a display <NUM> that shows a visual representation <NUM> of the mining wall <NUM> to an operator <NUM>. In one example, the processor <NUM> receives image data and depth data from image sensor <NUM> or depth sensor <NUM> via communications port <NUM>, such as by using a Wi-Fi network according to IEEE <NUM>. The Wi-Fi network may be a decentralised ad-hoc network, such that no dedicated management infrastructure, such as a router, is required or a centralised network with a router or access point managing the network.

In one example, the processor <NUM> receives and processes the image data and depth data in real time. This means that the processor <NUM> determines the data indicative of a change of the two-dimensional locations of the mining wall every time image data or depth data is received from sensors <NUM> and <NUM> and completes this calculation before the sensors <NUM> or <NUM> send the next update.

Although communications port <NUM> and user port <NUM> are shown as distinct entities, it is to be understood that any kind of data port may be used to receive data, such as a network connection, a memory interface, a pin of the chip package of processor <NUM>, or logical ports, such as IP sockets or parameters of functions stored on program memory <NUM> and executed by processor <NUM>. These parameters may be stored on data memory <NUM> and may be handled by-value or by-reference, that is, as a pointer, in the source code.

The processor <NUM> may receive data through all these interfaces, which includes memory access of volatile memory, such as cache or RAM, or non-volatile memory, such as an optical disk drive, hard disk drive, storage server or cloud storage. The computer system <NUM> may further be implemented within a cloud computing environment, such as a managed group of interconnected servers hosting a dynamic number of virtual machines.

It is to be understood that any receiving step may be preceded by the processor <NUM> determining or computing the data that is later received. For example, the processor <NUM> determines image data or depth data and stores the image data or depth data in data memory <NUM>, such as RAM or a processor register. The processor <NUM> then requests the data from the data memory <NUM>, such as by providing a read signal together with a memory address. The data memory <NUM> provides the data as a voltage signal on a physical bit line and the processor <NUM> receives the image data or depth data via a memory interface.

It is to be understood that throughout this disclosure unless stated otherwise, nodes, edges, graphs, solutions, variables and the like refer to data structures, which are physically stored on data memory <NUM> or processed by processor <NUM>. Further, for the sake of brevity when reference is made to particular variable names, such as "point in time" or "quantity of movement" this is to be understood to refer to values of variables stored as physical data in computer system <NUM>.

<FIG> illustrates a method <NUM> as performed by processor <NUM> for monitoring a mining wall. Processor <NUM> first receives <NUM> from image sensor <NUM> first image data <NUM> representing two-dimensional locations of multiple points of the mining wall <NUM> associated with a first point in time as described with reference to <FIG>.

Processor <NUM> further receives <NUM> from image sensor <NUM> second image data <NUM> representing the two-dimensional locations of the multiple points of the mining wall <NUM> associated with a second point in time, the second point in time being later than the first point in time as described with reference to <FIG>.

Processor <NUM> also receives <NUM> depth data <NUM> from depth sensor <NUM> indicative of a change in depth of the mining wall between the first point in time and the second point in time as described with reference to <FIG>. It is noted that the depth data may be recorded for a time period that is different to the time period between the first point in time and the second point in time. For example, the depth data may be indicative of a change in depth of the mining wall <NUM> between a point in time that is before the first image data <NUM> was captured and a point in time that is after the second image data <NUM> was captured. Such 'wider' depth data is still indicative of the change of depth for the time during the capture of the two images. Computer system <NUM> may interpolate the depth data or the image data to align the different time scales.

Finally, processor <NUM> determines <NUM> data indicative of a change of the two-dimensional locations of the multiple points of the mining wall between the first point in time and the second point in time based on the first image data, the second image data, and the depth data.

<FIG> is to be understood as a blueprint for the monitoring software program and may be implemented step-by-step, such that each step in <FIG> is represented by a function or class in a programming language, such as C++ or Java. The resulting source code is then compiled and stored as computer executable instructions on program memory <NUM>.

Mining wall <NUM> may also be referred to as a slope. The slope is deforming due to the development of a failure, and the deformation can be characterised by a 3D vector. Due to operational and other reasons, slope monitoring radar <NUM> may be placed in a sub-optimal location with respect to the deformation such that only a component of the 3D deformation vector (say at an angle θ of <NUM> degrees) is along the line of sight (LOS) of the radar <NUM>.

The other component (sin(<NUM>) or <NUM>%) corresponds to perpendicular to line of sight (PLOS) deformation, potentially detectable by the proposed imaging system <NUM>.

<FIG> illustrates an example deformation <NUM> of the wall <NUM> comprising a deformation along the LOS <NUM> and PLOS <NUM>. The rate of deformation is non-linear and may indeed progress through both 'progressive' and 'regressive' phases as described in <NPL>.

In one example, mining wall <NUM> shows constant velocity deformation and from day <NUM>, gradually accelerating over <NUM> more days. The movement may comprise a Gaussian 'spike' <NUM> at <NUM> days to represent some anomalous, unpredictable behaviour (e.g. stick -slip of the failure).

Finally, sub-millimetre noise may be present in the LOS data, which can be found in some radar deformation monitoring systems.

In one example, the calculations performed by processor <NUM> are based on an assumption that the true perpendicular to LOS (PLOS) deformations are a linear function of the LOS deformations, such as the case for a rigid block sliding with no rotation. In other words, the calculations are based on a positive correlation between the acceleration of the mining wall towards the depth sensor <NUM> and the acceleration of the two-dimensional locations of the multiple points of the mining wall <NUM>. As previously stated, the relationship of the PLOS and LOS components is given by<MAT>.

<FIG> illustrates the deformation measurements comprising additional noise <NUM> affecting the measurements of the PLOS <NUM> and <NUM>. The noise may be much greater than the noise on the radar data <NUM>. For an imaging system employing feature tracking to measure PLOS, processor <NUM> may use sub-pixel accuracy for the feature matching. In one example, imaging sensor <NUM> is a camera , such as a Nikon D800e and AF-S NIKKOR <NUM> f/<NUM>. 6E FL ED VR <NUM> x <NUM> Nikon FX format <NUM>,<NUM> x <NUM>,<NUM> pixels. The range to target may be <NUM> metres. Multiples of this pixel size (three times) is added as zero mean Gaussian noise to the PLOS data.

The significance of the noise may become apparent when measurements are undertaken as the accelerating phase begins. <FIG> illustrates an enlarged period of the LOS measurements <NUM>, PLOS measurements <NUM> and noise <NUM> from <NUM> to <NUM> days, which also includes the 'spike' event <NUM>.

In one example, an adaptive filtering method, such as a Kalman filter, may be used as a component of the data processing algorithm in method <NUM>.

A Kalman filter is a recursive data processing algorithm. It allows processing of all available measurements for estimation of the current variables being studied. The constraints for using a Kalman filter are sufficient for this demonstration, namely that the process model be linear (unless the extended version of the filter is used), the process noise be Gaussian and white (constants spectral density). It is defined as <MAT>.

The system being modelled assumes a measurement model applies that describes how the sensor data relates to the state variables: <MAT> where z, is the predicted measurement model after accounting for the estimated state and measurement noise vt ~ N(<NUM>,Rt) where R represents the covariance of the measurement noise.

Processor <NUM> applies a Kalman filter using the LOS signal, that is, the depth data, as a guide to the PLOS signal, that is, the image data. Let us define the system state as the PLOS true, that is, the data indicative of a change of the two-dimensional locations of the multiple points of the mining wall, and the LOS data, that is, the depth data, as a control signal. State <MAT> Measure <MAT> Control input: <MAT>.

In one example, the calibration constant B for the control signal is assumed to be known perfectly.

In another example, processor <NUM> performs the calculations with an inaccurate estimate (i.e. guess) of calibration constant B for the control signal. In some examples, processor <NUM> determines this constant such as through the use of iterative estimation algorithms, such as Expectation Maximisation, in a pre-processing step.

If the constant is not calibrated, there may be a systematic bias error in the filtered LOS data.

Processor <NUM> may also use a different filter where the state is defined using a velocity inferred from the LOS data: <MAT> that is, <MAT> (PLOSdot) is the derivative of the perpendicular to line of sight position with respect to time, which is the velocity perpendicular to the line of sight. This velocity is assumed to be proportional to the derivative of the line of sight movement with respect to time, which is the velocity along the line of sight.

In this method, no prior knowledge of the geometry (i.e. tan(θ) factor) is assumed. State <MAT>.

With the above definitions processor <NUM> can perform the predict and update steps of the Kalman filter as follows where t is replaced by k:.

In one example, the values of the above matrices are determined based on library functions providing the Kalman filter, such as Python libraries pykalman, scipy, filterpy or Matlab kalman class in the Control System Toolbox.

To summarise, the variable x holds data indicative of a change of the two-dimensional location (PLOS) and depth (LOS) of the multiple points of the mining wall and the calculation of x is based on the image data and depth data represented by the variable z and also potentially the variable u.

<FIG> illustrates the result of method <NUM> where the crosses <NUM> indicate the result of the Kalman filtered PLOS signal.

<FIG> illustrates a simplified block model <NUM> of mining wall <NUM>, where the mining wall <NUM> is spatially discretised in three dimensions resulting an multiple blocks spanning the three-dimensional shape of the mining wall <NUM>. By calculating the data indicative of the change of the two-dimensional locations (PLOS) processor <NUM> can use the depth data (LOS) to assign a three-dimensional vector of movement or acceleration to each block. These vectors are depicted as vectors in <FIG> such as example vector <NUM> for example block <NUM>.

Processor <NUM> may generate a graphical representation of each the multiple blocks of the block model of the mining wall similar to what is shown in <FIG>, which may be displayed on display device <NUM> in <FIG>. In the graphical representation, the colour of the blocks may be indicative of the speed of movement of that block. In the example of <FIG> a dark colour indicates a relatively fast movement and therefore a relatively unstable part of the wall while white colour represents a relatively slow movement and therefore, a relatively stable part of the wall.

Once the two dimensional movement of that block (PLOS) or the LOS movement satisfies a predefined criterion, such as maximum deformation, speed or acceleration, processor <NUM> initiates a mitigation process, such as by generating an alarm that may trigger the evacuation of the mine <NUM>, or evacuation from the area affected by a potential collapse of mining wall <NUM>, or modification of mine operations to mitigate the risk associated with wall collapse for people and machinery.

It will be appreciated by persons skilled in the art that numerous variations and/or modifications may be made to the specific embodiments without departing from the scope as defined in the claims. For example, while the above examples relate to monitoring pit walls in mining operations, the disclosed methods and systems may be equally applicable in other applications of monitoring walls including any inclined and vertical planes or surfaces, such as monitoring exposed rock slopes and may even be applicable when monitoring other structures such as dams, bridges or mobile plant.

In those examples, image sensor <NUM> and depth sensor <NUM> are located at a distance, such as <NUM> metres, from the structure and are directed at the structure. The objects in <FIG> may then include features of that structure, such as windows, stay cables, edges or gaps between bricks or concrete blocks. In one example, monitoring system <NUM> monitors a concrete dam while the concrete cures over an extended period of time.

Wall monitoring may also relate to monitoring of landslides etc. The mining wall <NUM> may relate to a subterranean / sub-surface wall. In other words, the methods and systems disclosed herein may be applicable to slope monitoring in general including sub-surface slope monitoring and in particular, to monitoring of mining walls.

It should be understood that the techniques of the present disclosure might be implemented using a variety of technologies. For example, the methods described herein may be implemented by a series of computer executable instructions residing on a suitable computer readable medium. Suitable computer readable media may include volatile (e.g. RAM) and/or non-volatile (e.g. ROM, disk) memory, carrier waves and transmission media. Exemplary carrier waves may take the form of electrical, electromagnetic or optical signals conveying digital data steams along a local network or a publically accessible network such as the internet.

It should also be understood that, unless specifically stated otherwise as apparent from the following discussion, it is appreciated that throughout the description, discussions utilizing terms such as "estimating" or "processing" or "computing" or "calculating", "optimizing" or "determining" or "displaying" or "maximising" or the like, refer to the action and processes of a computer system, or similar electronic computing device, that processes and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.

Claim 1:
A method (<NUM>) for monitoring a wall (<NUM>), the method comprising:
receiving from an image sensor (<NUM>) first image data (<NUM>) representing two-dimensional locations of multiple points of the wall associated with a first point in time;
receiving from the image sensor (<NUM>) second image data (<NUM>) representing the two-dimensional locations of the multiple points of the wall associated with a second point in time, the second point in time being later than the first point in time;
receiving depth data from a depth sensor (<NUM>) indicative of a change in depth (<NUM>) of the wall between the first point in time and the second point in time, and the depth data is further indicative of an acceleration of the wall towards the depth sensor (<NUM>); and
applying an adaptive filter to the first and second image data and the adaptive filter is parameterised based on the acceleration of the wall towards the depth sensor, to determine data indicative of a change of the two-dimensional locations of the multiple points of the wall between the first point in time and the second point in time perpendicular to a line of sight from the image sensor (<NUM>),
wherein determining the data indicative of the change of the two-dimensional locations is based on:
the acceleration of the wall; and
a positive correlation between the acceleration of the wall towards the depth sensor (<NUM>) and an acceleration of the two-dimensional locations of the multiple points of the wall.