Abstract:
A sensor compares frames of pixels representing a speckle pattern caused by interference of light through an optical fiber to detect magnitudes of deflection of the fiber. A coherent light source illuminates the optical fiber. An image sensor captures the speckle pattern and frames of pixels produced by the image sensor are processed to determine deflection. A baseline frame is generated from frames previously received. Each frame is compared to the baseline frame to determine the cumulative amount of deflection on the fiber. To compensate for drift and large-scale movements of the optical fiber, the baseline frame is updated as frames are received. The processed output from comparing to the baseline frame has larger amplitude signals than from comparing to adjacent frames due to larger deflections over time since the baseline frame. Signal-to-Noise ratio is improved, and the processed output better matches a plot of the actual total deflection.

Description:
FIELD OF THE INVENTION 
       [0001]    This invention relates to sensing devices, and more particularly to detecting changes in interference patterns from fiber optics. 
       BACKGROUND OF THE INVENTION 
       [0002]    Optical fibers are sometimes used to detect physical movement, deflection, or perturbation, such as bending of a security fence or of a mat with a fiber optics strand placed underneath a patient in a bed. Coherent light is used to illuminate the fiber at one end, while a photo detector at the other end reads an interference, stipple, or speckle pattern. The speckle pattern is created by multi-mode interference in the long optical fiber. This speckle pattern changes as the fiber is deflected. 
         [0003]      FIG. 1  shows a prior-art multi-mode optical-fiber sensor. Coherent light source  101  illuminates one end of optical fiber  102 . The light exiting the other end of optical fiber  102  produces modal interference pattern  103 , which is commonly called a speckle pattern. Interference pattern  103  is detected by image sensor or photo-detector  104  to produce a series of readings  105  that are processed by image processor  106  to produce processed output  107 . 
         [0004]    A multi-mode optical-fiber sensor may be used as an intrusion alarm system. A subset of the speckle pattern may be detected. The overall amount of light detected changes as the speckle pattern changes. The light detected may be compared against the previous reading to generate the sensor output. 
         [0005]    When an image sensor is used rather than a photo detector, a frame of pixels may be detected and generated from the speckle pattern. Subsequent video frames from the image sensor may be compared to detect changes in the speckle pattern. Sometimes only a portion of the speckle pattern or frame is detected or processed. 
         [0006]    The speckle pattern is not always used in detection. Interferometry between two optical fibers may be used to detect patient vital signs or as a security device for perimeter protection. A much simpler photo-detector may be used when the speckle pattern is not detected. 
         [0007]    Multi-mode fiber-optic sensors are distinguished by whether the speckle pattern is detected with a photo-detector, that only produces a single overall output, or with a Charge-Coupled Device (CCD), complementary metal-oxide-semiconductor (CMOS) sensor, or some other 2-D image sensor that produces a frame of pixel values. Fiber-optic sensors based upon image sensors tend to be significantly more sensitive than sensors based upon photo detectors because small displacements within the field of view of the photo detector do not register as a change in the overall amount of light received by the photo detector, whereas such small displacements in the speckle pattern can be detected on a 2-D image sensor that produces an array of pixel readings. 
         [0008]    Existing multi-mode fiber-optic sensors based upon image sensors typically compare subsequent speckle images to detect changes to the speckle pattern. In  FIG. 1 , changes over time to speckle pattern  103  are detected by image sensor  104  that produces a 2-D frame of pixels for each snapshot in time. Over a longer period of time, a stream of frames of pixels, readings  105 , may be processed by image processor  106  to produce processed output  107 . 
         [0009]      FIG. 2  shows image processing on a frame-by-frame basis. Incoming frames  200  from the image sensor are processed on a frame-by-frame basis and delayed by frame delay  206 . Current frame  201  is compared with previous frame  202  on a pixel-by-pixel basis, such as by subtraction of pixel values by pixel comparator  203 . Processed output  107  is the sum of the absolute value of the difference between each pixel and the corresponding pixel in the previous frame. Summer  205 , absolute value generator  204 , and pixel comparator  203  produce a sum-of-the-absolute difference (SAD) for all pixels between current frame  201  and previous frame  202 . 
         [0010]      FIG. 3  shows plots of raw and processed changes to a speckle pattern. Dotted curve  301  is a representation of the cumulative change to the speckle pattern over time, such as for changes in readings  105  from the image sensor. Frames are sampled periodically, so for every sample there is delay  302 , represented by a horizontal line, and sensor reading  303 , represented by a vertical step. In this figure, sensor reading  303  indicates the absolute value of difference between the corresponding pixel in a frame and in the previous frame summed across all pixels. It is the measured deflection of the optical fiber during the time period of delay  302 . 
         [0011]    Processed output  107 , shown as bar readings  304  in the lower graph, consists individual sensor readings  303  over time. Note that the shape of the processed output&#39;s bar readings  304  is not consistent with the change to the speckle pattern of dotted curve  301  because individual readings  303  measure the amount of change between frames. In fact, when cumulative change to the speckle pattern reaches a peak, such as at the middle peak of dotted curve  301  (top graph), processed output  107  has bar reading  304  that are close to zero (middle of bottom graph). 
         [0012]    Since the individual readings can drop to almost zero at the peak of the cumulative change to the speckle pattern, periodic signals such as breathing or vibration may be destroyed because every peak on the signal is transformed into multiple peaks in processed output  107 . This complicates using a FFT or a DFT to take a frequency response for determining breath rate, heart rate, or frequency of vibration. 
         [0013]    Furthermore, since individual reading  303  measures the incremental change between subsequent frames, every reading  303  is less than the cumulative change to the speckle pattern, shown by dotted curve  301 . Hence the peak output of processed output  107  is less than the peak change to the speckle pattern, the peak of dotted curve  301 , reducing the signal to noise ratio of the sensor. 
         [0014]    If the sampling frequency is increased, the sample period decreases and the difference between subsequent frames is reduced.  FIG. 4  shows plots of raw and processed changes to a speckle pattern for a higher sampling frequency. 
         [0015]    At a higher sampling frequency, delay  302  ( FIG. 3 ) is reduced to delay  402 . Note that sensor reading  403  is smaller than corresponding sensor reading  303  because the change to the speckle pattern, shown by dotted curve  301 , is smaller because of the smaller sample period. Note also that processed output  107  has bar readings  404  that are much smaller than bar readings  304  of  FIG. 3 . Thus increasing the sampling frequency decreases amplitude and the signal-to-noise ratio of the sensor. 
         [0016]    What is desired is better processing of frames of pixels from image sensors for optical fiber deflection detectors. A better signal-to-noise ratio is desired for the processed output of such detectors. A more direct and accurate measure of fiber deflection is desirable. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0017]      FIG. 1  shows a prior-art multi-mode optical-fiber sensor. 
           [0018]      FIG. 2  shows image processing on a frame-by-frame basis. 
           [0019]      FIG. 3  shows plots of raw and processed changes to a speckle pattern. 
           [0020]      FIG. 4  shows plots of raw and processed changes to a speckle pattern for a higher sampling frequency. 
           [0021]      FIG. 5  shows a baseline-comparing multi-mode optical-fiber sensor. 
           [0022]      FIG. 6  shows processing frames against a baseline image frame. 
           [0023]      FIG. 7  shows plots of raw and processed changes to a speckle pattern using baseline frame comparison. 
           [0024]      FIG. 8  shows plots of raw and processed changes to a speckle pattern for a higher sampling frequency using baseline comparison. 
           [0025]      FIG. 9  shows an updating-baseline multi-mode optical-fiber sensor. 
           [0026]      FIG. 10  shows poor baseline management. 
           [0027]      FIG. 11  shows processing frames against a baseline image frame with baseline updating. 
           [0028]      FIG. 12  shows a baseline update process. 
           [0029]      FIG. 13  is a graph of a processed output and a raw adjacent frame difference over time. 
       
    
    
     DETAILED DESCRIPTION 
       [0030]    The present invention relates to an improvement in optical-fiber deflection detectors. The following description is presented to enable one of ordinary skill in the art to make and use the invention as provided in the context of a particular application and its requirements. Various modifications to the preferred embodiment will be apparent to those with skill in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the particular embodiments shown and described, but is to be accorded the widest scope consistent with the principles and novel features herein disclosed. 
         [0031]    The inventor has developed a method for processing frames received from the image sensor in a multi-mode optical-fiber sensor to more directly measure deflection. Rather than compare adjacent frames, frames of pixels are compared against a baseline frame to produce a processed output that is consistent with cumulative changes to the speckle pattern, has an optimal signal to noise ratio, and that does not depend upon the sampling rate. 
         [0032]      FIG. 5  shows a baseline-comparing multi-mode optical-fiber sensor. Coherent light source  101  illuminates one end of optical fiber  102 . The light exiting the other end of optical fiber  102  produces modal interference pattern  103 , which is commonly called a speckle pattern. Interference pattern  103  is detected by image sensor  104  to produce a series of readings  105  or frames that are each compared to baseline image  508  by baseline image processor  516  to produce processed output  507 . 
         [0033]      FIG. 6  shows processing frames against a baseline image frame. Incoming frames  200  from the image sensor are processed and compared to baseline frame  503  on a frame-by-frame basis. Current frame  201  is compared with baseline frame  503  on a pixel-by-pixel basis, such as by subtraction of pixel values by pixel comparator  502 . Processed output  507  ( FIG. 5 ) is over-baseline value  506 , which is the sum of the absolute value of the difference between each pixel in the current frame and the corresponding pixel in the baseline frame. Summer  505 , absolute value process  504 , and pixel comparator  502  produce a sum-of-the-absolute difference (SAD) for all pixels between current frame  201  and baseline frame  503 . This SAD is not the difference between adjacent or sequential frames, but instead is the difference of the current frame to the baseline frame, over-baseline value  506 . 
         [0034]      FIG. 7  shows plots of raw and processed changes to a speckle pattern using baseline frame comparison. In this Figure, baseline frame  705  is selected to be the first frame received from the image sensor and every subsequent frame is compared to baseline frame  705 . Since baseline frame  705  is not compared to any prior frame, it has no difference or SAD with another frame and is represented by the horizontal base line. 
         [0035]    Dotted curve  301  is a representation of the cumulative change to the speckle pattern over time, the cumulative measured deflection of readings  105  from the image sensor. Frames are sampled periodically, so for every sample there is delay  702 , represented by a horizontal line segment, and over-baseline reading  703 , represented by a vertical step. In this figure, over-baseline reading  703  indicates the absolute value of difference between the corresponding pixel in a frame and in the baseline frame summed across all pixels. It is the measured deflection of the optical fiber during all time periods since the baseline frame. Thus the height of over-baseline readings  703  tend to be much larger than sensor readings  303  ( FIG. 3 ), since the over-baseline readings represent the total deflection of the fiber rather than the incremental deflection that occurred between frames. 
         [0036]    Processed output  507  ( FIG. 5 ), which is over-baseline value  506  ( FIG. 6 ), is shown as bar readings  704  in the lower graph, and consists of over-baseline readings  703  generated over time. Note that the shape of the processed output&#39;s processed readings  704  is consistent with the cumulative change to the speckle pattern of dotted curve  301  because individual over-baseline readings  703  measure the amount of change between a frame and the baseline frame, not adjacent frame. When cumulative change to the speckle pattern reaches a peak, such as at the middle peak of dotted curve  301  (top graph), processed output  507  has processed readings  704  that are also at a peak. The shape of the envelope of processed readings  704  matches the shape of dotted curve  301 . 
         [0037]      FIG. 7  shows that processed readings  704  are consistent with changes to the speckle pattern (dotted curve  301 ). Also, the amplitude of processed readings  704  is about the same as the amplitude of cumulative changes to the speckle pattern shown by dotted curve  301 . Processed readings  704  track changes to the speckle pattern. The signal-to-noise ratio of the sensor is optimal since over-baseline readings  703  have a large amplitude, as shown by the large heights. Periodic signals such as breathing or heartbeat may be accurately captured and easily processed using standard discreet signal post-processing algorithms. 
         [0038]      FIG. 8  shows plots of raw and processed changes to a speckle pattern for a higher sampling frequency using baseline comparison. If the sampling frequency is increased, the sample period decreases and the difference between subsequent frames is reduced. However, since each frame is compared to a baseline frame, not to adjacent frame, the time delay between frames is not critical. 
         [0039]    At a higher sampling frequency, delay  702  ( FIG. 7 ) is reduced to delay  402 . Although frame-to-frame differences are smaller due to the smaller time delay between frames, the difference to the baseline frame is not affected by the delay between frames and the insertion of additional frames for the higher sampling frequency. 
         [0040]    Processed readings  804  are shown assuming the same delay  402  as in  FIG. 4  and cumulative changes in the speckle pattern as shown by dotted curve  301 . Baseline frame  705  is selected to be the first frame received from the image sensor. Note that the amplitude of the processed readings  804  is the same as the amplitude of changes to the speckle pattern, dotted curve  301 . The signal-to-noise ratio of the sensor is optimal. 
         [0041]      FIG. 9  shows an updating-baseline multi-mode optical-fiber sensor. Coherent light source  101  illuminates one end of optical fiber  102 . Constructive and destructive interference of the light as it propagates through the fiber creates modal interference pattern  103 , a speckle pattern, that changes as the fiber is deflected. Changes to the speckle pattern are detected by image sensor  104  to produce a series of frames or readings  105  that are each compared to baseline image  508  by baseline image processor  516  to generate over-baseline value  506 . Over-baseline value  506  is output as processed output  510 . 
         [0042]    Baseline image  508  is periodically updated based upon the sensor output to compensate for dark-current noise in the image sensor, changes in geometry of the optical fiber, and other sources of sensor drift. Baseline updater  901  reads over-baseline value  506  and adds a weighted amount of over-baseline value  506  and the old value of baseline image  508  to generate a new updated value for baseline image  508 . 
         [0043]      FIG. 10  shows poor baseline management. In this example, baseline image  1005  (the horizontal line) is selected to be a frame intermediate to the minimum and maximum of changes to the speckle pattern. This can occur with a periodic signal if the baseline frame is incorrectly selected. Since the absolute difference is obtained by image processor  516  , below-baseline values of over-baseline readings  1003  are reflected along the axis of baseline image  1005  so that the processed output has large positive bars rather than large negative bars for these below-baseline readings. 
         [0044]    Because of this reflection of below-baseline readings, over-baseline readings  1003  of the processed output do not match changes to the speckle pattern, dotted curve  301 , and the signal-to-noise ratio of the sensor is reduced. In fact, processed output  1004  is not improved over sensor output  304  from the prior art. 
         [0045]      FIG. 11  shows processing frames against a baseline image frame with baseline updating. Incoming frames  200  from the image sensor are processed and compared to baseline frame  503  on a frame-by-frame basis. Current frame  201  is compared with baseline frame  503  on a pixel-by-pixel basis, such as by subtraction of pixel values by pixel comparator  502 . Processed output  510  ( FIG. 9 ) is over-baseline value  506 , which is the sum of the absolute value of the difference between each pixel in the current frame and the corresponding pixel in the baseline frame. Summer  505 , absolute value process  504 , and pixel comparator  502  produce a sum-of-the-absolute difference (SAD) for all pixels between current frame  201  and baseline frame  503 . This SAD is not the difference between adjacent or sequential frames, but instead is the difference of the current frame to the baseline frame, over-baseline value  506 . 
         [0046]    Incoming frames  200  from the image sensor are processed on a frame-by-frame basis and delayed by frame delay  206 . Current frame  201  is compared with previous frame  202  on a pixel-by-pixel basis, such as by subtraction of pixel values by pixel comparator  203 . Adjacent frame difference  1102  is the sum of the absolute value of the difference between each pixel and the corresponding pixel in the previous frame. Summer  205 , absolute value generator  204 , and pixel comparator  203  produce a sum-of-the-absolute difference (SAD) for all pixels between current frame  201  and previous frame  202 . 
         [0047]    Baseline updater  901  reads adjacent frame difference  1102  and determines when to update baseline frame  503 , and by how much. 
         [0048]      FIG. 12  shows a baseline update process. The current or a recent over-baseline value  506  and adjacent frame difference  1102  are evaluated by current frame weight function  1204  to produce current frame weight  1205 . If the current frame weight is greater than 0, pixels in baseline frame  503  are multiplied in multiplier  1203  by baseline weight  1202  and pixels in the current frame  201  are multiplied in multiplier  1206  by current frame weight  1205 . Corresponding weighted pixels in the baseline frame and in the current frame are added by pixel blender  1207  to produce updated baseline frame  503 . In practice, weights are typically generated from a user-specified parameter. Current frame weights may range from 0 to 100%, while the baseline frame weight is typically 80-95%. This weighting results in a blending of current frame  201  with previous baseline frame  503 . 
         [0049]    Weight function  1204  is dependent upon the particular application that the sensor is used for. For example, in security monitoring applications, the sensor is normally quiet with occasional perturbations that need to be reported. In patient monitoring applications, the sensor is normally not quiet but rather has to monitor a repeating signal, such as from a person&#39;s respiration. 
         [0050]    A variety of baseline weighting functions are possible and different functions may be preferred in different circumstances. Multiple baseline weighting functions  1204  can be compiled and used. Each baseline weight function may be tested in a priority order and the result of the first baseline weight function that returns a non-zero value is selected for use. 
         [0051]    Frame weighting function  1204  may monitor adjacent frame difference  1102  over a sliding window of time to determine a representative adjacent frame difference that indicates that the sensor is not currently excited by external stimuli. This identifies periods of time when the sensor is quiet, such as when an intrusion alarm is active but there is no activity. During quiet periods, adjacent frame difference  1102  is lower than the representative adjacent frame difference and the current frame is heavily weighted in the baseline image. When the sensor is not quiet, adjacent frame difference  1102  is greater than the representative internal reading and the current frame is given no weight in the baseline image. 
         [0052]    Another possible baseline weighting function  1204  monitors over baseline value  506  and keeps track of large minima in the reading over time. Frames corresponding to these minima are heavily weighted in the baseline image and other frames are not. This tends to add frames at the bottom of a repeating signal to the baseline image and produces an optimal baseline when the sensor is actively monitoring a signal such as respiration. 
         [0053]      FIG. 13  is a graph of a processed output and a raw adjacent frame difference over time. Curve  1302  shows the adjacent frame difference signal, which has small variations since frame-to-frame changes tend to be relatively small, especially for higher sampling frequencies. The small amplitude of signals in curve  1302  also produces a small signal-to-noise ratio. 
         [0054]    Curve  1301  shows a processed output signal that is generated by comparing each frame to a baseline frame. The baseline frame may be updated as needed. Since the differences in the speckle pattern are large for a current frame that is a relatively long distance in time from the baseline frame, a large amplitude signal is generated. This large signal has a better signal-to-noise ratio. Periodic variations due to real monitored behavior, such as breathing or vibrations of a security fence due to wind are visible in curve  1301 . Further post-processing, such as by a digital-signal processor (DSP) may be performed. For example, a Fast Fourier Transform (FFT) may be used to extract the breathing rate from the periodic peaks in curve  1301 . 
       Alternate Embodiments 
       [0055]    Several other embodiments are contemplated by the inventor. For example the output from the image sensor could be an array of pixel values of various pixel formats such as intensity or color. While each frame has been described as being compared to the baseline frame, only a subset of frames could be compared, such as every other frame, or every third frame, etc. 
         [0056]    Over baseline processing and baseline processing can occur on a subset of the pixels produced by the image sensor. Multiple fibers can point at different zones in the image sensor and each zone could be processed separately. 
         [0057]    The adjacent frame difference could run a slower frame rate than the over-baseline processing. At high frame rates the adjacent frame difference gets very small. Baseline update might per performed at a lower frame rate than the over-baseline processing. 
         [0058]    The processed output could be further post-processed, such as by a FFT, a Discrete Fourier Transform (DFT), or a wavelet transform to determine the rate of periodic signals. This can be used to determine heartbeat or breath rate in patient monitoring, or to determine frequency of vibration. 
         [0059]    The image sensor can be a CMOS image sensor, a CCD sensor or any other pixel-based image sensor. The light source may support multiple frequencies. For example, a combination of red, green and blue and/or Infrared lasers can be used as the light source. 
         [0060]    The particular image processed might not be the current image. For example, a baseline update might be against the current frame and the processed output against a previous frame. Registers may be added for pipelining or delaying operations. 
         [0061]    There are a large number of possible variations of weight function  1204 . Functions and processes may be performed by programming a general-purpose computer, or by dedicated hardware functions, firmware, or various combinations. The weight function can depend upon other factors such as a FFT or DFT transform of the processed output. 
         [0062]    While a sum-of-the-absolute difference (SAD) function has been described for comparing pixels, other compare functions could be used. Frames could be histograms of the number of pixels with a particular value. Pixels could be grouped (e.g. a ‘pixel’ could be a 4×4 patch of pixels from the image sensor. 
         [0063]    The background of the invention section may contain background information about the problem or environment of the invention rather than describe prior art by others. Thus inclusion of material in the background section is not an admission of prior art by the Applicant. 
         [0064]    Any methods or processes described herein are machine-implemented or computer-implemented and are intended to be performed by machine, computer, or other device and are not intended to be performed solely by humans without such machine assistance. Tangible results generated may include reports or other machine-generated displays on display devices such as computer monitors, projection devices, audio-generating devices, and related media devices, and may include hardcopy printouts that are also machine-generated. Computer control of other machines is another tangible result. Patient monitors, automatic generation of patient records, automatic alarms that are triggered when breathing or heart beat stops or is irregular (e.g. baby monitor) are other examples of tangible results. 
         [0065]    Any advantages and benefits described may not apply to all embodiments of the invention. When the word “means” is recited in a claim element, Applicant intends for the claim element to fall under 35 USC Sect. 112, paragraph 6. Often a label of one or more words precedes the word “means”. The word or words preceding the word “means” is a label intended to ease referencing of claim elements and is not intended to convey a structural limitation. Such means-plus-function claims are intended to cover not only the structures described herein for performing the function and their structural equivalents, but also equivalent structures. For example, although a nail and a screw have different structures, they are equivalent structures since they both perform the function of fastening. Claims that do not use the word “means” are not intended to fall under 35 USC Sect. 112, paragraph 6. Signals are typically electronic signals, but may be optical signals such as can be carried over a fiber optic line. 
         [0066]    The foregoing description of the embodiments of the invention has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. It is intended that the scope of the invention be limited not by this detailed description, but rather by the claims appended hereto.