Patent Description:
The embodiments described herein generally relate to health monitoring and more particularly to performing automated defect detection for a wire rope using image processing techniques.

Cables and wire ropes are used across all types of industries and applications. For example, cables are used in elevators, hoists, cranes, etc. These applications include safety-critical applications such as human carrying rescue missions. These cables are required to undergo periodic inspections to determine if any defects exist. There are various types of defects that may occur for example, broken wires, broken strands, etc. due to fatigue, heaving loading, or other events. There may be a need to consistently and efficiently perform cable inspections. <CIT> describes a system for checking ropes. <CIT> describes the testing of a carrier element for a lift installation. <CIT> describes a device for detecting the replacement state of wear of a high-strength fibre cable. <CIT> relates to a system and method for determining belt wear using images. <CIT> relates to method for non-destructively estimating a current physical condition of a cordage product while in service.

According to an embodiment, a system for performing automated defect detection for a flexible member using image processing is provided according to claim <NUM>.

In addition to one or more of the features described herein, or as an alternative, further embodiments include one or more sensors that are optical fibers.

In addition to one or more of the features described herein, or as an alternative, further embodiments include a flexible member that is at least one of rope, wire, belt, and chain.

In addition to one or more of the features described herein, or as an alternative, further embodiments include using reference image data that is based on the flexible member.

In addition to one or more of the features described herein, or as an alternative, further embodiments include a processing module that is configured to determine a location of the defect of the flexible member.

In addition to one or more of the features described herein, or as an alternative, further embodiments include a notification that includes a location of the defect, a number of defects, a type of defect, or a recommendation for performing a service relating to the flexible member.

According to another embodiment, a method for performing automated defect detection for a flexible member using image processing is provided according to claim <NUM>.

In addition to one or more of the features described herein, or as an alternative, further embodiments include reference image data that is based on the flexible member.

In addition to one or more of the features described herein, or as an alternative, further embodiments include determining a location of the defect of the flexible member.

In addition to one or more of the features described herein, or as an alternative, further embodiments include notification including a location of the defect, a number of defects, a type of defect, or a recommendation for performing a service relating to the flexible member.

Technical effects of embodiments of the present disclosure include utilizing optical fibers, lens and image sensor and image processors to consistently and efficiently monitor the health of cables and wire ropes.

The foregoing features and elements may be combined in various combinations without departing from the scope of the claims.

In today's environment ropes, cables, and wires (generally referred to as ropes) are used for various applications. The application the ropes are used for, can determine the frequency at which they are to be inspected and the tolerances allowed for the ropes for replacement/repair. Frequent inspections may be required for safety-critical application such as every <NUM> cycles or <NUM> hours of operation, where non-safety-critical applications can be inspected at longer intervals.

Ropes are flexible members and can be subjected to sudden loads and external disturbances, where the loading of the rope exceeds the rated load of the rope. In addition, the ropes can be subjected to repeated loads that can eventually lead to defects. These defects can include kinking, bird-caging (strand issue caused by metal-metal contact), dirt, corrosion, and a host of other factors. The ropes are inspected by trained technicians and the health of the rope can vary based on the manual inspection by different service technicians. The service technician's skill and experience can inform their assessment of the rope's health which can lead to a range of subjectivity for the same rope and/or defect.

In addition, the rope inspection cannot occur during the operation. Therefore, the device including the rope and/or flexible member may suffer downtime due to the time it takes to remove the rope and send it off for inspection. The length of the rope can vary based on their application. For example, ropes that are being used for elevator applications can range in length adding additional time to the inspection procedure. Also, different types of ropes exist which can exhibit different types of defects and the service technician must be knowledgeable of each type of rope, their tolerances, and associated defects.

The techniques described herein provide for a non-intrusive based diagnostic method for rope health monitoring by using image processing techniques. In addition, the techniques provide for a prognostic analysis of the rope or other flexible member. The techniques described herein can use one or more sensors to collect data for the rope and monitor the health of a rope where the sensors can include fiber optic sensors.

The techniques described herein also provide for defect detection by using an image processing and analysis method. Multiple algorithms can be used to analyze the rope for defects and the algorithms can be selected to best suit the application. One or more techniques include a defect detection method that classifies the type of failure and also ranks severity for operational clearance based on the application of the rope. For example, safety-critical applications such as rescue missions may require tighter threshold tolerances than non-safety critical applications.

The techniques described herein also provide for consistent defect detection in a rope or other flexible member be determining the number of defects, types of defects, the location of defects in the rope, and other information related to the rope. In one or more embodiments, the system can be updated based on the collected field failure data to improve the overall detection efficiency and correlating the defects for the different types of rope to the rope health.

In <FIG>, a system <NUM> for performing automated defect detection method for a rope using image processing in accordance with one or more embodiments is provided.

The system <NUM> includes an image capture device <NUM> which is located within the control electronics of the wire rope handling device <NUM>. The wire rope handling device <NUM> includes a mount <NUM> and configured to monitor a flexible member <NUM> such as a rope, belt, wire, etc. The mount <NUM> can be positioned near a location where the flexible member <NUM> is dispensed from a device that controls it. In one or more embodiments, the image capture device <NUM> includes one or more sensors such as optical fibers that are configured to detect sensor data from the flexible member <NUM> by providing a light on the surface of the flexible member <NUM> and analyzing the light that is reflected off of the surface of the flexible member <NUM>. The sensors are positioned to detect the entire surface of the flexible member <NUM>. In this non-limiting example, there are three optical fibers (L1, L2, and L3) that are used to detect the sensor data from the flexible member.

The system <NUM> also includes an image conversion system <NUM> that is configured to receive the sensor data from the image capture device <NUM> and convert the sensor data to image data. In a non-limiting example, the sensor data is optical light data and the light information is converted to form an image of the flexible member <NUM>. <FIG> also depicts an image processor <NUM>, where the image processor <NUM> is configured to enhance the image data from the conversion system <NUM> for further processing. The image processor <NUM> is configured to increase the sharpness of the image data, remove blurring, removing distortion, crop the image to a region of interest, monochrome image, etc. to proper the image for further processing and analysis for defect detection.

The processing module <NUM> is configured to execute a variety of algorithms for processing the image data received from the image processor <NUM>. The processing module <NUM> is also configured to receive reference image data from a reference image data source <NUM> and threshold setting information from a threshold setting source <NUM>. The reference image data and the threshold settings information corresponding to the type of flexible member <NUM> that is being monitored by the system <NUM> which is used to compare the current flexible member <NUM> to a healthy or new flexible member. The reference image information can include information such as rope type, length, diameter, strand information, the pattern of rope/strands, etc. The threshold setting information can include information such as defect types, defect classes that are mapped to the remaining life or strength of the rope. It should be understood that other types of information can be included in the reference image and threshold setting information and the information can vary for each type of rope, as well as the application the rope is being used for. For example, the same rope that is used for two different applications can have different acceptable tolerances and threshold for operations.

Responsive to performing the processing and analysis of the current image of the flexible member <NUM> with the reference image and the threshold setting information, the processing module <NUM> can transmit the results to a display <NUM>.

In one or more embodiments, a display <NUM> can be included in the system <NUM> to display the results of the analysis. The display <NUM> can present graphical data, textual data, visual image of the rope, etc. The display <NUM> can also present notification information for an operator that indicates information such as the health of the rope including the severity of any defects that exist or a recommendation as to when the rope should be repaired/replaced.

Responsive to the results of the analysis, an alert can be transmitted to an operator or other user through the alert module <NUM>. The alerts can include information related to the type of defect, the severity of defects of the rope, number of defects, a recommendation as to repair and/or replacement, estimated time for repair, visual of defects, etc. The results of the analysis performed by the system <NUM> can be stored in a storage device <NUM>. Machine-learning can optimize the types of defects and the remaining life of the rope and recommendations provided for a defect of the rope.

In <FIG>, a view of an arrangement <NUM> of the mount <NUM> used in the system <NUM> in accordance with one or more embodiments is shown. The device <NUM> includes a mount <NUM> that is configured to house one or more sensors such as the optical fibers for monitoring the flexible member <NUM>. Each optical fiber includes an optical fiber port 202A, B, and C as shown in <FIG>. Each optical fiber port <NUM> is configured to transmit and receive optical signals L1, L2, and L3 and has a field of view <NUM> as shown in <FIG>.

In one or more embodiments, the optical fibers and ports <NUM> are spaced apart at <NUM> degrees to cover the entire surface of the rope <NUM>. In other embodiments, a different number of sensors, sensor types, spacing, etc. can be used to monitoring the health of the rope. For example, in the case a belt is being monitored by the system <NUM>, one or two sensor devices can be used to monitor the rope.

Now referring to <FIG>, an arrangement <NUM> of an internal view of the optical port <NUM> is shown. The optical port <NUM> is configured to be attached to the housing/mount <NUM> shown in <FIG>. The optical port <NUM> receives the optical cable <NUM> that is configured to exchange optical signal data to the image processor <NUM> shown in <FIG>. The optical port <NUM> also includes one or more filters <NUM> and lenses <NUM>. The optical port <NUM> includes a light source <NUM> that is configured to transmit a light to the rope <NUM> and the optical port <NUM> is configured to receive the reflected light data that characterize the surface of the rope <NUM> for further processes.

Now referring to <FIG>, a processing flow <NUM> for processing the image data in accordance with one or more embodiments is shown. As shown, at block <NUM> several inputs are received at the processing module <NUM> to perform the automated defect detection for the rope using image processing techniques. In one or more embodiments, the inputs received at block <NUM> include the set of data <NUM> including the captured image data, image file name, the location of the rope segment, and reference image data. In one or more embodiments, the system <NUM> is configured to receive rope segment information from the hoist or rope dispensing system indicates a location of the length of rope that has been dispensed. This information can include the size and number of turns of a drum that is used to dispense the rope. It should be understood that other techniques that can be used. It should be understood that other data can be received and used in the processing of image data. At block <NUM>, image data is converted to black and white for further processing.

A first marking technique 408A utilizes a Canny transform that is used to compare the edges of the rope to the reference image. For example, loose strand information can be obtained using the Canny transform processing. A second marking technique 408B includes a differentiation filter for image gradient/image Laplacian magnitude which is used to compare the current image to the reference image. Each frame can monitor a certain length of the rope and splits the frame into smaller frames/windows for analysis against the reference frame. A third marking technique 408C includes using block-based pixel color analysis to each segment of the rope. During processing, the image of the current rope can be provided as a matrix that is compared to the reference image that is also provided as a matrix. The two matrices can be compared to each other further compared to the tolerances that are allowed by the threshold settings for that particular rope. At block <NUM>, a voting for a defect image of the various marking methods is performed. For example, the marking technique compares the reference image with the actual (defect) image. In addition, the voting process allows for a threshold tolerance for various applications to be used to determine whether the rope is fit for operations or requires repair/replacement. It should be understood that the selection can be selected according to a configurable selection. The resulting defect image <NUM> can be provided to the operator in a notification and/or further analysis.

Now referring to <FIG>, a flowchart of a method <NUM> for performing automated defect detection for a flexible member using image processing techniques is provided. The method <NUM> begins at block <NUM> and proceeds to block <NUM> which provides for monitoring, by one or more sensors, a flexible member to obtain sensor data. The method <NUM> continues to block <NUM> and provides for converting the sensor data from the one or more sensors to image data. Block <NUM> provides for receiving reference image data and threshold setting information. At block <NUM>, the method <NUM> provides for comparing the image data with the reference image data. Block <NUM> provides for determining a defect based on the comparison and the threshold setting information. The method at block <NUM> provides for transmitting a notification based on the defect. The method <NUM> ends at block <NUM>.

The technical effects and benefits include real-time rope health monitoring providing prognostic and diagnostic solutions. Therefore there is an increased reliability of the system due to onboard diagnostics. The technical effect and benefits include a reduced time for manual inspections and increased consistency in reporting rope health. There is also an increase in personnel safety when using wire ropes used for human and cargo applications. The system can be made more intelligent with machine learning techniques that are used to analyze the field service data to accurately identify defects and correlate the defects to the remaining life of the rope.

Claim 1:
A system (<NUM>) for performing automated defect detection for a flexible member using image processing, the system comprising:
one or more sensors configured to monitor a flexible member (<NUM>);
an image processor configured to convert the sensor data from the one or more sensors to image data; and
a processing module (<NUM>) configured to:
receive reference image data, compare the image data with the reference image data;
determine a defect based on the comparison and a threshold setting information for the flexible member;
and characterized in that:
the threshold setting information is based on an application type for the flexible member,
wherein the threshold setting information comprises, for the same flexible member, different acceptable thresholds and different tolerances for operation based on the application type; and
the processing module is further configured to:
categorize the defect into one or more classes based at least in part on the defect, wherein the one or more classes indicates remaining life of the flexible member based on an application for the flexible member; and
transmit a notification based on the defect.