Conveyance system video analytics

According to an aspect, a method includes capturing image data from a video camera at a conveyance system. Analytics of the image data can be initiated to determine a plurality of conditions of the conveyance system. A status of the conditions can be summarized as a metadata output. The metadata output can be transmitted to a support system operable to initiate a corrective action responsive to the status of the conditions.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of the IN Application No. 201811042193 filed Nov. 9, 2018, which is incorporated herein by reference in its entirety.

BACKGROUND

The embodiments herein relate to conveyance system operation and more particularly to conveyance system video analytics.

Conveyance systems, such as elevator systems, may be linked to video surveillance systems that stream video from one or more cameras from a location local to the conveyance system to a centralized surveillance station. Surveillance system operators may monitor the video feeds to determine whether abnormal conditions are present at one or more conveyance systems. In structures with multiple conveyance systems operating in parallel, the video feeds can consume a large amount of bandwidth and/or may require multiple dedicated video links. Further, it can be challenging for human observers to note more subtle changes in conditions of the conveyance systems.

SUMMARY

According to an embodiment, a method includes capturing image data from a video camera at a conveyance system. Analytics of the image data are initiated to determine a plurality of conditions of the conveyance system. A status of the conditions is summarized as a metadata output. The metadata output is transmitted to a support system operable to initiate a corrective action responsive to the status of the conditions.

In addition to one or more of the features described herein, or as an alternative, further embodiments can include where the conditions include a luminescence level of the conveyance system.

In addition to one or more of the features described herein, or as an alternative, further embodiments can include where the conditions include a status of or damage to one or more components of the conveyance system.

In addition to one or more of the features described herein, or as an alternative, further embodiments can include where the conditions include an operational status of a control operating panel of the conveyance system.

In addition to one or more of the features described herein, or as an alternative, further embodiments can include where the conditions include a state of occupancy of the conveyance system.

In addition to one or more of the features described herein, or as an alternative, further embodiments can include where the conveyance system includes a passenger enclosure, and the conditions include one or more of: door operation of the passenger enclosure and a door cycle count of the passenger enclosure.

In addition to one or more of the features described herein, or as an alternative, further embodiments can include where the conditions include entrapment of one or more occupants within the passenger enclosure.

In addition to one or more of the features described herein, or as an alternative, further embodiments can include where the conditions include vandalism, and the method includes outputting a suspected vandalism notification with an image of a suspected vandal based on the image data.

In addition to one or more of the features described herein, or as an alternative, further embodiments can include applying machine learning to identify a plurality of scenarios and using a plurality of feature images to establish one or more benchmarks.

In addition to one or more of the features described herein, or as an alternative, further embodiments can include adapting the image data for variations in arrangement of the conveyance system and lighting.

According to an embodiment, a system includes a video camera and a monitoring system operably coupled to the video camera. The monitoring system is configured to perform a plurality of operations including capturing image data from the video camera at a conveyance system and initiating analytics of the image data to determine a plurality of conditions of the conveyance system. The monitoring system is further configured to summarize a status of the conditions as a metadata output and transmit the metadata output to a support system operable to initiate a corrective action responsive to the status of the conditions.

Technical effects of embodiments of the present disclosure include performing video analytics to determine one or more conditions within an elevator car.

DETAILED DESCRIPTION

The tension member107engages the machine111, which is part of an overhead structure of the elevator system101. The machine111is configured to control movement between the elevator car103and the counterweight105. The position reference system113may be mounted on a fixed part at the top of the elevator hoistway117, such as on a support or guide rail, and may be configured to provide position signals related to a position of the elevator car103within the elevator hoistway117. In other embodiments, the position reference system113may be directly mounted to a moving component of the machine111, or may be located in other positions and/or configurations as known in the art. The position reference system113can be any device or mechanism for monitoring a position of an elevator car and/or counter weight, as known in the art. For example, without limitation, the position reference system113can be an encoder, sensor, or other system and can include velocity sensing, absolute position sensing, etc., as will be appreciated by those of skill in the art.

The controller115is located, as shown, in a controller room121of the elevator hoistway117and is configured to control the operation of the elevator system101, and particularly the elevator car103. For example, the controller115may provide drive signals to the machine111to control the acceleration, deceleration, leveling, stopping, etc. of the elevator car103. The controller115may also be configured to receive position signals from the position reference system113or any other desired position reference device. When moving up or down within the elevator hoistway117along guide rail109, the elevator car103may stop at one or more landings125as controlled by the controller115. Although shown in a controller room121, those of skill in the art will appreciate that the controller115can be located and/or configured in other locations or positions within the elevator system101. In one embodiment, the controller may be located remotely or in the cloud.

The machine111may include a motor or similar driving mechanism. In accordance with embodiments of the disclosure, the machine111is configured to include an electrically driven motor. The power supply for the motor may be any power source, including a power grid, which, in combination with other components, is supplied to the motor. The machine111may include a traction sheave that imparts force to tension member107to move the elevator car103within elevator hoistway117.

Although shown and described with a roping system including tension member107, elevator systems that employ other methods and mechanisms of moving an elevator car within an elevator hoistway may employ embodiments of the present disclosure. For example, embodiments may be employed in ropeless elevator systems using a linear motor to impart motion to an elevator car. Embodiments may also be employed in ropeless elevator systems using a hydraulic lift to impart motion to an elevator car.FIG. 1is merely a non-limiting example presented for illustrative and explanatory purposes.

In other embodiments, the system comprises a conveyance system that moves passengers between floors and/or along a single floor. Such conveyance systems may include escalators, people movers, etc. Accordingly, embodiments described herein are not limited to elevator systems, such as that shown inFIG. 1. Thus, the elevator system101may also be referred to as conveyance system101.

FIG. 2depicts an example of a system200according to an embodiment. The system200includes a passenger enclosure202, which may be the elevator car103ofFIG. 1. The system200also includes a monitoring system204operably coupled to a video camera206, where the video camera206is configured to capture image data at a conveyance system, such as conveyance system101ofFIG. 1. In the example ofFIG. 2, there is a single instance of the video camera206within the passenger enclosure202. In alternate embodiments, there may be multiple instances of the video camera206, for instance, to capture multiple angles within the passenger enclosure202, to perform depth measurements (e.g., in a stereoscopic configuration), and/or to observe for interior and exterior conditions relative to the passenger enclosure202. Embodiments with a single instance of the video camera206may be beneficial for applications (e.g., retrofit applications) where a lower complexity solution is sufficient to capture images of conditions of interest. Rather than integrating a large suite of sensors to determine various conditions within the passenger enclosure202, the monitoring system204can be configured to capture image data through a camera interface208and initiate analytics of the image data to determine a plurality of conditions of the conveyance system101. AlthoughFIG. 2depicts the video camera206at an angle, the location and orientation of the video camera206can vary depending upon camera type, installation geometry, and types of events or conditions to be captured. For instance, the video camera206may be mounted in a central location in the ceiling of the elevator car103, looking downwards. The monitoring system204can also include a processing system210, a memory system212, and a communication interface214, as well as other subsystems (not depicted). In some embodiments, the processing system210is configured to capture image data and perform on-board image processing to analyze the content of the image data. In other embodiments, the processing system210provides captured image data through the communication interface214for off-board image processing, such as image processing performed at the controller115ofFIG. 1, or another location, such as cloud-based image processing through a network218and/or other computing resources.

The processing system210may be but is not limited to a single-processor or multi-processor system of any of a wide array of possible architectures, including field programmable gate array (FPGA), central processing unit (CPU), application specific integrated circuits (ASIC), digital signal processor (DSP) or graphics processing unit (GPU) hardware arranged homogenously or heterogeneously. The memory system212may be a storage device such as, for example, a random access memory (RAM), read only memory (ROM), or other electronic, optical, magnetic or any other computer readable storage medium. The memory system212can include computer-executable instructions that, when executed by the processing system210, cause the processing system210to perform operations as further described herein.

The communication interface214can include wired, wireless, and/or optical communication links to establish communication with one or more support systems216either directly or through the network218. Examples of the support systems216can include a mobile device220or any type of computer system222, such as a personal computer, a workstation, a laptop computer, a tablet computer, wearable computer, or a custom-built computer system, and/or the controller115ofFIG. 1. The mobile device220may have a service interface for a mechanic or technician to further diagnose data associated with one or more conditions reported by the monitoring system204. The computer system222may be a central control computer that monitors the functionality of multiple instances of the conveyance system101, such as multiple elevators in the same building or structure. The computer system222may also or alternatively be part of a security system that monitors for safety conditions and/or potential damage to the passenger enclosure202. The computer system222may also be part of an elevator service system to monitor and control conditions pertaining to the elevator system101. The network218can also support cloud-based operations and processing to directly support or partially offload processing burdens of the processing system210.

In some embodiments, the support systems216can control one or more aspects of the passenger enclosure202as part of a corrective action responsive to the status of the conditions as reported by the monitoring system204. For example, if a luminescence level of one or more light fixtures224of the passenger enclosure202is reported below a minimum lighting threshold, the support systems216may disable operation of the passenger enclosure202until the light fixtures224can be repaired or otherwise serviced. The monitoring system204may observe an average luminescence level in the image data captured by the video camera206and monitor for changes over time. Where the interior of the passenger enclosure202is configured to receive exterior lighting, e.g., through windows, the time-of-day and/or external weather conditions may be considered in making health determinations with respect to the light fixtures224. Further, an opened/closed state of one or more doors226of the passenger enclosure202may also be considered in determining the luminescence level.

The embedded video analytics of the monitoring system204can monitor for various observable conditions of the passenger enclosure202. For instance, the monitoring system204can detect features such as a floor228, walls230, ceiling232, rails234, and a control operating panel236. By observing for changes occurring over time, the accumulation of dirt, debris, or damage may be detected through the image data. The image processing of the monitoring system204can include applying machine learning to identify a plurality of scenarios and using a plurality of feature images to establish one or more benchmarks. The image processing can also include adapting the image data for variations in arrangement of the conveyance system and lighting. For example, the monitoring system204may perform initial training by accessing a library of feature data locally within the memory system212or remotely over the network218to learn relative positions, sizing, color, illumination levels, and other features that define the light fixtures224, doors226, floor228, walls230, ceiling232, rails234, control operating panel236, and the like. Algorithms such as edge detectors, classifiers, and known machine learning techniques (e.g., linear regression, nearest neighbors, support vector machines, neural networks, and the like) can be implemented locally at the monitoring system204to establish benchmarks and observe variations from the benchmarks. Thus, different configurations of the passenger enclosure202and changes over time can be detected. The monitoring system204can distinguish, for instance, between the accumulation of dirt or debris that accumulates over time on one or more surfaces of the passenger enclosure202and the hanging of a sign or picture within the passenger enclosure202. Further, by distinguishing between various surfaces within the passenger enclosure202, the existence of a condition in need of a corrective action response can be determined. For instance, a change in shape, linearity, angular deflection, or other aspects of the rails234can be indicative of damage to the rails234that result in a service call. Some conditions may include a combination or time-based sequence to be established. For instance, a movement pattern of the doors226may include observing a sequence of multiple frames of image data to verify proper operation in terms of complete opening/closing, rate of travel, and the like. The algorithm can also be trained to count door open/door close cycles leading to improved door service. Illumination of or damage to the control operating panel236may also take multiple frames of image data to confirm.

Further, the monitoring system204can observe occupancy and activity of occupants within the passenger enclosure202. For instance, the monitoring system204can use known passenger counting techniques to track a number of occupants entering and exiting the passenger enclosure202. Occupant entrapment may be detected where the one or more occupants remain within the passenger enclosure202and the doors226do not open after a predetermined timeout period. Other approaches to tracking and entrapment detection are contemplated. Further, the monitoring system204may support real-time detection of vandalism within the passenger enclosure202by one or more occupants. Upon detecting at least one occupant and a change in one or more surface features, such as dents, scratches, paint, holes, broken buttons or ill-functioned lights around the buttons, and the like, the monitoring system204can incorporate a potential vandalism condition message in a notification to the one or more support systems216. Images of conditions and/or of a suspected vandal can be captured in image data and reported.

Referring now toFIG. 3with continued reference toFIGS. 1-2,FIG. 3depicts a flow chart of a method300in accordance with an embodiment of the disclosure. The method300can be performed, for example, by the monitoring system204ofFIG. 2.

At block302, the monitoring system204captures image data from a video camera206at a conveyance system101. As previously described, the video camera206can be mounted within the passenger enclosure202. The monitoring system204can be local to the passenger enclosure202and may travel with the passenger enclosure202.

At block304, the monitoring system204can initiate analytics of the image data to determine a plurality of conditions of the conveyance system101. Examples of conditions with respect to an elevator are further described with respect toFIG. 4.

At block306, the monitoring system204can summarize a status of the conditions as a metadata output. Rather than storing all of the image data, the metadata output can summarize observed conditions from the image data. Further, image data may be temporarily buffered in the memory system212, with clips or sequences captured and retained around events or conditions of interest. This can reduce the storage requirements of the memory system212and the communication bandwidth requirements of the network218.

At block308, the monitoring system204can transmit the metadata output to a support system216operable to initiate a corrective action responsive to the status of the conditions. Transmission of data and/or metadata output can include sending data to the controller115to perform one or more corrective actions and/or further analysis. Corrective actions can include disabling or removing the passenger enclosure202from service, changing scheduling of other conveyance systems101, sending the elevator car103to a certain floor, stopping movement of the elevator car103, triggering a security alert, initiating a maintenance request, initiating communication with occupants of the passenger enclosure202, and other such actions.

While the above description has described the flow process ofFIG. 3in a particular order, it should be appreciated that unless otherwise specifically required in the attached claims that the ordering of the steps may be varied.

Referring now toFIG. 4with continued reference toFIGS. 1-3,FIG. 4depicts a flow chart of a method400of condition checks of the conveyance system101, where implemented as elevator system101in accordance with an embodiment of the disclosure. The method400can be performed, for example, by monitoring system204(e.g., processing system210) ofFIG. 2. The sequence depicted inFIG. 4is not intended to convey a particular sequence of steps, which may be re-ordered, with other steps added, omitted, or combined. For instance, blocks depicted of the method400ofFIG. 4may be executed based on various scheduling, conditions, and/or events, such as when the elevator car103is stopped, timing conditions are met, etc.

At block402, the monitoring system204can check a luminescence level of the conveyance system101, for instance, based on image brightness data of image data from the video camera206. As one example, in a Red-Green-Blue (RGB) color space, RGB pixels can be averaged across an image and masking may be used to block out selected features when determining a luminescence level. Luminescence level checks can include establishing reference levels, tracking light bulb/fixture aging, normalizing for outside lighting effects, coordinating with door opened/closed status, and the like. Training with reference images can be used to establish reference levels and initial conditions.

At block404, the monitoring system204can check door226operation of the passenger enclosure202. Image data can be analyzed over time to ensure that the doors226are not stuck and can open and close smoothly. Door operation checks can also capture image data while the doors226are open to inspect for dirt, debris, or obstructions that are not otherwise visible when the doors226are closed. Visual inspection of the doors226may be performed each time that the doors226cycle between opened and closed when not otherwise visually obstructed.

At block406, the monitoring system204can check for a status or visible damage to one or more components of the conveyance system101, such as floor228, walls230, ceiling232, rails234, and the control operating panel236of the passenger enclosure202using image data. The status can include state data, such as confirming a current opened/closed state of the doors226, inspecting for dirt, and/or various types of damage, such as scratches, dents, defacement, and the like.

At block408, the monitoring system204can check for occupancy of the conveyance system101, such as determining whether or how many people are observed within the passenger enclosure202based on the image data. The check for occupancy can also include checking a condition of occupancy, such as whether someone has fallen, whether occupants are fighting, vandalism is in progress, and/or other such conditions related to occupants. Image data associated with occupancy may be captured and output, for instance, to assist in identifying a suspected vandal. General occupancy may be determined by subtracting a sequence of image frames to identify motion within the elevator car103while the doors226are closed. If motion is detected, one or more classifiers can be applied to the image data to search for facial features and/or other features of interest.

At block410, the monitoring system204can check for entrapment of one or more occupants within the passenger enclosure202based on the image data. Entrapment can be detected, for instance, based on motion detected within the elevator car103and an extended period of time (e.g., a timeout period) without opening of the doors226. Other algorithms are contemplated.

At block412, the monitoring system204can check an operational status of the control operating panel236of the conveyance system101based on the image data. The operational status can include detection of button illumination, button responsiveness, button damage, and other such features.

While the above description has described the flow process ofFIG. 4in a particular order, it should be appreciated that unless otherwise specifically required in the attached claims that the ordering of the steps may be varied.

The term “about” is intended to include the degree of error associated with measurement of the particular quantity and/or manufacturing tolerances based upon the equipment available at the time of filing the application.