Drone first responder assistance

Methods, systems, and apparatus, including computer programs encoded on computer-storage media, for receiving a request to deploy an aerial drone to assist an emergency vehicle from a first location to a second location; determining a route to be traversed by the emergency vehicle from the first location to the second location based on the request; and deploying the aerial drone to traverse the route in advance of the emergency vehicle.

BACKGROUND

Many properties are equipped with monitoring systems that include sensors and connected system components.

SUMMARY

Techniques are described for drone assistance. According to one implementation, first responders may use drones to collect data in order to aid in an emergency response. One or more drones may also be used to help first responders navigate to a scene of an emergency or other situation.

In some implementations, one or more drones are used to search on a property and provide data or to provide guidance to first responders. For example, drones or other automated devices of a property can be used to track one or more elements of the property, including residents, such that, in an emergency situation, the locations of the one or more elements of the property, including residents, can be provided to first responders. This is advantageous especially in scenarios where search and rescue operations are being considered. Information regarding the location of residents at a property or whether or not residents are at a property, can inform subsequent activities by first responders.

In some implementations, one or more drones direct first responders to a resident within a property. For example, data related to the location of a particular resident within a property can be used by a drone to accompany a first responder into the property. The drone can help pinpoint and aid in the rescue of the particular resident.

In some implementations, one or more drones provide guidance or traffic assistance. For example, one or more drones can fly ahead of a police car to alert cars on the road that a police car is coming and to move to the side of the road. One or more drones can be used to affect traffic. For example, a drone can be used as a traffic signal or red-light, stopping one or more cars and allowing an unobstructed path for one or more first responders to a particular property related to an emergency situation.

In some implementations, one or more drones are used for reconnaissance. For example, elements like fire hydrants can be spotted by a drone to help aid in the response of firefighters. A drone can go ahead of firefighters and illuminate elements, or otherwise make them readily visible, in order to facilitate efficient response when the first responders arrive.

In some implementations, other items may be highlighted by a drone. For example, in addition to a fire hydrant, fire department connections (FDC) on buildings, emergency key access boxes, alternative entrances to a building, among others may be highlighted by a drone. In some cases, a drone may highlight the location of an item by directing light on the item, sending a communication to a user of the drone indicating the location of an item after finding the item, or otherwise indicating the location of an item to personnel. In some cases, a drone may use a global position system (GPS), property plan details, or pre-incident survey data, such as data collected before a current emergency scenario, that indicates where one or more items are on the property. A drone may then be used to confirm or direct emergency personnel to the location of the item or to directly interact with the item. For example, a drone may retrieve a key box and return it to emergency personnel to allow personnel to access the key. A drone may be used to open a fire hydrant or perform other tasks. In general, a drone may interact with any found item as personnel would if personnel were able to interact with it. In this way, a drone may enable interaction with items even if the item is compromised (e.g., within a threshold of a fire, gas leak, or other dangerous incident).

In some implementations, one or more drones are used for hazards and scene assessment. For example, a drone can use visual sensors or other forms of sensors to detect elements of an emergency situation. The drone can then alert first responders of the particular hazards or other details of the scene before the first responders arrive. Data collection and dissemination to first responders may continue even after first responders arrive especially in situations where the view from an aerial drone is desired.

In some implementations, one or more drones are used to set up and maintain an exclusion zone around an emergency situation or other alarm. For example, a chemical spill on a road way may have fumes and debris related to a car crash on the scene. The one or more drones on the scene can gather data and determine an appropriate exclusion zone, or recommend an appropriate exclusion zone to first responders and alert first responders if any unauthorized entity enters. The one or more drones may also directly enforce the exclusion zone with speakers, lights or other indicators of danger.

DETAILED DESCRIPTION

FIG.1is a diagram showing an example of a system100for drone first responder assistance. The system100includes a property101, a fire102, a police car103, a network104, a control unit106, a first traffic signal117, a first drone120, a second drone122, a first car135, a second car140, a third drone142, a fire hydrant147, and a fourth drone144.

The property101is on fire as shown by the fire102. First responders in the form of the police car103are called to the scene by monitoring systems, including fire detectors, located on the property101. In some cases, a call placed to a first responder can prompt similar first responder action.

The police car103navigates to the property101on public roadways with the first traffic signal117as well as other cars such as the first car135and the second car140. The first drone120and the second drone122in the system100are used to help the police car103navigate safely and efficiently to the property101.

The police car103sends a signal to the network104that is a form of a request for drone assistance. The network is connected to the control unit106that interprets and processes this request.

The network104may be configured to enable exchange of electronic communications between the control unit106and the one or more elements of the system100including the police car103, the first drone120, the second drone122, the third drone142, and the fourth drone144. The network116may include, for example, one or more of the Internet, Wide Area Networks (WANs), Local Area Networks (LANs), analog or digital wired and wireless telephone networks (e.g., a public switched telephone network (PSTN), Integrated Services Digital Network (ISDN), a cellular network, and Digital Subscriber Line (DSL)), radio, television, cable, satellite, or any other delivery or tunneling mechanism for carrying data.

The request from the police car103is received by the control unit106in a receive request component110of the control unit106. Elements of the receive request component110of the control unit106include preparing the data sent from the police car103to the control unit106for additional processing. For example, the data received from the police car103inFIG.1contains data indicating a request to clear the roadways in a path to the property101.

The control unit106processes the request in a process request component112of the control unit106. Elements of the process request component112include determining available elements of the system100(i.e., drones, connected components, monitoring system elements, etc.) and actions, based on available elements that, when performed, satisfy the received request. In the example ofFIG.1corresponding to the intersection with the first traffic signal117, the actions include clearing the roadway in front of the police car103and controlling traffic flow.

The control unit106delegates action in a delegate action component114by sending data over the network104to the first traffic signal117, first drone120, and the second drone122. The first traffic signal117is to perform the action of activating a green light for the police car103to move through the intersection. The first drone120is to perform the action of adding a traffic signal to the intersection in order to stop the first car135. The second drone122is to perform the action of alerting the second car140and indicating to the second car140that it should make way for the police car103.

The first traffic signal117receives a signal from the control unit106that includes information used to change the light activated in the first traffic signal117. The first traffic signal117changes from red to green allowing the police car103to move through the intersection.

In some implementations, the first drone120receives signals from local devices. For example, the first drone120can receive signals from an emergency vehicle as it approaches an intersection. The first drone120can, in response to receiving a signal from the emergency vehicle, perform relevant actions. In some cases, the signal includes instructions for the first drone120. For example, the first drone120can receive instructions from the emergency vehicle to stop all traffic and give right of way to the emergency vehicle.

In some implementations, the first drone120uses locally captured data to perform operations. For example, the first drone120can capture one or more images of a vehicle approaching an intersection or exiting an intersection. In response to detecting the vehicle approaching the intersection, the first drone120can change one or more lights to give right of way to the vehicle. In response to detecting the vehicle exiting the intersection, the first drone120can perform relevant actions based on characteristics of the vehicle exiting the intersection. For example, if the vehicle turns east, the first drone120can preemptively move to a next intersection to the east of a current location of the vehicle. In this way, the first drone120can perform subsequent actions at the next intersection. Similarly, the first drone120can detect the vehicle and send updates to one or more other drones or other autonomous vehicles to perform subsequent actions.

In some implementations, the second drone122transmits a preemptive signal based on an approaching vehicle. For example, the second drone122can send a signal to a traffic light to change preemptively based on a vehicle approaching an intersection. In some cases, the second drone122can travel in front of the vehicle to change the lights preemptively. In some cases, the second drone122can perform actions that a vehicle may not be able to perform. For example, the second drone122can change traffic lights at an intersection for a vehicle that does not have an ability to change the traffic lights at the intersection.

In some implementations, the first drone120performs relevant actions for unplanned events detected locally. For example, the first drone120may detect the police car103coming from the direction of the first car135. In this case, the first drone120can detect the police car103and, based on a known emergency location, change the signals for other cars at the intersection to give the police car103right of way through the intersection. In this case, the first drone could send a signal to the first traffic signal117to display a red light to stop traffic from interfering with the police car103.

The first drone120is equipped with a light that resembles a standard traffic signal of the first traffic signal117. By activating the light, the first drone120is able to stop the first car135, and any other car that may proceed on the same road as the first car135. This method may be especially advantageous in situations, likeFIG.1, where the area occupied by the first drone120does not have a traffic signal. This method may also be advantageous in situations where the traffic signals are not connected to a system or a user of the system100is not recognized by traffic signals and therefore cannot control them.

In some cases, the first drone120is equipped with another object that is used to control the traffic flow on a roadway. For example, the first drone120can use a single red light to function as the red in a traditional traffic signal. The first drone120can similarly be equipped with a standard looking stop sign that is shown to one or more cars to affect the traffic flow of the roadway. Any object that is used to control the traffic flow on a roadway can alternately stop a car from moving or allow a car to move or keep moving.

The second drone122performs the action of alerting the second car140and indicating to the second car140that it should make way for the police car103. The second drone122uses an onboard speaker to alert the second car140and its driver that the police car103is approaching. The second drone122also uses visual cues, including lights to show where the second car140should proceed to in order to make way for the police car103. In this case, the second car140makes way by pulling off to the side of the road allowing the police car103to pass.

In some implementations, the second drone122uses other methods or devices to alert or indicate to the second car140. For example, in the case of the second car140being an autonomous driving vehicle, the second drone122may send one or more electronic signals to the one or more computers controlling the second car140. The electronic signals may contain data that, when interpreted by the one or more computers controlling the second car140, result in the second car140pulling to the side of the road to allow the police car103to pass or similar motion related to making an unobstructed path for the police car103.

One or more of the first traffic signal117, the first drone120, or the second drone122communicate with the control unit106after receiving one or more assigned actions in a monitor action component123.

The monitor action component123includes receiving data from the first traffic signal117that the light has been changed from red to green. Depending on factors such as the speed of the police car103and the distance between the police car103and the first traffic signal117, the first traffic signal117controls the switch from red light to green light. The first traffic signal117also includes a recommended time interval to be displaying the green light before the police car103arrives. In general, time intervals can be used to make sure that the intersection is clear before a first responder vehicle enters an intersection.

The monitor action component123includes receiving data from the first drone120. The data includes visual images captured by one or more cameras on board the first drone120, the current light activation of the light element carried by the first drone120, and other data (i.e., current location, battery level, etc.). The visual images are sent to the control unit106to be processed. Based on visual analysis performed by the control unit106on the received visual images, the control unit106determines that a car, the first car135, is in view. By comparing a sequence of images, the control unit106determines that the first car135is not moving and has stopped. Furthermore, based on the current light activation of the light element carried by the first drone120and other data, the control unit106determines the first drone120successfully activated the red light indicator, the current battery level is nominal, and the location matches the location of the intersection corresponding to the first traffic signal117.

In some implementations, the first drone120processes visual images locally. A determination based on the locally processed images is sent to the control unit106as part of the monitor action component123.

The monitor action component123includes receiving data from the second drone122. The data includes visual images captured by one or more cameras on board the second drone122and other data (i.e., current location, battery level, etc.). The visual images are sent to the control unit106to be processed. Based on visual analysis performed by the control unit106on the received visual images, the control unit106determines that a car, the second car140, is in view. By comparing a sequence of images, the control unit106determines that the second car140has pulled to the side of the road. Furthermore, based on other data, the control unit106determines the current battery level of the second drone122is nominal, and the location matches the location of roadway currently in front of the police car103.

In some implementations, the second drone122processes visual images locally. A determination based on the locally processed images is sent to the control unit106as part of the monitor action component123. In the case where the second car140is autonomously driven, the second drone122provides data indicating a transaction of data between the second drone122and the one or more computers controlling autonomous driving of the second car140.

The control unit106provides feedback to the police car103in response to request of the police car103and the actions performed by one or more elements of the system100in a provide feedback component130. The feedback data includes information regarding the first traffic signal successfully activating the green signal allowing the police car103to move through the intersection, the first drone120successfully deploying an ad hoc traffic signal to successfully stop the first car135, and second drone122successfully instructing the second car140to the side of the road to make a way for the police car103.

In some implementations, the police car103submits the request for drone assistance a mile or more away from the intersection corresponding to the first traffic signal117. The control unit106uses the location of the police car103and the location of various other elements in the request in order to perform actions relevant to the request. In some cases, the police car103sends multiple requests to the control unit106and the control unit106handles each request to enable to the police car to navigate freely to the property101. For example, the police car103is called to another property. The path from the current location of the police car103and the other property has three intersections. The police car103can submit requests detailing drone assistance to the other property which results in help being deployed sequentially at each of the three intersections. In some cases, drones may be used for more than one use. For example, it may be advantageous to use the second drone122again, in another part of the road, to alert another car to move to the side of the road and make way for the police car103.

In some implementations, if multiple emergency vehicles are in the vicinity of one another, the one or more drones involved can control traffic or alert appropriate personnel to avoid any collisions or obstructions. For example, if the first car135inFIG.1was an emergency vehicle, the first drone120may be used to stop and then allow the first car135to proceed on its route ahead of any other normal cars. A hierarchy of emergency vehicles or situations may be used to determine which vehicle should yield to another vehicle in cases where a vehicle must yield to another or a particular route is to be used for one vehicle and not another.

The police car103successfully navigates to the property101. The police car103submits an additional request to the control unit106requesting drone assistance at the property101.

The additional request is sent over the network104. The control unit106receives the request in the receive request component110of the control unit106. Elements of the receive request component110of the control unit106include preparing the data sent from the police car103to the control unit106for additional processing. For example, the data received from the police car103inFIG.1contains data indicating a request to perform property reconnaissance on the property101, collect data from the property101and components connected on the property101, and perform hazard and scene assessment on the property101.

The control unit106processes the additional request in a process request component112of the control unit106. Elements of the process request component112include determining available elements of the system100(i.e., drones, connected components, monitoring system elements, etc.) and actions, based on available elements that, when performed, satisfy the received additional request. In the example ofFIG.1, the actions include performing property reconnaissance on the property101, collecting data from the property101and components connected on the property101, and performing hazard and scene assessment on the property101.

The control unit106delegates action in a delegate action component114by sending data over the network104to the third drone142, the property101, and the fourth drone144. The third drone142is to perform the action of reconnaissance on the property101. The property101is to perform, together with connected components including monitoring systems installed, the action of collecting data from the property101. The fourth drone144is to perform the action of hazard and scene assessment on the property101.

The third drone142receives a signal from the control unit106that includes information used to start reconnaissance on the property101. The information includes the fact that the property101is on fire and flames, including102and145, need to be put out. The information includes objects of importance that should be found such as possible sources of water that are nearby. The third drone142, using the information, performs reconnaissance on the property101and in the vicinity of the property101. The third drone142finds the nearest fire hydrant, the fire hydrant147, a water source that will be important in putting out the fire at the property101.

The property101receives a signal from the control unit106to send data related to the property101. In the example ofFIG.1, the property101uses additional elements, including internal drones, to investigate the property101and persons on, or the vicinity to, the property101.

In some implementations, the property101or computer systems interfacing with elements of the property101such as a monitoring system, control one or more drones pre-installed on the property101or one or more drones deployed by one or more first responders. For example, the property101may have drones stationed and/or first responders indicated by the police car103may deploy additional drones.

In some implementations, drones from nearby properties may be used in emergency situations by first responders or other users. For example, if a forest fire broke out near the property101and if one or more drones were stationed at the property101, the drones stationed at the property101could be used to help fight the forest fire that broke out near the property101. The one or more drones may be used, in some cases, when the fire is not directly on the property101.

In some implementations, one or more drones at the property101or within the property101, communicate with the control unit106. The one or more drones gather information from the control unit106related to the alarm condition and the current or last-known locations of one or more persons in or at the property101. The one or more drones search for missing people using data about the property (i.e., three dimensional model representation, two dimensional models, vector maps, etc.), live and recorded data from one or more sensors in the monitoring system of the property101, and information about the occupants and their mobility. The one or more drones stream video and notifications to first responders, including responders associated with the police car103, or lead the first responders into the property101—either to the source of the alarm, such as a fire detector in the example ofFIG.1, or the likely locations of the one or more persons at the property.

In some implementations, one or more drones lead first responders to specific elements related to the alarm or request of a first responder. For example, one or more drones on the property101may lead firefighters on the most direct route to the alarming smoke detector or to the inferred location of a resident based on motion sensors or the topology of the home.

The fourth drone144receives a signal from the control unit106that includes information used to perform the action of hazard and scene assessment on the property101. The fourth drone144uses onboard sensors including cameras to locate the property101as well as fire102and145. By capturing multiple elements of the alarm condition and the situation at the property101, the fourth drone144establishes an exclusion zone146. Aspects of the exclusion zone146are informed by the control unit106. For example, the fourth drone144receives data related to the wind direction in the vicinity of the property101or uses onboard sensors to detect wind. In the case of the fire102and145inFIG.2, the direction of the wind can be used to include downwind section of lands in any exclusion zone as downwind sections of land have a greater potential of catching fire. The process of hazard and scene assessment is discussed also in reference toFIG.2.

One or more of the third drone142, the property101or connected systems, or the fourth drone144communicate with the control unit106after receiving one or more assigned actions in a monitor action component123.

The monitor action component123includes receiving data from the third drone142that includes one or more visual images. The one or more visual images are processed by the control unit106to determine that the third drone142has found the fire hydrant147. The third drone142receives a signal from the control unit106that the one or more visual images captured at a particular location represent the fire hydrant147. The third drone142uses illuminators on board to illuminate the location corresponding to the fire hydrant147. The illumination of the fire hydrant147is especially advantageous in situations of low visibility such as fog or at night.

In some implementations, the location of the fire hydrant147is known. For example, the location of the fire hydrant147can be associated with coordinates based on global positioning or other localizing system. Similarly, the fire hydrant147can be associated with a proximity to one or more other features on or near the property101. The third drone142can emphasize the location of the fire hydrant147. For example, the third drone142can detect low levels of light in the vicinity of the fire hydrant147. In response, the third drone142can activate lights to illuminate the fire hydrant147. In another example, the third drone142can detect that the fire hydrant147is visually obscured by surrounding elements. The third drone142can remove or move the obscuring elements. Alternatively, the third drone142can use the light or other equipped device to indicate a location of the fire hydrant147that may be visually obscured. In some cases, other autonomous devices or personnel at or near the property101can remove the visual obscurity.

In some implementations, the third drone142processes visual images locally. A determination based on the locally processed images is sent to the control unit106as part of the monitor action component123.

The monitor action component123includes receiving data from the property101or connected systems that includes data related to the property101and the alarm or situation shown inFIG.1. The control unit106receives information related to the area of the fire and whether or not any persons were at the property101at the time of the fire. The control unit106receives information from monitoring system elements and/or drones within the property101. The control unit106determines based on the received information that no persons were at the property101at the time of the fire.

The monitor action component123includes receiving data from the fourth drone144that includes one or more visual images. The one or more visual images are processed by the control unit106. The control unit106determines based on one or more elements in the one or more visual images (i.e., the fire102and145, the property101, the wind direction discussed above, etc.) that the exclusion zone146is appropriate for the situation. The fourth drone144receives a signal from the control unit106that a specific region related to the property101is to be the exclusion zone146. The fourth drone144uses elements on board such as speakers and lights to make sure that persons, things, and other possibly hazardous elements such as explosives or other dangerous materials, do not enter or are removed from the exclusion threshold. The fourth drone144communicates with the control unit106to maintain the exclusion threshold and to call for assistance if assistance is needed.

In some implementations, the fourth drone144processes visual images locally. A determination based on the locally processed images is sent to the control unit106as part of the monitor action component123.

The control unit106provides feedback to the police car103in response to the additional request of the police car103and the actions performed by one or more elements of the system100in a provide feedback component130. The feedback data includes information regarding the third drone142performing reconnaissance and successfully locating the fire hydrant147nearest to the property101, the property101performing, together with connected components including monitoring systems installed, the action of collecting data from the property101resulting in the determination that no persons are within the property101, and the fourth drone144performing the action of hazard and scene assessment on the property101resulting in the exclusion zone146determined by visual processing and other information (i.e., wind direction, etc.).

The first responders associated with the police car103and the additional request made to the system100, use the assistance of the drones142and144together with the connected systems monitoring the property101to aid in the response to the fire at the property101. Data related to the drone assistance as well as further assistance is made available to subsequent first responders such as firefighters. The drones142and144may be reassigned or maintain current roles. Specific additional data from the property101may be obtained from one or more connected systems related to the property101.

In some implementations, other forms of first responders are involved in the system100. For example,FIG.1shows the police car103that represents any first responder. Instead of the police car103, an ambulance, fire engine, private security firm, etc. can submit requests to the control unit106.

In some implementations, the request for assistance in navigating to the property101and assistance at the property101is a singular request and the control unit106organizes dispatching and timing of elements of the system100. For example, the police car103or another element of the system100sends a request to the control unit106for drone assistance related to a fire at the property101. The control unit106can use data such as the location of the property101, navigation routes of first responders to the property101, connected elements available at the property101, along the route to the property101, and in the vicinity of the property101to organize assistance.

In some implementations, drone assistance is automatically dispatched based on emergency need. For example, a car crash detected on a road way can automatically invoke one or more elements of the system100to assist first responders. The automatic dispatching of one or more elements can be accompanied by further direct requests made by users or first responders for specific actions to be taken by a drone or another element of the system100.

FIG.2is a diagram showing an example of a drone detecting a hazard.

FIG.2shows a system200that includes a drone202, a roadway204, a truck container206, a truck208, a spill210, initial fumes212, a toxic gas label214, secondary fumes216, an exclusion zone218, a first car230, and a second car232.

The drone202is dispatched to the area shown inFIG.2. The drone202is given information by a control unit similar to the control unit106ofFIG.1. The information includes a location of the crash involving the truck208and the truck container206on the roadway204. The drone202uses the location to navigate to the crash before first responders arrive. The drone202is assigned the action of establishing an exclusion zone.

The drone202captures one or more visual images of the crash scene. The drone202either processes the one or more visual images of the crash scene on board or sends the one or more images to an external entity to be processed, depending on implementation. The drone202determines that the crash involving the truck208has resulted in the spill210coming from the truck container206.

The drone202uses further visual processing to read the toxic gas label214on the truck container206. The toxic gas label214informs the drone202the likely nature of the spill210. The drone202detects the initial fumes212and determines based on the likely toxic nature of the spill210that the initial fumes212are toxic.

The drone202uses devices on board to detect other elements relevant to the situation. In the example shown inFIG.1, the drone202detects the initial fumes212and based on a calculated wind speed and direction determines the secondary fumes216. In some implementations, wind speed and direction can be determined by another element and sent to the drone202. For example, a control unit may have weather data of a region and pass relevant data to the drone202. In other implementations, the control unit receives the location of the initial fumes212and computes the location of the secondary fumes216. Data representing the location of the secondary fumes216is sent to the drone202. In other implementations, the drone202directly observes, with visual sensors, chemically sensitive sensors, or another type of sensor, a particular danger. For example, the drone202may use visual sensors to detect the secondary fumes216and direct locate them based on the visual data from the visual sensors.

The drone202, based on the elements of the crash including the truck208, the truck container206, and the spill210, the initial fumes212, the toxic gas label214, the secondary fumes216, and other computed or obtained information (i.e., wind speed direction/speed, etc.), determines the exclusion zone218.

In some implementations, the exclusion zone218is determined by a control unit associated with the drone202. For example, the drone202, or one or more other elements not shown, provides data to the control unit. The control unit determines the region for the exclusion zone218and sends information corresponding to the exclusion zone218region to the drone202.

In some implementations, the drone202automatically identifies hazardous materials and other dangers ahead of the first responders. For example, the drone202may automatically identify hazmat labels and placards, estimate the size of a container or spill, look up the material in National Institute for Occupational Safety and Health (NIOSH) or the Department of Transportation (DOT) guides. The drone202may then relay this information to first responders, along with recommended exclusion zones based on the observed wind patterns or weather.

In some implementations, the data obtained by the drone202is placed on a map or other interface. For example, data related to the crash and subsequent toxic gas spill210can be set on a generated map and be used for planning a strategy of containment by first responders.

The drone202sends information related to the crash of the truck208and the spill210to first responders. The first responders use this information to organize safety measures and arrange necessary clean-up.

The drone202monitors the exclusion zone218. The drone202observes the first car230coming into the exclusion zone218. Depending on implementation, the drone202either warns the encroaching party or alerts first responders. For example, the drone202can warn the first car230indicating that a toxic spill210is ahead and that the first car230must move back to beyond the threshold of the exclusion zone218. The drone202can use speakers, lights, or other elements on board to warn or otherwise indicate the situation and necessary actions to the first car230. For another example, the drone202can alert first responders that the first car230is within the exclusion threshold. In some cases, the first car230may have a role in the clean-up effort or investigation in which case, first responders or another entity receiving the alert can respond to the drone202that no action needs to be taken, that the first car230is allowed in the exclusion zone218. In some implementations, the drone202alerts both the first car230and the first responders.

In some implementations, the drone202navigates on the roadway204to perform actions subsequent to a hazard detection. For example, the drone202can move up the roadway204against the direction of traffic and close the lane in advance of the hazard or accident. In some cases, the drone202can perform actions automatically based on the detection of the hazard or can receive a signal from emergency personnel to perform relevant actions including closing lanes of oncoming traffic.

In some implementations, two or more drones work together. For example, the drone202can, together with one or more other drones, help to close a lane of traffic or create a perimeter. For example, the two or more drones can act as flares to direct traffic. The two or more drones can encircle an accident site and prevent unwanted or unauthorized access. Access attempts associated with unauthorized persons can prompt a warning action by the one or more drones, a preventative action, or signal sent to emergency personnel or person of authority.

In some implementations, the drone202provides an interface for the first car230to communicate with first responders. For example, the drone202can use speakers and microphones on board to allow the first car230to ask questions about the accident or provide alternate route information if a crash or other situation is blocking a pathway. The response to questions can either be automated with speech processing or addressed directly by a human.

The drone202also detects the second car232in the exclusion zone218. The drone202, depending on implementation can fly over and alert the second car232or first responders or, if the distance of the exclusion threshold is large, the drone202can communicate with other drones in the vicinity to check whether another drone is available to provide the alert to the second car232. In general, one or more elements of a system (i.e., drones, sensors, automated systems, etc.) can communicate with one another either directly or through a control unit to cover multiple aspects of a particular situation or emergency.

FIG.3is a flow diagram illustrating a process300for a drone assistance system. The process300is performed by one or more electronic devices or computers such as the system100ofFIG.1.

The process300includes receiving a request for drone assistance (304). For example, the police car103sends a request for traffic assistance to the control unit106. The control unit106receives the request.

In some implementations, the process300may include receiving a request to deploy an aerial drone to assist an emergency vehicle from a first location to a second location. For example, the second drone122can be deployed to travel in front of the police car103. In some cases, this may enable the second drone122to change the lights preemptively or prevent obstructions from blocking the route of the police car103.

In some implementations, an emergency vehicle may include any vehicle used in an emergency response situation, such as vehicles for law enforcement, firefighting, medical emergencies, or other emergencies. For example, an emergency vehicle may include a police car, a firetruck, an ambulance, a police bicycle, a motorcycle, a drone, a van, or any other form of transportation used, or controlled, by emergency personnel in responding to an emergency situation.

In some implementations, a request is automatically generated based on a given situation or emergency. For example, an armed monitoring system, upon detecting a robbery, can signal a request for drone assistance and emergency response. In some cases, emergency response automatically includes drone assistance. For example, in a situation where emergency first responders are required, drones may be deployed to assist.

The process300includes activating one or more drones based on the request (306). For example, the control unit106uses the delegate action component114to assign actions to the first traffic signal117to turn from red to green, to the first drone120to use on board traffic-signal-like elements to produce a stopping signal to stop the first car135, to the second drone122to indicate and otherwise warn the second car140that the police car103is coming and to move to the side of the road.

In some implementations, the process300may include determining a route to be traversed by the emergency vehicle from the first location to the second location based on the request. For example, the request for drone assistance sent by the police car103to the network104and the control unit106may include data indicating a request to clear a route to the property101from some initial starting point, such as a police department, other emergency headquarters, current location of the emergency vehicle, or expected location of the emergency vehicle at a time in the future.

In some implementations, the process300may include deploying the aerial drone to traverse the route in advance of the emergency vehicle. For example, if the request for drone assistance sent by the police car103to the network104and the control unit106includes data indicating a request to clear a route to the property101from some initial starting point, the control unit106may deploy one or more drones, such as the first drone120, the second drone122, the third drone142, or the fourth drone144, to traverse the route from the initial starting point to the property101in advance of the police car103.

In some implementations, deploying the aerial drone to traverse the route in advance of the emergency vehicle may include deploying the aerial drone to arrive at a third location along the route before the emergency vehicle. For example, the aerial drone, such as the first drone120or the second drone122, among others, may arrive at the intersection with the first traffic signal117before the police car103. After proceeding through the intersection, the police car103may be ahead of one or more deployed drones along the route but the deployed drones may proceed in advance of the police car103again to secure a subsequent intersection or aid in an emergency operation at the property101.

In some implementations, the process300may include deploying the aerial drone as a drone traffic signal, wherein the aerial drone is configured with one or more indicators for signaling one or more cars to stop. For example, as shown inFIG.1, the first drone120may use onboard traffic-signal-like elements to produce a stopping signal to stop the first car135.

In some implementations, the process300may include identifying a traffic signal of an intersection along the route; deploying the aerial drone to the intersection; and triggering the aerial drone to transmit one or more signals configured to change a state of the traffic signal. For example, after determining a route between an initial starting point and the property101, the control unit106may identify the intersection along the route where the first traffic signal117is directing traffic. The second drone122may send a signal to first traffic signal117to change a state of the first traffic signal117preemptively based on the police car103approaching the intersection. For example, the first traffic signal117may change from a red or stop state to a green or go state to give the police car103the right of way.

In some implementations, the one or more signals transmitted by the aerial drone may include one or more of: an infrared communication signal, an acoustic communication signal, or an electromagnetic signal.

In some implementations, the control unit106may send a signal to change a state of a traffic signal. For example, instead of a drone being deployed and triggered to transmit one or more signals configured to change a state of the traffic signal, the control unit106may send a signal directly to the first traffic signal117to perform the action of activating a green light for the police car103to move through the intersection.

In some implementations, a drone is used to perform an action. After the action is completed, the drone can then return to a previous action. For example, a drone may be monitoring a first house in a neighborhood when an emergency personnel requests the drone to circle another home that has caught fire. The drone can circle the other home to check for fire, smoke, or occupants. After circling the property, the drone can return to the first house or continue circling the property, depending on the request or system settings.

In some implementations, the drones to be activated are drones within a set distance from the particular incident. For example, all drones within a mile radius of the property101may be available to perform actions relevant to the fire102at the property101.

In some implementations, the process300may include receiving feedback from the one or more drones containing a status of the current action or data related to the current action (308). For example, the control unit106receives visual data from the drones120and122. The control unit106processes the visual data from the drones120and122to determine, in part, that the first car135and the second car140have stopped and moved to the side of the road, respectively.

In some implementations, based on determining the one or more signals configured to change the state of the traffic signal did not change the state of the traffic signal, the process300may include deploying the aerial drone as a drone traffic signal, wherein the aerial drone is configured with one or more indicators for signaling one or more cars to stop. For example, the control unit106may receive sensor data from the drones120and122. The control unit106processes the sensor data from the drones120and122to determine, in part, that the state of the first traffic signal117has not been changed from a red or stop state to a green or go state. Based on this determination, the control unit106can deploy one or more drones as a drone traffic signal at the intersection to either stop one or more cars or to clear a right of way for the emergency vehicle.

In some implementations, a drone may be deployed as a drone traffic signal. For example, the control unit106may identify that an intersection along the route to the property101does not have a functioning traffic signal. There may be no traffic signal or they may be a non-functioning traffic signal. The control unit106may deploy an aerial drone, such as the first drone120, as a drone traffic signal, wherein the aerial drone is configured with one or more indicators for signaling one or more cars to stop.

In some implementations, the one or more indicators include one or more of a colored light and a version of a road sign. For example, the first drone120may be equipped with a red light element that, when activated, illuminates red light similar to a red light on a traffic signal. In this way, the first drone120may stop one or more cars from entering an intersection. Similarly, the any of the drones in the system100may be equipped with another element to signal to a vehicle either stop or go instructions. For example, the first drone120may be equipped with a stop sign that may be revealed to cars in order to stop the car. The first drone120may be equipped with a sign indicating that a car proceed slowly. In general, any sign or indicator may be used in order to indicate a desired motion of one or more vehicles.

In some implementations, the process300may include providing feedback to the user based on the received status of the current action of data related to the current action (310). For example, the control unit106provides the police car103with information stating in some form that the first car135and the second car140are not obstructing the route to the property101.

In some implementations, the process300may include receiving sensor data from the aerial drone and based on processing the sensor data, determining that an intersection along the route is free from obstacles. For example, using one or more sensors, including a visual sensor, the second drone122may obtain one or more visual images of a section of the route to the property101, such as the intersection with the first traffic signal117. The second drone122may send data, including the visual images to the control unit106. The control unit106can determine that a car, the second car140has pulled to the side of the road. In some cases, the control unit106can compare a sequence of images to determine a change of position of the second car140and that the second car140has pulled to the side of the road.

In some implementations, the process300may include sending a signal to the emergency vehicle indicating that the intersection is free from obstacles. For example, the control unit106may provide the police car103with information indicating that the first car135and the second car140are not obstructing the route to the property101and the current route is free from obstacles. In this way, an emergency vehicle may traverse a route more efficiently as the status of a section of the route may be predetermined as clear allowing the emergency vehicle to travel at higher speeds with greater safety along the route to a target location, such as the property101.

In some implementations, the process300may include deploying one or more drones to the second location to aid in an emergency. For example, the third drone142may be deployed to the property101to aid in the emergency operations of the fire on the property101. In one instance, the third drone142may be deployed to identify and highlight key areas or objects on or near the property. For example, the third drone142may be deployed, in a fire emergency to identify and highlight, with onboard lights, the hydrant147to make it easier for emergency personnel to locate the hydrant147to be used in putting out the fire.

In some implementations, one or more drones may be used to communicate with one or more persons at a property. For example, a drone of the system100may be deployed at the property101to communicate with persons in a building of the property101. The communication may help to indicate an evacuation route or indicate where the emergency personnel will be entering or providing assistance. For example, a drone may use a speaker, screen, or other onboard equipment to communicate a message from emergency personnel to persons at the property101. The message may instruct the persons to go to a window to be rescued or proceed along another evacuation route depending on the situation or implementation.

FIG.4is a diagram illustrating an example of a property monitoring system. The network405is configured to enable exchange of electronic communications between devices connected to the network405. For example, the network405may be configured to enable exchange of electronic communications between the control unit410, the one or more user devices440and450, the monitoring server460, and the central alarm station server470. The network405may include, for example, one or more of the Internet, Wide Area Networks (WANs), Local Area Networks (LANs), analog or digital wired and wireless telephone networks (e.g., a public switched telephone network (PSTN), Integrated Services Digital Network (ISDN), a cellular network, and Digital Subscriber Line (DSL)), radio, television, cable, satellite, or any other delivery or tunneling mechanism for carrying data. The network405may include multiple networks or subnetworks, each of which may include, for example, a wired or wireless data pathway. The network405may include a circuit-switched network, a packet-switched data network, or any other network able to carry electronic communications (e.g., data or voice communications). For example, the network405may include networks based on the Internet protocol (IP), asynchronous transfer mode (ATM), the PSTN, packet-switched networks based on IP, X.25, or Frame Relay, or other comparable technologies and may support voice using, for example, VoIP, or other comparable protocols used for voice communications. The network405may include one or more networks that include wireless data channels and wireless voice channels. The network405may be a wireless network, a broadband network, or a combination of networks including a wireless network and a broadband network.

The control unit410includes a controller412and a network module414. The controller412is configured to control a control unit monitoring system (e.g., a control unit system) that includes the control unit410. In some examples, the controller412may include a processor or other control circuitry configured to execute instructions of a program that controls operation of a control unit system. In these examples, the controller412may be configured to receive input from sensors, flow meters, or other devices included in the control unit system and control operations of devices included in the household (e.g., speakers, lights, doors, etc.). For example, the controller412may be configured to control operation of the network module414included in the control unit410.

The network module414is a communication device configured to exchange communications over the network405. The network module414may be a wireless communication module configured to exchange wireless communications over the network405. For example, the network module414may be a wireless communication device configured to exchange communications over a wireless data channel and a wireless voice channel. In this example, the network module414may transmit alarm data over a wireless data channel and establish a two-way voice communication session over a wireless voice channel. The wireless communication device may include one or more of a LTE module, a GSM module, a radio modem, cellular transmission module, or any type of module configured to exchange communications in one of the following formats: LTE, GSM or GPRS, CDMA, EDGE or EGPRS, EV-DO or EVDO, UMTS, or IP.

The network module414also may be a wired communication module configured to exchange communications over the network405using a wired connection. For instance, the network module414may be a modem, a network interface card, or another type of network interface device. The network module414may be an Ethernet network card configured to enable the control unit410to communicate over a local area network and/or the Internet. The network module414also may be a voice band modem configured to enable the alarm panel to communicate over the telephone lines of Plain Old Telephone Systems (POTS).

The control unit system that includes the control unit410includes one or more sensors420. For example, the monitoring system may include multiple sensors420. The sensors420may include a lock sensor, a contact sensor, a motion sensor, or any other type of sensor included in a control unit system. The sensors420also may include an environmental sensor, such as a temperature sensor, a water sensor, a rain sensor, a wind sensor, a light sensor, a smoke detector, a carbon monoxide detector, an air quality sensor, etc. The sensors420further may include a health monitoring sensor, such as a prescription bottle sensor that monitors taking of prescriptions, a blood pressure sensor, a blood sugar sensor, a bed mat configured to sense presence of liquid (e.g., bodily fluids) on the bed mat, etc. In some examples, the health monitoring sensor can be a wearable sensor that attaches to a user in the home. The health monitoring sensor can collect various health data, including pulse, heart-rate, respiration rate, sugar or glucose level, bodily temperature, or motion data.

The sensors420can also include a radio-frequency identification (RFID) sensor that identifies a particular article that includes a pre-assigned RFID tag.

The system400also includes one or more thermal cameras430that communicate with the control unit410. The thermal camera430may be an IR camera or other type of thermal sensing device configured to capture thermal images of a scene. For instance, the thermal camera430may be configured to capture thermal images of an area within a building or home monitored by the control unit410. The thermal camera430may be configured to capture single, static thermal images of the area and also video thermal images of the area in which multiple thermal images of the area are captured at a relatively high frequency (e.g., thirty images per second). The thermal camera430may be controlled based on commands received from the control unit410. In some implementations, the thermal camera430can be an IR camera that captures thermal images by sensing radiated power in one or more IR spectral bands, including NIR, SWIR, MWIR, and/or LWIR spectral bands.

The thermal camera430may be triggered by several different types of techniques. For instance, a Passive Infra-Red (PIR) motion sensor may be built into the thermal camera430and used to trigger the thermal camera430to capture one or more thermal images when motion is detected. The thermal camera430also may include a microwave motion sensor built into the camera and used to trigger the thermal camera430to capture one or more thermal images when motion is detected. The thermal camera430may have a “normally open” or “normally closed” digital input that can trigger capture of one or more thermal images when external sensors (e.g., the sensors420, PIR, door/window, etc.) detect motion or other events. In some implementations, the thermal camera430receives a command to capture an image when external devices detect motion or another potential alarm event. The thermal camera430may receive the command from the controller412or directly from one of the sensors420.

In some examples, the thermal camera430triggers integrated or external illuminators (e.g., Infra-Red or other lights controlled by the property automation controls422, etc.) to improve image quality. An integrated or separate light sensor may be used to determine if illumination is desired and may result in increased image quality.

The thermal camera430may be programmed with any combination of time/day schedules, monitoring system status (e.g., “armed stay,” “armed away,” “unarmed”), or other variables to determine whether images should be captured or not when triggers occur. The thermal camera430may enter a low-power mode when not capturing images. In this case, the thermal camera430may wake periodically to check for inbound messages from the controller412. The thermal camera430may be powered by internal, replaceable batteries if located remotely from the control unit410. The thermal camera430may employ a small solar cell to recharge the battery when light is available. Alternatively, the thermal camera430may be powered by the controller's412power supply if the thermal camera430is co-located with the controller412.

In some implementations, the thermal camera430communicates directly with the monitoring server460over the Internet. In these implementations, thermal image data captured by the thermal camera430does not pass through the control unit410and the thermal camera430receives commands related to operation from the monitoring server460.

In some implementations, the system400includes one or more visible light cameras, which can operate similarly to the thermal camera430, but detect light energy in the visible wavelength spectral bands. The one or more visible light cameras can perform various operations and functions within the property monitoring system400. For example, the visible light cameras can capture images of one or more areas of the property, which the cameras, the control unit, and/or another computer system of the monitoring system400can process and analyze.

The system400also includes one or more property automation controls422that communicate with the control unit to perform monitoring. The property automation controls422are connected to one or more devices connected to the system400and enable automation of actions at the property. For instance, the property automation controls422may be connected to one or more lighting systems and may be configured to control operation of the one or more lighting systems. Also, the property automation controls422may be connected to one or more electronic locks at the property and may be configured to control operation of the one or more electronic locks (e.g., control Z-Wave locks using wireless communications in the Z-Wave protocol). Further, the property automation controls422may be connected to one or more appliances at the property and may be configured to control operation of the one or more appliances. The property automation controls422may include multiple modules that are each specific to the type of device being controlled in an automated manner. The property automation controls422may control the one or more devices based on commands received from the control unit410. For instance, the property automation controls422may interrupt power delivery to a particular outlet of the property or induce movement of a smart window shade of the property.

The system400also includes thermostat434to perform dynamic environmental control at the property. The thermostat434is configured to monitor temperature and/or energy consumption of an HVAC system associated with the thermostat434, and is further configured to provide control of environmental (e.g., temperature) settings. In some implementations, the thermostat434can additionally or alternatively receive data relating to activity at the property and/or environmental data at the home, e.g., at various locations indoors and outdoors at the property. The thermostat434can directly measure energy consumption of the HVAC system associated with the thermostat, or can estimate energy consumption of the HVAC system associated with the thermostat434, for example, based on detected usage of one or more components of the HVAC system associated with the thermostat434. The thermostat434can communicate temperature and/or energy monitoring information to or from the control unit410and can control the environmental (e.g., temperature) settings based on commands received from the control unit410.

In some implementations, the thermostat434is a dynamically programmable thermostat and can be integrated with the control unit410. For example, the dynamically programmable thermostat434can include the control unit410, e.g., as an internal component to the dynamically programmable thermostat434. In addition, the control unit410can be a gateway device that communicates with the dynamically programmable thermostat434. In some implementations, the thermostat434is controlled via one or more property automation controls422.

In some implementations, a module437is connected to one or more components of an HVAC system associated with the property, and is configured to control operation of the one or more components of the HVAC system. In some implementations, the module437is also configured to monitor energy consumption of the HVAC system components, for example, by directly measuring the energy consumption of the HVAC system components or by estimating the energy usage of the one or more HVAC system components based on detecting usage of components of the HVAC system. The module437can communicate energy monitoring information and the state of the HVAC system components to the thermostat434and can control the one or more components of the HVAC system based on commands received from the thermostat434.

In some examples, the system400further includes one or more robotic devices490. The robotic devices490may be any type of robot that are capable of moving and taking actions that assist in home monitoring. For example, the robotic devices490may include drones that are capable of moving throughout a property based on automated control technology and/or user input control provided by a user. In this example, the drones may be able to fly, roll, walk, or otherwise move about the property. The drones may include helicopter type devices (e.g., quad copters), rolling helicopter type devices (e.g., roller copter devices that can fly and/or roll along the ground, walls, or ceiling) and land vehicle type devices (e.g., automated cars that drive around a property). In some cases, the robotic devices490may be robotic devices490that are intended for other purposes and merely associated with the system400for use in appropriate circumstances. For instance, a robotic vacuum cleaner device may be associated with the monitoring system400as one of the robotic devices490and may be controlled to take action responsive to monitoring system events.

In some examples, the robotic devices490automatically navigate within a property. In these examples, the robotic devices490include sensors and control processors that guide movement of the robotic devices490within the property. For instance, the robotic devices490may navigate within the property using one or more cameras, one or more proximity sensors, one or more gyroscopes, one or more accelerometers, one or more magnetometers, a global positioning system (GPS) unit, an altimeter, one or more sonar or laser sensors, and/or any other types of sensors that aid in navigation about a space. The robotic devices490may include control processors that process output from the various sensors and control the robotic devices490to move along a path that reaches the desired destination and avoids obstacles. In this regard, the control processors detect walls or other obstacles in the property and guide movement of the robotic devices490in a manner that avoids the walls and other obstacles.

In addition, the robotic devices490may store data that describes attributes of the property. For instance, the robotic devices490may store a floorplan of a building on the property and/or a three-dimensional model of the property that enables the robotic devices490to navigate the property. During initial configuration, the robotic devices490may receive the data describing attributes of the property, determine a frame of reference to the data (e.g., a property or reference location in the property), and navigate the property based on the frame of reference and the data describing attributes of the property. Further, initial configuration of the robotic devices490also may include learning of one or more navigation patterns in which a user provides input to control the robotic devices490to perform a specific navigation action (e.g., fly to an upstairs bedroom and spin around while capturing video and then return to a home charging base). In this regard, the robotic devices490may learn and store the navigation patterns such that the robotic devices490may automatically repeat the specific navigation actions upon a later request.

In some examples, the robotic devices490may include data capture and recording devices. In these examples, the robotic devices490may include one or more cameras, one or more motion sensors, one or more microphones, one or more biometric data collection tools, one or more temperature sensors, one or more humidity sensors, one or more air flow sensors, and/or any other types of sensors that may be useful in capturing monitoring data related to the property and users at the property. The one or more biometric data collection tools may be configured to collect biometric samples of a person in the property with or without contact of the person. For instance, the biometric data collection tools may include a fingerprint scanner, a hair sample collection tool, a skin cell collection tool, and/or any other tool that allows the robotic devices490to take and store a biometric sample that can be used to identify the person (e.g., a biometric sample with DNA that can be used for DNA testing).

In some implementations, one or more of the thermal cameras430may be mounted on one or more of the robotic devices490.

In some implementations, the robotic devices490may include output devices. In these implementations, the robotic devices490may include one or more displays, one or more speakers, and/or any type of output devices that allow the robotic devices490to communicate information to a nearby user.

The robotic devices490also may include a communication module that enables the robotic devices490to communicate with the control unit410, each other, and/or other devices. The communication module may be a wireless communication module that allows the robotic devices490to communicate wirelessly. For instance, the communication module may be a Wi-Fi module that enables the robotic devices490to communicate over a local wireless network at the property. The communication module further may be a 900 MHz wireless communication module that enables the robotic devices490to communicate directly with the control unit410. Other types of short-range wireless communication protocols, such as Bluetooth, Bluetooth LE, Z-wave, Zigbee, etc., may be used to allow the robotic devices490to communicate with other devices in the property. In some implementations, the robotic devices490may communicate with each other or with other devices of the system400through the network405.

The robotic devices490further may include processor and storage capabilities. The robotic devices490may include any suitable processing devices that enable the robotic devices490to operate applications and perform the actions described throughout this disclosure. In addition, the robotic devices490may include solid state electronic storage that enables the robotic devices490to store applications, configuration data, collected sensor data, and/or any other type of information available to the robotic devices490.

The robotic devices490can be associated with one or more charging stations. The charging stations may be located at predefined home base or reference locations at the property. The robotic devices490may be configured to navigate to the charging stations after completion of tasks needed to be performed for the monitoring system400. For instance, after completion of a monitoring operation or upon instruction by the control unit410, the robotic devices490may be configured to automatically fly to and land on one of the charging stations. In this regard, the robotic devices490may automatically maintain a fully charged battery in a state in which the robotic devices490are ready for use by the monitoring system400.

The charging stations may be contact-based charging stations and/or wireless charging stations. For contact-based charging stations, the robotic devices490may have readily accessible points of contact that the robotic devices490are capable of positioning and mating with a corresponding contact on the charging station. For instance, a helicopter type robotic device490may have an electronic contact on a portion of its landing gear that rests on and mates with an electronic pad of a charging station when the helicopter type robotic device490lands on the charging station. The electronic contact on the robotic device490may include a cover that opens to expose the electronic contact when the robotic device490is charging and closes to cover and insulate the electronic contact when the robotic device is in operation.

For wireless charging stations, the robotic devices490may charge through a wireless exchange of power. In these cases, the robotic devices490need only locate themselves closely enough to the wireless charging stations for the wireless exchange of power to occur. In this regard, the positioning needed to land at a predefined home base or reference location in the property may be less precise than with a contact based charging station. Based on the robotic devices490landing at a wireless charging station, the wireless charging station outputs a wireless signal that the robotic devices490receive and convert to a power signal that charges a battery maintained on the robotic devices490.

In some implementations, each of the robotic devices490has a corresponding and assigned charging station such that the number of robotic devices490equals the number of charging stations. In these implementations, the robotic devices490always navigate to the specific charging station assigned to that robotic device. For instance, a first robotic device490may always use a first charging station and a second robotic device490may always use a second charging station.

In some examples, the robotic devices490may share charging stations. For instance, the robotic devices490may use one or more community charging stations that are capable of charging multiple robotic devices490. The community charging station may be configured to charge multiple robotic devices490in parallel. The community charging station may be configured to charge multiple robotic devices490in serial such that the multiple robotic devices490take turns charging and, when fully charged, return to a predefined home base or reference location in the property that is not associated with a charger. The number of community charging stations may be less than the number of robotic devices490.

Also, the charging stations may not be assigned to specific robotic devices490and may be capable of charging any of the robotic devices490. In this regard, the robotic devices490may use any suitable, unoccupied charging station when not in use. For instance, when one of the robotic devices490has completed an operation or is in need of battery charge, the control unit410references a stored table of the occupancy status of each charging station and instructs the robotic device490to navigate to the nearest charging station that is unoccupied.

The system400further includes one or more integrated security devices480. The one or more integrated security devices may include any type of device used to provide alerts based on received sensor data. For instance, the one or more control units410may provide one or more alerts to the one or more integrated security input/output devices480. Additionally, the one or more control units410may receive one or more sensor data from the sensors420and determine whether to provide an alert to the one or more integrated security input/output devices480.

The sensors420, the property automation controls422, the thermal camera430, the thermostat434, and the integrated security devices480may communicate with the controller412over communication links424,426,428,432, and484. The communication links424,426,428,432, and484may be a wired or wireless data pathway configured to transmit signals from the sensors420, the property automation controls422, the thermal camera430, the thermostat434, and the integrated security devices480to the controller412. The sensors420, the property automation controls422, the thermal camera430, the thermostat434, and the integrated security devices480may continuously transmit sensed values to the controller412, periodically transmit sensed values to the controller412, or transmit sensed values to the controller412in response to a change in a sensed value.

The communication links424,426,428,432, and484may include a local network. The sensors420, the property automation controls422, the thermal camera430, the thermostat434, and the integrated security devices480, and the controller412may exchange data and commands over the local network. The local network may include 802.11 “Wi-Fi” wireless Ethernet (e.g., using low-power Wi-Fi chipsets), Z-Wave, Zigbee, Bluetooth, “Homeplug” or other “Powerline” networks that operate over AC wiring, and a Category 4 (CATS) or Category 6 (CAT6) wired Ethernet network. The local network may be a mesh network constructed based on the devices connected to the mesh network.

The monitoring server460is one or more electronic devices configured to provide monitoring services by exchanging electronic communications with the control unit410, the one or more user devices440and450, and the central alarm station server470over the network405. For example, the monitoring server460may be configured to monitor events (e.g., alarm events) generated by the control unit410. In this example, the monitoring server460may exchange electronic communications with the network module414included in the control unit410to receive information regarding events (e.g., alerts) detected by the control unit410. The monitoring server460also may receive information regarding events (e.g., alerts) from the one or more user devices440and450.

In some examples, the monitoring server460may route alert data received from the network module414or the one or more user devices440and450to the central alarm station server470. For example, the monitoring server460may transmit the alert data to the central alarm station server470over the network405.

The monitoring server460may store sensor data, thermal image data, and other monitoring system data received from the monitoring system and perform analysis of the sensor data, thermal image data, and other monitoring system data received from the monitoring system. Based on the analysis, the monitoring server460may communicate with and control aspects of the control unit410or the one or more user devices440and450.

The monitoring server460may provide various monitoring services to the system400. For example, the monitoring server460may analyze the sensor, thermal image, and other data to determine an activity pattern of a resident of the property monitored by the system400. In some implementations, the monitoring server460may analyze the data for alarm conditions or may determine and perform actions at the property by issuing commands to one or more of the automation controls422, possibly through the control unit410.

The central alarm station server470is an electronic device configured to provide alarm monitoring service by exchanging communications with the control unit410, the one or more mobile devices440and450, and the monitoring server460over the network405. For example, the central alarm station server470may be configured to monitor alerting events generated by the control unit410. In this example, the central alarm station server470may exchange communications with the network module414included in the control unit410to receive information regarding alerting events detected by the control unit410. The central alarm station server470also may receive information regarding alerting events from the one or more mobile devices440and450and/or the monitoring server460.

The central alarm station server470is connected to multiple terminals472and474. The terminals472and474may be used by operators to process alerting events. For example, the central alarm station server470may route alerting data to the terminals472and474to enable an operator to process the alerting data. The terminals472and474may include general-purpose computers (e.g., desktop personal computers, workstations, or laptop computers) that are configured to receive alerting data from a server in the central alarm station server470and render a display of information based on the alerting data. For instance, the controller412may control the network module414to transmit, to the central alarm station server470, alerting data indicating that a sensor420detected motion from a motion sensor via the sensors420. The central alarm station server470may receive the alerting data and route the alerting data to the terminal472for processing by an operator associated with the terminal472. The terminal472may render a display to the operator that includes information associated with the alerting event (e.g., the lock sensor data, the motion sensor data, the contact sensor data, etc.) and the operator may handle the alerting event based on the displayed information.

In some implementations, the terminals472and474may be mobile devices or devices designed for a specific function. AlthoughFIG.4illustrates two terminals for brevity, actual implementations may include more (and, perhaps, many more) terminals.

The one or more authorized user devices440and450are devices that host and display user interfaces. For instance, the user device440is a mobile device that hosts or runs one or more native applications (e.g., the smart home application442). The user device440may be a cellular phone or a non-cellular locally networked device with a display. The user device440may include a cell phone, a smart phone, a tablet PC, a personal digital assistant (“PDA”), or any other portable device configured to communicate over a network and display information. For example, implementations may also include Blackberry-type devices (e.g., as provided by Research in Motion), electronic organizers, iPhone-type devices (e.g., as provided by Apple), iPod devices (e.g., as provided by Apple) or other portable music players, other communication devices, and handheld or portable electronic devices for gaming, communications, and/or data organization. The user device440may perform functions unrelated to the monitoring system, such as placing personal telephone calls, playing music, playing video, displaying pictures, browsing the Internet, maintaining an electronic calendar, etc.

The user device440includes a smart home application442. The smart home application442refers to a software/firmware program running on the corresponding mobile device that enables the user interface and features described throughout. The user device440may load or install the smart home application442based on data received over a network or data received from local media. The smart home application442runs on mobile devices platforms, such as iPhone, iPod touch, Blackberry, Google Android, Windows Mobile, etc. The smart home application442enables the user device440to receive and process image and sensor data from the monitoring system.

The user device450may be a general-purpose computer (e.g., a desktop personal computer, a workstation, or a laptop computer) that is configured to communicate with the monitoring server460and/or the control unit410over the network405. The user device450may be configured to display a smart home user interface452that is generated by the user device450or generated by the monitoring server460. For example, the user device450may be configured to display a user interface (e.g., a web page) provided by the monitoring server460that enables a user to perceive images captured by the thermal camera430and/or reports related to the monitoring system. AlthoughFIG.4illustrates two user devices for brevity, actual implementations may include more (and, perhaps, many more) or fewer user devices.

The smart home application442and the smart home user interface452can allow a user to interface with the property monitoring system400, for example, allowing the user to view monitoring system settings, adjust monitoring system parameters, customize monitoring system rules, and receive and view monitoring system messages.

In some implementations, the one or more user devices440and450communicate with and receive monitoring system data from the control unit410using the communication link438. For instance, the one or more user devices440and450may communicate with the control unit410using various local wireless protocols such as Wi-Fi, Bluetooth, Z-wave, Zigbee, HomePlug (ethernet over power line), or wired protocols such as Ethernet and USB, to connect the one or more user devices440and450to local security and automation equipment. The one or more user devices440and450may connect locally to the monitoring system and its sensors and other devices. The local connection may improve the speed of status and control communications because communicating through the network405with a remote server (e.g., the monitoring server460) may be significantly slower.

Although the one or more user devices440and450are shown as communicating with the control unit410, the one or more user devices440and450may communicate directly with the sensors420and other devices controlled by the control unit410. In some implementations, the one or more user devices440and450replace the control unit410and perform the functions of the control unit410for local monitoring and long range/offsite communication.

In other implementations, the one or more user devices440and450receive monitoring system data captured by the control unit410through the network405. The one or more user devices440,450may receive the data from the control unit410through the network405or the monitoring server460may relay data received from the control unit410to the one or more user devices440and450through the network405. In this regard, the monitoring server460may facilitate communication between the one or more user devices440and450and the monitoring system400.

In some implementations, the one or more user devices440and450may be configured to switch whether the one or more user devices440and450communicate with the control unit410directly (e.g., through link438) or through the monitoring server460(e.g., through network405) based on a location of the one or more user devices440and450. For instance, when the one or more user devices440and450are located close to the control unit410and in range to communicate directly with the control unit410, the one or more user devices440and450use direct communication. When the one or more user devices440and450are located far from the control unit410and not in range to communicate directly with the control unit410, the one or more user devices440and450use communication through the monitoring server460.

Although the one or more user devices440and450are shown as being connected to the network405, in some implementations, the one or more user devices440and450are not connected to the network405. In these implementations, the one or more user devices440and450communicate directly with one or more of the monitoring system components and no network (e.g., Internet) connection or reliance on remote servers is needed.

In some implementations, the one or more user devices440and450are used in conjunction with only local sensors and/or local devices in a house. In these implementations, the system400includes the one or more user devices440and450, the sensors420, the property automation controls422, the thermal camera430, and the robotic devices490. The one or more user devices440and450receive data directly from the sensors420, the property automation controls422, the thermal camera430, and the robotic devices490(i.e., the monitoring system components) and sends data directly to the monitoring system components. The one or more user devices440,450provide the appropriate interfaces/processing to provide visual surveillance and reporting.

In other implementations, the system400further includes network405and the sensors420, the property automation controls422, the thermal camera430, the thermostat434, and the robotic devices49are configured to communicate sensor and image data to the one or more user devices440and450over network405(e.g., the Internet, cellular network, etc.). In yet another implementation, the sensors420, the property automation controls422, the thermal camera430, the thermostat434, and the robotic devices490(or a component, such as a bridge/router) are intelligent enough to change the communication pathway from a direct local pathway when the one or more user devices440and450are in close physical proximity to the sensors420, the property automation controls422, the thermal camera430, the thermostat434, and the robotic devices490to a pathway over network405when the one or more user devices440and450are farther from the sensors420, the property automation controls422, the thermal camera430, the thermostat434, and the robotic devices490. In some examples, the system leverages GPS information from the one or more user devices440and450to determine whether the one or more user devices440and450are close enough to the monitoring system components to use the direct local pathway or whether the one or more user devices440and450are far enough from the monitoring system components that the pathway over network405is required. In other examples, the system leverages status communications (e.g., pinging) between the one or more user devices440and450and the sensors420, the property automation controls422, the thermal camera430, the thermostat434, and the robotic devices490to determine whether communication using the direct local pathway is possible. If communication using the direct local pathway is possible, the one or more user devices440and450communicate with the sensors420, the property automation controls422, the thermal camera430, the thermostat434, and the robotic devices490using the direct local pathway. If communication using the direct local pathway is not possible, the one or more user devices440and450communicate with the monitoring system components using the pathway over network405.

In some implementations, the system400provides end users with access to thermal images captured by the thermal camera430to aid in decision making. The system400may transmit the thermal images captured by the thermal camera430over a wireless WAN network to the user devices440and450. Because transmission over a wireless WAN network may be relatively expensive, the system400can use several techniques to reduce costs while providing access to significant levels of useful visual information (e.g., compressing data, down-sampling data, sending data only over inexpensive LAN connections, or other techniques).

In some implementations, a state of the monitoring system and other events sensed by the monitoring system may be used to enable/disable video/image recording devices (e.g., the thermal camera430or other cameras of the system400). In these implementations, the thermal camera430may be set to capture thermal images on a periodic basis when the alarm system is armed in an “armed away” state, but set not to capture images when the alarm system is armed in an “armed stay” or “unarmed” state. In addition, the thermal camera430may be triggered to begin capturing thermal images when the alarm system detects an event, such as an alarm event, a door-opening event for a door that leads to an area within a field of view of the thermal camera430, or motion in the area within the field of view of the thermal camera430. In other implementations, the thermal camera430may capture images continuously, but the captured images may be stored or transmitted over a network when needed.