Systems and methods for monitoring traffic lights using imaging sensors of vehicles

A traffic light monitoring system includes a vehicle. The vehicle includes an imaging sensor configured to capture images, one or more processors, one or more memory modules, and machine readable instructions stored in the one or more memory modules that cause the vehicle to perform at least the following when executed by the one or more processors: identify a traffic light based on the images, determine a first transition time when a first light of a traffic light turns off and a second light of the traffic light turns on based on the images, determine a second transition time when the second light of the traffic light turns off and a third light of the traffic light turns on based on the images, and transmit information about the first transition time and the second transition time to an edge computing device.

TECHNICAL FIELD

The present specification generally relates to systems and methods for monitoring traffic lights and, more specifically, to systems and methods for monitoring traffic lights using imaging sensors of vehicles and edge computing devices.

BACKGROUND

Traffic lights control the flow of traffic on roads. In many instances, drivers of vehicles have limited information about when the light of a traffic light will change. For example, drivers of vehicles may not know in advance when a green light is going to change to yellow or when a yellow light is going to change to red, etc. In this regard, a driver of a vehicle close to an intersection may need to press a brake abruptly when the driver sees the traffic light changing from green to yellow. Accordingly, a traffic light monitoring system that provides information about upcoming traffic light changes is needed.

SUMMARY

In one embodiment, a traffic light monitoring system includes a vehicle. The vehicle includes an imaging sensor configured to capture images, one or more processors, one or more memory modules, and machine readable instructions stored in the one or more memory modules that cause the vehicle to perform at least the following when executed by the one or more processors: identify a traffic light based on the images, determine a first transition time when a first light of a traffic light turns off and a second light of the traffic light turns on based on the images, determine a second transition time when the second light of the traffic light turns off and a third light of the traffic light turns on based on the images, and transmit information about the first transition time and the second transition time to an edge computing device.

In another embodiment, an edge computing device for monitoring traffic light is provided. The edge computing device includes one or more processors, one or more memory modules, and machine readable instructions stored in the one or more memory modules that cause the edge computing device to perform at least the following when executed by the one or more processors: receive a first transition time when a first light of a traffic light turns off and a second light of the traffic light turns on from a vehicle; receive a second transition time when the second light of the traffic light turns off and a third light of the traffic light turns on from the vehicle; calculate a period for the second light being on based on the first transition time and the second transition time; receive another transition time when the first light turns off and the second light turns on from another vehicle, the another transition time being after the first transition time; and broadcast a notification of an upcoming change of the second light based on the another transition time, the period, and a current time.

In yet another embodiment, an edge computing device for monitoring a traffic light is provided. The edge computing device includes one or more processors, one or more memory modules, and machine readable instructions stored in the one or more memory modules that cause the vehicle to perform at least the following when executed by the one or more processors: receive a first transition time when a first light of a traffic light turns off and a second light of the traffic light turns on from a first vehicle; receive a second transition time when the second light of the traffic light turns off and a third light of the traffic light turns on from a second vehicle; calculate a period for the second light being on based on the first transition time and the second transition time; receive a third transition time when the first light turns off and the second light turns on from a third vehicle, the third transition time being after the first transition time; and broadcast a notification of an upcoming change of the second light based on the third transition time, the period, and a current time.

DETAILED DESCRIPTION

FIG. 2generally depicts one embodiment of a traffic light monitoring system using imaging sensors of vehicles. The traffic light monitoring system includes an edge computing device and a vehicle. The vehicle includes an imaging sensor configured to capture images, one or more processors, one or more memory modules, and machine readable instructions stored in the one or more memory modules that cause the vehicle to perform at least the following when executed by the one or more processors: identify a traffic light based on the images, determine a first transition time when a first light of a traffic light turns off and a second light of the traffic light turns on based on the images, determine a second transition time when the second light of the traffic light turns off and a third light of the traffic light turns on based on the images, and transmit information about the first transition time and the second transition time to the edge computing device. Various embodiments of the systems and methods for monitoring traffic lights are described in greater detail herein.

Referring now toFIG. 1A, a system100for monitoring traffic lights is depicted.FIG. 1depicts a first vehicle102and a second vehicle110on a roadway101. The first vehicle102or the second vehicle110may be an automobile or any other passenger or non-passenger vehicle such as, for example, a terrestrial, aquatic, and/or airborne vehicle. In some embodiments, the first vehicle102or the second vehicle110is an autonomous vehicle that navigates its environment with limited human input or without human input. The first vehicle102and the second vehicle110are approaching an intersection103. A traffic light111and a traffic light113are located at two intersections, respectively. The first vehicle102and the second vehicle110are recording visual data with imaging sensors104(e.g., cameras). Each of the first vehicle102and the second vehicle110further includes network interface hardware106and an electronic control unit (“ECU”)108(SeeFIG. 2). The imaging sensor104, network interface hardware106, and ECU108are described in greater detail herein with respect toFIG. 2.

The system100may also include a first edge computing device112that includes network interface hardware116. The first edge computing device112may include a processor140(FIG. 2) and one or more memory modules for storing processor-readable instructions as described in greater detail with respect toFIG. 2. In some embodiments, the first edge computing device112may be a roadside unit (“RSU”). In some embodiments, the system100may include a second edge computing device114. The second edge computing device114may further include network interface hardware116. In some embodiments, the second edge computing device114may be an RSU. The first edge computing device112and the second edge computing device114may maintain a data connection with one another via the network interface hardware116and may be a part of a larger network of computing devices (e.g., a grid computing network). In some embodiments, the first vehicle102and the second vehicle110establish an edge server connection with one or more of the first edge computing device112and the second edge computing device114using the network interface hardware106of the first vehicle102and the second vehicle110and the network interface hardware116of the first edge computing device112and the second edge computing device114.

The first vehicle102, the second vehicle110, the first edge computing device112, and the second edge computing device114may form data connections with one another via their respective network interface hardware106,116. The first vehicle102, the second vehicle110, the first edge computing device112, and the second edge computing device114may transmit image data and other data over the data connections.

InFIG. 1A, the traffic light111has three lights including a red light105, a yellow light107, and a green light109. In this example, the red light105of the traffic light111is on. The first vehicle102may capture images of the traffic light111using the imaging sensor104and process the images to monitor the light of the traffic light111. When it is determined that the red light105turns off and the green light109turns on, the first vehicle102transmits the transition time when the red light105turns off and the green light109turns on to the first edge computing device112. For example, when the first vehicle102determines that the red light105turns off and the green light109turns on at 12 o'clock 31 minutes 9 seconds, the first vehicle102transmits the time stamp of 12 o'clock 31 minutes 9 seconds along with an indication of change from red to green.

InFIG. 1B, the first vehicle102has passed the intersection103and the second vehicle110is close to the intersection103. The green light109of the traffic light is turned on. Similar to the first vehicle102, the second vehicle110may capture images of the traffic light111using the imaging sensor104and process the images to monitor the light of the traffic light111. The second vehicle110may capture the images of traffic light111using the imaging sensor104and process the images to monitor the light of the traffic light111. When it is determined that the green light109turns off and the yellow light107turns on, the second vehicle110transmits the transition time when the green light109turns off and the yellow light107turns on to the first edge computing device112. For example, when the second vehicle110determines that the green light109turns off and the yellow light107turns on at 12 o'clock 31 minutes 51 seconds, the first vehicle102transmits the time stamp of 12 o'clock 31 minutes 51 seconds along with an indication of the change from green to yellow.

In a similar way, the second vehicle110may transmit a transition time when the traffic light111switches from yellow to red. For example, when the second vehicle110determines that the yellow light107turns off and the red light105turns on at 12 o'clock 31 minutes 55 seconds, the second vehicle110transmits the time stamp of 12 o'clock 31 minutes 55 seconds along with an indication of the change from yellow to red.

Referring now toFIGS. 1A, 1B, and 2, additional features and details of the image generating system100are described.FIG. 2is a schematic showing the various systems of each of the first vehicle102and the second vehicle110ofFIG. 1. It is to be understood that the first vehicle102and the second vehicle110are not limited to the systems and features shown inFIG. 2and that each may include additional features and systems. As shown, the first vehicle102includes a data unit118for generating, processing, and transmitting data. The second vehicle110may include a second data unit120which may be to the same as the data unit118of the first vehicle102in some embodiments.

The data unit118includes the ECU108, the network interface hardware106, the imaging sensor104, a screen122, a navigation module124, a speaker125, and one or more motion sensors136that may be connected by a communication path126. The network interface hardware106may connect the first vehicle102to external systems via an external connection128. For example, the network interface hardware106may connect the first vehicle102to one or more other vehicles directly (e.g., a direct connection to the second vehicle110) or to an external network such as a cloud network129.

Still referring toFIGS. 1A, 1B, and 2, the ECU108may be any device or combination of components comprising a processor132and a non-transitory processor readable memory module134. The processor132may be any device capable of executing a processor-readable instruction set stored in the non-transitory processor readable memory module134. Accordingly, the processor132may be an electric controller, an integrated circuit, a microchip, a computer, or any other computing device. The processor132is communicatively coupled to the other components of the data unit118by the communication path126. Accordingly, the communication path126may communicatively couple any number of processors132with one another, and allow the components coupled to the communication path126to operate in a distributed computing environment. Specifically, each of the components may operate as a node that may send and/or receive data. While the embodiment depicted inFIG. 2includes a single processor132, other embodiments may include more than one processor.

The non-transitory processor readable memory module134is coupled to the communication path126and communicatively coupled to the processor132. The non-transitory processor readable memory module134may comprise RAM, ROM, flash memories, hard drives, or any non-transitory memory device capable of storing machine-readable instructions such that the machine-readable instructions can be accessed and executed by the processor132. The machine-readable instruction set may comprise logic or algorithm(s) written in any programming language of any generation (e.g., 1GL, 2GL, 3GL, 4GL, or 5GL) such as, for example, machine language that may be directly executed by the processor132, or assembly language, object-oriented programming (OOP), scripting languages, microcode, etc., that may be compiled or assembled into machine readable instructions and stored in the non-transitory processor readable memory module134. Alternatively, the machine-readable instruction set may be written in a hardware description language (HDL), such as logic implemented via either a field-programmable gate array (FPGA) configuration or an application-specific integrated circuit (ASIC), or their equivalents. Accordingly, the functionality described herein may be implemented in any conventional computer programming language, as pre-programmed hardware elements, or as a combination of hardware and software components. While the embodiment depicted inFIG. 2includes a single non-transitory processor readable memory module134, other embodiments may include more than one memory module.

Still referring toFIGS. 1A, 1B, and 2, one or more imaging sensors104are coupled to the communication path126and communicatively coupled to the processor132. While the particular embodiment depicted inFIGS. 1A, 1B, and 2shows an icon with one imaging sensor and reference is made herein to “imaging sensor” in the singular with respect to the data unit118, it is to be understood that this is merely a representation and embodiments of the system may include one or more imaging sensors having one or more of the specific characteristics described herein.

The imaging sensor104may be any device having an array of sensing devices capable of detecting radiation in an ultraviolet wavelength band, a visible light wavelength band, or an infrared wavelength band. The imaging sensor104may have any resolution. In some embodiments, one or more optical components, such as a mirror, fish-eye lens, or any other type of lens may be optically coupled to the imaging sensor104. In embodiments described herein, the imaging sensor104may provide image data to the ECU108or another component communicatively coupled to the communication path126. The image data may include image data of the environment around the first vehicle102. In some embodiments, for example, in embodiments in which the first vehicle102is an autonomous or semi-autonomous vehicle, the imaging sensor104may also provide navigation support. That is, data captured by the imaging sensor104may be used by the navigation module124to autonomously or semi-autonomously navigate the first vehicle102.

The imaging sensor104may operate in the visual and/or infrared spectrum to sense visual and/or infrared light. Additionally, while the particular embodiments described herein are described with respect hardware for sensing light in the visual and/or infrared spectrum, it is to be understood that other types of sensors are contemplated. For example, the systems described herein could include one or more LIDAR sensors, radar sensors, sonar sensors, or other types of sensors and that such data could be integrated into or supplement the data collection described herein to develop a fuller real-time traffic image.

In operation, the imaging sensor104captures image data and communicates the image data to the ECU108and/or to other systems communicatively coupled to the communication path126. The image data may be received by the processor132, which may process the image data using one or more image processing algorithms. Any known or yet-to-be developed video and image processing algorithms may be applied to the image data in order to identify an item or situation. Example video and image processing algorithms include, but are not limited to, kernel-based tracking (such as, for example, mean-shift tracking) and contour processing algorithms. In general, video and image processing algorithms may detect objects and movement from sequential or individual frames of image data. One or more object recognition algorithms may be applied to the image data to extract objects and determine their relative locations to each other. Any known or yet-to-be-developed object recognition algorithms may be used to extract the objects or even optical characters and images from the image data. Example object recognition algorithms include, but are not limited to, scale-invariant feature transform (“SIFT”), speeded up robust features (“SURF”), and edge-detection algorithms.

The network interface hardware106may be coupled to the communication path126and communicatively coupled to the ECU108. The network interface hardware106may be any device capable of transmitting and/or receiving data with external vehicles or servers directly or via a network, such as the cloud network129. Accordingly, network interface hardware106can include a communication transceiver for sending and/or receiving any wired or wireless communication. For example, the network interface hardware106may include an antenna, a modem, LAN port, Wi-Fi card, WiMax card, mobile communications hardware, near-field communication hardware, satellite communication hardware and/or any wired or wireless hardware for communicating with other networks and/or devices. In embodiments, network interface hardware106may include hardware configured to operate in accordance with the Bluetooth wireless communication protocol and may include a Bluetooth send/receive module for sending and receiving Bluetooth communications.

In some embodiments, the first vehicle102may be communicatively coupled to a network such as the cloud network129. In embodiments, the cloud network129may include one or more computer networks (e.g., a personal area network, a local area network, grid computing network, wide area network, etc.), cellular networks, satellite networks and/or a global positioning system and combinations thereof. Accordingly, the first vehicle102can be communicatively coupled to the cloud network129via wires, via a wide area network, via a local area network, via a personal area network, via a cellular network, via a satellite network, or the like. Suitable local area networks may include wired Ethernet and/or wireless technologies such as, for example, wireless fidelity (Wi-Fi). Suitable personal area networks may include wireless technologies such as, for example, IrDA, Bluetooth, Wireless USB, Z-Wave, ZigBee, and/or other near field communication protocols. Suitable personal area networks may similarly include wired computer buses such as, for example, USB and FireWire. Suitable cellular networks include, but are not limited to, technologies such as LTE, WiMAX, UMTS, CDMA, and GSM.

Referring toFIGS. 1A, 1B, and 2, in embodiments, the first vehicle102may connect with one or more external vehicles (e.g., the second vehicle110) and/or external processing devices (e.g., the first edge computing device112) via a direct connection. The direct connection may be a vehicle-to-vehicle connection (“V2V connection”). The V2V connection may be established using any suitable wireless communication protocols discussed above. A connection between vehicles may utilize sessions that are time and/or location-based. In embodiments, a connection between vehicles may utilize one or more networks to connect (e.g., the cloud network129), which may be in lieu of, or in addition to, a direct connection (such as V2V) between the vehicles. By way of non-limiting example, vehicles may function as infrastructure nodes to form a mesh network and connect dynamically/ad-hoc. In this way, vehicles may enter/leave the network at will such that the mesh network may self-organize and self-modify over time. Other non-limiting examples include vehicles forming peer-to-peer networks with other vehicles or utilizing centralized networks that rely upon certain vehicles and/or infrastructure (e.g., the first edge computing device112). Still other examples include networks using centralized servers and other central computing devices to store and/or relay information between vehicles.

In embodiments, the data unit118may include one or more motion sensors136for detecting and measuring motion and changes in motion of the first vehicle102. Each of the one or more motion sensors136is coupled to the communication path126and communicatively coupled to the one or more processors132. The motion sensors136may include inertial measurement units. Each of the one or more motion sensors136may include one or more accelerometers and one or more gyroscopes. Each of the one or more motion sensors136transforms sensed physical movement of the first vehicle102into a signal indicative of an orientation, a rotation, a velocity, or an acceleration of the first vehicle102.

In embodiments, the data unit118includes a screen122for providing visual output such as, for example, maps, navigation, entertainment, seat arrangements or a combination thereof. The screen122may be located on the head unit of the vehicle such that a driver of the vehicle may see the screen122while seated in the driver seat. The screen122is coupled to the communication path126. Accordingly, the communication path126communicatively couples the screen122to other modules of the data unit118. The screen122may include any medium capable of transmitting an optical output such as, for example, a cathode ray tube, a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a liquid crystal display, a plasma display, or the like. In embodiments, the screen122may be a touchscreen that, in addition to visually displaying information, detects the presence and location of a tactile input upon a surface of or adjacent to the screen122.

In embodiments, the data unit118may include the navigation module124. The navigation module124may be configured to obtain and update positional information of the first vehicle102and to display such information to one or more users of the first vehicle102. The navigation module124may be able to obtain and update positional information based on geographical coordinates (e.g., latitudes and longitudes), or via electronic navigation where the navigation module124electronically receives positional information through satellites. In certain embodiments, the navigation module124may include a GPS system.

In embodiments, the data unit118includes the speaker125for transforming data signals into mechanical vibrations, such as in order to output audible prompts or audible information to a driver of the vehicle. The speaker125is coupled to the communication path126and communicatively coupled to the one or more processors132.

The components of the second data unit120of the second vehicle110may be exactly the same as the components of the data unit118of the first vehicle102in some embodiments.

Referring toFIGS. 1A, 1B, and 2, the first edge computing device112may include the network interface hardware116which may be communicatively coupled to a control unit138including a processor140and a non-transitory processor readable memory module142via a communication path127.

The network interface hardware116may be coupled to the communication path127and communicatively coupled to the control unit138. The network interface hardware116may be any device capable of transmitting and/or receiving data with external vehicles or servers directly or via a network, such as the cloud network129. Accordingly, network interface hardware116can include a communication transceiver for sending and/or receiving any wired or wireless communication. For example, the network interface hardware116may include an antenna, a modem, LAN port, Wi-Fi card, WiMax card, mobile communications hardware, near-field communication hardware, satellite communication hardware and/or any wired or wireless hardware for communicating with other networks and/or devices. In embodiments, network interface hardware116may include hardware configured to operate in accordance with the Bluetooth wireless communication protocol and may include a Bluetooth send/receive module for sending and receiving Bluetooth communications.

The control unit138may include the processor140and the non-transitory processor readable memory module142. The processor140may be any device capable of executing the processor-readable instruction set stored in the non-transitory processor readable memory module142. Accordingly, the processor140may be an electric controller, an integrated circuit, a microchip, a computer, or any other computing device. The processor140is communicatively coupled to the communication path127. Accordingly, the communication path127may communicatively couple any number of processors140with one another, and allow the components coupled to the communication path127to operate in a distributed computing environment. Specifically, each of the components may operate as a node that may send and/or receive data. While the embodiment depicted inFIG. 2includes a single processor140, other embodiments may include more than one processor.

The non-transitory processor readable memory module142is coupled to the communication path127and communicatively coupled to the processor140. The non-transitory processor readable memory module142may comprise RAM, ROM, flash memories, hard drives, or any non-transitory memory device capable of storing machine-readable instructions such that the machine-readable instructions can be accessed and executed by the processor140. The machine-readable instruction set may comprise logic or algorithm(s) written in any programming language of any generation (e.g., 1GL, 2GL, 3GL, 4GL, or 5GL) such as, for example, machine language that may be directly executed by the processor140, or assembly language, object-oriented programming (OOP), scripting languages, microcode, etc., that may be compiled or assembled into machine readable instructions and stored in the non-transitory processor readable memory module142. Alternatively, the machine-readable instruction set may be written in a hardware description language (HDL), such as logic implemented via either a field-programmable gate array (FPGA) configuration or an application-specific integrated circuit (ASIC), or their equivalents. Accordingly, the functionality described herein may be implemented in any conventional computer programming language, as pre-programmed hardware elements, or as a combination of hardware and software components. While the embodiment depicted inFIG. 2includes a single non-transitory processor readable memory module142, other embodiments may include more than one memory module.

FIG. 3depicts a flow chart for monitoring a traffic light, according to embodiments shown and described herein.

In block310, a first vehicle captures images of a traffic light using its imaging sensor. In embodiments, the first vehicle102captures images of the traffic light111using its imaging sensor104as shown inFIG. 1A. The processor132of the first vehicle102may process the captured images to identify which light of the traffic light111is on. In the embodiment, the red light105of the traffic light111is on, and the processor132of the first vehicle102may process the captured images to identify that the red light105is on.

In block320, the first vehicle determines a first transition time when a first light of a traffic light turns off and a second light of the traffic light turns on based on the captured images. In embodiments, the first vehicle102continuously captures images of the traffic light111, and determines the first transition time when the red light105of the traffic light111turns off and the green light109of the traffic light111turns on by processing the captured images. For example, the first vehicle102determines that the first transition time (i.e., when the green light109turns on) is 12 o'clock 31 minutes 9 seconds.

In block330, the first vehicle transmits information about the first transition time to an edge computing device. In embodiments, the first vehicle102transmits the first transition time of 12 o'clock 31 minutes 9 seconds to the first edge computing device112. In embodiments, the first vehicle102transmits the first transition time along with an indication of a color being turned off and a color being turn on. For example, the first vehicle102transmits an indication that the red light turns off and the green light turns on to the first edge computing device112along with the first transition time. In some embodiments, the first vehicle102may also transmit information about the location of the traffic light111. For example, the first vehicle102may determine the current location of the first vehicle102using the navigation module124. Based on the current location of the first vehicle102, the driving direction of the first vehicle102and the captured image of the traffic light111, the first vehicle102determines the location of the traffic light111. In some embodiments, the first vehicle102transmits the location of the intersection103where the traffic light111is located. For example, the first vehicle102may determine the current location of the first vehicle102using the navigation module124and determine a driving direction of the first vehicle102using the motion sensor136. Based on the current location of the first vehicle102and the driving direction of the first vehicle102, the first vehicle102determines the location of the intersection103.

In block340, a second vehicle captures images of a traffic light using its imaging sensor. In embodiments, the second vehicle110captures images of the traffic light111using its imaging sensor104as shown inFIG. 1B. The processor132of the second vehicle110may process the captured images to identify which light of the traffic light111is on. In the embodiment, the green light109of the traffic light111is on, and the processor132of the second vehicle110may process the captured images to identify that the green light109is on.

In block350, the second vehicle determines a second transition time when the second light of a traffic light turns off and a third light of the traffic light turns on based on one or more images captured by the second vehicle. In embodiments, the second vehicle110continuously captures images of the traffic light111, and determines the second transition time when the green light109of the traffic light111turns off and the yellow light107of the traffic light111turns on by processing the captured images. For example, the second vehicle110determines that the second transition time (i.e., when the yellow light107turns on) is 12 o'clock 31 minutes 51 seconds.

In block360, the second vehicle transmits information about the second transition time to the edge computing device. In embodiments, the second vehicle110transmits the second transition time of 12 o'clock 31 minutes 51 seconds to the first edge computing device112. In embodiments, the first vehicle102transmits the second transition time along with an indication of a color being turned off and a color being turn on. For example, the second vehicle110transmits an indication that the green light turns off and the yellow light turns on to the first edge computing device112along with the second transition time. In some embodiments, the second vehicle110may also transmit information about the location of the traffic light111. For example, the second vehicle110may determine the current location of the second vehicle110using the navigation module124. Based on the current location of the second vehicle110and the captured image of the traffic light111, the second vehicle110determines the location of the traffic light111. In some embodiments, the second vehicle110transmits the location of the intersection103where the traffic light111is located. For example, the second vehicle110may determine the current location of the second vehicle110using the navigation module124and determine a driving direction of the second vehicle110using the motion sensor136. Based on the current location of the second vehicle110and the driving direction of the second vehicle110, the second vehicle110determines the location of the intersection103.

The first edge computing device112may calculate the duration where the green light109is on (i.e., green time) based on the first transition time and the second transition time. For example, the first edge computing device112may calculate the green time based on the difference between the second transition time and the first transition time. In the example, the green time is 42 seconds, i.e., time difference between 12 o'clock 31 minutes 51 seconds and 12 o'clock 31 minutes and 9 seconds. The first edge computing device112may transmit an alert to other vehicles coming toward the traffic light111based on the calculated green time, which will be described in detail with reference toFIGS. 5 and 6below.

In some embodiments, the second vehicle determines a third transition time when the third light of a traffic light turns off and the first light of the traffic light turns on based on one or more images captured by the second vehicle. In embodiments, the second vehicle110continuously captures images of the traffic light111, and determines the third transition time when the yellow light107of the traffic light111turns off and the red light105of the traffic light111turns on by processing the captured images. For example, the second vehicle110determines that the third transition time (i.e., when the red light105turns on) is 12 o'clock 31 minutes 55 seconds. The first edge computing device112may calculate the duration where the yellow light107is on (i.e., yellow time) based on the second transition time and the third transition time. For example, the first edge computing device112may calculate the yellow time based on the difference between the third transition time and the second transition time. In the example, the yellow time is 4 seconds, i.e., time difference between 12 o'clock 31 minutes 55 seconds and 12 o'clock 31 minutes and 51 seconds. The first edge computing device112may transmit an alert to other vehicles coming toward the traffic light111based on the calculated yellow time, which will be described in detail with reference toFIGS. 5 and 6below.

WhileFIGS. 1A, 1B, and 3depicts an embodiment where the green time is calculated based on the first transition time and the second transition time communicated from the first vehicle and the second vehicle, respectively, the green time may be calculated based on the first transition time and the second transition time communicated from a single vehicle as shown inFIGS. 4A and 4B.

InFIG. 4A, the first vehicle102is located about 0.1 miles from the traffic light111. In embodiments, the first vehicle102captures images of the traffic light111using its imaging sensor104as shown inFIG. 4A. The processor132of the first vehicle102may process the captured images to identify which light of the traffic light111is on. In the embodiment shown inFIG. 4A, the red light105of the traffic light111is on, and the processor132of the first vehicle102may process the captured images to identify that the red light105is on. Then, the first vehicle102determines the first transition time when the red light105of the traffic light111turns off and the green light109of the traffic light111turns on by processing the captured images. For example, the first vehicle102determines that the first transition time (i.e., when the green light109turns on) is 9 o'clock 25 minutes 5 seconds. The first vehicle102transmits the first transition time of 9 o'clock 25 minutes 5 seconds to the first edge computing device112along with an indication of a color being turned off and a color being turn on (e.g., the red light105turns off and the green light109turns on). The first vehicle102may also transmit information about the location of the traffic light111and/or the location of the intersection103.

InFIG. 4B, the first vehicle102is close to the traffic light111. For example, the first vehicle102is driving toward the traffic light111and is now only 20 feet from the traffic light111. The first vehicle102captures images of the traffic light111using its imaging sensor104as shown inFIG. 4B. The processor132of the first vehicle102may process the captured images to identify which light of the traffic light111is on. In the embodiment, the green light109of the traffic light111is on, and the processor132of the first vehicle102may process the captured images to identify that the green light109is on. The first vehicle102continuously captures images of the traffic light111, and determines the second transition time when the green light109of the traffic light111turns off and the yellow light107of the traffic light111turns on by processing the captured images. For example, the first vehicle102determines that the second transition time (i.e., when the yellow light107turns on) is 9 o'clock 25 minutes 49 seconds. Then, the first vehicle102transmits the second transition time of 9 o'clock 25 minutes 49 seconds to the first edge computing device112along with an indication of a color being turned off and a color being turn on (e.g., the green light109turns off and the yellow light107turns on). The first vehicle102may also transmit information about the location of the traffic light111and/or the location of the intersection103along with the second transition time. In a similar way, the first vehicle102may determine a third transition time when the third light of a traffic light turns off and the first light of the traffic light turns on based on one or more images captured by the first vehicle102. In embodiments, the first vehicle102continuously captures images of the traffic light111, and determines the third transition time when the yellow light107of the traffic light111turns off and the red light105of the traffic light111turns on by processing the captured images.

While the figures depict a traffic light have three color lights, the traffic light may have more than three or less than three color lights. For example, the traffic light may have two colors: a red light and a green light. The first transition time may be when the red light turns off and the green light turns on. The second transition time may be when the green light turns off and the red light turns on. As another example, the traffic light may have four color lights: a red light, a yellow light, a green left turn light, and a green light. The first transition time may be when the red light turns off and the green left turn light turns on. The second transition time may be when the green left turn light turns off and the green light turns on. A third second transition time may be when the green light turns off and the yellow light turns on. A fourth transition time may be when the yellow light turns off and the red light turns on. Based on the first transition time, the second transition time, the third transition time, and the fourth transition time, the first edge computing device112may calculate a green left turn time (i.e., a duration while the green left turn light is on), the green time, and the yellow time.

In some embodiments, the first vehicle102and/or the second vehicle110capture images of a pedestrian traffic light170, for example, as shown inFIGS. 1A and 1B. The pedestrian traffic light170includes a do not walk light171and a walk light173. The first vehicle102and/or the second vehicle110may calculate a first transition time when the do not walk light171of the pedestrian traffic light170turns off and the walk light173of the pedestrian traffic light170turns on. The first vehicle102and/or the second vehicle110may calculate a second transition time when the walk light173of the pedestrian traffic light170turns off and the do not walk light171of the pedestrian traffic light170turns on. The first vehicle102and/or the second vehicle110transmits the first transition time and the second transition time to the first edge computing device112. The first edge computing device112may calculate a walk time of the pedestrian traffic light170based on the first transition time and the second transition time. The first edge computing device may communicate the walk time to other vehicles or mobile devices within a predetermined area. For example, a pedestrian at the intersection may receive information about the walk time of the pedestrian traffic light170or when the pedestrian traffic light170will switch, through her mobile device from the first edge computing device112.

FIG. 5depicts a flow chart for alerting an upcoming traffic light change, according to one or more embodiments shown and described herein. In block510, a first edge computing device receives a first transition time when a first light of a traffic light turns off and a second light of the traffic light turns on from a first vehicle. In embodiments, as depicted inFIG. 1A, the first edge computing device112receives, from the first vehicle102, the first transition time when the red light105turns off and the green light109turns on. For example, the first transition time is 12 o'clock 31 minutes 9 seconds.

In block520, the first edge computing device receives a second transition time when the second light of a traffic light turns off and a third light of the traffic light turns on from a second vehicle. In embodiments, as depicted inFIG. 1A, the first edge computing device112receives, from the second vehicle110, the second transition time when the green light109turns off and the yellow light107turns on. For example, the second transition time is 12 o'clock 31 minutes 51 seconds. In some embodiments, the first edge computing device112receives the second transition time from the first vehicle102instead of the second vehicle110.

In block530, the first edge computing device calculates a green time based on the first transition time and the second transition time. For example, the first edge computing device112calculates a green time based on the difference between the second transition time and the first transition time. The green time, in this example, is 42 seconds.

In block540, the first edge computing device receives a third transition time when the first light of the traffic light turns off and the second light of the traffic light turns on from a third vehicle. By referring toFIG. 6, the first edge computing device112receives a third transition time when the red light105turns off and the green light109turns on from the third vehicle610. For example, the third transition time is 12 o'clock 35 minutes 5 seconds.

In block550, the first edge computing device broadcasts a notification of upcoming change of the traffic light based on the third transition time, the green time, and current time. For example, the current time is 12 o'clock 35 minutes 26 seconds. The first edge computing device112may store the green time as 42 seconds that is calculated in block530. The first edge computing device112may determine that the green light109will turn off in 21 seconds based on the third transition time of 12 o'clock 35 minutes 5 seconds, the green time of 42 seconds, and the current time of 12 o'clock 35 minutes 26 seconds. Then, the first edge computing device broadcasts a notification that the green light109will turn off in 21 seconds. In embodiments, the first edge computing device112may broadcast the notification to vehicles within a predetermined distance from the first edge computing device112.

The third vehicle610and a fourth vehicle620inFIG. 6may receive the notification from the first edge computing device112. In response to receiving the notification, the third vehicle610and the fourth vehicle620may determine an arrival time to the intersection. For example, each of the third vehicle610and the fourth vehicle620may determine the arrival time at the intersection103based on the vehicle speed and the distance between the vehicle and the intersection103. The vehicle speed may be determined by the one or more motion sensors136. The distance may be determined based on the current location of the vehicle measured by the navigation module124and the location of the intersection103.

In embodiments, if the arrival time is after the green light109turns off, the vehicle may provide a notification to a driver. In this example, the third vehicle610will arrive at the intersection103in 5 seconds, and the fourth vehicle will arrive at the intersection103in 25 seconds. The third vehicle610may not provide a notification to the driver of the third vehicle610because the third vehicle610will cross the intersection before the green light109turns off. The fourth vehicle620may provide a notification to the driver of the fourth vehicle620because the green light109will turn off before the fourth vehicle620reaches the intersection103.

The notification may be different depending on the estimated arrival time. For example, if the green light turns off more than five seconds before the arrival time at the intersection103, the vehicle may provide a caution to the driver of the vehicle. If the green light turns off less than five seconds before the arrival time at the intersection, the vehicle may provide a warning to the driver of the vehicle. In this example, the green light will turn off less than five seconds before the fourth vehicle's arrival time at the intersection, and the fourth vehicle may provide a warning to the driver. For example, the warning may be displayed on the screen122of the vehicle. As another example, the warning may be an audible output through the speaker125.

It should now be understood that a traffic light is monitored by multiple vehicles in real time. An edge computing device receives a first transition time when a first light of a traffic light turns off and a second light of the traffic light turns on from a vehicle, and receive a second transition time when the second light of the traffic light turns off and a third light of the traffic light turns on from a vehicle. The edge computing device calculates a period for the second light being on based on the first transition time and the second transition time. Based the calculated period (e.g., a green time), the edge computing device transmits, to vehicles within a predetermined distance, a message regarding upcoming change of the traffic light. According to the present disclosure, traffic light change timing information is shared among vehicles via edge computing devices. Vehicles that receive the information may prepare to react in advance of the traffic light change.