Infrared-based apparatus for detecting gaps in mosquito netting

Systems and methods for detecting gaps in netting include constructing a depth map from electromagnetic signals associated with netting surrounding an area, the depth map including measurements of distance of each strand of the netting along an axis perpendicular to a plane formed by the netting relative to the detector. Each segment of netting is identified including each strand of the netting in a side of the netting facing the detector. Locations of gaps in the netting are identified according to the depth map. A user is alerted to the presence and locations of gaps by sending a communication to a computing device.

BACKGROUND

Technical Field

The present invention generally relates to infrared pattern detection, and more particularly to an infrared-based apparatus for detecting gaps in mosquito netting.

Description of the Related Art

Mosquito borne illnesses, such as malaria, affect a large amount of people, particularly in tropical regions. For many of these illnesses, medical treatments are expensive, ineffective, or otherwise infeasible for preventing the spread and contraction of these illnesses. However, some steps can be taken to prevent mosquito bites, and thus reduce the risks of contracting a mosquito borne illness. For example, mosquito netting, particularly insecticide treated mosquito netting, hung around a bed is an effective method of preventing mosquito borne illness while sleeping.

However, to effectively prevent mosquito bites, mosquito netting cannot have any openings large enough for a mosquito to penetrate. Thus, any gaps in the mosquito netting can result in a high risk of mosquito bites and thus contracting a mosquito borne illness.

SUMMARY

In accordance with an embodiment of the present invention, a method for detecting gaps in netting is described. The method includes constructing a depth map from electromagnetic signals associated with netting surrounding an area, the depth map including measurements of the distance of each strand of netting from the detector. Each segment of netting is identified including each strand of the netting in a side of the netting facing the detector. Locations of gaps in the netting are identified according to the depth map. A user is alerted to the presence and locations of gaps by sending a communication to a computing device.

In accordance with another embodiment of the present invention, a system for detecting gaps in netting is described. The system includes an electromagnetic detector that captures an electromagnetic signal from netting surrounding an area, the electromagnetic signal including a signal and an angle of the signal from each strand of the netting. A signal processor includes a memory device and a processor to process the captured electromagnetic signal. The signal processor includes a depth mapper that constructs a depth map from the signals, the depth map including measurements of distance of each strand of the netting from the detector, a pattern recognition engine that identifies the netting, and a gap locator that identifies locations of gaps in the netting according to the depth map of the netting. A notifier alerts a user to the presence and locations of gaps.

In accordance with another embodiment of the present invention, a non-transitory computer readable storage medium comprising a computer readable program for detecting gaps in netting is described. The computer readable program when executed on a computer causes the computer to perform the steps of constructing a depth map from electromagnetic signals associated with netting surrounding an area, the depth map including measurements of distance of each strand of the netting from the detector. Each segment of netting is identified including each strand of the netting in a side of the netting facing the detector. Locations of gaps in the netting are identified according to the depth map. A user is alerted to the presence and locations of gaps by sending a communication to a computing device.

DETAILED DESCRIPTION

According to an embodiment of the present invention, a system and method for detecting gaps in mosquito netting are contemplated that utilize electromagnetic (EM) sensors to detect irregularities in the netting.

In one possible embodiment of the present invention, the EM sensors include infrared (IR) sensors to detect IR signals. The IR sensors can be positioned in one or more locations around an area bounded by the netting, such as, e.g., around a bed. Each IR sensor can use IR imaging to sense a portion of the netting on a side of the area. For example, an IR sensor can project an IR signal towards the netting and analyze the reflected infrared signal. Every object radiates and reflects a characteristic amount of infrared radiation according to the shape, size, and molecular make-up of the objects. Thus, each object, including, e.g., the netting, furniture, sheets and blankets, among other objects in and around the area, can be fingerprinted with the spectrum and the geometry of the reflection of the IR signal. For example, the netting reflects the IR signal by reflecting the signal striking the individual strands of the netting, thus reflecting a grid shaped IR reflection pattern having a spectral signature in accordance with the material of the netting.

A depth map can be created from images captured by the IR sensor according to the reflected IR signal. Distances from the IR sensor to each object in and around the area can be determined by, e.g., using an angle of the sensed IR signal, using a parallax effect due to stereoscopic detection of the IR signal, incorporating patterned infrared sensing, among other depth sensing techniques. Thus, the depth of each strand of the netting, as well as each other object, can be determined. Using the depths, any depth-wise gaps in the netting can be determined. In particular, in portions of the depth map where two netting layers are detected, overlapping with respect to the IR sensor, a distance of each netting layer is determined. One or more strands of one of the netting layers is then traced until an end of the strand is reached. If the strand traverses both layers of the netting, then the two layers are the result of a wrinkle in the netting and a depth-wise gap does not exist. However, if the strand does not traverse both layers, then a depth-wise gap does exist.

Moreover, the depth map includes vertical and horizontal positions of each object including, e.g., each strand of the netting, where the vertical and horizontal directions are perpendicular to the depth direction, or parallel to a plane of the netting on a particular side of the area being bounded. Thus, a portion of the depth map where a space exists between netting strands that is larger than the size of a mosquito, then an x/y gap exists between segments of netting.

Upon identifying gaps in the netting, a user is notified. The notification is one or both of directing a visible beam towards the location of the gap, and sending a message to the user's phone. Thus, the user can be notified of a gap that might let a mosquito through thereby increasing the risk of contracting a mosquito borne illness.

Exemplary applications/uses to which the present invention can be applied include, but are not limited to: identifying gaps in mosquito netting, fishing nets, transparent or semi-transparent fabrics, among other materials.

Referring now to the drawings in which like numerals represent the same or similar elements and initially toFIG. 1, a diagram of a system for gap detection in netting is shown in accordance with an embodiment of the present invention.

According to an embodiment of the present invention, a bed122is bounded by netting120having at least a first netting segment124and a second netting segment126. The netting120can have any number of sides to bound an area such as the bed122. Thus, the netting120can form a perimeter in the shape of, e.g., a rectangle or other polygon, or a circle or ellipse, or any other shape to bound the area.

The first netting segment124and the second netting segment126can be on a same side of the bed122. The two segments of netting120can be used to open a gap between the first netting segment124and the second netting segment126to permit entry into the bed122area. However, to prevent mosquito entry, the two segments are brought back into an overlapping arrangement with each other. Sometimes, closing the gap in this manner is not fully effective and one or more gaps between the first netting segment124and the second netting segment126may remain. These gaps are potential entry points for mosquitoes. Similarly, holes and tears may develop in one or more segments of netting120, opening gaps which also can provide an entry point for mosquitoes. Each of these gaps can be detected using an infrared detector100.

The infrared detector100detects an infrared (IR) signal from objects in the vicinity of the bed122and netting124and126. The infrared detector100uses the IR signal to inspect a portion of the netting120including the first netting segment124and the second netting segment126, and a seam or overlapping region of the first and second netting segments124and126. Depending on the size of each segment and the size of the netting120in general, as well as the number of sides of the bounded area, multiple infrared detectors100can be used such that the entirety of the netting120can be inspected. Alternatively, infrared detectors100can be positioned to only inspect the sides of the bounded area that are adjacent to an open area. For example, infrared detectors100may not be needed for a side of a bed pushed up against a wall because the risk of a gap being opened on the side of the wall is low or negligible. Thus, infrared detectors100would be positioned to only view the sides of the bed122that are not adjacent to walls.

The use of IR facilitates depth measurements, however, other portions of the electromagnetic spectrum can be used to inspect the netting120, such as, e.g., ultraviolet (UV), visible light, or any other suitable range of the electromagnetic spectrum. Thus, the infrared detector100captures both an image of the first netting segment124, the second netting segment126and the bed122, as well as depth information of each object present within an area of detection according to a field of view of the infrared detector100. The infrared detector100can use, e.g., an IR emitter placed at a pre-determined offset from the infrared detector100to determine an angle of IR reflection off each object, and thus determine a depth of each object. Alternatively, two infrared detectors100can be used in conjunction at a pre-determined offset from each other such that the parallax of the IR images captured by each detector can be used to determine an angle, and thus a depth, of the IR signal from each object, similar to stereoscopic vision.

The depth information can be used to form a depth map that presents depth information for strands of each of the first netting segment124and the second netting segment126as well as any objects behind each netting segment such as the bed122. The depth information can include, e.g., a measurement of distance D from the infrared detector100, or a variation from a pre-established distance D between the infrared detector100and, e.g., the bed122. For example, the depth measurement can be determined as a difference between the pre-established distance D and a measured distance between the infrared detector100and one or both of the first netting segment124and the second netting segment126. However, the depth information can also include relative positioning without a precise measurement, for example, a depiction or representation that, e.g., the second netting segment126is in front of the first netting segment124, and both of the first netting segment124and the second netting segment126are in front of the bed122. Because the netting120has holes between strands, objects such as a bed122and a segment of the netting120can be seen through another segment of the netting120. Thus, the depth map can include the netting120as well as depth information for anything behind the netting120, including additional netting120.

The depth map can be assessed for any gaps in the netting120, including the first netting segment124and the second netting segment126. Gaps can include a separation at a seam between the first netting segment124and the second netting segment126, the separation being either in a depth-wise direction with respect to depth D (e.g., a z-direction), or in a direction perpendicular to depth D (e.g., an x-direction or y-direction), or a combination thereof. Gaps can also include holes or tears in the netting120. The assessing can include determining differences in depth between one layer of netting120and a second layer of netting120to determine that there are overlapping layers of netting120with respect to the perspective of the infrared detector100. Such an overlap can include a z-direction, or depth-wise, gap between the layers where there is a depth-wise separation between the two layers of the netting120. The x-direction and y-direction gaps can be assessed by identifying the edges of the both the first netting segment124and the second netting segment126and identifying a space between strands of the first and/or second netting segment124and126is, e.g., large enough for a mosquito to fit through. Such identification can include, e.g., measuring a distance between adjacent strands and comparing the measurement to a threshold value representative of the size of a mosquito.

The results of the depth map assessment can be communicated to a transmitter130to communicate information regarding a gap to a user device. The transmitter130can include, e.g., a wireless transmitter such as, e.g., a Wi-Fi transmitter, a Bluetooth transmitter, a radio frequency (RF) transmitter, a cellular network transmitter, or a wired transmitter such as, e.g., an ethernet connection, an optical fiber transmitter, or any other data communication device.

The transmitter130sends an alert to a user device concerning the identified gaps in the netting120so that the gaps can be closed. The user device can include, e.g., a smartphone232or other text enabled phone, a computer234, or any other device suitable for receiving a message by text, audio, video, images or other format. The alert can include a notice that a gap exists. Additionally, the alert can include, e.g., information regarding the location of the gap, the size of the gap, the type of gap (e.g., separation between netting120segments, tear, hole, separation between the netting120and a floor, or any other type of gap), as well as any other information.

Alternatively, the transmitter130can be a stand-alone alert device or notifier. Rather than communicating the alert to another device electronically, the transmitter130can produce a notification to alert the user via, e.g., a speaker, indicator light, display, or other user perceptible means of alert from the transmitter130itself.

Referring now toFIG. 2, a diagram of types of gaps in netting is shown in accordance with an embodiment of the present invention.

As described above, the netting120can include two or more segments, such as, e.g., a first netting segment124and a second netting segment126. While the first netting segment124and the second netting segment126can be connected at another location, the first netting segment124and the second netting segment126have ends that do not connect. Thus, an opening can be made between the first netting segment124and the second netting segment126such that a person can enter or exit a bounded area such as a bed122. At this area of separation, when the first netting segment124and the second netting segment126are brought together to close the opening, gap132, with corresponding net opening edge134, can form due to inadequate closure of the opening.

According to aspects of the present invention, at least two types of gaps can be identified in the netting120. The gaps can be in any of the x-, y- or z-directions as indicated by the coordinate axes. The x-direction is a horizontal direction perpendicular to the axis of the view of the infrared detector. Similarly, the y-direction is perpendicular to the axis of the view of the infrared detector, only in a vertical direction relative to the axis. The z-direction is parallel to the axis of the view of the infrared detector. Thus, the z-direction is related to the depth in a depth map generated by the infrared detector and is related to a distance from the infrared detector.

Therefore, the first gap132is an illustration of a gap in the x-direction relative to a view from an infrared detector, such as, e.g. the infrared detector100described above. The first gap132can occur, e.g., due to improper closing of an opening between the first netting segment124and the second netting segment126. The first gap132can be measured in width to have a gap size of Gx. Where Gx is greater than a threshold amount, such as, e.g., greater than a size of a mosquito, the first gap132can be identified as a risk of permitting entry of mosquitoes.

The second gap134can be, e.g., a depth-wise gap in the z-direction. Thus, even though the ends of each of the first netting segment124and the second netting segment126extend past each other, the opening may still be present due to improper closing the in the z-direction. Similar to the first gap132, the second gap134can be identified as a risk of permitting the entry of mosquitoes.

Referring now toFIG. 3, a top view from cross-section Z-Z ofFIG. 2is illustrating showing the detection of types of gaps in netting in accordance with an embodiment of the present invention.

This second gap134can be a z-direction, or depth-wise gap relative to the infrared detector100. Thus, the second gap134can include a depth-wise separation of the first netting segment124behind the second netting segment126of Gz. Because the second netting segment126has strands with holes between the strands, the second netting segment126is, in effect, semi-transparent. Thus, the infrared detector100can detect an infrared signal of the strands to the first netting segment124through the holes of the second netting segment126. As a result, a depth of the strands of the first netting segment124and the second netting segment126can be compared to determine the size of Gz. If Gz is greater than a threshold amount, such as, e.g., the size of a mosquito, the second gap134can be identified as a risk of permitting mosquito entry.

However, there can be multiple layers of netting120that do not include depth-wise gaps, such as, e.g., wrinkle136. The wrinkle136in the first netting segment124has three layers of netting120which the infrared detector100can detect as described above in reference to the second gap134. However, the layers of netting120are a result of the netting120of the first netting segment124folding over on itself. The infrared detector100can identify the wrinkle136as a folding of the netting120by, e.g., tracing a strand of the first netting segment124that passes through the area of the wrinkle136in the x-direction and y-direction. By tracing the strand through the area of the wrinkle136, the infrared detector100can determine if the strand passes through the area in each layer of the wrinkle136, or if the strand ends before passing the area in another layer. In the event that the strand does pass through the wrinkle136more than once, then the wrinkle136can be identified as a wrinkle or fold in the first netting segment124to produce the multiple layers of the netting120.

Referring now toFIG. 4, a side view of a netting gap detection device and method is shown in accordance with an embodiment of the present invention.

According to an embodiment of the present invention, an infrared detector100can include a signal processor110, a lens102, an IR beam emitter104and a visible beam projector106. The IR beam emitter104emits an IR signal towards a field of view of the lens102. The IR signal encounters objects within the field of view and the lens102collects reflections of the IR signal reflecting off of the objects. The infrared detector100can have a resolution sufficient to resolve IR reflections from individual strands of netting, such as, e.g., about 13 megapixels or more of resolution at a distance of 2 meters.

The signal processor110can include, e.g., a depth mapper112that can be, e.g., a hardware or software module, such as, e.g., a module stored in a memory and executed by a processor. Because the IR beam emitter104is offset from the lens102, a beam of the IR signal that is emitted from the IR emitter104and reflected back to lens102has an angle between the emitted signal and the reflected signal. Thus, by determining the angle of the reflected signal through the lens102, the depth mapper112can determine a distance of the object reflecting the IR signal. Each reflected IR signal collected by the lens102can, therefore, be analyzed to determine a distance of each object reflecting each corresponding reflected IR signal. Each distance measurement can be aggregated into a map of depths across the field of view of the lens102, thus creating a three-dimensional (3D) representation of the objects in the field of view, such as, e.g., netting and objects in an area bounded by the netting, including, e.g., a bed.

The depth map can be analyzed by an object recognition engine114. The object recognition engine114can include a suitable mechanism for identifying the netting120, the bed122, and any other objects in the field of view of the lens102, such as, e.g., a hardware or software module, including, e.g., a module stored in a memory and executed by a processor. The object recognition engine114can take into account size, shape, distance and spectrum signatures of each object. Thus, a fingerprint for each object can be determined according to the object location, shape and the spectrum of the IR signal reflected back to the lens. For example, the object recognition engine114can identify netting based on the configuration of strands and holes as well as the spectrum of IR light reflected back by the material of the strands. Individual segments of netting can be identified as separate segments based on distance and the configuration of the strands and holes. The object recognition engine114can similarly analyze the IR reflections in the depth map of each of the objects present within the field of view of the lens102.

Relative positions of multiple segments of netting can then be determined to identify gaps with a gap locator116. The gap locator116can be, e.g., a hardware or software module, such as, e.g., a module stored in a memory and executed by a processor. The gap locator116can identify locations of interest in the depth map. The locations of interest can include areas where more than one layer of netting has been identified in a common area projected from a plane parallel to both the x-direction and y-direction, or where no netting exists in an area projected from the plane that is larger than a hole in the netting.

The gap locator116assess the netting at each location of interest to identify a gap. Where no netting exists in an area, the gap locator116identifies the area is between ends of segment of the netting and located in front of an area to be bounded by the netting. Thus, the gap locator116determines whether the location of interest is in a location where netting should be present to prevent mosquito entry into the area to be bounded. If the netting should be present, the gap locator116then identifies the location where no netting exists as a x/y gap. The gap locator116can then determine a gap centroid of the x/y gap.

The gap locator116also assess the netting at a location of interest having multiple layers of netting. Using the depth map, the gap locator116determines a difference in depth between a first layer and one or more subsequent layers at the location of interest. Where the difference in depth is greater than a threshold, such as, e.g., a size of a mosquito, the gap locator116identifies the location as a depth-wise gap.

To prevent false positives in depth-wise gap identification, the gap locator116checks whether the depth-wise gap is the result of separate segments of netting or a wrinkle in a single segment of netting. Thus, a strand tracer117selects a netting strand in one layer of netting that traverses the depth-wise gap. The strand tracer117traces the strand from a location outside of the location of the depth-wise gap until the strand comes to an end or traverses through the depth-wise gap in the other layer or layers of netting present in the location of the depth-wise gap. A strand that traverses the depth-wise gap in more than one layer indicates that the layers are part of a same segment of netting. Thus, the located depth-wise gap is the result of a fold or wrinkle in the netting, and thus a false positive depth-wise gap. However, if the strand ends before traversing the depth-wise gap in the other layer or layers then the layers are separate segments of netting and the depth-wise gap is a true positive. Thus, the gap locator116can accurately determine that a depth-wise gap exists at a location. The gap centroid of the depth-wise gap can then be determined.

An alert generator118can then notify a user of each of the gaps present within the field of view of the infrared detector100. The alert generator118can include a communication to a communication device, such as, e.g., a smartphone, tablet, smartwatch, computer, cell phone, or other device. The communication can include, e.g., a text based notification including a warning that a gap exists and the location of the centroid of each gap. Alternatively or in addition, the notification can include an image of the gaps such that the user can easily find the gaps.

Moreover, the alert generator118can send a signal to the visible beam projector106. The signal can include a command to project a beam in visible light towards the identified gap. Thus, the alert generator118uses the visible beam projector106as a visual alert to a user that facilitates easy discovery of the gaps in the netting.

Referring now toFIG. 5, a block/flow diagram of a system/method to detecting and identifying gaps in netting is shown in accordance with an embodiment of the present invention.

At block501, capture an infrared (IR) image of a patch of netting, the netting bounding an area, such that each of the IR images includes IR data for the patch of the netting.

At block502, construct a depth map from the IR data corresponding to the IR image, the depth map including measurements of a natural distance of the netting.

At block503, identify strands of the netting and background objects captured by each IR sensor and a corresponding side of the netting to distinguish between netting and objects visible through holes in the netting.

At block504, determine a depth difference between netting and background objects, including a depth difference between a first layer of netting and a second layer of netting.

At block505, identify x/y gaps in the netting, the x/y gaps being parallel to a plane formed by the patch of the netting, by identifying a hole between the strands greater than the size of the smallest mosquito of concern.

At block506, identify depth-wise gaps in the netting of size greater than that of the smallest mosquito of concern by identifying first strands of netting having a depth greater than second strands of netting at a same horizontal and vertical location.

At block507, compute a centroid of each of the x/y gaps and the depth-wise gaps.

At block508, send a notification to user identifying a presence of one or more gaps and locations thereof.

Referring now toFIG. 6, a block/flow diagram of a system/method detecting and identifying depth-wise gaps in netting is shown in accordance with an embodiment of the present invention.

At block507, depth-wise gaps in the netting are identified by identifying first strands of netting having a depth greater than second strands of netting at a same horizontal and vertical location. According to an embodiment of the present invention, the depth-wise gaps are identified according to blocks601and602.

At block601, identify a portion of the IR image depicting more than one layer of netting according to at least one strand having a depth measurement different by greater than a threshold from a depth measurement of at least one other strand within a radius of each other in the IR image.

At block602, determine that the more than one layer are separate segments of netting separated by a depth-wise gap by tracing the one or more strands in the portion from end to end to determine that the at least one strand and the at least one other strand are not connected.

Referring now toFIG. 7, an exemplary processing system700to which the present invention may be applied is shown in accordance with one embodiment. For example, the processing system700can be used with or as the signal processor110described above. The processing system700includes at least one processor (CPU)704operatively coupled to other components via a system bus702.

The processing system700includes components for processing an EM signal to determine if gaps are present. A depth mapper707is in communication with bus702to utilize the EM signal with associated depth measurement to generate a depth map. The depth map can be communicated via the bus702to an object recognition engine710that is configured to recognize netting in the depth maps, as well as any nearby objects. A gap locator708can then determine whether the netting identified in the object recognition engine710has any gaps. Identified gaps are communicated to an alert generator709via the bus702. The alert generator702can provide commands to other components of the processing system700to generate an alert regarding the gaps. Each of the depth mapper707, object recognition engine710, gap locator708and alert generator709can be, e.g., separate processors with associated memory to store models or programs for performing the associated functions. Alternatively, each of the depth mapper707, object recognition engine710, gap locator708and alert generator709can be, e.g., stored in memory devices such as read only memory (ROM) and executed by the CPU704.

Also, operatively coupled through the system bus702are a cache106, an input/output (I/O) adapter720, a sound adapter730, a network adapter740, a user interface adapter750, and a display adapter760.

A first storage device722and a second storage device724are operatively coupled to system bus702by the I/O adapter720. The storage devices722and724can be any of a disk storage device (e.g., a magnetic or optical disk storage device), a solid state magnetic device, and so forth. The storage devices722and724can be the same type of storage device or different types of storage devices.

A speaker732is operatively coupled to system bus702by the sound adapter730. A transceiver742is operatively coupled to system bus702by network adapter740. A display device762is operatively coupled to system bus702by display adapter760. The speaker732and/or the display device762can generate a user perceptible alert, such as, e.g., an audible alert from the speaker732or a visual alert produced by the display device762.

A first user input device752, a second user input device754, and a third user input device756are operatively coupled to system bus702by user interface adapter750. The user input devices752,754, and756can be any of a keyboard, a mouse, a keypad, an image capture device, a motion sensing device, a microphone, a device incorporating the functionality of at least two of the preceding devices, and so forth. Of course, other types of input devices can also be used, while maintaining the spirit of the present invention. The user input devices752,754, and756can be the same type of user input device or different types of user input devices. The user input devices752,754, and756are used to input and output information to and from system700.

Of course, the processing system700may also include other elements (not shown), as readily contemplated by one of skill in the art, as well as omit certain elements. For example, various other input devices and/or output devices can be included in processing system700, depending upon the particular implementation of the same, as readily understood by one of ordinary skill in the art. For example, various types of wireless and/or wired input and/or output devices can be used. Moreover, additional processors, controllers, memories, and so forth, in various configurations can also be utilized as readily appreciated by one of ordinary skill in the art. These and other variations of the processing system700are readily contemplated by one of ordinary skill in the art given the teachings of the present invention provided herein.