Visual privacy protection system

A visual privacy system unilaterally blocks rolling shutter cameras by providing a periodically interrupted ambient light source imposing bright and dark bands on the image yet at a frequency imperceptible to human observers in the environment. Communication of the modulation pattern to authorized cameras allows unaffected imaging in these regions by authorized individuals. The system also contemplates operation in a barcoding mode with a light modulation providing embedded barcodes that can be extracted from images taken in the region to indicate that the images were improperly acquired and to block the images in certain applications.

CROSS REFERENCE TO RELATED APPLICATION

BACKGROUND OF THE INVENTION

The present invention relates to a system for controlling the use of cameras to record visual images in given locations and in particular to a privacy protection system that may work with standard rolling shutter cameras.

Electronic cameras are pervasive, being found on common consumer devices including smart phones, tablets, drones, smart glasses, and the like. The ubiquity of these cameras, paired with widely available wireless access, raises significant visual privacy issues including individual privacy, for example, in locker rooms and the like, the prevention of copying of visual works such as art or the like, and the protection of other visual information such as trade secrets.

A variety of privacy enhancing systems have been proposed that require the cameras to sense and abide by signals indicating that the camera is in a “no photography zone” or is focused on a subject that should not be photographed. The signals may be visual, for example, a QRS code. Currently very few, if any, cameras respect such signals, and accordingly these “bilateral” visual privacy systems, requiring coordination between the holder of privacy rights and the camera owner, are limited in use.

A desire to enforce visual privacy against a wide variety at cameras without special modification to those cameras (“unilateral” visual privacy systems) has been addressed, for example, using camera-blinding light sources, for example, in the infrared or near infrared region. Such systems only work if the light sources are in the camera's field of view and therefore special hardware must be placed in close proximity to every object to be visually protected. Such systems are relatively costly when there are many objects to protect, for example, multiple pictures in a museum, or impractical for moving objects or in situations where a premeditated location of camera-blinding lights would be difficult, for example, in a factory.

SUMMARY OF THE INVENTION

The present invention provides a unilateral visual privacy system that can be implemented to protect an arbitrary number of items in a given area at low cost by making use of LED lights that may also provide for area illumination. The LED lights are modulated within a frequency range designed to interact with the rolling shutter electronics of common electronic cameras to “print” obscuring bands or colors over the image. The system can cooperate with authorized cameras allowing them to work freely in the protected area and, in cases where there is substantial ambient light that cannot be controlled, can switch to a “watermarking” mode which embeds a “barcode” in the image that can be detected automatically if the image is posted or published.

Specifically, in one embodiment, the invention provides a visual privacy system providing a privacy-enforced environment having a floor area of at least 100 square feet containing a plurality of LED light fixtures synchronously modulatable to provide at least 50 percent of the ambient illumination of the environment and its contents. A modulator communicates with the light fixtures to synchronously modulate the light of the light fixtures between on and off values at a frequency above the flicker fusion rate to impose a banding in images taken by rolling frame shutter cameras relying on the ambient illumination for image acquisition, the banding completely obscuring portions of the image.

It is thus a feature of at least one embodiment of the invention to provide a “unilateral” privacy enforcement mechanism that does not require coordination with the unauthorized camera and that can operate to protect an entire region.

The modulator may synchronously modulate the light of the light fixtures between on and off values at a frequency in excess of 400 Hz.

It is thus a feature of at least one embodiment of the invention to provide a simple modulation system (binary) that can be readily implemented with existing LED hardware used for ambient lighting without degrading the ambient light experience through perceivable flicker.

The modulator may alternatively or in addition synchronously modulate the light of the light fixtures between on and off values at a frequency of less than 2,000 Hz.

It is thus a feature of at least one embodiment of the invention to provide thick obscuring, bands in an acquired image that prevent ready visual interpolation and thus which substantially obscure the image.

In addition, the modulator may synchronously modulate the light of the light fixtures between on and off values with a duty cycle greater than 40 percent.

It is thus a feature of at least one embodiment of the invention to provide a balance between producing ambient illumination and wide obscuring bars in the image.

The ambient illumination may provide an illumination within the environment of no less than 2000 Lux.

It is thus a feature of at least one embodiment of the invention to provide substantial ambient illumination and to maximize the ability of the invention to block picture taking in the environment between the limits of under and over exposure.

The modulation may vary modulation among different frequencies in a nonrepeating pattern over an interval less than 1 second so that time adjacent different frequencies are not related by an integer multiple.

It is thus a feature of at least one embodiment of the invention to prevent simple defeat of the privacy system of the present invention through adjustment of the exposure of the unauthorized camera. While a given exposure setting may defeat the ability to obscure the image at some modulation frequencies, shifting modulation frequencies makes this exposure adjustment strategy impractical.

The LED light fixtures may provide separate red, green, and blue light sources that are independently modulatable, and the modulator provides different modulation frequencies to the red, green, and blue light sources.

It is thus a feature of at least one embodiment of the invention to separately modulate different color channels to greatly increase the difficulty of defeating the privacy system.

The visual privacy system may include at least one authorized camera positioned to view contents of the environment, the authorized camera intercommunicating with the modulator to exchange a timing signal controlling timing of an image acquisition. The timing signal may synchronize the modulation of the light fixtures and the image acquisition by the authorized camera to eliminate banding in images acquired by the authorized camera. For example, when the authorized camera provides a rolling shutter in which rows of an image are sequentially exposed, the timing signal may allow the authorized camera to coordinate an exposure of each row with a time period to produce an exposure spanning a constant cumulative on-time of the plurality of light fixtures.

It is thus a feature of at least one embodiment of the invention to prevent interference with authorized cameras, for example, security cameras in the environment.

The timing signal may indicate at least one of a frequency and a period of repetition of a modulation pattern or at least one of a beginning of a period of repetition and a phase of a modulation pattern.

It is thus a feature of at least one embodiment of the invention to provide a relatively simple communication between the modulator and the authorized camera (e.g., frequency) requiring only low bandwidth or to permit phase locking of the modulator and authorized camera, for example, when transmitting obscuring data during the interframe blanking period of the camera.

The modulator may further provide a barcoding modulation pattern when an illumination in the environment is below a predetermined percentage, the barcoding modulation pattern imposing a non-obscuring banding in images taken by rolling frame shutter cameras relying on the ambient illumination for image acquisition and encoding a predetermined data pattern.

It is thus a feature of at least one embodiment of the invention to provide “bilateral” type privacy protection when “unilateral” type protection is not available because of adverse environmental conditions.

The predetermined data pattern may be a ratio between a first and second frequency of the barcoding modulation pattern.

It is thus a feature of at least one embodiment of the invention to provide an encoding in an image taken by an unauthorized camera that does not require synchronization with that camera.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring now toFIG. 1, a privacy system10of the present invention may establish visual privacy in a room12or other illuminated area where individuals14carrying unauthorized cameras16may be present. The unauthorized camera16may be a rolling shutter camera employing a CMOS detection element, for example, having a shutter speed of 1/24 of a second or faster and typically on the order of 1/30 of the second. Such an unauthorized camera16may be found, for example, in a standard cellular phone or tablet, and may be expected to be provided in other wearable or easily carried camera systems.

Typically, the room will have an area of greater than 100 square feet and ceiling heights ranging from 8 to 20 feet although no size limitation is contemplated by the present invention so long as an adequate ambient illumination level may be obtained. In this regard, the room will generally be illuminated by a set of light fixtures18to provide a lighting level equal to or greater than that of a standard office (700 Lux) and preferably at a higher level in excess of 2000 Lux to provide the invention with greater efficacy.

The room12may contain contents20including both objects or people for which visual privacy is desired, for example, a painting in a museum where no photograph should be taken, although it is contemplated that the invention may be used in any environment where ambient lighting may be controlled.

Each of the light fixtures18may provide a monochrome (white) LED array or preferably a three-color LED array22providing red, green, and blue LEDs that may be independently modulated by a control circuit24according to a received signal-over-signal channel26as will be discussed below. The control circuit24may provide for controlled current to the LEDs according to techniques well known in the art, for example, for color balancing, as well as to allow the LEDs to be turned to either an on or off state according to the signal channel26. In this regard, the control circuit24may make use of standard commercial technology providing LED control coupled with circuitry for receiving modulation patterns over signal channel26. The invention also contemplates operation in a monochrome mode using LEDs producing white light, for example, with a phosphor and ultraviolet source, as mentioned above.

A synchronizing modulator30may provide a modulation signal over signal channel26to each of the light fixtures18, for example, through a daisy-chained conductor system, carrier current communicating over the light fixture power supply lines, wireless signaling (including radiofrequency and light frequency communication), or the like. The signal-over-signal channel26may simply provide on and off state which the light fixtures18follow or may indicate at a higher level a timing and pattern providing instructions allowing the modulation patterns to be developed in a distributed fashion among each of the light fixtures18. Generally, each of the light fixtures18will be synchronized to provide their respective on and off states at the same time.

The synchronizing modulator30may also communicate with one or more authorized cameras32positioned in the room12and intended to provide for imaging of objects including contents20in the room12without interference. Such authorized cameras32, for example, may be security cameras where unobscured imaging is essential. In this regard, the modulator30may provide a signal-over-signal channel26coordinating acquisition by the authorized camera32and providing either the on and off pattern directly or frequency and phase information necessary for the authorized camera32to coordinate acquisition of visual information without interference. In some embodiments, the signal-over-signal channel26may be identical to that provided over channel26to the light fixtures18. This signal on channel26may be encrypted or otherwise protected against unauthorized interception.

The central modulator30may, for example, provide for a processor36executing a stored program38as will be discussed below. Each of the unauthorized camera16and authorized camera32may provide for a similar computer processing system.

Referring still toFIG. 1, generally an unauthorized camera16held by an individual14will provide for a CMOS array40having light sensors each sensing one pixel of an image. These light sensors will be arranged in rows42and rectilinear columns44and are usually scanned in a row by row basis (termed a rolling shutter) shown by dotted line39in which a first row is exposed and then read out by an associated camera processor48, followed by exposure and read out of the next row and so on. Sequential rows may have periods of overlapping exposure but generally the exposure times of successive rows are increasingly delayed.

The camera processor48will include a memory49holding image files51collected from the CMOS array40and may communicate with the display and user controls or the like (not shown) depending on the camera application.

Referring now toFIGS. 1 and 2, in a simple example, each of the light fixtures18will be illuminated periodically as indicated by the waveform50at a periodicity62during illumination periods52of duration63. These durations63of illumination periods52are adjusted so that the illumination periods52will fall into the exposure windows54of some CMOS rows42and not others. Thus, for example, an exposure window54of a first row42amay occur before an illumination period52athus providing for a low exposure for the light sensors of that row42and a corresponding dark row58ain the resulting image60.

In contrast, a next row42bmay have an exposure window54fully aligned with the illumination period52aproducing a bright row58bin the resulting image60caused by a full exposure of the light sensors of that row42b. A next row42cmay provide an exposure window54partially aligned with illumination period52athereby providing a somewhat darker row58c.

Generally, each of the exposure windows54of succeeding rows42will be staggered in time (either at disjoint times or overlapping times) so as to continue in this pattern to produce a set of dark and light rows58in the image60. The speed of the scanning through the rows42will be such that the exposure of the camera will be essentially constant in this time meaning that the rows58will include over and underexposed regions blocking or significantly reducing information in the image60. The exposure separation in the over and underexposed regions, such as controls the obscuring quality of the bands, may be accentuated through the use of high-intensity LEDs in the light fixtures18. High-intensity LEDs tend to produce overexposure in rows42having alignment of the illumination periods52and exposure windows54by increasing the average exposure of all rows42, and tend to create severe underexposure when the illumination periods52are unaligned with a given exposure window54.

Referring, now toFIGS. 2 and 3, generally when the camera exposure window54is equal to or shorter than the dark time between illumination periods52(periodicity62minus duration63) there will be completely dark bands55. Likewise, depending on the strength of the light during illumination periods52(its peak value and the length of illumination periods52) and the sensitivity of the unauthorized camera16, the space in the image between the dark bands55may also be overexposed providing a set of bright bands57also obscuring data. The invention contemplates that for a given exposure setting of the unauthorized camera16, there will be either completely dark bands55with intelligible image showing between the dark bands55or completely overexposed bands57with the dark bands55revealing some intelligible image, although there can be situations where the image is completely obscured by both bands55and57. The bands55and57are shown vertically reflecting the fact that the rows of the CMOS sensor, and hence the scanning direction, can be oriented arbitrarily.

The bands55and57in the image not only redact image information in the same way that black lines (or erasures) would work in text, but the banding may also upset the camera's autofocus function because of the variations in exposure which upset the focusing algorithm and/or because the sharpness of the bands does not depend on the lens focus (being a time domain phenomenon) suggesting to the autofocus circuit that the camera is in focus even when the image scene is blurred.

In order to maximize the obscuring effect of the dark bands55and57, the periodicity62of the illumination periods52should be much shorter than the response time of a camera auto exposure circuit but long enough to provide a limited number of bands55or57of substantial width within the image60. Larger bands55and57prevent the missing image information from being visually interpolated when some information is visible in the dark bands55or light bands57. Desirably the modulation frequency of the light fixtures18(the inverse of the periodicity62) will be low enough to produce a limited number of bands55and57within a single scan of the CMOS array40but also high enough, to be above the flicker fusion rate typically being at a frequency above 100 Hz. When the duty cycle of the illumination period52is approximately 50 percent, three to ten bands may be desired in an image.

Lower duty cycles of the illumination periods52(duration63divided by periodicity62) provide increased degradation of the image60by generating fully black bands but reduce the ability of the system to provide for overexposure bands57during the on time of the illumination periods52which can also obscure the image. In one embodiment, the invention may adopt a relatively modest duty cycle (between 0.4 and 0.6) and high peak intensity of the light during illumination period52, for example, greater than typical office illumination of 700 Lux and desirably above 2000 Lux.

Desirably the frequency provided by the modulation of the illumination at periodicity62will be such as to produce multiple bands within a single scan of the CMOS array40as well as to be above the flicker fusion rate typically being at a frequency above 100 Hz. This lower frequency bound prevents modulation of the light fixtures18(which also provide ambient illumination) from being visible or distracting to a human observer. The flicker fusion rate is a rate of flicker for a given brightness above which a human observer no longer perceives a flashing. For most people, frequencies above 400 Hz are free from flicker and thus are above the flicker fusion rate. Lower frequencies may be accommodated under particular illumination conditions that may be empirically determined.

Referring again toFIG. 2, it will be appreciated that when exposure window54is long enough to equal an integer multiple of the periodicity62of the illumination periods52, the camera will effectively integrate or average the waveform50and in this way, eliminate the bands55and57preventing the system from obscuring the image60. For this reason, it is possible that a user could adjust the unauthorized camera16to eliminate banding problem, for example, by adjusting the exposure time of that unauthorized camera16to match an integer multiple of the periodicity62, for example, by observing banding in the image and adjusting the exposure setting accordingly.

Referring now toFIG. 4, in order to prevent this circumvention of the present privacy system through simple exposure rate control in the unauthorized camera16, the invention contemplates varying the time periodicity62between illumination periods52to increase the difficulty to the user of an unauthorized camera16in selecting a static exposure rate that will defeat the present invention. In this regard, the modulator30may switch between two (or more) different periods, for example, periodicity62and longer periodicity62′, essentially changing the frequency at which the LEDs of the light fixtures18are illuminated. Ideally these different frequencies are not integer multiples of each other, avoiding the possibility that an exposure time adjusted to be equal to one time periodicity62would likewise defeat the second time periodicity62′.

Preferably, switching between the different periodicities62and62′ occurs at a rate selected to prevent the formation of strong low-frequency side bands that might be below the flicker fusion rate mentioned above. In one embodiment, for example, the modulation rate may be approximately 200 Hz meaning that a new periodicity62is selected every 0.005 seconds. The central modulator30may communicate the new frequencies to each of the light fixtures18or may communicate a schedule to the individual light fixtures18that may then be used to generate the frequency pattern on a distributed basis.

Referring still toFIG. 4, the operation of this frequency switching can be understood by considering, for example, an unauthorized camera16having an exposure window54aat a first time during modulation periodicities62awhere the exposure window54acoincidentally equals an integer multiple of the modulation periodicity62a. In a first time interval (I) during exposure window54a, the unauthorized camera16may receive and integrate light from two illumination periods52indicated by cumulative exposure59which climbs a fixed amount for each illumination period52for each imaging row42, ultimately providing, in this example, an exposure that is neither underexposed (as indicated by level70a) or over exposed (as indicated by level70b) during scanning of a single row42in the image (shown inFIG. 1). By coincidence, this could be an acceptable exposure level on the unauthorized camera16to effectively decode the obscured image. This small exposure window54a, however, when applied to a later modulation periodicity62bat time interval (II) may receive light from no illumination periods5for rows42′ providing no exposure of the resulting image row42′ (at level70a) or underexposure at intervening rows42″. The result is the banding described with respect toFIG. 3with black bands55and a general underexposure of the image60.

Referring still toFIG. 4, conversely, it will be understood that if the unauthorized camera16coincidentally selects an exposure window54cequal to the latter modulation periodicity62b, then during time interval (I) the exposure window54cwill capture so many illumination periods52so as to drive the cumulative exposure59into overexposure level70bas it integrates an additional illumination period52.

Additional robustness to the above-described banding system may be obtained by applying different and independent periodicities62to each different color channel of red, green, and blue of the image60. This approach provides a set of obscuring bands of different colors and may further affect the image by interfering with the color balance system of the unauthorized camera16.

Referring now toFIGS. 1 and 4, the present invention permits an authorized camera32, for example, shown mounted in the room12but conceivably held by another individual, to take images without interference by the system10of the present invention. This may be done most simply when there is a constant periodicity62(in the case ofFIG. 2) by ensuring that the exposure window54of the authorized camera32exactly equals an integer multiple of the periodicity62. This information may be communicated from the modulator30to the authorized camera32either in the form of exposure time value, or start and stop signals having a separation of that time. It will be appreciated that the authorized camera32may conversely control the modulator30to similar effect.

Referring toFIG. 4, in the case of a varying modulation frequency, the authorized camera32may receive a set of different values for periodicity62aand periodicity62bas those periodicities change under control of the modulator30. That is, the authorized camera32may receive periodicity period62aduring time period (I) and set its exposure window54bto equal in length to a predetermined integer multiple of this periodicity62aand may receive periodicity period62bduring time period (II) setting its exposure window54′bequal to the same integer multiple of this periodicity period62b. In both time intervals (I) and (II), the cumulative exposures59′ will be identical. Note that the authorized camera32need not align its exposure window54bwith the periodicity62aand62b(that is, phase alignment is not required) but rather only frequency matching is necessary.

In this way, the authorized camera32may provide for consistently exposed CMOS rows42despite the widely varying illumination frequencies. Again, it will be appreciated that this information can be communicated to the authorized camera32by a variety of means including a wire or wireless, signal from the modulator30or maybe derived by observing the light fixtures18, for example, to count illumination periods52to provide consistent exposure for each CMOS row42.

Referring now toFIG. 5, after the exposure windows54have been completed for each row of the image60there may be a blanking interval76separating a second scan of each of the rows of the image60. This blanking interval76is a normal part of camera operation and is a time during which there are no exposures of any of the rows42and may be used by the camera for processing or data transfer. In one embodiment, the present invention contemplates that during this blanking interval76, the light fixtures18may be activated to provide a random or pseudorandom pulse sequence78(or any arbitrary pulse sequence) matching the duty cycle occurring during regular exposure of the sensor. The cumulative exposure59′ of the authorized camera32will not be affected by this pulse sequence which occurs during its blanking interval76, but an unauthorized camera16out of phase with the authorized camera32will capture these pulses78driving its cumulative exposure59″ into overexposure indicated by the crosshatched portion of the graph of cumulative exposure59″. Coordination of the blanking interval76with the operation of the light fixtures18may be done by communication from the authorized camera32to the central modulator30or vice versa with either device acting as a master.

The invention is ideally suited for a closed environment where 40 percent or more and typically 50 percent or more, or desirably 80 percent or more, of the light in the room12is provided by the light fixtures18. In this way, high contrast between the on and off times of the light fixtures18is ensured. Nevertheless, the invention also contemplates use in environments that may not consistently provide this level of illumination control, for example, because of sunlight coming through windows or auxiliary light sources or the like. In such cases, the modulator30may move from an image blocking mode, in which the images acquired by the unauthorized camera16are blocked through dark or overexposed bands55and57as described above, to a barcode mode where the mechanism of producing the dark and overexposed bands55and57is used to produce low contrast and possibly non-obscuring bands in the image collected by the unauthorized camera16, in this latter mode, the bands55and57form a “barcode” that can be used later, after the images are displayed, to identify pictures taken in restricted areas either for policing or through automatic systems that refuse to display or transmit such pictures.

Referring toFIG. 8, this barcoding may be accomplished in one embodiment through a set of relatively low contrast bars79providing two regions80aand80bdistributed across the image60and corresponding to different times during the acquisition of that image60. The bars79in the region80amay have a first frequency that is in a first fixed frequency ratio to the frequency of the bars in region80b. This first fixed frequency ratio R may uniquely encode the fact that indicates that the image60was taken in a restricted area.

The bars79may be either slightly overexposed or under exposed and may be sensitively recovered from any image60, even in low contrast, through the use of a Fourier transform that averages all columns of data (rows in the depicted image ofFIG. 8) and analyzes frequency content to isolate particular frequencies. In this regard, the bars79may be created through extremely short duty cycles of illumination periods52of the light fixtures18such as enforces a high-frequency content on the image60that may exceed the optical bandwidth of the camera thus being readily distinguishable from the image60itself.

It will be appreciated that multiple different frequency pairs may be used for regions80aand80bencoding the same or a set of frequency ratios Rito provide either greater noise immunity or to encode a series of values, for example, expressing text messages or the like. It will further be understood that the regions80aand80bmay be distributed in any fashion in the image60and thus there is no need to coordinate this encoding with the acquisition timing of the unauthorized camera16. It will further be appreciated that because a ratio is being considered, this information is not lost when the unauthorized camera16employs a range of different exposure times which may affect the physical band size and spacing in the image60but not the frequency ratio.

Additional robustness can be obtained by encoding the same or independent frequencies in each of the different color channels of red, green, and blue of the image by separate modulation of the LEDs of the light fixtures18. Each of these different channels may be individually decoded to determine the necessary values of R or multiple values of R.

Using the barcoding system, the present invention may switch from a unilateral to a bilateral mode of operation that works even when the external light source greatly exceeds the power of the light fixtures18. Such a system could conceivably be used in daylight with a directed beam of light from a fixture18on the object to be barcoded for protective purposes.

Referring now toFIG. 5, the system10ofFIG. 1may operate through a program executed on the modulator30to first check the ambient light as indicated by decision block90. This check may be done through a separate light detector or may make use of a signal from the authorized camera32to determine whether the ambient light is primarily provided by the fixtures18or whether there is additional unmodulated light from an external source that would corrupt the process of providing privacy protection. In this regard, the sensor or authorized camera32may measure light in the room at the time the light fixtures18are turned off. If there is an interfering light source illuminating the room beyond a predetermined value, the latter of which may be determined for each environment, such as would prevent obscuring of images taken by unauthorized camera16in the room12, the system10may move to a barcoding mode.

In the barcoding mode and at process block92, the barcoding frequencies and phase or the like may be provided to the authorized camera32so that the authorized camera32may coordinate acquisitions to eliminate any obscuring barcode in the images acquired. This coordination adjusts the exposure window54of the authorized camera32in a manner described above with respect to obscuring bands55and57. In some embodiments, the barcoding may be faint enough so that this step need not be performed.

After any updating of the authorized camera32, the central modulator30may modulate the light of the fixtures18to multiple frequencies as indicated by process block94and96producing the barcoding bands of regions80aand80bshown inFIG. 8and described above. These frequency pairs may then be cycled with new frequency pairs having the same or different ratios as indicated by process block98and this process repeated indefinitely (so long as the interfering external light signal detected at decision block90is present) to provide for continuous barcoding. The resulting low contrast bars79in the barcoding mode may allow greater than four bits of image information to be communicated through the bar79.

If at decision block90, the ambient light is primarily provided by the fixtures18(for example, in excess of 50 percent) then, again, authorized camera32may be updated with modulation information that will be used for blocking normal photography, as indicated by process block100, so that the authorized camera32can avoid having obscuring bands55and57in its image. Next, the modulation pattern may be imposed on the light fixtures18indicated by process block102as discussed above with respect toFIGS. 2 and 4. At least one of the obscuring bands55and57may prevent more than four bits of image information from being viewed through the bar over an area of at least 20 percent of the image60.

Referring toFIG. 6, the authorized camera32receiving the update information at either process block92or100, as indicated by process block104, may then synchronize its acquisition of image data as discussed with respect toFIG. 4as indicated by process block106.