Patent ID: 12256192

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Before the presently disclosed system is described in further detail, it is to be understood that the invention is not limited to the particular embodiments described, as such may, of course, vary. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting, since the scope of the present invention will be limited only by the appended claims.

Where a range of values is provided, it is understood that each intervening value, to the tenth of the unit of the lower limit unless the context clearly dictates otherwise, between the upper and lower limit of that range and any other stated or intervening value in that stated range is encompassed within the invention. The upper and lower limits of these smaller ranges may independently be included in the smaller ranges is also encompassed within the invention, subject to any specifically excluded limit in the stated range. Where the stated range includes one or both of the limits, ranges excluding either or both of those included limits are also included in the invention.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although any methods and materials similar or equivalent to those described herein can also be used in the practice or testing of the present invention, a limited number of the exemplary methods and materials are described herein.

It must be noted that as used herein and in the appended claims, the singular forms “a”, “an”, and “the” include plural referents unless the context clearly dictates otherwise.

All publications mentioned herein are incorporated herein by reference to disclose and describe the methods and/or materials in connection with which the publications are cited. The publications discussed herein are provided solely for their disclosure prior to the filing date of the present application. Nothing herein is to be construed as an admission that the present invention is not entitled to antedate such publication by virtue of prior invention. Further, the dates of publication provided may be different from the actual publication dates, which may need to be independently confirmed.

FIG.1shows a personally tunable custom audio system101which may be suitable for Adaptive Noise Cancellation. The system may be implemented in a housing102. The housing may be portable and have a clip for attaching to a belt, garment or exercise equipment.

Alternatively, the housing may be integrated with a case for a personal electronic device such as a smartphone or tablet.

The system may be implemented in a personal electronic device such as a smartphone or tablet.

The system may have or be connected to a noise-detecting sensor or microphone110. The sensor may be integrated with the housing or be remote. In the case of a personal electronic device, the system may have a jack103for a remote noise-detecting sensor.

The system may be connected to or integrated with a sound reproduction device such as one or more speakers or headphones. The connection may be by a speaker jack104.

The system may be connected to an audio source, for example, a personal media player such as an MP3 player. The connection may use jack105.

The system may be provided with an on/off switch106and one or more user controls107. The controls may be for one or more channels such as a left channel tune adjustment108and a right channel tune adjustment109. There may be one or more controls for frequency bands per channel. Alternatively, the controls may be for degree in balance in one or more frequency bands.

FIG.2shows an embodiment implemented on a personal electronic device,201, such as a tablet or smartphone. The device may have a touch screen202and a mechanical control203. The device shown inFIG.2may be implemented in an application.FIG.2shows three level sliders204,205and206for three frequency bands for the left channel and three level sliders207,208and209for three frequency bands for the right channel. There is an on/off switch210that is also a touch control. The tablet201may have an on-board microphone211and a stereo headphone jack212. Audio input may be provided by an onboard radio player or an external input.

FIG.3shows an embodiment with a housing301. The housing provided with an input jack302which may be connected to an audio source such as an MP3 player303. The housing301is provided with an audio output jack304. Headphones305may be connected by a cable to the jack304. The housing may be connected to two noise-sensing microphones307and308. The microphones may be hard-wired or connected with a jack.

The microphones307and308may be affixed to the headphone earpieces in a manner to approximate location of the user's ears. The housing may also include a left channel control309, a right channel control310, and an on/off switch311.

The system may be used with or without an audio source. The system may enhance the user's listening experience by reducing the impact of external and ambient noise and sounds when used with an audio source. When used without an audio source, the system still operates to reduce the impact of external sounds and ambient noise.

FIG.4shows a schematic of an embodiment of the custom audio system according to the system which may be an adaptive noise cancellation system.

According to an embodiment of the system, audio is delivered to a user with a perceived reduction of noise. In addition, the audio characteristics may be tailored according to a profile selected by a user, a profile determined by audio analysis, a profile indicated by a non-audio input, and/or a preset profile.

Customized audio according to an embodiment of the system may be implemented by the use of an adaptive filter. The adaptive filter may be hardware or software implemented. A software implementation may be executed using an appropriate processor and advantageously by a digital signal processor (DSP).

An adaptive filter is a filter system that has a transfer function controlled by variable parameters. According to embodiments of the system, an adaptive filter may allow improved control over the adjustment of the parameters.

User controlled adjustment; audio analysis driven adjustment; and/or non-audio analysis driven adjustment may be used to customize audio input. The adjustment types can be used individually, in combination with each other and/or in combination with other types of adjustment.

According to an embodiment illustrated inFIG.4, an adaptive noise cancellation system401may receive a source audio signal402from an audio source403which may provide live or pre-recorded audio. Live audio may be obtained from an audio signal generator or an audio transducer, such as a microphone and analog to digital converter.

The adaptive noise cancellation system may receive an ambient audio signal404from an ambient audio source405.

The ambient audio source may include one or more audio transducers such as a microphone(s) for detecting noise. According to one embodiment, two microphones may be used in positions corresponding to a user's ears. According to a different embodiment, a single microphone may be used. The single microphone may be in or connected to the system housing102, associated with headphones in the form of a headset, or remotely located in a fixed or mobile position.

Alternatively, the ambient audio source may be an artificial source designed to provide a signal that acts as the base of the cancellation.

The active noise reduction system has a control unit406. The control unit406provides parameters which define or influence the transfer function.

FIG.5shows a more detailed illustration of the adaptive filter505and filter control system506. The filter control system506responds to user variable input parameter control501, audio analysis based variable control502, and identification based variable parameter control503.

The filtration control unit504mixes the variable parameters to create an adaptive filter control signal507. The adaptive filter control signal defines the transfer function used by the adaptive filter505.

User-set variable input parameter controls501are useful to tune the transfer function by the user to the preference of the user. The user set variable input parameter controls501may be established to permit the user to select a profile for the transfer function. Various profile controls can be provided to the user. For example, a profile specifically tuned to the environment inside of a passenger train. A profile specifically tuned to the environment in a jet airliner, a profile specifically tuned to the environment inside a subway train. The user adjustable controls may be a single control or multiple controls. They may correlate to conventional audio parameters such as bass, treble, frequency response. The user control parameters may be specifically engineered to modify the response of the adaptive filter according to conventional or non-conventional parameters. The user set variable input parameter controls may be controlled through switches and/or knobs on a connected interface or through a software implemented display interface such as a touchscreen. The touchscreen may be on a dedicated interface device or may be implemented in a personal electronic device such as a smart phone.

Audio analysis based variable controls may be based on a computerized assessment of the ambient audio source signal. The analysis of the ambient source audio may provide input to the filtration control unit504to modify the adaptive filter response based on analysis of background noise and/or dominant noise. For example, the audio analysis may assess the background noise typically present on a city street and the result of that analysis is used to influence the filtration control unit504. The audio analysis may also detect dominant noise, in this example a jackhammer being operated at a construction site, to further influence the filtration control to provide an input to the adaptive filter to compensate for the dominant noise source.

The identification based variable parameter input unit503may provide input to the filtration control unit504to influence the response of the adaptive filter505. Identification based variable parameters are further described in connection withFIG.6.

The environmental identification may be provided in the form of a local radio beacon transmitting identification based variables. The local beacon may be transmitting Bluetooth, Wi-Fi or other radio signals. The identification may also be based on location services such as those available in an iOS or Android device. The available variables are provided to the filtration control unit504which combines or mixes the signals to generate an adaptive filter control signal507. The adaptive filter control signal507is provided to the adaptive filter505and defines the transformation applied to the audio source403.

FIG.6illustrates identification based adaption non-audio-based variable parameter input unit503in order to provide an input to the filtration control unit504. The identification based variable parameter input unit503obtains non-audio environmental identification signals. These non-audio environmental identification signals may serve as an index to noise profile compensation control. The noise profile compensation control may be generic or specific to a particular location. Examples of generic profiles include a passenger train, a bus, a city street, etc. Examples of specific profiles, for example, the main dining in Del Frisco's restaurant in New York City. Or inside of a 1970 Chevelle SS with a well-tuned 396 cubic inch V8 engine.

FIG.7shows an audio customization system. The system includes an audio divider701. The audio divider has one or more audio inputs702. The audio inputs may be digital or analog signals. According to the preferred embodiment, analog signals may be digitized using an analog to digital converter. The analog inputs may be connected to microphones, instruments, pre-recorded audio or one or more audio source inputs like a board feed. The audio divider701may include one or more demultiplexers in order to separate different audio signals on the same input. The audio divider701also includes the capacity to divide input signals into multiple channels, for example, frequency domain channels.

The audio divider701may be implemented in a multi-channel audio processor such as an STA311B available from ST Microelectronics. The STA311B has an automode that may divide an audio signal into eight frequency bands. Audio input signals may be divided, shaped or transferred according to controllable frequency bands or in any other manner that may be accomplished by a digital signal processor or other circuitry. The audio divider may have matrix switching capabilities to allow control of selecting which input(s) is connected to which channel output(s)703.

The audio divider701may be connected to an audio controller704which may dictate the manner in which the audio input signals702are handled. Alternatively, the audio divider701may be static and transform the audio inputs702to channel outputs703according to a predefined scheme. In addition, the audio divider701is connected to a storage unit705which may contain pre-recorded audio or audio profiles. The channel outputs703of the audio divider701are connected to the inputs706of an audio processing unit707. The audio processing unit707is responsive to audio controller704, and contains one or more adaptive filters to combine audio input signals706. The audio controller704dictates which inputs are combined and the manner of combination. The audio processing unit707is connected to a mixing unit708which combines the channel outputs703of the audio processing unit707in a manner dictated by audio controller704. The mixing unit708has one or more audio outputs (709). According to one embodiment, the mixing unit708may have a two-channel output for connection to a headphone (not shown).

Mixing may be accomplished using a digital signal processor. For example, a Cirrus Logic C54700xx Audio-System-on-a-chip (ASOC) processor may be used to mix the outputs710of audio processing unit707.

In practical implementation a single digital signal processor may be used to perform the functions of the audio divider701, audio processing unit707and mixing unit708.

FIG.8shows an illustration of an embodiment of the system.FIG.8Ashows an integrated input/output headset801. The headset may include left speaker802and right speaker803. Speakers802and803may advantageously be connected by a headband804. A microphone array805may be carried on the headband804and may include multiple microphones806. Advantageously, the microphones806are directional.

FIG.8Bshows an alternative embodiment of an input/output unit with microphones806located in a neckpiece housing807and including earphones808.

A third embodiment is illustrated inFIG.8C. Conventional headphones810may be used as an audio output device. A microphone array809carrying a plurality of directional microphones806may be attached to the headband of a headphone810.

FIG.8Dshows an interface with a housing811designed to be connected to a belt or other structure by clip812. The housing811may include one or more microphones806, an input jack813, and an output jack814. The input jack813may be connected to an audio source such as an mp3 player. The output jack814may be connected to speakers, an earphone set or a headphone set.

A further embodiment shown inFIG.8Eincludes a housing815configured for connection to a smartphone such as an iPhone or Android phone. The housing815may be integrated with or connected to a smartphone case. The device shown inFIG.8Emay include one or more sensor microphones806. Advantageously, a plurality of directional microphones may be used. Alternatively, one or more omni-directional microphones may be used. The housing815may have a connector816suitable for electrically connecting the device to a smartphone. In the smartphone embodiment shown inFIG.8E, the smartphone or other portable electronic device (not shown) may include application software operating as a user control. The signal processing capability may be incorporated into the smartphone or be performed by a separate processor located in the housing.

In each of the embodiments8A,8B,8C,8D, and8E, user controls may be provided for in a connected input/output device such as a smartphone or by controls mounted on any of housings805,807,809,811or815. In addition, an audio divider702and mixing unit708may be provided for either within the microphone housings or control unit. In addition, connections between the input/output devices, audio inputs, audio processing unit, and mixing unit may be by wired or wireless connections. The same holds true for the controller and audio divider and/or storage if utilized.

FIG.9A-Gshows alternative aspects of a user control interface for use and connection with the audio optimization system according to the system.

FIG.9Ashows a user control interface useful to control noise cancellation according to direction of noise source.

FIG.9Bshows a user control interface suitable for controlling direction and distance of audio subject to noise cancellation.

FIG.9Cshows a user control interface to facilitate a user capturing audio to serve as a model for enhancement or cancellation. The interface ofFIG.9Bto record a sample audio that is to be exempted from cancellation, enhanced or specifically subject to cancellation. For example, a particular ringtone or alarm may be recorded and stored to serve as a profile to permit the same or similar audio to be transferred to the audio output.

The user control interface may also include controls for channels, volume, bass, treble, midrange, other frequency ranges, selection of cancellation algorithm or profile, selection of enhancement algorithm or profile, feature on/off switches, etc.

FIG.9Dshows a user control interface including a display of a representation of an ambient sound and sliders to change or customize audible parameters in an audio library.

FIG.9Eshows a user control interface designed for microphone selection.

FIG.9Fshows a user control interface including a display allowing selection of distance from ambient sound source and/or microphone array.

FIG.9Gshows a user control interface including a display corresponding to a noise cancellation algorithm and user input controls.

FIG.10shows a system layout according to an embodiment of the system. An adaptive noise controller1001is provided. The adaptive noise controller1001may be connected to a reference microphone array1002and to a set of digital filters1003. The reference microphone array1002may also be connected to the digital filters1003. The digital filters1003may rely on ambient sound profiles stored in an ambient sound library1004also connected to the adaptive noise controller1001. A source signal1005may be connected to digital filters1006which in turn are connected to ambient sound library1004and adaptive noise controller1001. Output devices such as earphone/headphone1007may be connected to the adaptive noise controller1001and may be connected to a speaker driver1008. One or more error microphones1009may be connected to the adaptive noise controller1001and/or the headphone/earphone array1007.

An embodiment of the system may operate to allow a user to select audio received in a headphone. The system may include a programmable audio processor which transmits audio selected by a user to an audio transducer, such as a headphone. The selection of audio can be by audio source and can be particular aspects or portions of an audio signal. It is a recognized problem that when audio is being played through headphones a user can become isolated from his audio environment. Noise canceling headphones designed to increase the perceived quality of audio to a user increase the level of isolation. The embodiment of the system may be designed to allow a user to selectively decrease audio isolation from the user's environment.

The system may include audio profiles that are selected to control customization of audio provided to a user.FIG.11shows a system for management, acquisition and creation of audio profiles for use in customizing audio.

The system may include an audio customization engine1101. One or more audio sources1102may be connected to the audio customization engine1101. The audio sources advantageously include local audio sensor(s) such as one or more microphones or microphone arrays. The system may have microphones to detect local audio which may be used by the audio customization engine1101for active noise control.

One or more active profiles1103may be used by the audio customization engine1101to customize audio signals provided to an audio output device1104, for example, headphones.

A user control interface1105operates with a profile manager1106to designate a set of active profiles. The profile manager1106can assemble audio profiles to be in active profiles1103. The active profiles1103may be from one or more sources. The active profiles1103may include one or more default profile such as car horns or police sirens.

The system may have a user profile storage cache1107containing profiles obtained or generated by a user. Selected audio profiles may be from user profile storage cache1107, may be transferred or copied to the active profiles1103for use by the audio customization engine. Another potential source of audio profiles is library1108. The library1108may contain audio profiles indexed by a directory to allow a user to select an audio profile from a remote source. The library1108may contain profiles for individuals, environments, specified sounds or other audio components.

Audio profiles may also be stored in the contacts for a user or organization. The profile manager1106may access a contacts application to obtain audio profiles contained in a contacts application.

A profile generator1110may be present and connected to profile manager106. The profile generator1110may sample audio from a microphone1111and process the sampled audio to generate an audio profile. The generated profile may be placed directly in the active profiles1103, added to a contact1109or stored in user profile storage cache1107or library1108. The audio profiles may be associated with appropriate metadata to facilitate location, identification and use.

An invitation system1112may be connected to the profile manager1106in order to invite another user or system to provide an audio profile or sample audio to generate a profile. The user control interface1105may control operation of the profile manager1106and audio customization engine1101.

The system described herein may be implemented in a personal electronic device such as a smartphone or tablet. The system may be implemented and computation allocated between server and client devices depending on computational, communications, and power resources available.

The system may have or be connected to one or more microphones or microphone arrays, integrated with the housing of a user device or be remote. In the case of a personal electronic device, the system may have a jack to connect an audio sensor. The system may be connected to or integrated with a sound reproduction device such as one or more speakers or headphones. The connection may be by a speaker jack1104. The system may be connected to an audio source, for example, a personal media player such as an MP3 player. The connection may use jack105.

The system may be provided with an on/off switch and one or more user controls. The controls may be for one or more channels such as a left channel tune adjustment and a right channel tune adjustment. There may be one or more controls for frequency bands per channel. Alternatively, the controls may be for degree in balance in one or more frequency bands. The user controls may be applied to control operations on a server or local operation on a user device.

FIG.12shows a schematic of an embodiment of the custom audio system using an adaptive filter1201as an audio customization engine.

The adaptive filter1201may act on one or more audio input signals1202,1204to condition the audio information for delivery of a modified or customized audio signal to a user. The audio characteristics may be tailored according to a profile selected by a user, a profile determined by audio analysis, a profile indicated by a non-audio input, and/or a preset profile. The adaptive filter may be hardware or software implemented. A software implementation may be executed using an appropriate processor and advantageously by a digital signal processor (DSP). An adaptive filter is a filter system that has a transfer function controlled by variable parameters. An adaptive filter may allow improved control over the adjustment of the parameters.

One or more sources1203,1205may be connected to adaptive filter1201to provide audio signals1202,1204. Audio source1203may be local or remote. Audio source1205may provide local ambient audio information from one or more audio transducers such as microphones or microphone arrays. Other audio sources may be from remote or specialized audio transducers, mp3 or other audio players, or audio streams, or any other audio source.

The adaptive filter1201may be connected and responsive to a control unit206. The control unit1206may provide parameters which define or influence the transfer function executed by the adaptive filter1201.

FIG.13shows an embodiment of an audio customization system1306showing profile manager1304. The profile manager1304may be associated with profiles1301,1302,1303.

The profiles1301,1302, and1303may be mixed and used to control the adaptive filter to create an adaptive filter control signal1307. The profile manager1304may perform this function. The adaptive filter control signal1307defines the transfer function used by the adaptive filter1305. For illustration,FIG.13shows an audio source(s)1308which is representative of one or more audio inputs, including, but not limited to, local microphone(s)/microphone array(s); local audio player; cloud-based audio player; and/or network connected devices etc. The system is not limited by the source(s) or type of source(s). The adaptive filter1305applies the transfer function defined by the profile manager1304to the audio sources1308and outputs to an audio output1309. The mixing function may also be performed in the adaptive filter itself, depending on implementation choices.

FIG.14shows a system layout. An adaptive audio controller1401may be provided. The adaptive audio controller1401may be connected to an audio source(s)1402which may be one or more microphones or other audio sources including an ambient microphone array. The adaptive audio controller may also be connected to a set of active audio profiles1403. The active audio profiles1403may be selected from profiles stored in the sound library1404. The sound library1404may contain audio profiles created by sampling audio information detected by the ambient microphone. If a user wants to establish a profile for certain characteristic audio, the audio may be sampled and characterized in order to create a profile. The sample audio may be used to create an audio profile such as a specific voice, machinery, or other noise. Profiles for a noise, such as a jackhammer or a person the user does not want to hear may be created, as well as profiles to a noise or person the user especially want to hear may be created by isolating and analyzing the specified audio to characterize the audio and establish a profile that can be used by the adaptive audio controller1401, to either enhance or attenuate audio corresponding to the characteristics of the sample.

The adaptive audio controller1401may be implemented in a multi-channel audio processor, a digital signal processor, for example an Audio-System-On-A-Chip (ASOC) processor. The audio processor may have an auto mode that may divide an audio signal into eight frequency bands. Audio input signals may be divided, shaped or transferred according to controllable frequency bands or in any other manner that may be accomplished by a digital signal processor or other circuitry.

The audio divider may be connected to an audio controller implemented by the DSP which may dictate the manner in which the divided audio input signals are handled. The processed audio channels may then be mixed down to a mono or stereo output. The stereo or two-channel output may connect to a headphone.

Output device1407may be connected to the adaptive audio controller1401. The audio source(s)1402may also include one or more error microphones1405for noise detection and cancellation purposes.

The customization may be used and managed in a networked system.FIG.15illustrates an embodiment of a networked communications system for establishing and providing preferred audio. According to an embodiment of the system, a social networking system may be established where members of the network may authorize and/or request access to enhanced communication with others in the network. The communications may occur over a network or may occur in a non-networked fashion, i.e., people talking within “earshot” of each other. One system implementation is shown inFIG.15. The system is managed by a control processor1501. A subscriber interface502may be utilized by the subscriber's or network members. The subscribers may establish a transformation to be used for their own accessible audio. Subscribers may create their own audio profiles. Subscribers may authorize others to include the subscribers in transformations. A network connection1503is illustrated, however, processing and communications resources may suggest whether indicated processes are performed centrally on servers or distributed to user devices.

An audio acquisition system1504may be connected to the control processor1501. The audio acquisition system is used to sample audio. The subscriber interface may include a microphone and a subscriber advantageously will record voice samples which will be processed through the audio acquisition system1504and provided to the profile generation system1505. The profile generation system is utilized to characterize the nature of the acquired audio in order to establish a generalized filter useful for distinguishing audio content having the same characteristics for use in specifying a transformation. Certain audio signals may exhibit characteristic properties which facilitate establishment of a profile for use in transformation. For example, a telephone dial tone may have a particular narrow frequency which could be measured and profiled. The profile would be used in the transformation in order to filter out that particular frequency. Other audio sources are more complex but may still be characterized for filter generation. Complex audio sources such as individual voices will typically require substantial processing, and as such, centralized server processing may be appropriate. Profiles generated by the profile generation system may be stored in a profile library1506. The subscriber interface1502may be utilized to identify and select profiles contained in the profile library for incorporation in a subscriber transformation. Advantageously a profile library may include subscriber profiles and generic profiles which may be useful such as police siren profiles, car horn profiles, alarm profiles, etc.

FIGS.16A,16B, and16Cillustrate operations of an embodiment of the communications system.FIG.16Aillustrates the registration process for the system. Registration is initiated by acquisition operations1601. The acquisition operations acquire information for use in the system for each subscriber. The acquisition process includes acquiring subscriber identification and registering credentials. The acquisition process also involves setting permissions. Setting permissions as a process to establish which subscribers may have access to subscriber profiles. The acquisition process1601also includes acquiring audio samples from the subscriber. Process1602serves to generate an audio profile on the basis of audio acquired in process1601. Process1603generates a subscriber record which includes or links subscriber identifications, subscriber permissions and subscriber audio profiles. Process1604operates to store the subscriber record in a library for use by the subscribers and those authorized by the subscriber.FIG.16Billustrates the configuration operation for subscribers. Configuration is initiated when a subscriber connects and submits acceptable credentials for identification and establishing authorization to access the system. The credentials are submitted and verified at process1605. Process1606illustrates operations to manage profiles. A subscriber, once connected to the configuration system, can manage the profiles which are utilized to generate the subscriber audio transformation. The manage profile operation1606includes search; request authorization; add profiles; and delete profiles. The search function is a mechanism for a subscriber to search for other subscribers and available profiles. The request authorization function may be initiated on the basis of the results of a subscribers search, or on the basis of input on a subscriber identification. The request authorization function initiates an authorization request to another subscriber for access to the other subscriber's audio profile. Once a subscriber has access to the audio profile of another subscriber, the first subscriber may use that audio profile in a transformation to enhance or attenuate audio information having matching characteristics.

The request authorization operation initiates an authorization request to another subscriber. Once that subscriber receives the request, it may be accepted, rejected, or ignored. According to an embodiment, once the request is accepted, the subscriber record of the accepting subscriber is updated to reflect permission granted to the request of the subscriber for use of the audio profile.

The managed profile operation also includes an add profile function whereby a subscriber can select profiles to be activated for that subscriber. Profiles including permissions which are added by a subscriber are then included in the active profiles and utilized to generate a transformation that will be applied to audio information received by that subscriber.

The manage profiles operation1606also includes a delete profiles function. The delete profiles function serves to deactivate and remove a particular profile from the subscriber's active profiles. The update active lists function1607operates to modify the subscriber's active audio profiles in accordance with the add profiles function and delete profiles function of the manage profiles operation1606.

FIG.16Cillustrates the operations function of the communications system. Operations are initiated by acquisition of the subscriber's active profiles1608. Once the active profiles are acquired for a session, the system carries out a configure transformation operation1609. The configure transformation operation1609combines the active profiles into a transformation which may be used by the adaptive audio profiler1401, the adaptive filter1305, or the audio customization engine1101. The system includes a sample audio operation610which advantageously utilizes one or more microphones to “listen” to the ambient environment and may include local or networked audio signals combined with the ambient signals.

One or more of the audio signals are provided to an audio processor which provides the audio transformation1611which is created by the configure transformation operation1609. The transformed audio may be provided to a transducer such as a speaker, and preferably headphones.

The techniques, processes and apparatus described may be utilized to control operation of any device and conserve use of resources based on conditions detected or applicable to the device.

Headphones are a pair of small speakers that are designed to be held in place close to a user's ears. They may be electroacoustic transducers which convert an electrical signal to a corresponding sound in the user's ear. Headphones are designed to allow a single user to listen to an audio source privately, in contrast to a loudspeaker which emits sound into the open air, allowing anyone nearby to listen. Earbuds or earphones are in-ear versions of headphones.

The system may be controlled so that a particular communication station will be in audio communication with one or more other communications stations1701. The control station1702may require permissions from one or more of the communications stations1701to establish and maintain audio communications. The permissions may be designated at a control station1702. Advantageously the control stations1702may be client applications running on a desktop or other computing platform. A user may log into a control station1702in order to manage and control audio communications to stations which the user is authorized to manage.

The control station may be connected by a network1705such as the internet to a connection manager1706. The connection manager1706may contain logic facilitating the identification of audio sources that each communications station has requested. The audio sources may be other subscriber stations which must be set up by their users to authorize communications. In addition, the audio sources may include static audio sources such as radio stations or other broadcast facilities and signaling stations to provide information of a more general interest. Examples of signaling stations may include weather alerts, AMBER alerts, or school closing notifications. A control station1702may be utilized to program the connection manager1706to designate the sources that the communications station1701is requesting.

Each individual computing device may have a physical or logical identification. The physical or logical identifications may be IP addresses, MAC addresses, telephone numbers, user numbers or any other identification token. When the communication manager1706has received sufficient permissions to authorize a communication connection, the connection manager informs the connection matrix1707of the enabled connection. The connection matrix1707is connected to and controls a matrix switching system1708which establishes authorized connections between communications stations1701.

It may be desirable to control the nature of or aspects of audio information which is communicated between communications stations1701.FIG.17illustrates an audio suppression system1709between the communications station1701and the matrix switching system1708. The audio suppression system1709may advantageously be controlled according to instructions from a control station1702provided to a communications manager1706. The communications manager1706may provide control instructions to the audio suppression system1709.

Audio suppression system1709may be in place to attenuate background noise or other portions of the audio information being communicated. Depending on the application, the audio suppression may be applied to inbound communications to a communications station1701or outbound communications from a communication system1701.

The control station1702may be used to populate a communications table1710as shown inFIG.18. The communications table1810may have a set of records1834that include a requesting station ID field1831, a requested station ID field1832and a mutual authorization flag field1811.FIG.19shows an authorization table1912with records containing a transmitting station ID field1933and transmit authorization flag1935. The control station1702may provide an identification of a station and an identification of each station that the station wishes to include in its communications group. The communications table1810may also include a flag field1811to signal a mutual authorization to establish communications. The mutual authorization field1811is activated when a station initiates a communication request to a second station which has previously been authorized by the second station. An authorization table1912may include records identifying communications stations that do not require explicit authorization to establish communications. For example, a radio station could be set up so that it does not require authorization, for example, a subscription-based station. A radio station may also be set up so that it does require authorization. The radio station's subscription management system1913would be responsible for communicating authorized identifications to the communications manager1706.

An entry may be created in a communications table1810when an authorized request is made for a first communications station to be in communication with a second communications station. The entry1834will include the ID of the first station as the requesting station ID1831and the ID of the second station as a requested station ID1832. If an authorized request for the second station to be in communication with the first station had not been previously made an entry is created in the communications table1810, an invitation may be transmitted to the second station to establish communication. If that invitation is accepted, a second entry may be created in the communications table1810indicating the ID of the second station seeking authorization to establish communications. A process may be used to determine when complementary entries exist in the communications table110, and if so, set the authorization flags1811to authorize communications and having an authorized field set.

If a station requests communication authorization with a second station which had previously authorized communication, a record may be entered in the communications table1810indicating the communication pair and setting the authorization flag1811. The communications manager1706identifies all communication pairs which have been mutually authorized either by specific action or by default and places an entry in the connection matrix1707. The connection matrix107controls the matrix switching system1708to establish a communication channel between the stations of the communication pair.

According to an advantageous feature, an address book may be provided in or in connection with each station. The address book may be a personal look-up table to identify a correlation between a user-identifiable information, like a name, and a logical identification like a station identification number.

In this fashion, a system can be established where a group of friends request communications. Each friend can listen in on audio originating from a paired communications station. The friends may modify the authorizations on an ad hoc basis.

According to an advantageous feature, each station may include a communication activation control. In this fashion, the user of each station may control whether the station broadcasts, receives broadcasts, broadcasts and receives or does not broadcast and does not receive. The control interface may be an application.

FIG.20shows an embodiment of a mutual permission audio connection system acting in cooperation with a social networking system. In the embodiment ofFIG.20a mutual permission audio communication system is shown working in connection with a social network platform. An example of an established social network platform is the Facebook platform. The Facebook platform facilitates add-on systems which may take advantage of the Facebook functionality for certain operations such as registration and log-on. In the embodiment illustrated inFIG.20an established social network platform2001may be controlled or operated through a user interface2002. The user interface may include an audio communication control station user interface2003, along with the intrinsic social network user interface2004.

The operation of the communication system may be controlled through an audio communication subsystem2005which may be associated with the established social network platform2001or may be independent, connected through a communications network2008. In either case the audio communication control station user interface2003may be separate from the social network user interface2004, freestanding and connected through communications network2008. Communication stations1701, previously described, may be connected through communications network2008. A connection matrix1707and matrix switching system1708along with audio suppression system1709, all previously described, may also be connected to the communications stations and audio communication subsystem through a communications network2008. The established social network platform2001may be connected to an intrinsic permissioning system2006. The connection manager2007, having the functionality previously described for connection manager1706, may be incorporated in the permissioning system2006of the established social network platform2001, or connected to connection matrix1707.

FIG.21andFIG.22show a pair of headphones with an embodiment of a microphone array.FIG.22shows a top view of a pair of headphones with a microphone array.

The headphones2101may include a headband2102. The headband2102may form an arc which, when in use, sits over the user's head. The headphones2101may also include ear speakers2103and2104connected to the headband2102. The ear speakers2103and2104are colloquially referred to as “cans.” A plurality of microphones2105may be mounted on the headband2102. There may be three or more microphones where at least one of the microphones is not positioned co-linearly with the other two microphones in order to identify azimuth.

The microphones in the microphone array may be mounted such that they are not obstructed by the structure of the headphones or the user's body. Advantageously the microphone array is configured to have a 360-degree field. An obstruction exists when a point in the space around the array is not within the field of sensitivity of at least two microphones in the array. An accelerometer2106may be mounted in an ear speaker housing2103.

FIG.23andFIG.24show a collar-mounted microphone array2301.

FIG.24illustrates the collar-mounted microphone array2301positioned on a user. A collar-band2302adapted to be worn by a user is shown. The collar-band2302is a mounting substrate for a plurality of microphones2303. The microphones2303may be circumferentially-distributed on the collar-band2302, and may have a geometric configuration which may permit the array to have a 360-degree range with no obstructions caused by the collar-band2302or the user. The collar-band2302may also include an accelerometer2304rigidly-mounted on or in the collar band2302.

FIG.25illustrates a hat-mounted microphone array.FIG.25illustrates a hat2501. The hat2501serves as the mounting substrate for a plurality of microphones2502. The microphones2502may be circumferentially-distributed around the hat or on the top of the hat in a fashion that avoids the hat or any body parts from being a significant obstruction to the view of the array. The hat2501may also carry on accelerometer2504. The accelerometer2504may be mounted on a visor2503of the hat2501. The hat mounted array inFIG.25is suitable for a 360-degree view (azimuth), but not necessarily elevation.

FIG.26shows a further embodiment of a microphone array. A substrate is adapted to be mounted on a headband of a set of headphones. The substrate may include three or more microphones2702. A substrate2603may be adapted to be mounted on headphone headband2102. The substrate2603may be connected to the headband2102by mounting legs2604and2605. The mounting legs2604and2605may be resilient in order to absorb vibration induced by the ear speakers and isolate microphones and an accelerometer in the array.

FIG.27shows a top view of a mounting substrate2603. Microphones2702are mounted on the substrate2603. Advantageously an accelerometer2701is also mounted on the substrate2603. The microphones alternatively may be mounted around the rim2604of the substrate2603. According to an embodiment, there may be three microphones2702mounted on the substrate2603where a first microphones is not co-linear with a second and third microphone. Line2705runs through microphone2702B and2702C. As illustrated inFIG.27, the location of microphone2702A is not co-linear with the locations of microphones2702B and2702C as it does not fall on the line defined by the location of microphones2702B and2702C. Microphones2702A,2702B and2702C define a plane. A microphone array of two omni-directional microphones2702B and2702C cannot distinguish between locations2706and2707. The addition of a third microphone2702A may be utilized to differentiate between points equidistant from line2705that fall on a line perpendicular to line2705.

According an advantageous feature, a motion detector such as Gyroscope, and/or a compass may be provided in connection with a microphone array. Because the microphone array is configured to be carried by a person, and because people move, a motion detector may be used to ascertain change in position and/or orientation of the microphone array. It is advantageous that the motion sensor, for example accelerometer, be in a fixed position relative to the microphones502in the array, but need not be directly mounted on a microphone array substrate. An accelerometer304may be mounted on the collar-band2302as illustrated inFIG.24. An accelerometer may be mounted in a fixed position on the hat2501illustrated inFIG.25, for example, on a visor2503. The accelerometer may be mounted in any position. The position2504of the accelerometer is not critical.

FIG.28shows a microphone array2801in an audio source location and isolation system. A beam-forming unit2803is responsive to a microphone array2801. The beamforming unit2803may process the signals from two or more microphones in the microphone array2801to determine the location of an audio source, preferably the location of the audio source relative to the microphone array. A location processor2804may receive location information from the beam-forming system2803. The location information may be provided to a beam-steering unit2805to process the signals obtained from two or more microphones in the microphone array2801to isolate audio emanating from the identified location. A two-dimensional array is generally suitable for identifying an azimuth direction of the source. An accelerometer2806may be mechanically coupled to the microphone array2801. The accelerometer2806may provide information indicative of a change in location or orientation of the microphone array. This information may be provided to the location processor2804and utilized to narrow a location search by eliminating change in the array position and orientation from any adjustment of beam-forming and beam-scanning direction due to change in location of the audio source. The use of an accelerometer to ascertain change in position and/or change in orientation of the microphone array2801may reduce the computational resources required for beam forming and beam scanning.

FIG.29shows a front view of a headphone fitted with a microphone array suitable for sensing audio information to locate an audio object in three-dimensional space.

An azimuthal microphone array2603may be mounted on headphones. An additional microphone array2906may be mounted on ear speaker2103. Microphone array2906may include one or more microphones2702and may be acoustically and/or vibrationally isolated by a damping mount from the earphone housing. According to an embodiment, there may be more than one microphone2702. The microphones may be dispersed in the same configuration illustrated inFIG.27.

A microphone array2907may be mounted on ear speaker2104. Microphone array2907may have the same configuration as microphone array2906.

Microphones may be embedded in the ear speaker housing and the ear speaker housing may also include noise and vibration damping insulation to isolate or insulate the microphone arrays2906and2907from the acoustic transducer in the ear speakers2103and2104.

Three non-co-linear microphones in an array may define a plane. A microphone array that defines a plane may be utilized for source detection according to azimuth, but not according to elevation. At least one additional microphone108may be provided in order to permit source location in three-dimensional space. The microphone108and two other microphones define a second plane that intersects the first plane. The spatial relationship between the microphones defining the two planes is a factor, along with sensitivity, processing accuracy, and distance between the microphones that contributes to the ability to identify an audio source in a three-dimensional space.

In a physical embodiment mounted on headphones, a configuration with microphones on both ear speaker housings reduces interference with location finding caused by the structure of the headphones and the user. Accuracy may be enhanced by providing a plurality of microphones on or in connection with each ear speaker.

FIG.30shows an audio source location tracking and isolation system. The system includes a sensor array3001. Sensor array3001may be stationary. According to a particularly useful embodiment the sensor array3001may be body-mounted or adapted for mobility. The sensor array3001may include a microphone array. The microphone array may have two or more microphones. The sensor array may have three microphones in order to be capable of a 360-degree azimuth range. The sensor array may have four or more microphones in order to have a 360-degree azimuth and an elevation range. The 360-degree azimuth requires that the three microphones be non-co-linear and the elevation-capable array must have at least three non-co-linear microphones defining a first plane and at least three non-co-linear microphones defining a second plane intersecting the first plane provided that two of the three microphones defining the second plane may be two of the three microphones also defining the first plane.

In the event that the sensor array3001is adapted to be portable or mobile, it is advantageous to also include a motion sensor rigidly-linked to the sensor array.

A wide source locating unit3002may be responsive to the sensor array. The wide source locating unit3002is able to detect audio sources and their general vicinities. Advantageously the wide source locating unit3002has a full range of search. The wide source locating unit may be configured to generally identify the direction and/or location of an audio source and record the general location in a location table3003. The system is also provided with a narrow source locating unit3004also connected to sensor array3001. The narrow source locating unit3004operates on the basis of locations previously stored in the location table3003. The narrow source locating unit3004will ascertain a pinpoint location of an audio source in the general vicinity identified by the entries in a location table3003. The pinpoint location may be based on narrow source locations previously stored in the location table or wide source locations previously stored in the location table. The narrow source location identified by the narrow source locating unit3004may be stored in the location table3003and replaced the prior entry that formed a basis for the narrow source locating unit scan. The system may also be provided with a beam steering audio capture unit3005. The beam steering audio capture unit3005responds to the pinpoint location stored in the location table3003. The beam steering audio capture unit3005may be connected to the sensor array3001and captures audio from the pinpoint locations set forth in the location table3003.

The location table may be updated on the basis of new pinpoint locations identified by the narrow source locating unit3004and on the basis of an array displacement compensation unit3006and/or a source movement prediction unit3007. The array displacement compensation unit3006may be responsive to the accelerometer rigidly attached to the sensor array3001. The array displacement compensation unit3006ascertains the change in position and orientation of the sensor array to identify a location compensation parameter. The location compensation parameter may be provided to the location table3003to update the pinpoint location of the audio sources relative to the new position of the sensor array.

Source movement prediction unit3007may also be provided to calculate a location compensation for pinpoint locations stored in the location table. The source movement prediction unit3007can track the interval changes in the pinpoint location of the audio sources identified and tracked by the narrow source locating unit3004as stored in the location table3003. The source movement prediction unit3007may identify a trajectory over time and predict the source location at any given time. The source movement prediction unit3007may operate to update the pinpoint locations in the location table3003.

The audio information captured from the pinpoint location by the beam steering audio capture unit3005may be analyzed in accordance with an instruction stored in the location table3003. Upon establishment of a pinpoint location stored in the location table3003, it may be advantageous to identify the analysis level as gross characterization. The gross characterization unit3008operates to assess the audio sample captured from the pinpoint location using a first set of analysis routines. The first set of analysis routines may be computationally non-intensive routines such as analysis for repetition and frequency band. The analysis may be voice detection, cadence, frequencies, or a beacon. The audio analysis routines will query the gross rules3009. The gross rules may indicate that the audio satisfying the rules is known and should be included in an audio output, known and should be excluded from an audio output or unknown. If the gross rules indicate that the audio is of a known type that should be included in an audio output, the location table is updated and the instruction set to output audio coming from that pinpoint location. If the gross rules indicate that the audio is known and should not be included, the location table may be updated either by deleting the location so as to avoid further pinpoint scans or simply marking the location entry to be ignored for further pinpoint scans.

If the result of the analysis by the gross characterization unit3008and the application of rules3009is of unknown audio type, then the location table3003may be updated with an instruction for multi-channel characterization. Audio captured from a location where the location table3003instruction is for multi-channel analysis, audio may be passed to the multi-channel/multi-domain characterization unit3010. The multi-channel/multi-domain characterization unit3010carries out a second set of audio analysis routines. It is contemplated that the second set of audio analysis routines is more computationally intensive than the first set of audio analysis routines. For this reason, the second set of analysis routines is only performed for locations which the audio has not been successfully identified by the first set of audio analysis routines. The result of the second set of audio analysis routines is applied to the multi-channel/multi-domain rules3011. The rules may indicate that the audio from that source is known and suitable for output, known and unsuitable for output or unknown. If the multi-channel/multi-domain rules indicate that the audio is known and suitable for output, the location table may be updated with an output instruction. If the multi-channel/multi-domain rules indicate that the audio is unknown or known and not suitable for output, then the corresponding entry in the location table is updated to either indicate that the pinpoint location is to be ignored in future scans and captures, or by deletion of the pinpoint location entry.

When the beam steering audio capture unit3005captures audio from a location stored in location table3003and is with an instruction as suitable for output, the captured audio from the beam steering audio capture unit3005is connected to an audio output3012.

As illustrated inFIG.31, the location of microphone2702A is not co-linear with the locations of microphones2702B and2702C as it does not fall on the line defined by the location of microphones2702B and2702C. Microphones2702A,2702B and2702C define a plane. A microphone array of two omni-directional microphones2702B and2702C cannot distinguish between locations2706and2707. The addition of a third microphone2702A may be utilized to differentiate between points equidistant from line2705that fall on a line perpendicular to line2705.

A motion sensor may be provided in connection with a microphone array. The motion sensor may be an accelerometer2701. The motion sensor may include an accelerometer, a gyroscope and/or a magnetometer/compass. A 9-axis motion sensor may be used. Because the microphone array is configured to be carried by a person, and because people move, a motion sensor may be used to ascertain change in position and/or orientation of the microphone array. It is advantageous that the motion sensor be in a fixed position relative to the microphones2702in the array, but need not be directly mounted on a microphone array substrate. A microphone array is useful as an audio sensor capable of multi-directional sensing. Other multi-directional sensors may be used.

FIG.31shows an audio source location tracking and isolation system. The system includes a sensor array3001. Sensor array3001may be stationary. The sensor array3001may also be body-mounted or adapted for mobility. The sensor array3001may include a microphone array or other multi-directional acoustic sensor. The multi-directional acoustic sensor may be two or three dimension capable.

In the event that the sensor array3001is adapted to be portable or mobile, it is advantageous to also include a motion sensor rigidly-linked to the sensor array.

A wide source locating unit3002may be responsive to the sensor array. The wide source locating unit3002is able to detect audio sources and their general vicinities. Advantageously the wide source locating unit3002has a full range of search. The wide source locating unit may be configured to generally identify the direction and/or location of an audio source and record the general location in a location table3003. The system is also provided with a narrow source locating unit3004also connected to sensor array3001. The narrow source locating unit3004operates on the basis of locations previously stored in the location table3003. The narrow source locating unit3004will ascertain a pinpoint location of an audio source in the general vicinity identified by the entries in a location table3003. The pinpoint location may be based on narrow source locations previously stored in the location table or wide source locations previously stored in the location table. The narrow source location identified by the narrow source locating unit3004may be stored in the location table3003and replace the prior entry that formed a basis for the narrow source locating unit scan. The system may also be provided with a beam steering audio capture unit3005. The beam steering audio capture unit3005responds to the pinpoint location stored in the location table3003. The beam steering audio capture unit3005may be connected to the sensor array3001and captures audio from the pinpoint locations set forth in the location table3003.

The location table may be updated on the basis of new pinpoint locations identified by the narrow source locating unit3004and on the basis of an array displacement compensation unit3006and/or a source movement prediction unit3007. The array displacement compensation unit3006may be responsive to the accelerometer rigidly attached to the sensor array3001. The array displacement compensation unit3006ascertains the change in position and orientation of the sensor array to identify a location compensation parameter. The location compensation parameter may be provided to the location table3003to update the pinpoint location of the audio sources relative to the new position of the sensor array. The location table3003output may be used for the directional cues3101stored in the digital audio storage unit3307.

Source movement prediction unit3007may also be provided to calculate a location compensation for pinpoint locations stored in the location table. The source movement prediction unit3007can track the interval changes in the pinpoint location of the audio sources identified and tracked by the narrow source locating unit3004as stored in the location table3003. The source movement prediction unit3007may identify a trajectory over time and predict the source location at any given time. The source movement prediction unit3007may operate to update the pinpoint locations in the location table3003.

The audio information captured from the pinpoint location by the beam steering audio capture unit3005may be analyzed in accordance with an instruction stored in the location table3003. Upon establishment of a pinpoint location stored in the location table3003, it may be advantageous to identify the analysis level as gross characterization. The gross characterization unit3008operates to assess the audio sample captured from the pinpoint location using a first set of analysis routines. The first set of analysis routines may be computationally non-intensive routines such as analysis for repetition and frequency band. The analysis may be voice detection, cadence, frequencies, or a beacon. The audio analysis routines will query the gross rules3009. The gross rules may indicate that the audio satisfying the rules is known and should be included in an audio output, known and should be excluded from an audio output or unknown. If the gross rules indicate that the audio is of a known type that should be included in an audio output, the location table is updated and the instruction set to output audio coming from that pinpoint location. If the gross rules indicate that the audio is known and should not be included, the location table may be updated either by deleting the location so as to avoid further pinpoint scans or simply marking the location entry to be ignored for further pinpoint scans.

If the result of the analysis by the gross characterization unit3008and the application of rules3009is of unknown audio type, then the location table3003may be updated with an instruction for multi-channel characterization. Audio captured from a location where the location table3003instruction is for multi-channel analysis, audio may be passed to the multi-channel/multi-domain characterization unit3010. The multi-channel/multi-domain characterization unit3010carries out a second set of audio analysis routines. It is contemplated that the second set of audio analysis routines is more computationally intensive than the first set of audio analysis routines. For this reason, the second set of analysis routines is only performed for locations which the audio has not been successfully identified by the first set of audio analysis routines. The result of the second set of audio analysis routines is applied to the multi-channel/multi-domain rules3011. The rules may indicate that the audio from that source is known and suitable for output, known and unsuitable for output or unknown. If the multi-channel/multi-domain rules indicate that the audio is known and suitable for output, the location table may be updated with an output instruction. If the multi-channel/multi-domain rules indicate that the audio is unknown or known and not suitable for output, then the corresponding entry in the location table is updated to either indicate that the pinpoint location is to be ignored in future scans and captures, or by deletion of the pinpoint location entry.

When the beam steering audio capture unit3005captures audio from a location stored in location table3003and is with an instruction as suitable for output, the captured audio from the beam steering audio capture unit3005is connected to an audio output3012.

FIG.32shows a pair of headphones with multi-planar multi-directional acoustic sensors such as microphone arrays.FIG.33shows a top view of a substrate with a microphone array which may be part of the headphones ofFIG.32.

The headphones3201may include a headband3202. The headband3202may form an arc which, when in use, sits over the user's head. The headphones3201may also include ear speakers3203and3204connected to the headband3202. The ear speakers3203and3204are colloquially referred to as “cans.”

A substrate is adapted to be mounted on a headband of a set of headphones. The substrate may include three or more microphones3208.

A substrate3205may be adapted to be mounted on headphone headband3202. The substrate3205may be connected to the headband3202by mounting legs3206and3207. The mounting legs3206and3207may be resilient in order to absorb vibration induced by the ear speakers or otherwise and isolate acoustic transducers and an accelerometer. A beacon3216may be mounted on the headphones3201. The beacon may be an acoustic or radio beacon. Acoustic beacons may be audible or inaudible. An inaudible beacon may emit ultrasound. A radio beacon may be a Bluetooth Low Energy (BLE) beacon, for example, according to the iBeacon standard.

FIG.33shows a microphone array3301in an audio source location and isolation system. A beam-forming unit3303is responsive to a microphone array3301. The beamforming unit3303may process the signals from two or more microphones in the microphone array3301to determine the location of an audio source, preferably the location of the audio source relative to the microphone array. A location processor3304may receive location information from the beam-forming system3303. The location information may be provided to a beam-steering unit3305to process the signals obtained from two or more microphones in the microphone array3301to isolate audio emanating from the identified location. A two-dimensional array is generally suitable for identifying an azimuth direction of the source. An accelerometer3306may be mechanically coupled to the microphone array3301. The accelerometer3306may provide information indicative of a change in location or orientation of the microphone array. This information may be provided to the location processor3304and utilized to narrow a location search by eliminating change in the array position and orientation from any adjustment of beam-forming and beam-scanning direction due to change in location of the audio source. The use of an accelerometer to ascertain change in position and/or change in orientation of the microphone array3301may reduce the computational resources required for beam forming and beam scanning.

FIG.34shows an audio source imaging system.

A location table3003as described in connection withFIG.30stores, inter alia, the location of audio sources being tracked by an audio source location system in a format suitable for the audio source location and isolation system. The format of the data indicating relative location stored in location table3003is not suitable for output directly to a display device. A display image translation unit3401is connected to the location table3003. The display image translation unit3401transforms the data contained in location table3003to a format which is suitable for output directly or indirectly to an image display. The display image translation unit3401has an output suitable for use by an image display. The output of the display image translation unit3401is or may be converted in a conventional manner to an image3402referenced to sensor array position. Image3402is particularly suitable for displaying to a user the tracked audio sources from the point of view of the sensor array. The image may be a two-dimensional, a simulated three-dimensional image, or an actually three-dimensional image display. Such images may be suitable to display on a wearable display such as a wrist-mounted display, a Google Glass-style display or any heads-up display.

The images referenced to the sensor array position3402may also be provided to an audio source station translation unit3403. The audio source station translation unit3403may translate the image3402referenced to the sensor array position to an image3404referenced to one of the audio sources tracked in location table3003. The audio source translation station may use a vector inversion process to translate the sensor array referenced image3402to an audio source referenced image3404. For example, the image3402referenced to sensor array position may express the location of each audio source contained in location table3003as a vector with its origin at the sensor array and each source being expressed in terms of a direction and distance. If, for example, the sensor array is located at Point A and the location of an audio source B is identified by direction and distance, for example, the image3402referenced to sensor array position may reflect that audio source B is in the northwest direction at a distance of 20 feet. Audio source translation unit3403may transform the origin of the vector to a location referenced to the location of audio source B. For example, the sensor array would therefore be located 20 feet from audio source B in the southeast direction. This type of translation may be accomplished to translate an image3402referenced to a sensor array position to an image3404referenced to any audio source location contained in location table3003.

According to an alternative or additional feature, the image3402referenced to a sensor array position can be translated to a referenced image3407for any known position. A mapping station translation unit3405may utilize information obtained from an array position sensor3406and the image3402referenced to the sensor array in order to transform the image3402referenced to sensor array to a referenced image3407referenced to any position correlated to a location identified by an array position sensor3406.

Array position sensor3406may utilize transducers in order to identify the position of the sensor array in relation to a known reference point. The position sensor3406may be co-located with the sensor array and may utilize location services or other position sensitive transducers in order to sense the position of the sensor array. The array position sensor may be responsive to a beacon located in a known position. An example of the transformation of an image3402referenced to an array to an image3407referenced to Point O is, the position sensor determines that the sensor array is 10 feet to the west of Point O and determines that the location of audio source B is 20 feet west of the sensor array, then the mapping station translation unit may select Point O as a reference point and determine that the location of audio source B is 30 feet west of Point O. In a similar fashion the mapping station translation unit3405may translate the image3402referenced to the sensor array position to an image3407referenced to any location in a known direction and distance from the origin, Point O.

The image generated by the audio source imaging system may be useful for any application where a particular reference position is desirable. For example, the image reference to the sensor array where the sensor array is mounted on the headband of headphones may be utilized for a heads-up image projection from a wearable display such as a Google Glass-type display unit or as an image for a wrist-mounted display unit. An image referenced to an audio source may be useful for any application where the audio source is the desired point of view. For example, an operative or team member may be outfitted to emit an audio signal as a beacon. The image referenced to the sensor array will include the position of the audio beacon and the audio source station translation unit3403may output the image reference to the audio source to a heads-up display worn or carried by the operative at Location B. In this manner, the operative receives a display of the audio sources being tracked by the location table3003but from its own point of view.

Using the sensor array and known distance between a first sensor location and a second sensor location, the distance to an audio source can be ascertained by one of ordinary skill knowing (i) the angles between a line extending from a first sensor location to a second sensor location (the “base line”), and a line extending from said second sensor location to an audio source, (ii) the angle between a line extending from said first sensor location to the audio source and the base line, and (iii) the distance between the first sensor location and the second sensor location. Because of the inherent nature of sensor elements, beamforming identifies a direction in terms of a range of directions the variations within the range affects accuracy of the determinations. The distance determinations may be enhanced by increasing the distance between the sensor locations. This is done using at least a known distance between sensor locations that is large enough to overcome uncertainty in the distance caused by uncertainty in the directions.

FIG.35shows an adaptive audio spatialization system. The system may be responsive to an audio source3501. The audio source may be live or pre-recorded. Audio from the source may be captured with a multi-directional acoustic sensor, also referred to as a directionally discriminated acoustic sensor. An example of a multi-directional audio sensor is a microphone array. Audio from the audio source3501is processed by the audio spatialization engine3502. The audio spatialization engine may apply a perceived spatial component to the audio obtained from the direction of the source. The application of the perceived spatial component may use head-related transfer functions (HRTF) applied to the audio so that the user perceives the audio source as emanating from the applied direction. The audio spatialization engine3502may be responsive to audio source directional cues3503. The audio source directional cues may be provided on the basis of the relative position of an audio source or on an artificial position or direction. The audio spatialization engine3502may also be responsive to a listener position/orientation unit3503. The listener position/orientation unit3503generates a signal representative of the listener position/orientation and is responsive to a motion sensor3505. The motion sensor3505may advantageously be rigidly linked to the personal audio output device and provides a signal indicative of the position or orientation of a user or changes in the position or orientation of the user. The motion sensor may be one or more of a compass, a gyroscope, and/or an accelerometer. According to one embodiment, a nine-access motion sensor may be utilized.

The audio spatialization engine3502has an output representing a spatialized audio signal. The output is connected to an audio output stage3506. The audio output stage3506may operate as a pre-amplifier and/or amplifier for the audio signal. In addition, the audio output stage3506may mix other audio signals so that audio information from more than one audio source is provided to the personal speakers. The audio source directional cues3503may be a location table as shown inFIG.30.

It is possible that the audio cues provided are not as specific as the location specified by the location table. The reason for this is that the beam steering functionality is optimized by having a very accurate location or direction to isolate. By contrast, in many applications, the precision of the spatialization is less important to a listener than the precision required for optimum beam steering functionality. The use of less precise directionality in the monitoring of user position and orientation and application of spatialization can conserve computational resources and may not be perceptually significant to a user.

The system may be used, for example, amongst a group of people each using a personal communication device linked to a customized audio delivery system in a multifaceted event. In an exemplary environment they may be participating in an event that may be spread across a large geographic area. In other cases participants may be densely assembled. Examples of multifaceted events include, but are not limited to arena venues, festival events, fairs, and conventions/exhibitions. Information may be passed between personal communication devices of the participants using point-to-point wireless communication, a distributed network of computers such as the Internet, a wireless communication network, small cell LTE, Wi-Fi, and so on. In any case, information received at the personal communications devices can include an identification of the event and an indication of available content or identification of one or more other participants possibly according to some specified criteria that can be passed to a participant's personal communication device. The system can be implemented as part of a communication system for establishing and providing preferred audio and/or a mutual permission customized audio source connection system

In the described embodiments, the personal communication device can take the form of a portable media player, cellular phone, or as a handheld computing device such as a tablet computer. In any case, the personal communication device can be configured to wirelessly receive and in some cases may send a signal that can contain information that can include a menu of available content, requests for content and/or communication with or to facilitate communications with other participants and/or event updates or news flashes (announcements). The information can include a snippet or chunk of data that can be broadcasted by one or more devices to other devices that are within the transmission range of the broadcasting device(s). In one embodiment, the snippet or chunk of data can take the form of a token that can be used to seed a group of personal communication devices with the menu of available content. The token can be stored in a personal communication device and concurrently broadcasted to any other personal communication device using, for example, short message service (SMS) messaging or a Wi-Fi RF transmission. In this way, by broadcasting the information, each personal communication device can be made aware of the available content, event updates, and announcements at about the same time.

In the described embodiments, the signal received at the personal communication device can include information other than the available content, event updates, and announcements. Such information can include any personal communication device identifiers, or PCDIDs, indicating the identity of those personal communication devices that have already received the information. In this way, a personal communication device can retrieve not only information related to the available content, event updates, and announcements, but other information related to those personal communication devices participating in the multifaceted event. One of the features of the PCDID is the ability to facilitate social networking within the group. In any case, the unique identifier (including any personalized information associated therewith) can be associated with the PCDID of the personal communication device and be passed between various other personal communication devices. In this way, a dynamic social network can be formed independent of or in conjunction with the available content, event updates, and announcements.

In addition to available content, event updates, and announcements, and any PCDIDs used to identity personal communication devices, the information (or the token for that matter) can include other information such as a time counter used to specify a start time and a stop time for a particular music session.

The menu of available content can be used to select audio content, event updates, and announcements stored or cached on each of the personal communication devices. The selection of available content, event updates, and announcements can be carried out in any number of different ways. For example, one of the ancillary services provided by the communication application can include categorizing content and/or stored on the personal communication device based upon various values of a particular music characteristic or content previously cached or individual identifications of participants. The communication application can create an alert to the presence of other participants selected on the basis of a specified criteria to facilitate ad hoc social networking connection. The criteria may be “fiends” or “contacts” within a certain distance. The criteria may also be based on common interests or other factors or information accessible to the system. The selected information may be prepared for private playing to a user of the personal communication device by way of a private listening accessory, such as headphones. In one embodiment, the music item(s) selected can be added to a playlist for private playing. The playlist can be presented for viewing on the personal communication device and in some cases, made available to the user for manual selection of specific content or connections. It should be noted that the individuals selected can be prequalified according to a specified criterion.

These and other embodiments of an environment where the lighting subsystem may be deployed are discussed below with reference toFIGS.36,37and38. However, those skilled in the art will readily appreciate that the detailed description given herein with respect to these figures is for explanatory purposes only and should not be construed as limiting.

FIG.36shows group3600participating in a multifaceted event. Along the lines of a music festival, group3600can congregate at the event. The congregating can occur in separated areas, for example, at a first stage3620, a second stage3622, a food court exhibition area, etc. The participants can each be apprised of event updates by, for example, SMS messaging, emails (similar to a silent disco), instant messages, or a dedicated communication app such as the aforementioned audio communication or preferred audio systems. An event update might be an announcement that a particular act is about to perform at an identified stage. Each personal communication device (PCD) can privately play content for the associated member of group3600. The member can select the content it will receive. By privately playing it is meant that only the member in possession of the personal communication device can hear the privately played content. This audio privacy can be accomplished using private listening accessory3602along the lines of a head phone, ear bud, and so on. The members may be listening to the same content broadcast, or listening to customized and/or selected content. The lighting display may be correlated to the selected content.

The members may be listening to the same content broadcast, or listening to customized and/or selected content.

In order to participate in the multifaceted event communications, each of PCD3614-PCD3618must include communications infrastructure and a control interface to select and play appropriate content. In order to assure that each of the personal communication devices in group3600has access to the content, a communication application (not shown) can be provided and stored on each of the personal communication devices. In one embodiment, the communication application can be part of an operating system provided upon the original purchase of a personal communication device. Alternatively, the communication application can be obtained after-market using, for example, remote media management services along the lines of iTunes. On the other hand, the communication application can be obtained in an ad hoc manner during, for example, an initial invitation session whereby part of an individual acceptance of an invitation to participate in the shared music session (using email, SMS messaging, Facebook, and so on) involves downloading and installing the communication application with a subsequent verification and acceptance.

In some cases, the system may communicate over an ad hoc P2P network, or by direct by broadcast3640communications. It should be noted that broadcast3640can take the form of a wireless RF transmission using any number and combination of available wireless protocols. For example, broadcast3640can take the form of conventional over the air (OTA) AM or FM broadcast in which case the user can be instructed to manually input the appropriate tuning instruction to their respective personal communication device. Alternatively, broadcast200can take the form of a Wi-Fi or Bluetooth RF signal that the communication application can recognize as including the updated music characteristic information.

If the system utilizes an ad hoc P2P network a limited number of members of group3600(referred to as initiators) can be identified to seed the P2P network with announcements or a menu of available content. For a more detailed description of the heuristics of distributing information in an ad hoc P2P network please refer to “On Disseminating Information Reliably Without Broadcasting”, Proc. 7thInt. Conf. on Distributed Computing Systems (ICDCS-7), pp. 74-81 Berlin, September 1987 by Alon, N., Barak, A. and Manber, U and “An Asynchronous Algorithm for Scattering Information Between the Active Nodes of a Multicomputer System”, Journal of Parallel and Distributed Computing, Vol. 3, No. 3, pp. 344-351, September 1986 by Drezner, Z. and Barak each incorporated by reference in their entireties. Assuming that member3606has been designated as an initiator, member3606can seed ad hoc P2P network with the event information. Member3606may be replaced by an initiation server acting as a control station.

It is foreseeable that due to local conditions, it may not be possible to reliably send information from one node directly to another node in P2P network. For example, PCD3614belonging to member3606(initiator) can broadcast token T that can be received by PCD3612and PCD3616belonging to members3604and3608, respectively. However, member3610may be too far away or may be in an area (such as behind a wall) where direct reception by PCD3618is unlikely. Therefore, each node of network can be instructed to retransmit the information wirelessly upon receiving information wirelessly. For example, when PCD3616(as well as PCD3612) wirelessly receives the event information each can generate re-broadcast a signal that includes the event information received from member3606. In this way, PCD3618can receive re-broadcast content information from PCD3616(as well as that from PCD3612).

In some cases, a multifaceted event can have session rules. The session rules can define various relationships and actions that can occur between the members of the group during a specific session. For example, the session rules can provide criteria for identifying networking proposals for individual members to connect during the session. In this way, by setting the session networking rules individual members can be identified to each other and establish social networking communications.

FIG.37shows a block diagram of a representative personal communication device (PCD)3700in accordance with the described embodiments. PCD3700can be formed to include at least housing3702configured to enclose and support various operational circuits. In some cases, PCD3700can include controller3704used to control data storage device206that can be used for storing a plurality of data files that can take the form of, for example, audio data, textual data, graphical data, image data, video data and multimedia data. The stored data files can be encoded either before or after being stored using a variety of compression algorithms. It should be noted that a user can interact with manager3712through an interface. For example, audio content can be compressed using MP3, AAC and Apple Lossless compression protocols. Other data may be compressed using protocols appropriate to such data. The audio content can include, for example, auxiliary content files3708stored in memory510controlled by the content manager3712. Content manager3712can be embodied as software executed by processor3714or as a separate hardware component. In any case, content manager3712can control the audio output of content files3708stored in memory3710. The content may also include available content menus, in audio or graphic form as well as social networking criteria and/or identification.

During operation, for example, content manager3712can select content item3716from auxiliary content3708which can be decoded using an appropriate codec. The decoded content file can then by output as audio signal3718to audio output interface3720. In accordance with one embodiment, content manager3712can select content items3716identified by a user through a guide or by voice command. Furthermore, content manager3712may receive transmission of content and play such content substantially in real time, subject to loading, buffering and decoding delays and subject to any user control such as pause or rewind or replay.

Content may include a tag3722to identify content type or other characteristic of the auxiliary content. For example, in a music festival the tag may indicate that the content is a commercial advertisement or offer. The tag may indicate information regarding purchase of the content, or may identify the facet of the multifaceted event that the content relates to. For example, the tag may indicate that the content relates to a performance on stage.

User input interface3724can assist a user of PCD3700in controlling various functions performed by PCD3700. For example, user interface3724can include a touch sensitive layer (not shown) that can facilitate the use of a user touch event for inputting control instructions or the user interface may be an audio interface for voice commands. In the case where PCD3700includes speakers, then audio signal3718can be broadcast to the external environment via the speakers. However, in those situations where PCD3700does not include speakers, or the speakers can be bypassed, PCD3700can include private listening interface3726suitable for directing audio signal3718to an external transducer associated with a personal listening accessory, such as earphones, ear buds, and so on. The personal/listening device may also include a microphone for detecting and sensing audio. In this way, the user of PCD3700can privately listen to audio output by music manager3712. PCD3700can also include wireless interface3728arranged to both receive and transmit information by way of any suitable wireless protocol such as, for example, Wi-Fi, Bluetooth, and so on capable of accessing various configurations of wireless networks, such as WLAN or peer to peer (P2P). It should be noted that even though only a limited set of components are shown this does not imply a limitation on the functional components that can be included in PCD3700. For example, in addition to the components shown inFIG.37, embodiments of PCD3700can also include a power connector, a data transfer component, voice recognition circuits, and so on.

Content manager3712can customize the audio experience of the user. The audio may be processed to enhance and/or mask aspects of the audio to be delivered to the user, for example, in accordance with the techniques described herein.

In another implementation, content manager3712can control social networking functionality. Selective networking may be provided by identifying participants in the event that satisfy a selection criteria. The system may allow a user the option of establishing networking communications with other participants who satisfy the selection criteria and designated by one or both users.

A communication application3728can provide instructions executable by processor3714for controlling the operations of PCD200. In the described embodiment, the communication application can be downloaded from an online data store automatically or as a result of a user selection at user interface3724from a central media management application (such as iTunes™) or from Apps Store maintained by Apple Inc. Alternatively, communication application3728can be present at the time of original purchase. In any case, communication application3728maintains a connection table to be periodically updated. The updating can occur, for example, during a synchronization operation performed between PCD3700and a central media management application (such as iTunes™). The updating can also occur on an ad hoc basis.

Communication application3728can provide a mechanism by which a user of PCD3700can participate in a social networking experience provided that a connection between two users satisfies a criteria identifying a suggested connection. In addition to providing services required for participation in the social networking experience, communication application3728can provide PCD3700with at least the appropriate network protocols required to exchange information with other personal communication devices in a P2P network. In addition to providing the requisite communication protocols, communication application3728can provide services related to categorizing music items stored on PCD3700based upon various values of a particular music characteristic. The selection and networking function can be based in or distributed among PCDs or be server based. In a server-based system, the server may be local (logically) to the multifaceted event or remote such as a server connected through a wide area network including, without limitation, the Internet.

In any case, PCD3700can obtain a connection token T by way of RF transmission3730. It should be noted that if PCD3700is a node in a P2P network, RF transmission3730can originate from another personal communication device within the network. In this situation, upon receiving token T, PCD3700can generate re-broadcast signal3732that includes at least token T while storing only tokens designated for that user. In this way, other personal communication devices with the P2P network can receive connection tokens applicable to other devices. Tokens can be transmitted by way of RF transmission3730that originates from a central broadcaster unit. It is also possible that PCD3700does not have wireless capabilities, in which case the token T can be provided by the communication application3728. In this way, a more limited session can be held since only those personal communication devices that have the same version of communication application3728can participate. For example, in order to participate, PCD3700may require the latest version of token T which can be obtained during, for example, a synchronization operation performed between the personal communication device and a central media management application.

Once token T has been received, processor3714can determine if token T has an indication of supplemental content. For example, token T can indicate availability of content which might be background information, coupon or commercial offers, or schedules. In this case, the user may have the option to listen to the supplemental content which may be requested or accessed and can be privately played by PCD3700. Accordingly, content3730,3732, and3734each tagged as an ID that corresponds to token t1 may be accessed. In the described embodiment, a content venue3736can be visually displayed at interface3724.

FIG.38shows an event-centric networking matching system3800. The system includes a connection server3801connected to a plurality of user personal communication devices3802by a network3803. The personal communication devices3802may have an interface for users to control, provide instructions, and provide information to the system. Alternatively, the instruction and information interface may be a separate terminal also connected to the network3803. The network3803may be a wired or wireless local area network or wide area network. The connections may be by Bluetooth, peer-to-peer connections, small cell LTE or any other connection mechanism. The system is not specific to a particular network. The communication server3801may be connected to data store3804.

FIG.38illustrates a single data store3804in the form of a database management system however individual tables or distributed tables may be utilized. The data may be distributed among the users3802or centrally located. The data may include user profile data3805composed of a user ID3809associated with a profile3810. The profile may include any information used by the system related to the user, for example, user name, password, gender, musical tastes, playlist, age, geographic location and any other demographic information. The system may also include a matching criteria table3806. The criteria table may include a plurality of rules3811, each associated with a rule number3812. In addition, the system may include a participation table3807which includes a user ID3813as an index and a rule number3814correlating to rule numbers3812of the matching criteria table3806. The participation table3807includes a list of user IDs correlated to the rule numbers and the matching criteria table3806includes those rule numbers correlated to matching criteria. Each user may be subscribed to one or more of the criteria as indicated by entries in the participation table3807. The matching criteria may include one or more requirements such as an identification of an event, a location service matching criteria, demographic matching criteria, a flag indicating appearance in a contact or approved list, and other criteria. In the example of a multi-faceted event such as a concert festival, the system may first identify all users who are participating in the event, i.e. are attending the music festival. This may be accomplished by determining which users have purchased tickets or have a token on their PCD indicating they have been admitted to the event. Alternatively, participation may be determined by location services. Each user may establish or subscribe to criteria which, if satisfied, suggests a connection. A matched status connection table3808may be established in order to identify connections approved in accordance with the proper operation of the system. The system may go through each entry in participation table3807. For each entry the rule corresponding to the user ID may be utilized to evaluate all of the entries in the user profile table. When an entry in the user profile table satisfies a user ID rule designation, an entry may be placed in the matched status connection table3808of the user ID in the user1field3815. The ID of the user who satisfied the criteria may be placed in user2field3816. The system may use different logic or sequences, but the idea is to create a table which has an entry for each pair of users who both satisfy the other's designated criteria. The designated criteria may be customized by each user and/or established by the system. An additional feature may permit each participant in a connection to approve or deny access even though the established criteria have been satisfied. Alternatively, one of the criteria may be approval of the matching user.

The system may also be able to establish communication groups so that connections may be one-to-many or even one-to-all. This may be established by user ID corresponding to a group criteria and each individual user who matches the group criteria is connected in the group. The system may impose an artificial limitation of allowing participation in only a single group.

FIG.39shows an audio play system3901. The audio play system3901has an output representative of one or more aspects of the audio selection. A display attribute generation unit3903may be provided and is responsive to the signal representative of content3902. The customized audio play system3901may be connected to personal audio speakers3906. The personal audio speakers3906may be headphones, earphones, or any other device for converting electrical signals to audio.

The display3905may constitute one or more light elements. The light elements may be LED light elements or any other light emitting element. The display3905may be monochrome or controllable to vary the color, intensity, and image of the lighting output. The display3905may have one or more color points such as the Pixmob or Xyloband displays. The display3905may be suitable to display image or video. The display3905may be mounted on a headphone or may be wearable in some other fashion, although it is not necessary for the display3905to be mounted on or even co-located with a user. The signal representative of content3902must be derived in part from the operational parameters of the customized audio play system. While the display3905may in part be controlled by audio intensity in the fashion of a light organ, the signal representative of content must include, in part, a signal representative of operating parameters. The operating parameters may include audio source selection, non-audio control signals, user-selected parameters, system-selected parameters, content-type parameters or other non-audio parameters.

A display attribute generation unit3903may be provided to generate signals to be displayed. Those signals may be provided to the display driver3904.

As an example, the light display system might be utilized in connection with a system shown inFIGS.36-38for a multi-stage concert event. In such a multi-stage concert event, each user may customize the audio being provided to a headphone by selection of one stage to be included in the user's customized audio. The light attribute to be displayed will in some way correspond the selected stage. For example, a country music stage may be designated by the color red, a rock and roll stage may be designated by the color white, and a techno stage may be designated by blue. When a user selects which stage to include in a customized audio feed, the display3905may be illuminated with the corresponding color.

The invention is described in detail with respect to preferred embodiments, and it will now be apparent from the foregoing to those skilled in the art that changes and modifications may be made without departing from the invention in its broader aspects, and the invention, therefore, as defined in the claims, is intended to cover all such changes and modifications that fall within the true spirit of the invention. For the sake of clarity, D/A and A/D conversions and specification of hardware or software driven processing may not be specified if it is well understood by those of ordinary skill in the art. The scope of the disclosures should be understood to include analog processing and/or digital processing and hardware and/or software driven components

Thus, specific apparatus for and methods of a customized audio display system have been disclosed. It should be apparent, however, to those skilled in the art that many more modifications besides those already described are possible without departing from the inventive concepts herein. The inventive subject matter, therefore, is not to be restricted except in the spirit of the disclosure. Moreover, in interpreting the disclosure, all terms should be interpreted in the broadest possible manner consistent with the context. In particular, the terms “comprises” and “comprising” should be interpreted as referring to elements, components, or steps in a non-exclusive manner, indicating that the referenced elements, components, or steps may be present, or utilized, or combined with other elements, components, or steps that are not expressly referenced.

FIG.40shows a schematic of a narrowcast messaging system. In particular,FIG.40illustrates the receiver side of the messaging system. A transducer4001is provided to convert acoustic signals to electrical signals. The transducer ofFIG.40may be suitable for detecting acoustic waves in the ultrasound frequency band. The transducer may also be suitable to detect acoustic signals in the audible frequency range. The transducer4001may be connected to an ultrasound isolation unit4002. The ultrasound isolation unit4002may be responsive to a channel control unit4003. The channel control unit4003may be responsive to a permissioning subsystem4004. A frequency transposition unit4005may be responsive to the ultrasound isolation unit4002and the channel control unit4003. The frequency transposition unit4005may have an output of an electrical signal corresponding to audio information. The audio information may be provided to an audio signal processing unit4006.

The audio signal processing unit4006may be provided to output audio information to a user. In one embodiment the audio signal processing unit may be a preamp connected to a speaker such as an earphone or headphone. In another embodiment the audio signal processing may be an audio customization unit.

In operation, an ultrasonic beacon system may be provided. An example of a beacon system is the iBeacon compatible transmitters. See https://developer.apple.com/iBeacon/. The Apple iBeacon system use Bluetooth LE. A beacon system may include an ultrasonic transmitter. Beacons, such as the iBeacon have localized transmission and are designed to assist in determining proximity of a receiving device to the beacon.

A drawback to a proximity sensing system is that it can only determine proximity to a particular beacon and to some extent distance from a particular beacon. The beacon may be designed to work with a directional sensing audio receiver.

An embodiment may include a microphone array having two or more spaced microphones. The microphones may receive the signal emitted by a beacon and determine the direction to that beacon. The direction may be represented in the form a vector. One or more additional beacons may be provided to facilitate the direction-sensing microphone to identify one or more vectors indicating the direction of the one or more additional beacons.

FIG.41shows a location generation unit4101which may be used with the narrowcast messaging system andFIG.42shows an embodiment of a location generation system which may be utilized. A position map4201may be a digital representation of the absolute or relative locations of two or more beacons.

A directionally discriminating acoustic sensor4202may be connected to a directional vector generation unit4203. The directional vector generation unit4203may operate to determine the direction of a beacon4204relative to the acoustic sensor4202. The directional vector generation unit4203may also determine a vector representing the direction of a second beacon4205relative to the acoustic sensor4202which may be a microphone array. A position processor4206may be responsive to the position map4201and the directional vector generation unit4203. The position map is a digital representation of information sufficient to specify the relative positioning of beacons4204and4205. The relative positioning of the beacons and directionality of the beacons relative to the directionally discriminating acoustic sensor4202is sufficient to determine the location of the array relative to the beacons. In addition, if the absolute position of one or more of the beacons is known the relative location of the array is sufficient to determine the absolute location of the array. A rule set4102may be responsive to the location generation unit4101and a user ID4103corresponding to the sensor4202. The location generation unit4101as described in connection withFIG.42may base the location, in part, on information reflecting the site location4104and a site identification4105.

The rule set4102includes logic that facilitates generation of a channel ID4106. The channel ID represents content or instructions to be played or executed by a personal communication device on the basis of the location of sensor4202coinciding with a designated location subject to qualifications (contingencies) as applied by the rule sets4102. The channel control unit4003may provide the channel ID4106to the ultrasound identification unit4002and the frequency transposition unit4005.

In operation, a user wearing or carrying a microphone array, may obtain transmissions of selected information based upon positioning in or traversal of a beacon field. One example of a beacon field may be installed in a retail department store. As the array moves through the department store the system facilitates determining the precise location of the array. iBeacon technology determines proximity and utilizes signal strength to infer some measure of confidence and distance. An iBeacon has no directional sensitivity. Thus, if an iBeacon infers a distance of 3 meters, the sensor is inferred to lie on the circumference of a circle that is 6 meters in diameter. An iBeacon is unable to determine if the device is at an exact position of interest or up to six meters away. The location may be utilized along with other parameters such as user preferences and system preferences to determine what information to provide to a user. For example, a user may select to enable messaging for special offers related to a particular type of product, for example, men's clothing. The retail outlet may establish a message that communicates a special offer for certain golf shirt. As the microphone array reaches a predetermined location, which may be a location immediately adjacent to the golf shirt, the system may communicate a special offer to the user triggered by being in that location. The message may be a promotional offer for the nearby golf shirt, for example, other types of offers may also be suitable such as a promotional offer for a golfing vacation package or a promotional offer for a different related or unrelated product. The position in this example is important as the message may not be relevant to a position up to 6 meters away.

Having determined the position of an array and permissioning for a particular message, the message may be transmitted to the user. It is desirable to have the ability to restrict the message to the individual user. One embodiment is the transmission of an inaudible ultrasonic wave containing the message. Various mechanisms can be provided to allow the user to receive and isolate an ultrasonic transmission. For example, the user system may be informed of the direction of the ultrasonic transmission source relative to the microphone array. The microphone array may use beamforming techniques to isolate that direction.

Another embodiment may provide for multi-channel ultrasonic transmissions. The transmission information may be modulated at different frequencies or may be provided in a specified frequency band. The isolation system may be provided to isolate the modulated transmission on the basis of its modulation frequency or filter communications outside of the specified frequency band.

Once the desired ultrasonic frequency is received and isolated, it remains an inaudible signal. The inaudible signal may be subject to frequency transposition converting the signal from an inaudible frequency to an audible frequency, for example, a frequency in the voice band. In this manner a personalized narrowcast message may be transmitted to a user on the basis of being in or having been in a particular location.

FIG.43shows a multi-directional acoustic sensor integrated into a ski helmet4300. Multi-directional acoustic sensors may be similarly integrated into other types of headgear, particularly protective headgear. For example, but without limitation, construction hardhats, bicycle helmets, football helmets, hockey helmets, skateboarding helmets, batting helmets, combat helmets, or any other kind of protective headgear. The elements described herein may be integrated directly into the outer surface of protective headgear integrated into a shell attached to the protective headgear.

The headgear may include a plurality of microphones4301mounted onto a surface of the headgear4300. Because of the typical dimensions of protective headgear it is possible to position microphone element4301at a greater distance from each other than microphone elements integrated into the headband of a pair of headphones. The accuracy of the sensing array is dependent in part upon the distance between the microphone elements, and as such implementation of a multi-directional acoustic sensor on protective headgear may enhance the accuracy of the directional location and isolation.

One or more additional microphone elements4302may be attached to the protective headgear4300at a position that is not coplanar with microphone element4301. Advantageously, microphone element4301may be positioned around the crown of the headgear and additional microphones4302may be positioned at a location corresponding to a wearer's ears or lower. The protective headgear4300may also be provided with a motion sensor4303. The location of the motion sensor is not critical.

The protective headgear4300may also be provided with an ultrasonic transmitter4304. The ultrasonic transmitter4304is useful to generate an ultrasound signal operating as a beacon. The ultrasound signal may be inaudible and may also be coded for identification purposes. In an alternative configuration, an audible acoustic transmitter or radio frequency transmitter, such as an iBeacon or other BLE beacon may be used. The transmitter facilitates identification and location of the protective headgear.

FIGS.44A and44Bshow a multi-directional acoustic sensor integrated into a ski jacket4400. Multi-directional acoustic sensors may be similarly integrated into other types of outerwear, particularly activewear. For example, but without limitation, ski jackets, sports jerseys, jumpsuits, flack jackets, biker jackets, bomber jackets, dusters, water ski vests, live preservers, or any other garment to be worn on a torso. The acoustic sensor elements described herein may be integrated directly into the outer surface of the outerwear or integrated into a shell worn over the outerwear.

The jacket may include a plurality of microphones4401mounted onto a surface of the jacket4400. Because of the typical dimensions of outerwear, it is possible to position microphone element4401at a greater distance from each other than microphone elements integrated into the headband of a pair of headphones. The accuracy of the sensing array is dependent in part upon the distance between the microphone elements, and as such implementation of a multi-directional acoustic sensor on outerwear may enhance the accuracy of the directional location and isolation. Microphone element4401may be positioned directly on the jacket4400or microphone elements4401may be positioned on a base4405attached by a fastener4406. The fastener4406may be hook and loop buttons, snaps, or other fasteners.

One or more additional microphone elements4402may be attached to the jacket4400at a position that is not coplanar with microphone element4401. Advantageously, microphone element4401may be positioned on the shoulders or around the collar and neckline and additional microphones4402may be positioned at a location lower than the microphone elements4401. The jacket4400may also be provided with a motion sensor4403. The location of the motion sensor is not critical.

The jacket4400may also be provided with an ultrasonic transmitter4404. The ultrasonic transmitter4404is useful to generate an ultrasound signal operating as a beacon. The ultrasound signal may be inaudible and may also be coded for identification purposes. In an alternative configuration, an audible acoustic transmitter or radio frequency transmitter, such as an iBeacon or other BLE beacon may be used. The transmitter facilitates identification and location of the protective outerwear. The techniques, processes and apparatus described may be utilized to control operation of any device and conserve use of resources based on conditions detected or applicable to the device.

The techniques, processes and apparatus described may be utilized to control operation of any device and conserve use of resources based on conditions detected or applicable to the device. For the sake of clarity, D/A and A/D conversions and specification of hardware or software driven processing may not be specified if it is well understood by those of ordinary skill in the art. The scope of the disclosures should be understood to include analog processing and/or digital processing and hardware and/or software driven components.

The invention is described in detail with respect to preferred embodiments, and it will now be apparent from the foregoing to those skilled in the art that changes and modifications may be made without departing from the invention in its broader aspects, and the invention, therefore, as defined in the claims, is intended to cover all such changes and modifications that fall within the true spirit of the invention.

Thus, specific apparatus for and methods of audio signature generation and automatic content recognition have been disclosed. It should be apparent, however, to those skilled in the art that many more modifications besides those already described are possible without departing from the inventive concepts herein. The inventive subject matter, therefore, is not to be restricted except in the spirit of the disclosure. Moreover, in interpreting the disclosure, all terms should be interpreted in the broadest possible manner consistent with the context. In particular, the terms “comprises” and “comprising” should be interpreted as referring to elements, components, or steps in a non-exclusive manner, indicating that the referenced elements, components, or steps may be present, or utilized, or combined with other elements, components, or steps that are not expressly referenced.