Patent ID: 12225648

DETAILED DESCRIPTION

FIG.1shows a block diagram of an example light show system102and related elements with which light show system102is in communication. Light show system102may provide automated light shows based on metadata in a music feature file (MFF), as described inFIG.2.

Light show system102may include a computing system104, a music file106, a lighting controller108, one or more lights110, and one or more speakers112. It is to be understood that any suitable number and/or combination of interconnects may be used to provide communication between various components of light show system102and related elements, including but not limited to Ethernet, Bluetooth, Wi-Fi, Controller-Area-Network (CAN), Media Oriented Systems Transport (MOST) buses, and so on. Further, light show system102may communicate with a network155wirelessly. The network155, which may be part of a server or other device, may communicate with an external device114and one or more remote servers116, via wireless and/or wired connections. In one example, external device114is a mobile phone, which may include an app188. App188may provide user inputs (e.g., user inputs regarding an automated light show) via a wireless connection with the computing system.

Computing system104may include one or more processors including a processor118, a communication module120, a user interface124, a non-volatile memory122, and a volatile memory126. Computing system104may be a standalone device (e.g., a laptop, a desktop computer, a mobile phone, and so on), while in other examples computing system104may be integrated with another component of light show system102. As an example, computing system104may be integrated with lighting controller108. As another example, computing system104may be integrated with a light or a speaker.

A non-volatile memory122, may be included in the computing system104to store data such as instructions executable by the processor118in non-volatile form. The non-volatile memory122may store application data to enable the computing system104to run an application for connecting to a cloud-based server and/or collecting information for transmission to the cloud-based server. The application may retrieve information gathered by input devices (e.g., a user interface124), devices in communication with the computing system (e.g., a mobile device connected via a Bluetooth link), and so on. Non-transitory storage devices, such as non-volatile memory122and/or volatile memory126, may store instructions and/or code that, when executed by the processor118, controls the computing system104to perform one or more of the actions described in the disclosure.

Processor118may execute programs stored in memory122of computing system104, control input/output, and interface with the other components of light show system102. Further, communication module120of computing system104may output data to the lighting controller108, the speakers112, and the lights110, while also receiving data input from other the music input106, the external devices114, the remote server116, and the lighting controller108.

User interface124of computing system104may enable the computing system to receive information from users, such as user preferences for light shows. In one example, a smartphone provides the user interface for light show system102, while in other examples, a dedicated touch-screen, keypad, or other input method allows a user to transmit preferences and/or commands to computing system104. The display may be a touch-screen display. Additionally or alternatively, the user interface may include an input hardware element, such as an input switch. The user interface may be electrically connected to the computing system104. The controller may include a digital signal processor DSP with a static or adaptive algorithm solution, a graphic processing unit GPU, a system-on-a-chip SOC, and/or another integrated circuit IC. For example, the controller may include frequency impulse response (FIR) control filters and/or adaptive algorithm controls that alter or re-calculate the FIR control filters. In other examples, displays may not provide a user interface, but may provide a televisual screen for projecting images (e.g., a music video synchronized to a music file).

In some embodiments, computing system104may be a mobile device. The mobile device may include a smart phone, a tablet, a laptop computer, a portable media player, and/or any suitable mobile computing system, and may be connected to other components of lighting system102via a communication link. The communication link may be wired (e.g., via Universal Serial Bus [USB], Mobile High-Definition Link [MHL], High-Definition Multimedia Interface [HDMI], and so on) or wireless (e.g., via BLUETOOTH, WI-FI, Near-Field Communication [NFC], and so on) and configured to provide two-way communication between the mobile device and the lighting system102. For example, the communication link may provide control and/or sensor signals from the mobile device to other components of the lighting system.

Computing system104may receive a music input106. In some examples, the music input may be a music file saved locally in memory non-volatile memory122, or may be provided via an AUX cable connection, a USB connection, a Bluetooth connection, and so on, from an external device114. As another example, the music input may be a streaming service, wherein a remote server116digitally streams a music file on command. In some examples, streaming a music file may include receiving a complete music file, while in other examples, streaming a music file may include receiving a plurality of packets from a remote server. In some embodiments, the computing system104may send the music input106to one or more of the plurality of speakers112in order to provide a synchronized audio-visual experience.

In some embodiments, the lighting controller108may adjust the lights110according to instructions received from the computing system104. However, in other embodiments, a lighting controller may not be included in lighting system102, and the computing system104may directly adjust one or more lights110. Further, in some embodiments the lighting controller108may transmit an audio signal to the speakers112according to instructions received from the computing system104, while in other embodiments, the computing system104may directly transmit an audio signal to the speakers112.

The speakers112may control a volume of audio output, a distribution of sound among the individual speakers of a plurality of speakers, an equalization of audio signals, and/or any other aspect of the audio output. In some examples, one or more of the speakers112may include a computing system, such as computing system104. Further, the lights110may include one or more lights, and may include a variety of light types, such as a spotlight, a globe light, a laser, a strobe light, and so on. In some examples, one or more of the lights may include a computing system, such as computing system104.

FIG.2shows a schematic representation of an example protocol governing a music feature file (MFF)200. As an example, MFF200may be a file containing characteristics of the music file, along with one or more time markers aligned with song events. The characteristics of the music file may include a time signature, a number of beats per minute (BPM), a musical key, a genre, and the like. Further, a timeline204may correspond to Music file202. In example MFF200, a plurality of song events, such as song events206may include a song beginning, a song end, a transition between verse and chorus, a transition between chorus and verse, a transition between verse and bridge, a climactic moment within the song (referred to herein as a “drop”), and so on. In one example, the song events206are categorized into a break, an intro, a verse, a chorus, and a build-up. It will be appreciated that other categorizations may be included and/or excluded from the song events206. MFF200may further include a plurality of trigger beats, such as trigger beats208, which may index song events206to timeline204. The MFF200may be supported multiple times during the track. In some embodiments, the MFF may be combined with the corresponding music file into a single file protocol, while in other embodiments, the MFF and the MP3 may be separate file protocols, and may be aligned by a separate controller. MFF200may be generated based on a music file, such as a corresponding music file202. In other examples, MFF200may be generated based on another type of music file, such as a WAV file or an MP4 file. Further, in some examples, MFF200may be generated in real-time, responsive to an analog or streaming music source.

FIG.3Ashows schematic representation of another example MFF300. MFF300may be generated based on corresponding music file302. Further, MFF300comprises a plurality of packets304, each of the plurality of packets304including a complete set of metadata for music file302, such as a time signature, a BPM, a musical key, a genre, a plurality of song events, a plurality of trigger beats, a timeline, and an indication of location relative to music file302. With a MFF comprised of timed packets including the complete set of metadata for the music file, a light show system may use data from the entire music file after receiving one packet.

Next,FIG.3Bshows a schematic representation of another example MFF330. The MFF330may be generated based on a corresponding music file332, and comprises a plurality of song events336and a plurality of trigger beats338. Further, the MFF332includes a karaoke video340, which may be aligned with the music file332such that a music video and song lyrics are timed to correspond with music file332. The karaoke video340may comprise one or more of lyrics, a background, a singer, or the like, wherein the karaoke video340is segmented based on one or more of the plurality of song events336and the plurality of trigger beats338.

Next,FIG.3Cshows a schematic representation of another example MFF350. The MFF350may be generated based on a corresponding music file352, and comprises a plurality of song events356and a plurality of trigger beats358. Further, the MFF may include a timing360for specific light show devices (e.g., lights and speakers) corresponding to trigger beats and song events. In the embodiment ofFIG.3C, MFF350may be a protocol combining the music file and the music feature file, which may improve processing time while providing a light show. The timing360may provide instructions for one or more lights, such as lights included in the light show system102ofFIG.1. The timing360may be used to activate or deactivate one or more lights based on the plurality of song events356and/or the plurality of trigger beats358. In one example, as a tempo of the song increases (e.g., increased number of trigger beats within a predetermined duration), the timing360may signal to activate more lights and/or to increase an intensity of activated lights. Additionally or alternatively, as the tempo decreases, such as during a break in the plurality of song events356, the timing i360may signal to deactivate the lights.

Turning now toFIG.4, it shows a flow chart of an example method400for acquiring a music feature file (MFF). The method400may be performed by a computing system, such as computing system104ofFIG.1. Instructions stored on memory of the computing system104may be executed and enable the computing system to acquire the music feature file.

At402, method400includes receiving a music input. In some examples, the music input may be a music file stored in controller memory. In another example, the music input may be a music file stored on an external device, such as an external hard drive, a CD-ROM, a DVD, and so on. In yet another example, the music input may be a streaming service, which may provide the music file to the controller in real-time. In still another example, the music input may be an analog signal, provided via an AUX cable and/or a microphone. For example, an AUX cable connected to an independent music player (e.g., a record player, a radio, a portable music player, or the like) may deliver an analog music file to the controller in real time. As another example, the controller may receive an analog audio signal via a microphone, such as microphone130ofFIG.1.

At404, method400includes determining a music input source. For example, the controller may determine whether the music file is available as a retrievable file in local memory or in an external device. The controller may further determine whether the music file is being provided by a streaming service. The controller may further determine whether the music file is being provided via an AUX cable and/or a microphone. For example, if the music input source is not a music streaming service, method400may determine that the music input source is a local file or an analog signal.

At406, method400includes determining whether an existing MFF is available for the music file. In one example, the MFF may have been generated and saved previously. As another example, the MFF may have been downloaded onto the computing system via another source. As yet another example, the MFF may be stored on the external device (e.g., an external hard drive, a personal computer, and so on), and may be available for download.

If method400determines that the MFF corresponding to the music input is available (“YES”), method400proceeds to408. At408, method400retrieves the existing MFF corresponding to the music input. After408, method400returns.

Referring again to406, if method400determines that an existing MFF is not available (“NO”), method400proceeds to410. At410, method400determines if the MFF is available from a streaming service. For example, if the music input source is a streaming service, the MFF may be available for download or for streaming via the streaming service.

If method400determines that the existing MFF is available from a streaming service (“YES”), method400proceeds to412. At412, method400requests the MFF from the streaming service. Continuing at413, method400receives the MFF from the streaming service. In some examples, the streaming service may transmit packets, such as packets304ofFIG.3A, at pre-determined intervals during transmission of the music file. In another example, the streaming service may transmit a complete MFF, such as the MFF200ofFIG.2, when computing device104begins streaming the music file. After413, method400returns.

Referring again to410, if method400determines that the MFF for the corresponding music input source is not available through a streaming service (“NO”), method400proceeds to414. At414, method400analyzes the music input and generates metadata for the MFF file. As an example, analyzing the music input may include determining song metadata such as a length, a number of BPM, a tempo, a genre, a plurality of song events, a mood, and a musical key. In one example, the music file is an MP3, and method400may input the MP3 file into one or more look-up tables, functions, or algorithms, which may output the song characteristics, for example. In one example, the music input is an analog music signal (e.g., via an AUX connection or a microphone), and method400may analyze the music file in real time to determine the song characteristics. For example, method400may input the analog music signal into one or more look-up tables, function, or algorithms, which may output the song characteristics. If the music input is an analog signal, the MFF protocol may include a plurality of discrete packets (e.g., such as the MFF protocol ofFIG.3A), and may update the song characteristics in real-time based on changes in the analog music signal.

At416, method400generates the MFF. The MFF may be generated by storing the song metadata determined at412according to a pre-determined protocol, such as the MFF protocol200ofFIG.2. In some examples, the MFF may be transmitted to a lighting controller, and additionally or alternatively be saved to controller memory. In other examples, the MFF may be transmitted to a remote server, such as remote servers116ofFIG.1. After414, method400returns.

FIG.5shows a view of an Integrated Light Show Hub500, which may provide an interface between a music source, a plurality of speakers, and a plurality of lights. In some examples, lighting controller108ofFIG.1may be an Integrated Light Show Hub. The Integrated Light Show Hub500includes a computing system, and may communicate with other devices, such as the music source, a speaker, and a light, via one or more communication links. The communication link may be wired (e.g., via Universal Serial Bus [USB], Mobile High-Definition Link [MHL], High-Definition Multimedia Interface [HDMI], and so on) or wireless (e.g., via BLUETOOTH, WI-FI, Near-Field Communication [NFC], and so on) and configured to provide two-way communication between the Integrated Light Show Hub500and other devices. As shown inFIG.5, Integrated Light Show Hub500includes a USB connector502, an outgoing HDMI connector504, an AUX cable506, an incoming HDMI connector508, a microphone connector510, a LIGHTNING connector512, and a WI-FI antenna514. In one example, the Integrated Light Show Hub may receive a MFF from a music source, such as via a wireless connection, and may adjust a signal to one or more lights and one or more speakers to provide a synchronized lighting show. In another example, the Integrated Light Show Hub may receive a music file, such as a MP3, from a music source, and may generate a MFF according to the method ofFIG.4. After generating a MFF, the Integrated Light Show Hub500may adjust a signal to one or more lights and to one or more speakers to provide a synchronized lighting show. In yet another example, the Integrated Light Show Hub may transmit the MFF to one or more lights and to one or more speakers, and may provide time synchronization between each device.

In the embodiment ofFIG.5, the Integrated Light Show Hub500may include beat, phrase, and key recognition, video color picking, and mapping information. Further, the Integrated Light Show Hub500ofFIG.5may include a processor capable of algorithmic processing, speech recognition, and control of a mobile application. For example, the Integrated Light Show Hub500may receive a music file via an analog music signal (e.g., from the AUX connector506), and may provide a light show based on the analog music signal in real-time. The Integrated Light Show Hub500may apply predictive learning algorithms to the analog music signal to determine characteristics of the music file, such as a number of BPM and a genre. The Integrated Light Show Hub may transmit packets to one or more connected devices at a pre-determined interval, such as every 20 milliseconds (ms), for example, with an average latency of 20 ms, for example. Further, the Integrated Light Show Hub may receive packets from one or more connected devices at a pre-determined interval, such as every 5 seconds, for example.

Next,FIG.6shows an example method for providing an automated light show based on a music input and a corresponding music feature file (MFF). The method600may be performed by a light show system such as light show system102ofFIG.1, which may include a computing system such as computing system104. The light show system may further include a lighting controller, which may be an Integrated Light Show Hub, such as the Integrated Light Show Hub show inFIG.5. In examples including an Integrated Light Show Hub, the computing system may be integrated with the Integrated Light Show Hub. In examples not including an Integrated Light Show Hub, the computing system may be integrated into the lighting controller, a mobile device, a light, or another external device.

At602, method600includes determining available equipment included in the light show system. For example, a light show system may include a plurality of speakers and a plurality of lights. In some examples, a light show system may further include a monitor for a karaoke video, a mobile device, a lighting controller (e.g., an Integrated Light Show Hub), and so on. Further, method600may determine the characteristics of the available equipment, such as a type of light (e.g., strobe, laser, spotlight, and so on), a connection method (e.g., wired, Bluetooth, WiFi, and so on), a color-change ability, a frequency, a location, pan and tilt area and an intensity.

At604, method600includes assigning a master device and one or more dependent devices (e.g., slave devices). For example, if an Integrated Light Show Hub is detected at602, the Integrated Light Show Hub may be designated as the master device. If an Integrated Light Show Hub is not detected at602, the first device including a processor to respond may be designated as the master device. All devices in the light show system not designated as the master device may be designated as dependent devices. Thus, method600may be executed by a controller included in a speaker, an Integrated Light Show Hub, or a light. However, in some examples, a separate computing device may be designated the master, and may execute method600.

At606, method600determines whether a light show is requested for a provided music file. For example, a user input may trigger a request for a light show. For example, a user may press a key or button in a user-interface connected to the lighting system, and may request a light show. In some embodiments, selecting a music file may trigger a request for a light show. For example, a user input may trigger a first music file selection, and a light show may be automatically provided for a selected music file.

If method600determines that a light show is not requested (“NO”), method600proceeds to608. At608, method600does not provide a light show. For example, the light show system may not acquire a MFF for a corresponding music file, may not adjust a light according to the MFF, and may not transmit a music signal to a speaker. After608, method600returns.

However, referring again to606, if method600determines that a light show is requested (“YES”), method600proceeds to610. At610, method600acquires a MFF for the first music file. For example, method600may acquire the MFF for the first music file according to the method400described hereinabove.

Continuing at612, method600optionally acquires user inputs for lighting. In some embodiments, a user may modify a number of light show pre-sets, which may change the method by which lights are adjusted. For example, a user may enter a color preference by pressing a button on a user interface, which may modify an algorithm for providing the light show. A user may further input a number of lights, a desired efficiency, a desired pulsing, and the like. For example, if the user inputs to a relatively high desired efficiency, then the light show may reduce a light intensity and number of lights activated compared to a light show with a lower desired efficiency.

Continuing at614, method600adjusts one or more lights and/or speakers according to the metadata in the MFF for the first music file. If a user input was acquired at612, method600may also adjust one or more lights based on the user input. In some examples, the master device may directly adjust the one or more lights and/or speakers. In other examples, if one or more dependent devices includes a computing system, the master device may transmit a protocol for adjustment, referred to herein as a “lighting feature file” (LFF), to the one or more dependent devices. Based on metadata in the LFF, a controller of each dependent device with a computing system may make a series of adjustments. Thus, in some examples, adjusting one or more lights and/or speakers according to the metadata in the MFF for the first file at614may include transmitting a LFF to one or more dependent devices. The LFF may include general instructions for adjusting a light show device, a position of the dependent device relative to the master device, and so on. Additionally or alternatively, colors, pulsations, brightness, contrast, intensity, and the like may be adjusted. Intensity may correspond to a color, such that a dark red when may be more intense than a light red. Brightness may refer to a visual sensation of a light. For example, a yellow light may be brighter than a purple light. In one example, the yellow light illuminates a greater area than the purple light. Contrast may refer to an amount of blending between light and dark areas. For example, low contract may include a greater amount of blending between light and dark areas relative to high contrast, which may more clearly define light and dark areas.

At616, method600determines whether a second music file is provided. For example, a user may request a light show that accompanies a sequence of music files. A user may provide a list of files before beginning the light show, or may add a new music file to the sequence of music files while the first music file is playing.

If the method determines that the second music file is not provided (“NO”), method600proceeds to618and includes not adjusting lighting to transition to the next music file. For example, method600may not adjust a tempo or color of lights to provide a transition between the first music file and the second music file. Method600then returns.

However, referring again to616, if method600determines that a next music file is provided (“YES”), method600proceeds to620and acquires the MFF for the next music file. For example, method600may acquire the MFF for the next music file according to the method400described hereinabove with regard toFIG.4.

After acquiring the MFF for the next music file, method600continues to622. At622, method600adjusts one or more lights to provide a transition between the first music file and the second music file. For example, the method may input a MFF corresponding to the first music file and a MFF corresponding to the second music file into one or more look-up tables, algorithms, or functions, which may output one or more adjustments for transitioning between the first music file and the second music file. In some embodiments, the method may include fading out the first music file on the speakers, and fading in the second music file on the speakers, in order to provide a more satisfying transition. After622, method600returns.

FIG.7is a view of a light show system700. Light show system700includes a plurality of lights701, including a master light702and dependent lights704,706,708,710,712. In the example ofFIG.7, the master light702may broadcast an automated light show. The automated light show may be a pre-determined sequence of adjustments to the master light702and the dependent lights, for example. In another example, the automated light show may be generated based on one or more music feature files.

The plurality of lights701are depicted as emitting differently colored light beams. In one example, a striped beam represents a first color and the dotted beam represents a second color, different than the first color. It will be appreciated that less than or greater than two color may be emitted by the plurality of lights701based on the acquired MFF.

FIG.8is a view of a light show system800, including a plurality of lights801and an audio source806. In one example, the audio source806is a speaker806. The plurality of lights801includes a master light802, and dependent lights804,808,810,812, and814. In the example ofFIG.8, the master light802may broadcast an automated light show separately from the speaker806. That is to say, the speaker806and the plurality of lights801may not be in communication. However, each of the speaker806and the plurality of lights801are instructed to play an identical audio file, wherein the master light802signals to the dependent lights804,808,810,812, and814to play a light show in conjunction with the MFF while the speaker806plays an audio file. In one example, audio of the audio file is identical to audio of the MFF.

FIG.9is a view of a light show system900, including a plurality of lights901, a speaker906, and a mobile device908. The plurality of lights901includes a master light902, and dependent lights904,910,912,914, and916. The mobile device908may include a smart phone, a tablet, a laptop computer, a portable media player, and/or any suitable mobile computing system. The mobile device908may be connected to the master light902and the speaker906via a communication link. The communication link may be wired or wireless, and may be configured to provide two-way communication between the mobile device908, the master light902, and the speaker906. In the example ofFIG.9, the mobile device908may transmit a MFF and/or a series of lighting adjustments to the master light902, and may further transmit a music file to the speaker906. The master light902may broadcast an automated light show to the plurality of dependent lights904based on one or more music files and/or MFFs provided by the mobile device908, and may be calibrated to correspond to the music file as transmitted by the speaker906. Further, a user may adjust parameters of the automated light show via mobile device908. Similar to the example ofFIG.8, the speaker906is not in communication with the mobile device908and/or the plurality of lights901. As such, the speaker906does not include wireless communication abilities.

FIG.10is a view of a light show system1000, including a master speaker1002, a plurality of lights1001, a dependent speaker1006, and a mobile device1008. The mobile device1008may include a smart phone, a tablet, a laptop computer, a portable media player, and/or any suitable mobile computing system. The mobile device1008may be connected to the master speaker1002and the plurality of lights1001via a communication link. The communication link may be wired or wireless, and may be configured to provide two-way communication between the mobile device1008, the master speaker1002, and the dependent speakers1006. In the example ofFIG.10, the mobile device1008may transmit a MFF and/or a series of lighting adjustments to the master speaker1002, and may further transmit a music file to the master speaker1002. The master speaker1002may broadcast an automated light show to the plurality of dependent lights1001based on one or more music files and/or MFFs provided by the mobile device1008, and may further broadcast a music file to the dependent speakers1006. Further, a user may adjust parameters of the automated light show via mobile device1008.

The master speaker1002comprises wireless communication capabilities, such as Wi-Fi, in order to communicate with the dependent speaker1006and the plurality of lights1001. The master speaker1002may relay instructions from the mobile device1008with regard to a light show corresponding to a MFF to the plurality of lights1001while relaying an audio file to the dependent speaker1006. Additionally or alternatively, the master speaker1002and the dependent speaker1006may comprise one or more lighting fixtures configured to provide further light sources for the light show.

FIG.11is a view of a light show system1100, including an Integrated Light Show Hub1102, a plurality of dependent lights1101, a dependent speaker1106, and a mobile device1108. In such a light show system, the Integrated Light Show Hub1102may be designated as the master. The mobile device1108may include a smart phone, a tablet, a laptop computer, a portable media player, and/or any suitable mobile computing system. The Integrated Light Show Hub1102may be connected to the mobile device1108, the dependent speakers1106, and the dependent lights1101via a communication link. The communication link may be wired or wireless, and configured to provide two-way communication between the Integrated Light Show Hub1102, the dependent lights1101, the dependent speakers1106, and the mobile device1108. In the example ofFIG.11, the mobile device1108may transmit a MFF and/or a music file to the Integrated Light Show Hub1102. The Integrated Light Show Hub1102may broadcast an automated light show to the plurality of dependent lights1101based on one or more music files and/or MFFs provided by the mobile device1108, and may further broadcast a music file to the dependent speakers1106. Further, a user may adjust parameters of the automated light show via mobile device1108. As another example, the Integrated Light Show Hub may receive a music signal via a microphone, and may generate a light show synchronized to the music signal in real-time.

FIG.12shows a view of a light show system1200, including an Integrated Light Show Hub1202, a plurality of dependent lights1204, a plurality of dependent speakers1206, a mobile device1208, a music source1210, and a graphical display unit1212. In the example ofFIG.12, the Integrated Light Show Hub1202may receive a music signal and a video signal from the music source1210via a HDMI communication link, and may further receive one or more user inputs from the mobile device1208via a communication link (e.g., a wired communication link or a wireless communication link). Further, the Integrated Light Show Hub1202may transmit a video signal to the graphical display unit1212via a HDMI communication link. Based on a music file from the music source1210, the Integrated Light Show Hub may receive, download, or generate a MFF for providing a light show corresponding to the music file. For example, the Integrated Light Show Hub may generate the MFF according to the method400ofFIG.4. Further still, the Integrated Light Show Hub1202may transmit a sequence of lighting adjustments to the dependent lights1204, and may transmit an audio signal to the dependent speakers1206, according to the method described hereinabove with regard toFIG.6. Thus, the Integrated Light Show Hub1202may broadcast an automated light show to the plurality of dependent lights1204based on one or more music files and/or MFFs provided by the mobile device1208, and may further broadcast a music file to the dependent speakers1206. Further, a user may adjust parameters of the automated light show via mobile device1208.

According to the present disclosure, a light show may be improved based on a mapping of relative locations of light show components (e.g., lights, speakers, lighting controllers, mobile devices, and so on). For example, utilizing features in a BLUETOOTH protocol, such as an Angle of Arrival (AoA) and an Angle of Departure (AoD), it is possible to perform a two-dimensional and/or three-dimensional mapping of an enclosed space including a plurality of lights and speakers. As another example, mobile devices, such as smartphones, may include integrated motion sensors, such as microelectromechanical systems (e.g., MEMS), which may provide fast and reliable signals for characterizing motion. In particular, sensor fusion, which combines signals from a gyroscope, an accelerometer, and a compass, may accurately determine the position of a smartphone over time. A filtering algorithm, such as an algorithm including Kalman filtering, may be applied to improve an accuracy of the locations of the plurality of light and sound devices. In one example, a Zero Velocity Update (ZUPT) algorithm may be applied, for example.

FIG.12further illustrates the Integrated Light Show Hub1202being connected to a karaoke player1212. In one example, the karaoke player1212may include a display device, such as a television, a projector, a monitor, or other display device. Additionally or alternatively, a media player1210may be connected to the Integrated Light Show Hub1202. The media player1210may be configured to read CDs, videotapes, cassette tapes, DVDs, Blu-ray, and the like. The media player1210may transmit song data to the Integrated Light Show Hub1202, wherein a MFF may be generated via the method ofFIG.4and transmitted to one or more of the plurality of lights1204and the speakers1206. In this way, the Integrated Light Show Hub1202may receive inputs from a plurality of devices and send outputs to a plurality of lights and speakers.

In one example, a user may use the mobile device1208of the Integrated Light Show Hub1202to adjust one or more settings of the light show, including color, sound, intensity, brightness, contrast, and the like.

FIG.13shows an example method1300for mapping locations of light show components, and adjusting an automated light show based on the locations of the light show components. By indexing locations of the light show components to locations of a mobile device including sensor fusion, a controller may build a virtual map of a light show system, including relative locations of a plurality of light show components. Based on the relative locations of the light show components, an automated light show may be adjusted to increase customer satisfaction. Method1300may be performed by a controller equipped with a plurality of sensors (e.g., an accelerometer, a gyroscope, a compass, GPS, and the like), such as a mobile device (e.g., a smartphone), which may be communicatively linked with one or more light show device (e.g., a “smart” speaker, an Integrated Light Show Hub, a light, a lighting controller, and so on). Further, each light show component may include a button. When the button of a component is depressed, the button may establish a communication link with the controller, and may further signal that a location of the component corresponds with a location of the mobile device. For example, when a button on a strobe-light is pressed, a controller may determine that a location of the mobile device corresponds to a location of the strobe-light. Note that method1300may be described with reference toFIGS.14A-14D, which show views of a light show system during a light show component mapping. The light show systems ofFIGS.14A-14Dare identical, and will be numbered identically. Components introduced inFIG.14Awill not be reintroduced inFIGS.14B-14D.

At1302, method1300includes receiving an indication of a first location of a first light show component. In one non-limiting example, a user may wirelessly connect via a mobile device to the first light show component and depress a button on the first light show component, while in physical proximity to the first light show component. Depressing the button on the first light show component may cause the wireless device to transmit its location to the controller. Responsive to receiving the location of the first light show component, the controller may establish a first mapping point. In the example embodiment ofFIG.14A, light show system1400includes a speaker1402, a plurality of globe lights1404, including a first globe light1404a, and a plurality of spotlights1406. A user1408and a smartphone1410are in physical proximity to the speaker1402, which may be the first light show component, and may include a first button1414. Speaker1402may be a “smart” device including at least one computing system, and may connect to other light show components and the smartphone1410via a communication link (e.g., a wireless or wired connection).

At1304, method1300includes receiving an indication of a second location of a second light show component relative to the first location. In one non-limiting example, the user may wirelessly connect to the first light show component and depress a button on a second light show component, while in physical proximity to the second light show component. Depressing the button on the second light show component may cause the wireless device to transmit its location to the controller. Responsive to receiving the location of the second light show component, the controller may establish a first mapping point. Further, using sensor fusion (e.g., integrated data from a gyroscope, an accelerometer, and a compass), method1300may determine a position of the second light show component relative to the first mapping point. The position of the second light show component relative to the first mapping point may be the second mapping point. An example light show system configuration during1304is shown graphically inFIG.14B. InFIG.14B, the user1408, with the smartphone1410, are in physical proximity to a first globe light1404a, which may be the second light show component. Further, the first globe light1404aincludes a second button1416.

At1306, method1300includes receiving indications of locations of other light show components to compile a set of mapping locations. Thus, method1300may assemble a set of mapping points corresponding to the location of each light show component relative to the first light show component. As shown inFIG.14C, a set of mapping points is determined based on the relative locations of each light show component received as the user1408interacts with each component while traversing a path1416around the light show system1400. The set of mapping points may be stored in a non-volatile memory location in the controller, or may be stored in a remote server. For example, the computing system of the speaker1402may transmit the set of mapping points to smartphone1410, which may store the mapping points in local memory, or may transmit the mapping points to a remote server for processing.

At1308, method1300includes indexing the set of mapping points corresponding to the location of each light show component relative to the first light show component to a grid. For example, method1300may construct a virtual space, and may divide the virtual space into a number of cells, as shown inFIG.14D. In some examples, the virtual space is three-dimensional, while in other examples, the virtual space is two-dimensional. As an example, the light show system1400ofFIG.14Dincludes a virtual space comprising a two-dimensional grid1418of cells, onto which the set of mapping points ofFIG.14Care assigned or indexed.

In this way, method1300may determine relative locations of a plurality of light show components, and may index these locations (e.g., mapping points) to a grid. The relative locations of the plurality of light show components may be transmitted to a lighting controller, such as the lighting controller108ofFIG.1.

Continuing at1310, method1310generates a light show based on the mapping points and the grid. For example, the lighting controller may modify a sequence of adjustments to the plurality of light show components according to the relative locations of the light show components. For example, the lighting controller may adjust a light show component based on its location relative to one or more cells in the virtual space. As an example, the lighting controller may determine to illuminate cell A-5ofFIG.14Dby adjusting globe light1404a, causing it to illuminate. As another example, the lighting controller may determine to illuminate cell B-6ofFIG.14Dby adjusting a direction and intensity of spotlight1406asuch that a beam of light from spotlight1406ais directed at cell B-6. Such adjustments may be coordinated by the lighting controller in order to achieve a variety of aesthetically pleasing effects while providing an automated light show, which may increase customer satisfaction. In particular, the lighting controller may synchronize adjustments to lights in order to highlight a certain feature of a music file, which may further increase customer satisfaction during an automated lighting show. Such dynamic sequences of lighting adjustments may enhance the harmony of an audio-visual synthesis provided by the light show. Method1300then returns.

It should be appreciated that the various systems and methods referred to herein for providing an automated light show may be combined. For example, mapping may be provided in a system including an Integrated Light Show Hub, which may generate a MFF and provide an automated light show based on a digital or analog music signal.

In one embodiment, a method comprises generating a music feature file based on selected audio, the music feature file comprising metadata for the selected audio including a plurality of time markers indicating audio events in the selected audio, and automatically controlling one or more lights according to the music feature file.

In a first example of the method, the method further comprises outputting the selected audio to one or more speakers, and synchronizing the automatic control of the one or more lights according to the music feature file with the output of the selected audio. In a second example of the method optionally including the first example, automatically controlling the lights according to the music feature file includes adjusting one or more of a color, state, beam movement, and intensity of the one or more lights based on one or more characteristics of the selected audio stored as the metadata of the music feature file. In a third example of the method optionally including one or more of the first and second examples, the metadata for the selected audio further includes a genre of the selected audio. In a fourth example of the method optionally including one or more of the first through third examples, the music feature file further comprises the selected audio. In a fifth example of the method optionally including one or more of the first through fourth examples, the music feature file further comprises a synchronized video file. In a sixth example of the method optionally including one or more of the first through fifth examples, the selected audio comprises an analog signal including the selected audio. In a seventh example of the method optionally including one or more of the first through sixth examples, the method further comprises receiving the selected audio from a remote streaming server, and generating the music feature file while receiving the selected audio. In an eighth example of the method optionally including one or more of the first through seventh examples, the selected audio comprises a music file stored in a memory of a controller. In a ninth example of the method optionally including one or more of the first through eighth examples, the method further comprises receiving a position for each of the one or more lights in a light show environment, mapping the position for each of the one or more lights to a grid corresponding to the light show environment, and further adjusting control of the one or more lights based on the mapped position for the one or more lights in the grid. In a tenth example of the method optionally including one or more of the first through ninth examples, the position for each of the one or more lights in a light show environment is determined based on a signal from a motion sensing device. In an eleventh example of the method optionally including one or more of the first through tenth examples, the motion sensing device is a mobile phone including BLUETOOTH. In a twelfth example of the method optionally including one or more of the first through eleventh examples, the motion sensing device is a mobile phone including a plurality of sensors.

In another embodiment, a light show system comprises a plurality of lights, and a controller communicatively coupled to the plurality of lights and configured with instructions stored in non-transitory memory that when executed by a processor of the controller cause the controller to control one or more lights of the plurality of lights based on a music feature file for a selected audio.

In a first example of the system, the system further comprises one or more speakers communicatively coupled to the controller, wherein the controller is further configured with instructions that when executed cause the controller to output the selected audio to the one or more speakers while controlling the one or more lights of the plurality of lights based on the music feature file for the selected audio. In a second example of the system optionally including the first example, the controller is further configured with instructions that when executed cause the controller to: receive a second selected audio; retrieve a second music feature file for the second selected audio; and control one or more of the plurality of lights based on the second music feature file while outputting the second selected audio to the one or more speakers. In a third example of the system optionally including one or more of the first and second examples, the controller is further configured with instructions that when executed cause the controller to control one or more of the plurality of lights based on the music feature file and the second music feature file while transitioning between outputting the selected audio and the second selected audio to the one or more speakers. In a fourth example of the system optionally including one or more of the first through third examples, the plurality of lights are distributed throughout a light show environment, and the controller is further configured with instructions that when executed cause the controller to: receive a position of each light of the plurality of lights within the light show environment; map the position of each light to a virtual grid corresponding to the light show environment; and adjust control of the one or more lights based on the music feature file and the mapped position of each light in the virtual grid. In a fifth example of the system optionally including one or more of the first through fourth examples, the position of each light of the plurality of lights within the light show environment is determined based on a signal from a motion sensing device, the motion sensing device including a plurality of sensors.

The description of embodiments has been presented for purposes of illustration and description. Suitable modifications and variations to the embodiments may be performed in light of the above description or may be acquired from practicing the methods. For example, unless otherwise noted, one or more of the described methods may be performed by a suitable device and/or combination of devices, such as the vehicle systems and cloud computing systems described above with respect toFIGS.1-14D. The methods may be performed by executing stored instructions with one or more logic devices (e.g., processors) in combination with one or more hardware elements, such as storage devices, memory, hardware network interfaces/antennas, switches, actuators, clock circuits, and so on. The described methods and associated actions may also be performed in various orders in addition to the order described in this application, in parallel, and/or simultaneously. The described systems are exemplary in nature, and may include additional elements and/or omit elements. The subject matter of the present disclosure includes all novel and non-obvious combinations and sub-combinations of the various systems and configurations, and other features, functions, and/or properties disclosed.

As used in this application, an element or step recited in the singular and proceeded with the word “a” or “an” should be understood as not excluding plural of said elements or steps, unless such exclusion is stated. Furthermore, references to “one embodiment” or “one example” of the present disclosure are not intended to be interpreted as excluding the existence of additional embodiments that also incorporate the recited features. The terms “first,” “second,” “third,” and so on are used merely as labels, and are not intended to impose numerical requirements or a particular positional order on their objects. The following claims particularly point out subject matter from the above disclosure that is regarded as novel and non-obvious.