Abstract:
In a first illustrative embodiment, a computer implemented method includes detecting a plurality of context providing devices accessible to a vehicle computing system. The illustrative method further includes evaluating context relevant information from each of the detected devices to compile a display. The illustrative method also includes outputting a compiled display to a vehicle display, including at least one portion of the display initially dedicated to vehicle-based service content and at least one portion of the display dedicated to content from a context providing device.

Description:
TECHNICAL FIELD 
       [0001]    The illustrative embodiments generally relate to a method and apparatus for context adaptive multimedia management. 
       BACKGROUND 
       [0002]    Portable, wireless devices are used in countless aspects of everyday life. From the basic phones carried by millions of people, to advanced technological devices such as laptops and tablet computers, to wireless medical devices, at any given time a person can be singly transporting a veritable cornucopia of computing technology. 
         [0003]    At the same time, vehicle computing capabilities and output devices have also been rapidly advancing and becoming more prevalent in the vehicles that people use in everyday transit. Advanced multimedia displays can be found, sometimes in several places, within vehicles, and wireless communication systems in conjunction with wireless devices and these displays allow access to resources far beyond those accessible in the typical automobile of twenty years past. 
         [0004]    It would not be unusual for a family of four traveling in a vehicle to have four cellular phones, at least one portable computer or computer-like device, such as a tablet, and a portable game system. Further, the vehicle itself may have multiple media outputs, such as speakers, rear visual screens and a center stack mounted navigation display. 
         [0005]    With so many media sources and outputs available, it could be difficult to manage content delivery to a particular display. If each device was independently operated, then this may be less of an issue, but there has been a push in modern multimedia environments to allow various devices to share content. In a vehicle, as much as anywhere else, the number and type of devices may be subject to frequent change, as passengers enter and exit a vehicle, and ride in varying configuration. In a non-mobile environment, such a problem could be addressed by requiring user configuration of device communication on a per-use instance, but in a vehicle environment this could lead to unwanted distractions for a driver. 
       SUMMARY 
       [0006]    In a first illustrative embodiment, a computer implemented method includes detecting a plurality of context providing devices accessible to a vehicle computing system. The illustrative method further includes evaluating context relevant information from each of the detected devices to compile a display. The illustrative method also includes outputting a compiled display to a vehicle display, including at least one portion of the display initially dedicated to vehicle-based service content and at least one portion of the display dedicated to content from a context providing device. 
         [0007]    In a second illustrative embodiment, a computer implemented method includes assembling an initial vehicle display based on context relevant information, the context relevant information including at least information relating to context sources including wireless devices present in the vehicle, occupants present in the vehicle, and vehicle-based services. The initial vehicle display includes at least information content relating to a plurality of the context sources. 
         [0008]    The illustrative method also includes outputting the initial vehicle display to a vehicle display source and monitoring the context sources for changes in information relating to the sources. The illustrative method further includes evaluating the changes in information to assemble a new, varied display, the variance based at least in part on changes in information. 
         [0009]    In a third illustrative embodiment, a computer readable storage medium stores instructions that, when executed by a processor, cause the processor to perform the method including detecting a plurality of context providing devices accessible to a vehicle computing system. The illustrative method also includes evaluating context relevant information from each of the detected devices to compile a display. Also, the illustrative method includes outputting a compiled display to a vehicle display, including at least one portion of the display initially dedicated to vehicle-based service content and at least one portion of the display dedicated to content from a context providing device. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0010]      FIG. 1  shows an illustrative vehicle computing system; 
           [0011]      FIG. 2  shows an illustrative process for context relevant content delivery; 
           [0012]      FIG. 3  shows an illustrative example of a vehicle/passenger configuration and content output; 
           [0013]      FIG. 4  shows an illustrative example of a content varying process; 
           [0014]      FIG. 5  shows an illustrative example of another content varying process; and 
           [0015]      FIG. 6  shows an example of a dynamic content management process. 
       
    
    
     DETAILED DESCRIPTION 
       [0016]    As required, detailed embodiments of the present invention are disclosed herein; however, it is to be understood that the disclosed embodiments are merely exemplary of the invention that may be embodied in various and alternative forms. The figures are not necessarily to scale; some features may be exaggerated or minimized to show details of particular components. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for teaching one skilled in the art to variously employ the present invention. 
         [0017]      FIG. 1  illustrates an example block topology for a vehicle based computing system  1  (VCS) for a vehicle  31 . An example of such a vehicle-based computing system  1  is the SYNC system manufactured by THE FORD MOTOR COMPANY. A vehicle enabled with a vehicle-based computing system may contain a visual front end interface  4  located in the vehicle. The user may also be able to interact with the interface if it is provided, for example, with a touch sensitive screen. In another illustrative embodiment, the interaction occurs through, button presses, audible speech and speech synthesis. 
         [0018]    In the illustrative embodiment 1 shown in  FIG. 1 , a processor  3  controls at least some portion of the operation of the vehicle-based computing system. Provided within the vehicle, the processor allows onboard processing of commands and routines. Further, the processor is connected to both non-persistent  5  and persistent storage  7 . In this illustrative embodiment, the non-persistent storage is random access memory (RAM) and the persistent storage is a hard disk drive (HDD) or flash memory. 
         [0019]    The processor is also provided with a number of different inputs allowing the user to interface with the processor. In this illustrative embodiment, a microphone  29 , an auxiliary input  25  (for input  33 ), a USB input  23 , a GPS input  24  and a BLUETOOTH input  15  are all provided. An input selector  51  is also provided, to allow a user to swap between various inputs. Input to both the microphone and the auxiliary connector is converted from analog to digital by a converter  27  before being passed to the processor. Although not shown, numerous of the vehicle components and auxiliary components in communication with the VCS may use a vehicle network (such as, but not limited to, a CAN bus) to pass data to and from the VCS (or components thereof). 
         [0020]    Outputs to the system can include, but are not limited to, a visual display  4  and a speaker  13  or stereo system output. The speaker is connected to an amplifier  11  and receives its signal from the processor  3  through a digital-to-analog converter  9 . Output can also be made to a remote BLUETOOTH device such as PND  54  or a USB device such as vehicle navigation device  60  along the bi-directional data streams shown at  19  and  21  respectively. 
         [0021]    In one illustrative embodiment, the system  1  uses the BLUETOOTH transceiver  15  to communicate  17  with a user&#39;s nomadic device  53  (e.g., cell phone, smart phone, PDA, or any other device having wireless remote network connectivity). The nomadic device can then be used to communicate  59  with a network  61  outside the vehicle  31  through, for example, communication  55  with a cellular tower  57 . In some embodiments, tower  57  may be a WiFi access point. 
         [0022]    Exemplary communication between the nomadic device and the BLUETOOTH transceiver is represented by signal  14 . 
         [0023]    Pairing a nomadic device  53  and the BLUETOOTH transceiver  15  can be instructed through a button  52  or similar input. Accordingly, the CPU is instructed that the onboard BLUETOOTH transceiver will be paired with a BLUETOOTH transceiver in a nomadic device. 
         [0024]    Data may be communicated between CPU  3  and network  61  utilizing, for example, a data-plan, data over voice, or DTMF tones associated with nomadic device  53 . Alternatively, it may be desirable to include an onboard modem  63  having antenna  18  in order to communicate  16  data between CPU  3  and network  61  over the voice band. The nomadic device  53  can then be used to communicate  59  with a network  61  outside the vehicle  31  through, for example, communication  55  with a cellular tower  57 . In some embodiments, the modem  63  may establish communication  20  with the tower  57  for communicating with network  61 . As a non-limiting example, modem  63  may be a USB cellular modem and communication  20  may be cellular communication. 
         [0025]    In one illustrative embodiment, the processor is provided with an operating system including an API to communicate with modem application software. The modem application software may access an embedded module or firmware on the BLUETOOTH transceiver to complete wireless communication with a remote BLUETOOTH transceiver (such as that found in a nomadic device). Bluetooth is a subset of the IEEE 802 PAN (personal area network) protocols. IEEE 802 LAN (local area network) protocols include WiFi and have considerable cross-functionality with IEEE 802 PAN. Both are suitable for wireless communication within a vehicle. Another communication means that can be used in this realm is free-space optical communication (such as IrDA) and non-standardized consumer IR protocols. 
         [0026]    In another embodiment, nomadic device  53  includes a modem for voice band or broadband data communication. In the data-over-voice embodiment, a technique known as frequency division multiplexing may be implemented when the owner of the nomadic device can talk over the device while data is being transferred. At other times, when the owner is not using the device, the data transfer can use the whole bandwidth (300 Hz to 3.4 kHz in one example). While frequency division multiplexing may be common for analog cellular communication between the vehicle and the internet, and is still used, it has been largely replaced by hybrids of with Code Domian Multiple Access (CDMA), Time Domain Multiple Access (TDMA), Space-Domian Multiple Access (SDMA) for digital cellular communication. These are all ITU IMT-2000 (3G) compliant standards and offer data rates up to 2 mbs for stationary or walking users and 385 kbs for users in a moving vehicle. 3G standards are now being replaced by IMT-Advanced (4G) which offers 100 mbs for users in a vehicle and 1 gbs for stationary users. If the user has a data-plan associated with the nomadic device, it is possible that the data-plan allows for broad-band transmission and the system could use a much wider bandwidth (speeding up data transfer). In still another embodiment, nomadic device  53  is replaced with a cellular communication device (not shown) that is installed to vehicle  31 . In yet another embodiment, the ND  53  may be a wireless local area network (LAN) device capable of communication over, for example (and without limitation), an 802.11g network (i.e., WiFi) or a WiMax network. 
         [0027]    In one embodiment, incoming data can be passed through the nomadic device via a data-over-voice or data-plan, through the onboard BLUETOOTH transceiver and into the vehicle&#39;s internal processor  3 . In the case of certain temporary data, for example, the data can be stored on the HDD or other storage media  7  until such time as the data is no longer needed. 
         [0028]    Additional sources that may interface with the vehicle include a personal navigation device  54 , having, for example, a USB connection  56  and/or an antenna  58 , a vehicle navigation device  60  having a USB  62  or other connection, an onboard GPS device  24 , or remote navigation system (not shown) having connectivity to network  61 . USB is one of a class of serial networking protocols. IEEE 1394 (firewire), EIA (Electronics Industry Association) serial protocols, IEEE 1284 (Centronics Port), S/PDIF (Sony/Philips Digital Interconnect Format) and USB-IF (USB Implementers Forum) form the backbone of the device-device serial standards. Most of the protocols can be implemented for either electrical or optical communication. 
         [0029]    Further, the CPU could be in communication with a variety of other auxiliary devices  65 . These devices can be connected through a wireless  67  or wired  69  connection. Auxiliary device  65  may include, but are not limited to, personal media players, wireless health devices, portable computers, and the like. 
         [0030]    Also, or alternatively, the CPU could be connected to a vehicle based wireless router  73 , using for example a WiFi  71  transceiver. This could allow the CPU to connect to remote networks in range of the local router  73 . 
         [0031]    In addition to having exemplary processes executed by a vehicle computing system located in a vehicle, in certain embodiments, the exemplary processes may be executed by a computing system in communication with a vehicle computing system. Such a system may include, but is not limited to, a wireless device (e.g., and without limitation, a mobile phone) or a remote computing system (e.g., and without limitation, a server) connected through the wireless device. Collectively, such systems may be referred to as vehicle associated computing systems (VACS). In certain embodiments particular components of the VACS may perform particular portions of a process depending on the particular implementation of the system. By way of example and not limitation, if a process has a step of sending or receiving information with a paired wireless device, then it is likely that the wireless device is not performing the process, since the wireless device would not “send and receive” information with itself. One of ordinary skill in the art will understand when it is inappropriate to apply a particular VACS to a given solution. In all solutions, it is contemplated that at least the vehicle computing system (VCS) located within the vehicle itself is capable of performing the exemplary processes. 
         [0032]      FIG. 2  shows an illustrative process for context relevant content delivery. In context relevant content delivery, the vehicle computing system considers a number of contextual sources before providing a media output to one or more vehicle outputs. A non-limiting list of context sources includes, number of passengers, identities of passengers, wireless phones in the vehicle, wireless medical devices in the vehicle, environmental factors, wireless computing devices in the vehicle, etc. 
         [0033]    For example, a vehicle computing system may be able to determine that there are currently three people riding in a vehicle, one in a back seat and two in a front seat. The system may also know that there is a media display in the front center stack and a media display in each of the headrests. Since there is at least one passenger in the back seat, the system may consider use of at least one of the back seat displays. If a vehicle media system has both speaker and headphone outputs capable of playing different audio streams, it may consider assigning a headphone output for use in the back seat in conjunction with the one or more displays, and using the vehicle&#39;s general speaker system for a front radio or navigation output source. 
         [0034]    Further, advanced vehicle computing systems may be able to identify particular passengers and retrieve known information about the preferences of those passengers. Favorite media can be queued up for delivery, radio presets can be changed, and preferences can be set based on previously entered setting preferences for each passenger. The preferences may further vary based on which passengers are sitting where in a vehicle. 
         [0035]    Additionally, the vehicle computing system may be able to communicate with one or more wireless devices in the passengers&#39; possession. Applications, multi-media, and communication functions can be temporarily integrated in dynamic fashion based on the presence of devices and/or preset user preferences. 
         [0036]    Environmental considerations, such as traffic, weather, time of day, etc. can also be taken as context inputs, and may affect brightness of displays, levels of volume for outputs, and which devices/functions are given priority on one or more of the vehicle outputs. For example, a nice drive through the countryside on a sunny afternoon may allow a first set of output preferences, wherein a DvD is played in a back seat, through the vehicle speakers, and a front center display shows local tourist information and handles phone calls from a driver and/or passenger phone. 
         [0037]    If, however, that same vehicle was being driven in a thunderstorm at 2 AM in heavy traffic, the front display may shift focus to weather/traffic information and the vehicle audio may be prioritized to navigation/weather related output. The decision to make these changes may also be based on which passengers are in a vehicle. For example, in the sunny afternoon scenario, the display may still be focused on traffic and weather if only a single occupant, the driver, is present. 
         [0038]    In  FIG. 2 , the process enables a context aware output control mode  201 . This mode will evaluate one or more contexts to determine the configuration, presentation and usage of vehicle outputs (such as, but not limited to, audio systems, multimedia displays, radio displays, etc.). The particular contexts may be manufacturer determined and/or they may be preset by users of the vehicle if changes are desired to standard configurations. 
         [0039]    In this exemplary process, the process will, using existing vehicle systems, detect a number of occupants  203 . In vehicles with appropriate identification capability, the process may also detect the specific identities of the vehicle occupants. In another embodiment, the vehicle may “guess” information about unknown occupants based on predictions stemming from, for example, height and/or weight considerations (e.g., without limitation, a four foot tall, seventy pound passenger is probably a child). Vehicle camera systems, weight sensors and other devices can be used to obtain this information. 
         [0040]    In addition to determining the number of and/or identities of the passengers, the process may determine what devices are present in the vehicle  205 . These can include, but are not limited to, wireless phones, wireless/BLUETOOTH capable medical devices, portable computers, tablets (tablet PCs, IPADs, etc.), wireless gaming systems, etc. Some or all of the devices may have been paired with the system at a previous point, depending on the communication medium used, and other devices may have the capability to pair with the system or transmit information to the system in the absence of pairing. 
         [0041]    Also, the process may determine what services are available to the passengers  207 . These can include, but are not limited to, weather related services, navigation services, traffic based services, content delivery services, medical assistance services, etc. 
         [0042]    In this example, although not shown as separate steps, as each source of context is determined, the process may also log relevant information relating to that context source. For example, if certain passengers are detected, known information about passenger preferences may be loaded. If certain media devices are loaded, information relating to the preferred display/output of data relating to those media devices may be loaded. 
         [0043]    Detection of certain services may cause information from those services to be determined/loaded. For example, without limitation, if a traffic service is detected, traffic for a travel route may be loaded to determine if the volume of traffic provides a context state that could adjust content output/display. The same can be done for weather, medical assistance, and any other relevant services. 
         [0044]    Other sources of context may also be detected and the examples discussed with respect to this process are for example only. 
         [0045]    Once all the relevant context sources have been detected and analyzed, the process determines if a preconfigured match for output exists  209 . For example, without limitation, if a driver is driving alone, with a cellular phone and a blood sugar monitor, the driver may want any incoming calls displayed on a small portion of a display, a majority of a display dedicated to navigation, and another portion of the display to show a blood sugar level. Incoming calls may cause the portion of the display dedicated to the call to temporarily enlarge to notify the driver of the call, and may also cause the vehicle audio system to shift into a call-handling mode. 
         [0046]    At the same time, if the driver is in the car with his wife, her phone may be given some portion of priority on the display. Additionally, her multimedia device (IPOD, etc.) may be given play priority and have a portion of the display dedicated to that. If traffic or weather worsens, the display may shift to a more nav/weather focused format. If the driver&#39;s blood sugar monitor sends an alarm, the entire display may be temporarily dedicated to the alert. 
         [0047]    If a pre-configured match/preference setting exists, the process provides the appropriate display and other outputs  217 . If not, the system checks to see if there is a set of settings within a reasonable tolerance that would allow a good “guess” as to which display was preferred. For example, in the above instance, if all devices except for a blood sugar monitor were present, the system could reasonably approximate the preferred display without the medical device shown. On the other hand, if the context doesn&#39;t correspond to any recognizable configuration, the process can “predict” what the driver would prefer. The prediction can be based on other configurations already stored, driver changes to previous predictions, and other known information about contexts and driver preferences. The driver could even be given the option to “save” a configuration for a given overall context so that the configuration would be used in future situations when that same context occurred. 
         [0048]      FIG. 3  shows an illustrative example of a vehicle/passenger configuration and content output. In this illustrative example, there is a front driver seat  301 , a front passenger seat  305  and a rear passenger seat  311 . For purposes of this example, it is assumed that occupants are sitting in the driver seat, front passenger seat, and rear seat behind the driver. 
         [0049]    In this example, the driver has a wireless device, such as a cellular phone  303 . The passenger in the front seat has both a cellular phone  307  and a wireless medical device  309 . The passenger in the back rear has a tablet device  313 . Position of the various wireless devices can be detected using known techniques, such as, but not limited to, near field communication. In at least one embodiment, certain passengers under driving ages are associated with wireless devices and are known in advance to be non-drivers based on a personal profile. 
         [0050]    Also in this vehicle, there are multiple media displays. There is a center stack display  315  and a rear display  325 . A second rear display (not shown) may also be present. According to, in this embodiment, predefined settings, the vehicle computing system executes a process that gathers all the relevant context and causes the displays to operate as shown (at least initially). In this embodiment, that corresponds to the primary center stack display showing driver phone information  317 , a navigation display  319  and a medical device display  321 . This display is capable of changing as relevant context changes occur. For example, if the driver disables his wireless phone, the display may no longer display phone information. Or, for example, if the driver does not utilize navigation, the navigation display may be removed and greater display space may be dedicated to the wireless phone(s) and the medical device. If there is a traffic tracking service available, however, and the driver&#39;s current location and/or estimated heading is likely to encounter traffic, the navigation display may be restored as long as the traffic situation persists in relevant proximity to the vehicle position. 
         [0051]    If the wireless medical device  309  reports an emergency condition, one or both of the displays may display a report of the condition in greater detail, to warn the driver/passengers about the onset of the condition. In the meantime, in this example, the rear display replicates the tablet output (if desired)  325 , or possible displays a movie playing on the tablet or on a vehicle content delivery system. 
         [0052]      FIG. 4  shows an illustrative example of a content varying process. In this illustrative example, an initial content display has been originated at some point prior to the initiation of this process. This process begins by initially monitoring system context  401 . In this example, the system is monitoring at least one context source for a priority alert  403 . This priority alert could correspond, for example, without limitation, to a medical device alert, a severe weather alert, a traffic accident alert, an amber alert, etc. 
         [0053]    If an emergency condition occurs  405 , the process may update all displays  407 . Different alerts may warrant a change to varying displays, with some conditions corresponding to an update to all displays, whereas other, less severe or driver relevant only conditions may only cause an update of one or more displays provided to the front of the vehicle. 
         [0054]    If the condition is not an emergency condition, the severity (and/or relevance) of the alert condition is determined  409 . Relevant displays are then updated  411 . In both display update instances, the process considers if the alert conditions have ended  413  before resuming the initial display state (or other now-relevant display state). 
         [0055]      FIG. 5  shows an illustrative example of another content varying process. In this example, one or more displays may be updated due to a change in context. For example, without limitation, a phone or other device may be disabled or enabled, and incoming call or message may occur, a navigation system may be enabled, etc. 
         [0056]    Again, the process monitors the various context sources and also considers the possibility of new context sources  501  (such as, for example, if a new device is enabled that was not previously detected). When a context change is detected  503 , relevant outputs may be updated  505 . 
         [0057]    For example, without limitation, a driver&#39;s phone may have a priority of display associated therewith. But, when the driver enters the vehicle, the phone may be disabled, so a lower priority context display may occur. In this instance, the vehicle may not even be aware that the driver&#39;s phone is present in the vehicle (depending on detection techniques employed). 
         [0058]    At some point during a journey, the driver may enable the device, thus changing the context priority settings. Since the driver&#39;s phone would have been initially displayed, a context content setting corresponding to that display may now be the relevant or higher ranked setting. Accordingly, given the context shift, the process would update, for example, a primary center stack display  505  to show information relating to the driver&#39;s wireless phone. 
         [0059]    It also may be the case, in this example, that the display shift is not desired (for example, if the primary display is being used in a desired manner at the time, and the driver or other passenger does not want the shift in content to occur). Accordingly, in this example, an over-ride option is presented in conjunction with the content shift  507 . If the over-ride is not desired, the new content display will be maintained, otherwise the display may revert to display of the previous set of content  509 . 
         [0060]    This is just one example of how a content shift may occur based on a change of context while the vehicle is in motion. Other examples include, but are not limited to, change in other devices, alert conditions, service updates (e.g., without limitation, traffic, weather, etc.), time changes, etc. 
         [0061]      FIG. 6  shows an example of a dynamic content management process. In this illustrative example, the system has completed an analysis of all the available contexts, but cannot determine a preset display state corresponding to these contexts. This could be because display states have not been preset, or because the preset states do not account (in a precise or a suitably approximate manner) for the existing, detected, analyzed contexts. 
         [0062]    In this example, the process initiates a prediction routine designed to “guess” at a suitable use of vehicle outputs  601 . First, in this example, the process considers whether medical devices are present  603 . Although a predetermined list of considered devices is employed in this process, the order of considered devices could vary based on, for example, previously observed or set vehicle preferences. 
         [0063]    If one or more medical devices are present, the system may allocate some portion of a display to relevant outputs of those devices  605 . Prior to consideration of this, some portion of the display may have also been allocated or set aside for common functions such as, but not limited to, a navigation display. Again, predictive algorithms can be based on input preferences and/or preset conditions. 
         [0064]    Next, the process determines if any driver devices are present  607 . As previously noted, determination of whether a device belongs to a driver can be determined by, for example, recognition of a known device, near field communication, etc. If driver devices are present, space corresponding to those devices may be provided in one or more displays or outputs. If no space is available, due to, for example, all available space having been used in a reasonable manner to account for other services and devices, the driver device space may not be provided, but could, for example, be given a priority of display if a context shift occurs (for example, without limitation, if navigation is no longer needed). 
         [0065]    Next, in this process, the system determines whether additional space remains  611  (although this decision could be made between all device checks, or at any needed time). Although remaining device inputs are not considered in this example, if no space remains (or other output capacity), the devices were all previously considered as context sources, so the system remains aware of the devices and can adjust content as needed based on context changes. 
         [0066]    If there is still usable display/output available, the system checks to see if any primary passenger devices are available  613 . If so, and/or if there are devices that are “usable” or desirable for output, the system allocates the appropriate display/output  615 . 
         [0067]    Also, in this example, the process determines if rear displays or outputs are available  617 . These could include, but are not limited to, rear displays, rear speakers (on a different channel from front speakers), rear headset outputs, etc. If there is a rear device  619  and usable outputs desirable for content provision based on known context (which could also include non-rear devices, such as, for example, a dvd player), the process will provide the appropriate content to the suitable outputs  621 . 
         [0068]    While exemplary embodiments are described above, it is not intended that these embodiments describe all possible forms of the invention. Rather, the words used in the specification are words of description rather than limitation, and it is understood that various changes may be made without departing from the spirit and scope of the invention. Additionally, the features of various implementing embodiments may be combined to form further embodiments of the invention.