Patent Description:
Modern vehicles have many components and sensors. Such components and sensors may be referred to as data objects, and be queried and combined to provide insights into the vehicle and the operation of the vehicle.

However, utilizing current tree like structures to define the data objects is rigid, and can provide data files that are large and difficult to create. Further such tree-like data structures grow exponentially based on associations between the data objects.

<CIT> relates to a system and methods for detecting occupancy state in a vehicle having an interior passenger compartment. <CIT> relates to a display input device for correcting horizontal coordinates of an object to be detected. <CIT> relates to a user interface and a user interface client engine to render the user interface.

Accordingly there is provided a method, a computing device, and a computer program as detailed in the claims that follow.

The present disclosure provides a method at a computing device for finding an association between data objects in a structure, the method comprising: selecting a coordinate location from a grid, the grid defining locations within the structure; finding whether a data object having a data object type is located at the coordinate location; when the data object is located at the coordinate location, associating the data object with other data objects at the coordinate location; and when the data object is not located at the coordinate location: determining through an explicit association that the data object at a second location services the first location; and associating the data object with the coordinate location.

The present disclosure further provides a computing device configured for finding an association between data objects in a structure, the computing device comprising: a processor; and a communications subsystem, wherein the computing device is configured to: select a coordinate location from a grid, the grid defining locations within the structure; find whether a data object having a data object type is located at the coordinate location; when the data object is located at the coordinate location, associate the data object with other data objects at the coordinate location; and when the data object is not located at the coordinate location: determine through an explicit association based on coordinates that the data object at a second location services the first location; and associate the data object with the coordinate location.

The present disclosure further provides a computer readable medium for storing instruction code, which, when executed by a processor of a computing device configured for finding an association between data objects in a structure causes the computing device to: select a coordinate location from a grid, the grid defining locations within the structure; find whether a data object having a data object type is located at the coordinate location; when the data object is located at the coordinate location, associate the data object with other data objects at the coordinate location; and when the data object is not located at the coordinate location: determine through an explicit association that the data object at a second location services the first location; and associate the data object with the coordinate location.

In a modern vehicle, information from one or more physical sensors may be processed to create an "Insight" that may be valuable in a system. Such one or more physical sensors and the processing associated therewith may be referred to logically as a micro-service or a Synthetic Sensor (SS). The terms micro-service and synthetic sensor are used interchangeably herein.

Synthetic Sensors may exist in other types of applications, including but not limited to medical applications, manufacturing applications, Internet of Things applications, among others, and the present disclosure is not limited to vehicle applications. Vehicle applications are provided for illustration below.

Insight is the term used herein to describe any computer created sensor data based on an interpretation of a set of basic sensor data generated by objects representing vehicle components (data objects). Insights can be as straight forward as data aggregation or correlation or as complex as artificial intelligence and machine learning. For example, a temperature sensor providing high and low watermarks for notification may be considered an "insight". For location services, geofencing is an insight. For cameras, occupant recognition may be an insight. The use of a combination of sensors such as temperature sensors and cameras, may be used with an artificial intelligence model to determine whether a car seat is occupied in a hot vehicle, which may be an insight. Many other examples of insights are possible.

In one embodiment, the vehicle applications may be implemented in a system providing consistent access to vehicle data and intelligent insights in a way that is familiar and accessible to developer communities. Such environment may allow cloud developers to extend their reach to the edge within the vehicle through the development of Synthetic Sensors that derive intelligent insights on vehicle data using common cloud development technologies and paradigms. Such environment may provide consistent access to vehicle data such that Synthetic Sensors can be written and deployed to a broad vehicle base without bespoke customization.

However, in order for such vehicle data to be easily accessed, the data objects within the vehicle need to be identified, located, and associations between various data objects within the vehicle need to be easily identified.

This may be complicated by different vehicle manufacturers using different naming conventions and relationship conventions. In order to overcome this, the Vehicle Signal Specification (VSS) is currently being used. VSS defines itself as a way to create a common understanding of vehicle signals in order to reach a common language independent of the protocol or serialization used.

The VSS naming convention is, for example, described in Table <NUM> below.

As seen from Table <NUM> above, a dot notated name path is utilized in the VSS standards to identify the component and the last component in the notated name path is a signal or attribute.

Utilizing the dot notated name path, a tree structure can therefore be built for the various data objects within the vehicle. For example, reference is now made to <FIG>.

In the embodiment of <FIG>, an object tree <NUM> has the vehicle <NUM> as the root. Below this root, an example in the tree could be a cabin <NUM>.

Cabin <NUM> may have doors, shown as door <NUM>.

The doors may be defined based on the row in the vehicle, namely row1 <NUM> and row2 <NUM>.

Further, the door in each row may be specified based on the position in the vehicle. Specifically, the door in row1 <NUM> could be a left door <NUM> or a right door <NUM>. Similarly, a door in row2 <NUM> could be a left door <NUM> or a right door <NUM>.

The localization of instances of components in a tree structure is very rigid and has various shortcomings. Specifically, vehicles have components that have multiple instances. This causes the VSS tree to grow extremely large. The VSS specification is also not well suited for non-standard vehicles, such as motorcycles with side cars, buses, double decker buses, center drive vehicles, auto rickshaws, among others. As used herein, components are specific elements of a vehicle, and sets of sensors from these components may be referred to as data objects.

Further, some instances of these components are associated with other components. For example, there may be several seats in vehicle, and also several heating, ventilation and air conditioning (HVAC) units in the vehicle. The seats and HVAC units may be related to each other, but not necessarily be in the same position. For example, a middle seat in a second row of a vehicle may be serviced by HVAC units that are under the left and right front seats.

Modern vehicle systems need a way to represent components and associate them with other components, such that the users of vehicle services application program interfaces (APIs) can query the components and their associations easily, and therefore derive data from them.

Utilizing a tree structure such as that described with VSS, the listing of the relations between all pairs becomes exponential based on the number of components. Since modern vehicles have many components, the listing of such relations becomes cost prohibitive.

Therefore, in accordance with the embodiments of the present disclosure, a three dimensional grid structure to define the position of the component is provided. Such grid assigns a position to all components.

In particular, a three dimensional grid system would support any kind of vehicle, from a sedan to a double Decker bus, as well as motorcycles and oddly shaped vehicles such as the McLaren™ <NUM> seater.

As a consequence, a default relation also is created. Components in the same grid location may be automatically associated by default. For example, in a cabin API, given a seat, an API user can find the HVAC, screen, seat belt, window, among other objects that services the seat simply by the fact that they are located in the same grid position.

The grid system provides three separate coordinates to locate components in a vehicle. A single grid system for the entire vehicle or separate grid systems for various parts of the vehicles such as the cabin could be implemented. The three separate coordinates could be defined as a triplet such as an (x,y,z) triplet. However, in other embodiments, other ordering or coordinate systems could be utilized.

In one embodiment, the first coordinate could correspond with the concept of rows, the second coordinate could correspond with the concept of columns and the third coordinate could correspond with the concept of levels. In other embodiments, the coordinates could represent the rows, columns and levels in different orders.

Further, the rows could be in ascending order from front to back, the columns could be in ascending order from left to right, and the levels could be ascending from lowest to highest in some cases. However, this is merely provided for illustration and the ordering could be done in different ways.

Further, when defining a coordinate system, an index origin may be designated. In some cases, the index may be a zero or a one, for example. In some cases, a decision needs to be made on whether to support negative indices.

These and other elements are described below.

Reference is now made to <FIG>. In <FIG>, a vehicle <NUM> has a grid system <NUM> for identification of the location of data objects within the vehicle. As can be seen in the embodiment of <FIG>, the grid <NUM> includes two rows and three columns. Further, a single level exists within grid <NUM>.

The grid is organized in the example of <FIG> to be numbered from the front of the vehicle to the back of the vehicle. Therefore, the (x,y,z) triplet identifying each location within the grid utilizes the "x" value to represent the row, "y" value to represent the column, and the "z" value to represent the level. The "x" value counts from the front of the vehicle to the back of the vehicle, starting with an index of "<NUM>". This allows for an index of "<NUM>" to indicate "all rows", for example.

Similarly, columns are counted from left to right starting with an index value of "<NUM>".

Further, the levels in a multi level vehicle would start at level <NUM> for the level closest to the ground, level <NUM> for the next level, and so on.

Various components or data objects may be defined within grid <NUM>. For example, the embodiment of <FIG> shows an HVAC system <NUM> located in the rear middle seat, at grid position (<NUM>,<NUM>,<NUM>). As explained below, HVAC system <NUM> may automatically be associated with the rear middle seat based on the fact that it is colocated with the rear middle seat in the grid system. However, HVAC system <NUM> may further service the rear left seat at grid position (<NUM>,<NUM>,<NUM>), and this is shown with an association <NUM>, HVAC system <NUM> may further service the rear right seat at grid position (<NUM>,<NUM>,<NUM>), and this is shown with association <NUM>.

Similarly, an HVAC system <NUM> may service the seat at grid location (<NUM>,<NUM>,<NUM>) based on the fact that it is collocated with such seat. HVAC system <NUM> may service the seat at grid location (<NUM>,<NUM>,<NUM>) based on the fact that it is collocated with such seat.

A display system <NUM> may be found at grid location (<NUM>,<NUM>,<NUM>). In this regard, the display system <NUM> may service the seat at grid location (<NUM>,<NUM>,<NUM>).

The rear centre seat at grid location (<NUM>,<NUM>,<NUM>) does not have a display system at its location. However, the passenger in the rear centre seat could make use of either display system <NUM> or display system <NUM>. In this regard, an association <NUM> is provided for display system <NUM> at grid location (<NUM>,<NUM>,<NUM>) and an association <NUM> is provided for display system <NUM> at grid location (<NUM>,<NUM>,<NUM>).

The example of <FIG> is merely a simplified example of a grid system showing various components. Each component or data object may either be collocated with other data components, thereby creating a first implicit association between data objects in some cases, or may be associated with other grid locations using a second, explicit association. The embodiment of <FIG> shows only a cabin portion of a vehicle and the grid applied thereto. In other cases, the grid could be applied to the entire vehicle, or the vehicle could have multiple grid systems for the various spaces within the vehicle. Other options are possible.

While the embodiment of <FIG> defines the grid based on seats and rows in a cabin, in some cases, an Original Equipment Manufacturer (OEM) could decide to define a more granular grid for their vehicle. For example, the chosen granularity may be made by balancing the level of precision or granularity it provides against the loss of simplicity provided by collocation. Further, in some cases, an implicit "servicing" relationship created by collocation may not be what is wanted, necessitating the use of a more granular grid. That is, selecting a low granularity grid might create implicit associations that are not warranted in some cases, and thus a more granular grid may be chosen.

For example, a window may be clearly associated with the seat. However, position wise, other components may not be as clearly defined with regard to the seat.

Reference is now made to <FIG>, which shows a more granular grid <NUM> for a vehicle <NUM> than the grid of <FIG>. Specifically, the example of <FIG> shows a grid with four rows, three columns and one level.

In this case, the vehicle manufacturer could assign seat positions in the second and fourth rows. Specifically, front row seats <NUM> and <NUM> could be assigned to positions (<NUM>,<NUM>,<NUM>) and (<NUM>,<NUM>,<NUM>) respectively. Rear seats <NUM>, <NUM> and <NUM> could be assigned to positions (<NUM>,<NUM>,<NUM>), (<NUM>,<NUM>,<NUM>), and (<NUM>,<NUM>,<NUM>) respectively.

Row one could indicate the dashboard position and row three could be associated with the back of the seats or back of the front central console. In this regard, HVAC systems <NUM> and <NUM> could be positioned at (<NUM>,<NUM>,<NUM>) and (<NUM>,<NUM>,<NUM>). Such HVAC systems are for the seats at (<NUM>,<NUM>,<NUM>) and (<NUM>,<NUM>,<NUM>) respectively, and this is shown with HVAC associations <NUM> and <NUM>. As will be appreciated by those in the art, the creation of the more granular grid of <FIG> loses the implicit associations between the seats and HVAC units when compared with the embodiment of <FIG>. However, more specificity is gained.

Similarly, an HVAC system <NUM> that is centered in row <NUM> at position (<NUM>,<NUM>,<NUM>) could be used to serve the entire back row. This is shown with HVAC association <NUM> at position (<NUM>,<NUM>,<NUM>), HVAC association <NUM> at position (<NUM>,<NUM>,<NUM>), and HVAC association <NUM> at position (<NUM>,<NUM>,<NUM>).

A vehicle may, in some cases, include entertainment screens on the back of the front seats for the benefit of the rear passengers. This is shown in the embodiment of <FIG> as entertainment screen <NUM> at position (<NUM>,<NUM>,<NUM>) and entertainment screen <NUM> at position (<NUM>,<NUM>,<NUM>).

Screen <NUM> may be used by the occupant of seat (<NUM>,<NUM>,<NUM>), as shown with screen association <NUM>. Screen <NUM> may be used by the occupant of seat (<NUM>,<NUM>,<NUM>), as shown with screen association <NUM>. The occupant of the seat positioned at (<NUM>,<NUM>,<NUM>) may use either screen <NUM> or <NUM>, and therefore position (<NUM>,<NUM>,<NUM>) is shown to have screen associations <NUM> and <NUM>.

Therefore, the grid system as illustrated in <FIG> and <FIG> could be expanded to the granularity that is needed by the vehicle manufacturer or the entity defining the positions of data objects within the vehicle/space/structure.

In accordance with embodiments of the present disclosure, data objects at a particular location may be intrinsically associated with each other by being in the same grid location. Specifically, a seat belt sensor located in the same grid location as a seat would be intrinsically associated with the seat based on its location.

However, in some cases the intrinsic relationship based on the grid position may not be wanted, and therefore the intrinsic relationship is optional in some cases.

Further, as described with regards to <FIG> and <FIG>, in some embodiments there is a need to create associations or relationships between components that cannot easily be expressed based on location. While a window can easily be associated with a specific seat simply on the basis of both having the same coordinates, the association or relationship is not always positional. Referring to <FIG> and <FIG>, the HVAC system for the rear seat may be positioned in a middle location but be oriented towards a specific seat. A query within a computing system such as "Which HVAC is providing service to the seat at (<NUM>,<NUM>,<NUM>)" cannot be answered by checking if there is an HVAC at (<NUM>,<NUM>,<NUM>) when it is located centrally at (<NUM>,<NUM>,<NUM>).

Further, in some embodiments the "service" tags may not be static, but may rather be dynamic based on conditions. For example, in the embodiment of <FIG>, display <NUM> may be tilted such that it can be seen at positions (<NUM>,<NUM>,<NUM>) and (<NUM><NUM>,<NUM>). Display <NUM> may be oriented such that it is only viewable at location (<NUM>,<NUM>,<NUM>). The associations at position (<NUM>,<NUM>,<NUM>) may be dynamic and, for example, based on a tilt sensor within displays <NUM> and <NUM>.

In this regard, a different approach could be used, and the OEM may have the ability to define specific associations in addition to ones based on colocation. Referring to <FIG> and <FIG>, the associations are shown, for example, as associations <NUM> and <NUM> for HVAC system <NUM> in <FIG> and associations <NUM>, <NUM>, and <NUM> for HVAC system <NUM> in the embodiment of <FIG>. This may be done using "service" tags in one embodiment, as described below.

Utilizing the grid approaches described above with regard to <FIG> and <FIG>, a mechanism is provided that allows a vehicle manufacturer or other service provider to define the cabin content and coordinate assignment. More generally, a mechanism is provided that allows data objects to be assigned locations within a grid for a service area such as a vehicle.

The encoding of such coordinates could be done in various ways. The examples below provide JavaScript Object Notation (JSON) implementations. However, such JSON implementations are merely provided for illustration and those skilled in the art would appreciate that other notations could equally be used.

In the notations below, coordinates are a triplet expressed as a JSON array of three positive integers.

Further, in the notations below, the grid is being established for a cabin of the vehicle. Again, this is for illustration purposes only and the grid could be used with any number of structures. In the examples below, a "cabin" tag is used to represent a root of a cabin structure. If the grid is used for other purposes, the root could be something such as "chasis", "engine_compartment", among others for vehicle grids, "home" for home automation applications, among other examples.

A "coordinate" tag can be used anywhere to define specific coordinates. Its value is an array of three integers corresponding, respectively, to row, column and level. However, in some embodiments, the coordinate may only be two dimensional.

Typically, such coordinates tag would be used not directly under the "cabin" tag, but instead by other text to specify where such data object is located.

For example, reference is now made to Table <NUM> below:.

As seen in Table <NUM> above, the tag for a data object is referred to as "some-tag", and refers to a data object that is either predefined in the system or that can be created by various end users. In this case, the coordinate of the data object is specified as a triplet indicating that the object is located in row <NUM>, column <NUM>, and level <NUM>.

In addition to providing a coordinate, a data object may have information about the locations that the digital object services. For example, referring to Table <NUM> below, the data object may be specified to be in a particular coordinate location and to service various locations.

As seen from Table <NUM> above, the data object is in location (<NUM>,<NUM>,<NUM>), meaning that it is in the second row, second column and second level. The data object services locations (<NUM>,<NUM>,<NUM>), (<NUM>,<NUM>,<NUM>) and (<NUM>,<NUM>,<NUM>).

In the example of Table <NUM>, the object's own location is enumerated in the locations that it services. However, in some embodiments the own location can be omitted as the data object likely services its own location. However, in other cases, there are situations where a component or data object may not service its own grid location and therefore in some embodiments the locations that the data object services can explicitly include the data object's own location.

In the example of Table <NUM>, the serviced locations are provided as an array of triplets. However, other notations are possible.

In some cases, it may be beneficial to specify the size of the grid for the platform, vehicle or service area. This may be done, for example, using a "dimensions" tag, as provided for example in Table <NUM> below.

Thus, from Table <NUM>, the "dimensions" tag can be used to define the size and granularity of the grid for the cabin. Such dimensions tag can then be used by a program to find the maximum values, for example, for a row or column. This could be used to find the rear row of seats, the front passenger seat, among other options. For example, in a left drive vehicle, the driver may be a position (<NUM>,<NUM>,<NUM>). The dimensions may be used to find that there are <NUM> columns, and therefore the passenger would be at (<NUM>,<NUM>,<NUM>), where column <NUM> is the maximum value for the columns.

Further, in some embodiments when utilizing the grid for vehicles, it may be beneficial to specify the location of the driver's seat. This may be done, for example, utilizing a "drivers" tag. In a standard vehicle, the identification of the driver's seat may be done in accordance with Table <NUM> below.

From Table <NUM>, assuming the grid is specified in columns from left to right, the driver is indicated to be on the left side of the vehicle. For right hand drive vehicles, assuming the grid is specified in columns from left to right, the driver location would be specified as the maximum column value in some embodiments (assuming that an additional column for the area beside the seat is not added).

However, utilizing the grid coordinate system of the present disclosure, the specification is robust enough to capture outlying situations. For example, in the case of a street sweeper which may have two driving positions, this may be specified in accordance with Table <NUM> below.

In Table <NUM>, the drivers tag is specified with two locations, namely the front left and the front right seat locations.

Similarly, the seats can be specified using a "seats" tag to define the coordinates for each seat. An example is provided in Table <NUM> below.

As seen in Table <NUM>, various seats are defined in different locations.

Similarly, the doors on the vehicle can be specified utilizing a "doors" tag as provided in Table <NUM> below.

Table <NUM> shows the example of a four door vehicle. In a two door vehicle the coordinates may be associated with the front seats. On a bus the door may be located near the front and in the centre of the vehicle along one side, depending on whether the bus is a right hand or left hand drive bus, for example. Other options are possible.

Similarly, the windows may be specified using a "windows" tag. An example of such windows tag is shown with Table <NUM> below.

The windows in Table <NUM> may be those that can be raised or lowered in some embodiments. In this case, other windows within the vehicle may not be specified. This could be used, for example, to determine by a synthetic sensor whether the window is rolled up or rolled down.

In some embodiments it may be more efficient programmatically to label at least a subset of the various digital objects within the vehicle. For this reason, an ID tag could be used. In some embodiments the ID tag could be unique for a vehicle to allow the digital objects to be properly addressed. For example, in Table <NUM> below, the rear seats of the vehicle can be provided with an ID which could then be used by programs to find specific seats in the vehicle, find associations for a specific seat, or designate actions for specific seats in the vehicle.

As seen in Table <NUM> above, the rear seats are identified as "seat-RL", meaning rear left seat, "seat-RM", meaning rear middle seat, and "seat-RR", meaning rear right seat.

The HVAC system could be specified using an "hvac" tag, as provided in Table <NUM> below.

Similarly, the display screens could be specified using a "displays" tag, as provided in Table <NUM> below.

As seen in Table <NUM>, the displays could be provided with coordinates as well as identifiers.

Further, in order to define associations, various options may be implemented. In one case, an association may be explicitly defined through an "associations" tag. For example, if the HVAC system for the rear has been defined with an ID of "hvac-RM" (rear-middle HVAC system) utilizing the tags defined in Tables <NUM> and <NUM> above, indicating a rear middle HVAC system, then the association tag may be defined as follows in Table <NUM>.

As seen in Table <NUM>, each line specifies a single association. In this case, the association is specified based on the seat and an indication of the components or data objects that are servicing the seat. Thus, the rear middle seat, specified with ID tag "seat-RM", has an association with the rear middle HVAC system, the rear left display system, and the rear right display system.

However, other representations for associations are possible. For example, Table <NUM> and below shows a case in which multiple associations can be provided on a single line.

Therefore, in the example of Table <NUM>, the first element in each line defines the data object that is being serviced, and the remaining data objects in the line specify the services being provided. Thus, the rear middle seat is shown to be serviced by the rear middle HVAC system, the rear left display system and the rear right display system.

In other cases, a more explicit differentiation between the data object being serviced and the data objects providing the services can be enumerated in the association. For example, in Table <NUM> below, the data objects providing the services can be provided in an array.

Thus, the arrays for the data objects providing services/functionality in Table <NUM> can be a single element array or can have multiple elements that are providing services/functionality to the various seats.

In still further embodiments, the notation for the associations can be in reverse order. Specifically, the data object providing the functionality or service can indicate to whom such service or functionality is being provided. This is illustrated for example in Table <NUM> below.

Thus, as seen in Table <NUM>, the data object with the ID tag "hvac-RM" service is the data objects with ID tags "seat-RL", "seat-RM", and "seat-RR".

In further cases, the association may be defined as either unilateral or bilateral. A unilateral association would be in a single direction such as a HVAC service providing services to a seat. A bilateral association would be each data object providing services to the other data object, and may for example include sensors within the vehicle.

Further, in Tables <NUM> to <NUM> above, the associations were defined with regard to two data objects. For example, the associations indicate that an HVAC system services a particular seat. Rather than specify a particular data object, instead the association could be from a data object to a particular location. An example of a unilateral association specification to a location is shown with Table <NUM> below.

Thus, from Table <NUM>, the HVAC system located at (<NUM>,<NUM>,<NUM>) services locations (<NUM>,<NUM>,<NUM>), (<NUM>,<NUM>,<NUM>), and (<NUM>,<NUM>,<NUM>).

While Table <NUM> enumerates the associations as a single association on each line, in some cases the associations could be grouped. This is, for example, shown with Table <NUM>.

In the example of Table <NUM>, the first element after the HVAC tag could indicate the element providing the functionality or service and the remaining elements could indicate the locations that the service is being provided to.

Further, in some embodiments, the JSON files defining the data objects within the platform could include various information together. For example, the coordinates and the services could be specified together as shown in Table <NUM>.

Thus, in Table <NUM>, both the coordinates for the location of the data object and the locations that such data object services are specified together. In the example of Table <NUM>, components without a "services" tag are assumed to service their own location only.

Therefore, Tables <NUM> to <NUM> provide examples of various notations that could be utilized with the grid system of <FIG> and <FIG> to provide a robust system for locating and utilizing data objects within a system or platform. The notations provided in Tables <NUM> to <NUM> are merely provided for illustration and are not limiting. Those skilled in the art will realize that other notations could equally be used, and more or fewer tag or different tags could be used in a practical system.

Using the HVAC example of <FIG>, although the rear seat HVAC is located in the middle position, it's "area of effect" is clearly the full rear seat. Thus, in certain embodiments, it may be valuable to consider an area of effect independently of the position. In this case, the assignment of the HVAC system to the grid would be done based on the area of effect instead of the actual position in the cabin. In this case, the concept of a span may be supported.

In this case, a vehicle manufacturer may wish to place HVAC stations into the grid system at locations (<NUM>,<NUM>,<NUM>), (<NUM>,<NUM>,<NUM>) and (<NUM>,<NUM>-<NUM>,<NUM>) in the embodiment of <FIG>.

The term (<NUM>,<NUM>-<NUM>,<NUM>) indicates that in row <NUM>, level <NUM>, the HVAC system applies to all of columns <NUM> to <NUM>.

Thus, in one case, in addition to or instead of providing associations, spans may be provided to indicate the locations that the HVAC station services.

The grid provided above with regard to <FIG> and <FIG> and Tables <NUM>-<NUM> could be created by anyone responsible for a particular structure or space. For example, in a vehicle setting, an Original Equipment Manufacturer could be responsible for defining the grid and populating it with the data objects representing the various data objects for the vehicle. In other cases the provider of the synthetic sensors could create the vehicle definition in the grid structure. Other parties could also be involved.

In some cases, the grid could be added to by third parties after it has been created. For example, in a vehicle scenario, aftermarket sales of components may lead to the grid for a particular vehicle being updated. Other options are possible.

One simplified example of creating a vehicle definition in a grid structure is provided in <FIG>. The process of <FIG> starts at block <NUM> and proceeds to block <NUM> in which the entity creating the vehicle definition may define a grid granularity for such vehicle. For example, the entity may specify that the grid shown with regard to <FIG> is sufficient in some cases. In other cases, the grid specified with regard to <FIG> may be used. In other cases, other grids may also be used.

From block <NUM>, the proceeds to block <NUM> in which each data object that is within the structure may be assigned a grid position. The syntax or notation for assigning such grid position could for example use the examples in Tables <NUM>-<NUM> above.

Further, various data objects may be associated with, or service, other areas, and such associations or service areas could be defined at block <NUM>. Block <NUM> shows the explicit creation of an association for data objects. The association may be between two particular data objects, between a data object and a specific location, for a data object and a span of locations, among other options.

From block <NUM>, this is proceeds to block <NUM> and ends.

Various examples are provided below to indicate how the vehicle definition using the grid system of the present disclosure could be used in practice.

In one example, a system may need to know whether the driver's seat is occupied. The system would care about whether any of the driver's seats are occupied in the case that multiple driver seats could exist. For example, in a street sweeper case, an indication of whether driver is present should check whether the right hand or left hand drivers' seats are occupied.

Therefore, utilizing the grid system of the present disclosure and the example of Table <NUM>, an inquiry could be made to find all the driver seats and then to do a systematic check to find whether any of them are occupied.

Reference is now made to <FIG>, which shows an example process for determining whether a driver is present. The process starts at block <NUM> and proceeds to block <NUM> in which the coordinates of all possible driver's seats are determined. This could be based on the coordinates specified under the "drivers" tag, as for example defined in Table <NUM> above.

Once the coordinates of the drivers' seats are found, the identification of a seat itself could be made at block <NUM>. Block <NUM> may choose the first coordinate in the list of coordinates for driver's seats if multiple coordinates exist, and find the seat for that coordinate.

The process then proceeds to block <NUM> in which a check is made to determine whether the seat found at block <NUM> is occupied. This may occur, for example, utilizing a seat occupancy sensor such as a pressure sensor within the seat itself. The seat occupancy sensor could be associated with the coordinate for the seat or as an attribute of the seat itself.

If, at block <NUM>, it is determined that the seat is not occupied, the process proceeds to block <NUM> in which a check is made to determine whether coordinates for other driver's seats were returned at block <NUM>.

If yes, the process proceeds back to block <NUM> in which another possible driver's seat coordinate is chosen and the seat for that coordinate location is found.

Conversely, at block <NUM>, if no other driver's seats are found, the process proceeds to block <NUM> in which an indication is returned indicating that the driver's seat is not occupied. The process then proceeds to block <NUM> and ends.

Conversely, from block <NUM>, if the occupancy sensor for the seat finds that the seat is occupied, then the process may proceed from block <NUM> to block <NUM> in which the process may return that the driver's seat is occupied and provide information as to which seat that is (in cases where there may be more than one driver seat). Instead of returning a Boolean, a Seat object or "null" may be returned, with "null" representing "false" or "no driver seat is occupied". The process may then proceed to block <NUM> and end.

The embodiment of <FIG> may be expressed in pseudocode as shown in Table <NUM> below.

Thus, Table <NUM> and <FIG> provide an example in which the coordinates can be specified for which seat is a driver's seat and a systematic check can be made to determine whether such seat is occupied. This allows for cases where the vehicle is not a standard vehicle and therefore provides a flexible system.

From a vehicle abstraction layer perspective, the front passenger window may be on the left side of the vehicle or the right side of the vehicle, depending on the whether the vehicle is a right hand drive vehicle or a left hand drive vehicle.

In this regard, the grid coordinate system can be leveraged to find the correct window and determine whether such window is open.

Reference is now made to <FIG>, which shows an example process for determining whether a passenger window is open. The embodiment of <FIG> starts at block <NUM> and proceeds to block <NUM> in which the coordinates of the driver's seat can be found. This can be for example done by querying the configuration files for the vehicle and utilizing a configuration such as that shown with Table <NUM> or Table <NUM> above.

Based on the coordinates found at block <NUM>, the process proceeds to block <NUM> and finds the row and column for a driver's seat. In the example of <FIG> an assumption is made that there is only a single driver's seat. However, the process of <FIG> could easily be expanded to include multiple driver seats. In this case, the passenger side window may be determined based on embodiment of <FIG> and finding the side opposite an active driver, for example.

From block <NUM> the process proceeds to block <NUM> in which the dimensions of the grid for the vehicle or the cabin are found. This is done based on the assumption that the driver and passenger front seats are on the edges of the grid and may utilize configuration files for the vehicle specified by, for example, the original equipment manufacturer.

Based on the dimensions of the grid and the location of the driver's seat, the passenger seat may be found. For example, in a left hand drive vehicle, the driver may be at position (<NUM>,<NUM>,<NUM>). In this case, the passenger may be at the maximum column width of the grid. Utilizing the embodiment from <FIG> and dimensions as for example specified in Table <NUM>, the maximum column is <NUM> and therefore the passenger seat would be found at (<NUM>,<NUM>,<NUM>).

Conversely, in a right hand drive vehicle, the driver may be at position (<NUM>,<NUM>,<NUM>). Since the driver is at the maximum column value, in this case it may be assumed that the passenger seat is at the minimum column value, or at (<NUM>,<NUM>,<NUM>).

Based on the determination at block <NUM>, the process proceeds to block <NUM> in which a window is chosen in the driver's row, and at the column that is opposite the driver.

At block <NUM>, a check may be made on the chosen window to determine whether it is open. In some embodiments, a window sensor may indicate how much a window is open, from <NUM>% indicating that the window is closed to <NUM>% indicating that the window is fully open. In this case, determination at block <NUM> may find that the window is open if the percentage is anywhere over <NUM>%. However, in other cases, the check may allow for slightly ajar windows and the check at block <NUM> may determine whether the window is open more than <NUM>% for example. Other options are possible.

From block <NUM>, if the window is open, the process proceeds to block <NUM> and returns true. Conversely, if it is determined that the window is closed, the process proceeds from block <NUM> to block <NUM> in which the process returns a false for the window being open.

From blocks <NUM> and <NUM>, the process proceeds to block <NUM> and ends.

The embodiment of <FIG> may, for example, be expressed in pseudocode as shown in Table <NUM> below.

In Table <NUM>, the column is determined to be either a "<NUM>" or the "maxColumn" based on the side that the driver is on.

The window is classified as a window object. However, in some cases the window may be a subset of another object such as an automobile.

The pseudocode and example of <FIG> provide for the ability to utilize coordinates to easily find the correct passenger side window.

In another example use case, a determination may be made of which HVAC station is providing service to a particular seat. This may, for example, utilize the "associations" tag provided for in Tables <NUM> to <NUM> above.

Reference is now made to <FIG>, which shows an example of the use of associations to find the HVAC system associated with a seat. In particular, the process starts at block <NUM> and proceeds to block <NUM> in which the coordinates of the seat of interest are found.

The system next proceeds to block <NUM> in which any HVAC system with the same coordinates is retrieved.

The process then proceeds to block <NUM> in which a check is made to determine whether any HVAC systems were found at block <NUM>. In particular, referring to <FIG>, if the coordinates for the seat of interest were (<NUM>,<NUM>,<NUM>), this would indicate the driver's seat and an HVAC system would be returned. Conversely, if the seat of interest had coordinates (<NUM>,<NUM>,<NUM>), this is the rear left seat and no HVAC system having the same coordinates would be found.

If the check at block <NUM> finds that one or more HVAC systems are retrieved at block <NUM>, the process proceeds to block <NUM> in which pointers or data links to the one or more HVAC systems are returned. The process then proceeds to block <NUM> and ends.

Conversely, if at block <NUM> it is found that zero HVAC systems were retrieved, the process proceeds from block <NUM> to block <NUM> in which the seat object for the coordinates is found. For example, this could involve finding the identifier tag for the seat if identifier tags are used in associations.

The process then proceeds to block <NUM> in which associations for the seat object are retrieved from the list of associations stored for the vehicle. At this point, pointers or data links to the associated HVAC system could be returned, as shown at block <NUM>.

The process then proceeds to block <NUM> and ends.

As can be seen in <FIG>, a first association based on colocation can be checked. If no first association exists, a second association based on an explicit association table or list can be found. In each case, the coordinates of the seat of interest are utilized as the basis for the search.

Example pseudocode for the embodiment of <FIG> is provided below in Table <NUM>.

As will be appreciated by those skilled in the art, the examples of <FIG> and Table <NUM> are simplified versions of the code that would likely be used in practice. In particular, the example of <FIG> relies on a unilateral association and does not handle the possibility that an identifier can be associated with more than one component. Specifically, if an identifier is associated with more than one component, a check to find the component of interest may be needed. This may rely, in some cases, on the association between identifiers and component types.

However, the embodiment of <FIG> utilizes coordinates to find data objects of interest that are collocated as a first check. The embodiment of <FIG> then utilizes associations that are enumerated to find associated data objects of interest.

Similar to associations, a span may be used to define the area that a data object services. In this regard, in some embodiments a span may be used to find which HVAC unit services a seat.

Reference is now made to <FIG>. In the embodiment of <FIG>, the process starts at block <NUM> and proceeds to block <NUM> in which the coordinates of the seat of interest are found.

The process then proceeds to block <NUM> in which any HVAC units with the same coordinates are retrieved. At block <NUM> a check is made to determine whether an HVAC system with the same coordinates exists.

If an HVAC system with the same coordinates exists, the process proceeds to block <NUM> in which a pointer or data reference to the HVAC system is returned. The process then proceeds to block <NUM> and ends.

Conversely, if at block <NUM> it is found that zero HVAC systems with the same coordinates exist, the process proceeds from block <NUM> to block <NUM> in which HVAC spans are found. The process then proceeds to block <NUM> in which an HVAC for the seat coordinates based on the spans is identified.

The process then proceeds to block <NUM> in which a pointer or data link to the HVAC system found at block <NUM> is returned.

From block <NUM>, the process proceeds to block <NUM> and ends.

Pseudocode for the example of <FIG> is provided below with regard to Table <NUM>.

Therefore, <FIG> and table <NUM> provide options for using spans to create the associations.

The embodiments of <FIG> can be generalized for the finding of the associations between data objects. In particular, reference is now made to <FIG>.

In the embodiment of <FIG> the process starts at block <NUM>. Typically, this occurs when an abstraction layer or program needs to know what data objects are associated with a particular data object or a particular location. As indicated above, this may be "what HVAC system services this seat", but may include "what audio source services these speakers", among other options. Such question may be generalized as "what second data object services this first data object".

The process proceeds to block <NUM> in which the coordinates of the first data object are found. As will be appreciated by those in the art, if rather than a first data object, the question asked by the program or abstraction layer was "what second data object services this coordinate location" then block <NUM> may be optional as the coordinate is already known.

The process then proceeds to block <NUM> in which a check is made to determine whether a second data object is collocated with the first data object. In particular, the collocation may provide a first, implicit, association between the data objects.

The process then proceeds to block <NUM> in which a check is made to determine whether any second data objects were found. If one or more data objects are found then, based on the colocation, the second data object may be returned at block <NUM>. The returning the second data object may involve a pointer, data reference, among other options.

The process proceeds from block <NUM> to block <NUM> and ends.

Conversely, if zero second data objects are found to be collocated with the first data object at block <NUM> (or zero second data objects are found at a particular coordinate location) the process proceeds to block <NUM> in which associations for the first data object or location may be found. Such finding at block <NUM> may use the explicit associations defined in the vehicle definition utilizing the grid structure of the present disclosure.

The process then proceeds block <NUM> in which the associated second data object may be returned. This may be done, utilizing a pointer, data reference, among other options.

From block <NUM> the process proceeds to block <NUM> and ends.

The present disclosure is described with regard to an automotive system with nodes. However, this is merely provided for illustration purposes and the methods and systems described herein could equally be used with any other systems.

For example, reference is now made to <FIG> which shows a node <NUM>. A node, as used herein, may be one or a group of electronic control units, central processing units, or kernel controls, among other options, and can be considered as a single computing unit.

In the example of <FIG>, node <NUM> includes a services manager <NUM> which may interact with drivers for sensors that the node is connected to. For example, the node <NUM> may have access to a location sensor such as a Global Positioning System (GPS) chipset, as shown at block <NUM>.

In order to allow node <NUM> to interact with modules on other nodes, and to provide functionality with a computing system, a Hardware Abstraction Layer (HAL) may be provided on node <NUM>, which comprises a HAL service <NUM>. Each HAL service <NUM> is responsible for the integration of a sensor and may provide various functions, including: integration to the underlying sensor; normalization of the sensor data; and/or, if required, providing a barrier between the safety certified and noncertified software. Other functions for the HAL service are also possible.

In the example of <FIG>, the HAL is provided for camera information, as shown with block <NUM>.

While the example of <FIG> shows a node <NUM> with a single service and a single HAL, this is merely provided for illustration purposes. A node <NUM> could have a single service without a HAL, a single HAL without a service, a plurality of services with no HAL, a plurality of HALs with no service, and/or a combination of services and HALs.

One example of a system that could use node <NUM> would be an applications development environment for vehicles. Such applications development environment may develop applications for user experience including comfort, navigation, infotainment, among others, applications for safety, applications for fleet management; applications for performance monitoring; or other such applications for a vehicle environment. In particular, vehicles provide a plurality of sensors and different makes, models or brands may use different sensors with different data formats or values, creating fragmented sensor readings depending on such sensor. This fragmentation impedes the fostering of an application ecosystem that makes use of vehicle data. In addition, low level sensor data is often too granular to be readily useful to applications.

In this regard, the Hardware Abstraction Layer can be used to provide hardware independent sensor interfaces/abstractions which may encapsulate interaction with underlying sensor drivers. The use of the hardware abstraction in the various computing nodes creates a platform that is extensible, can provide barriers between modules to, for example, enforce safety certified systems being separate from other systems, among other options.

Applications do not interact directly with sensor hardware to access sensor data, instead they leverage the Hardware Abstraction Layer. This separation provides a clear distinction between the responsibilities of the HAL (sensor integration and normalizing sensor data) and other abstractions such as a Vehicle Abstraction Layer (VAL), used for managing access to vehicle data and providing value-added insights.

Specifically, insights may leverage sensor data from multiple HALs in order to provide vehicle abstraction and value-added insights. Vehicle insight services in the VAL may control access to a normalized form of vehicle data and provide value-added inferences. Examples of insight services may include a Location Service, which may provide coordinate location data in a consistent format as well as insights such as geofencing; a Seat Service, which may provide a myriad of seat information such as belt status, weight, position, and child lock status; a Camera Service which may provide the video stream for in-cabin camera and possibly functions such as conversions and/or clipping; a Battery Service which may provide insights and access to battery such as charge state/consumption/projected hours remaining/projected range; a Door Service which may provide an abstraction for vehicle doors and door status; among others.

Insight Services may leverage sensor data from multiple HALs in order to provide vehicle abstraction and value-add insights. Higher level insights on data enable application developers to create future automotive experiences. Insight is the term used to describe any value-added interpretation of basic sensor data. Insights can be as straight forward as data aggregation or correlation or as complex as artificial intelligence and machine learning. For example, for a temperature sensor, providing high and low watermarks for notification may be considered an "insight". For location services - geofencing is an insight, for cameras - occupant recognition may be considered an insight.

Nodes with services and HAL managers as described in <FIG> are therefore needed for such application development environment. Node <NUM> will, in many systems, need to communicate with other nodes, such as other ECUs, CPUs, or computing systems, where such ECUs, CPUs, or computing systems may use a different operating system from that of node <NUM>.

Such system may use a grid system with the vehicle to find data objects and data object associations. In this regard, a vehicle abstraction layer can easily provide information regardless of the specific make and model of the vehicle for which information is sought.

The above network elements, nodes, computing devices and other computing platforms may be implemented using any computing device. One simplified diagram of a computing device is shown with regard to <FIG>. The computing device of <FIG> could be any fixed or mobile computing device.

In <FIG>, device <NUM> includes a processor <NUM> and a communications subsystem <NUM>, where the processor <NUM> and communications subsystem <NUM> cooperate to perform the methods of the embodiments described above. Communications subsystem <NUM> allows device <NUM> to communicate with other devices or network elements and may vary based on the type of communication being performed. Further, communications subsystem <NUM> may comprise a plurality of communications technologies, including any wired or wireless communications technology.

Processor <NUM> is configured to execute programmable logic, which may be stored, along with data, on device <NUM>, and shown in the example of <FIG> as memory <NUM>. Memory <NUM> can be any tangible, non-transitory computer readable storage medium which stores instruction code that, when executed by processor <NUM> cause device <NUM> to perform the methods of the present disclosure. The computer readable storage medium may be a tangible or in transitory/non-transitory medium such as optical (e.g., CD, DVD, etc.), magnetic (e.g., tape), flash drive, hard drive, or other memory known in the art.

Alternatively, or in addition to memory <NUM>, device <NUM> may access data or programmable logic from an external storage medium, for example through communications subsystem <NUM>.

In the example of <FIG>, one or more internal sensors <NUM> or external sensors <NUM> may be associated with the computing device. However, this is optional and in some cases computing device <NUM> will not be associated with sensors.

Communications between the various elements of device <NUM> may be through an internal bus <NUM> in one embodiment. However, other forms of communication are possible.

While operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be employed. Moreover, the separation of various system components in the implementation described above should not be understood as requiring such separation in all implementations, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

While the above detailed description has shown, described, and pointed out the fundamental novel features of the disclosure as applied to various implementations, it will be understood that various omissions, substitutions, and changes in the form and details of the system illustrated may be made by those skilled in the art. In addition, the order of method steps are not implied by the order they appear in the claims.

Typically, storage mediums can include any or some combination of the following: a semiconductor memory device such as a dynamic or static random access memory (a DRAM or SRAM), an erasable and programmable read-only memory (EPROM), an electrically erasable and programmable read-only memory (EEPROM) and flash memory; a magnetic disk such as a fixed, floppy and removable disk; another magnetic medium including tape; an optical medium such as a compact disk (CD) or a digital video disk (DVD); or another type of storage device. Note that the instructions discussed above can be provided on one computer-readable or machine-readable storage medium, or alternatively, can be provided on multiple computer-readable or machine-readable storage media distributed in a large system having possibly a plurality of nodes. Such computer-readable or machine-readable storage medium or media is (are) considered to be part of an article (or article of manufacture). The storage medium or media can be located either in the machine running the machine-readable instructions, or located at a remote site from which machine-readable instructions can be downloaded over a network for execution.

Claim 1:
A method at a computing device for finding an association between data objects in a vehicle, the method comprising:
assigning, to each of a plurality of components of the vehicle, a respective data object, each respective data object having a location in a grid structure (<NUM>) based on a position of a corresponding component (<NUM>, <NUM>, <NUM>) in the vehicle, wherein the grid structure defines locations (<NUM>,<NUM>,<NUM>; <NUM>,<NUM>,<NUM>; <NUM>,<NUM>,<NUM>) within the vehicle (<NUM>);
selecting a coordinate location from the grid structure;
finding whether a data object having a data object type is located at the coordinate location;
when the data object is located at the coordinate location, associating the data object with other data objects at the coordinate location;
when no data object is located at the coordinate location,
determining through an explicit association (<NUM>, <NUM>) that a second data object at a second location services the coordinate location and associating the second data object with the coordinate location;
obtaining an associated data object using an association of one of the data objects in the vehicle and the second data object; and
querying the associated data object.