Displaying a window in an augmented reality view

For displaying a window in an augmented reality view, a processor detects a new augmented reality placetime that includes a new augmented reality position and/or a new augmented reality time of an augmented reality device. The processor calculates new window characteristics for a window at the new augmented reality placetime based on previous window characteristics. The processor further displays the window with the new window characteristics.

FIELD

The subject matter disclosed herein relates to displaying a window and more particularly relates to displaying a window in an augmented reality view.

BACKGROUND

Augmented reality views change when the environment of an augmented reality device changes.

BRIEF SUMMARY

An apparatus for displaying a window in an augmented reality view is disclosed. The apparatus includes a processor and a memory that stores code executable by the processor. The processor detects a new augmented reality placetime comprising a new augmented reality position and/or a new augmented reality time of an augmented reality device. The processor calculates new window characteristics for a window at the new augmented reality placetime based on previous window characteristics. The processor further displays the window with the new window characteristics. A method and program product also perform the functions of the apparatus.

DETAILED DESCRIPTION

More specific examples (a non-exhaustive list) of the storage device would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

Code for carrying out operations for embodiments may be written in any combination of one or more programming languages including an object oriented programming language such as Python, Ruby, R, Java, Java Script, Smalltalk, C++, C sharp, Lisp, Clojure, PHP, or the like, and conventional procedural programming languages, such as the “C” programming language, or the like, and/or machine languages such as assembly languages. The code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, entirely or partly on a head mounted display such as an augmented reality device, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).

Reference throughout this specification to “one embodiment,” “an embodiment,” or similar language means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, appearances of the phrases “in one embodiment,” “in an embodiment,” and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment, but mean “one or more but not all embodiments” unless expressly specified otherwise. The terms “including,” “comprising,” “having,” and variations thereof mean “including but not limited to,” unless expressly specified otherwise. An enumerated listing of items does not imply that any or all of the items are mutually exclusive, unless expressly specified otherwise. The terms “a,” “an,” and “the” also refer to “one or more” unless expressly specified otherwise. The term “and/or” indicates embodiments of one or more of the listed elements, with “A and/or B” indicating embodiments of element A alone, element B alone, or elements A and B taken together.

FIG.1Ais a perspective drawing illustrating one embodiment of an augmented reality (AR) device120. In the depicted embodiment, the AR device120includes a camera125. The AR device120may be worn by a user who views a display within the AR device120. The AR view of the AR device120may include both the environment and images that are presented on the display by the AR device120.

FIG.1Bis a perspective drawing illustrating one embodiment of an AR view100. The AR view100is presented to the user by a display of the AR device120. The AR view100includes the environment103a. The AR view100further includes a window105that comprises video, graphics, text, and the like that is superimposed over the environment103aby the AR device120. Although the window105is depicted as a flat screen, the window105may be presented with any shape and in either two or three dimensions. In one embodiment, the window105is presented as though disposed on a surface of the environment103a. For example, a television window105may be presented as though disposed on a wall of a room.

FIG.1Cis a perspective drawing illustrating one alternate embodiment of an AR view100. In the depicted embodiment, a new environment103bis captured by the camera125at a different AR position and/or AR time from the given AR position and time of the environment103aofFIG.1B. The new environment103bincludes objects110that were not in the previous environment103a. As a result, displaying the window105at the same location in the new environment103bas was used in the original environment103amay not be desirable.

Unfortunately, the user may wish to view the window105. Because presenting the window105at the same location of the previous environment103awould degrade the user's viewing experience, the embodiments determine a new location and other window characteristics for the window105in the new environment103b. The embodiments detect a new AR placetime for the AR device120and calculate new window characteristics for the window105at the new AR placetime based on previous window characteristics at a previous environment103a. In addition, the embodiments display the window105with the new characteristics. As a result, the window105is automatically repositioned to a new location within the new environment103b, improving the efficiency and efficacy of the AR device120as will be described hereafter.

FIG.2Ais a schematic block diagram illustrating one embodiment of AR data200. The AR data200may be used to calculate the new window characteristics203for the window105and display the window105with the new window characteristics203. The AR data200may be organized as a data structure in a memory. In the depicted embodiment, the AR data200includes window preferences201, one or more window characteristics203, thresholds225, and current activity227.

The window preferences201may record user preferences for the placement of windows105within an AR view100. The window preferences201may include a preferred window angular size, a docking preference, a preferred window angular orientation, and/or a preferred window density. The preferred window angular size may specify a maximum window angular size, a minimum angular window size, and/or an average window angular size. The window angular size is an apparent maximum window dimension measured in degrees of the window105as seen from the user's eye. Alternatively, the window angular size may be measured in pixels.

The docking preference may specify whether a window105is presented as docked, wherein the window105appears disposed on a surface such as a wall, or presented undocked as though floating in the air. The preferred window angular orientation may specify a maximum angular difference between a vector from the user's eye to the center of the window105, and an apparent normal vector radiating from the center of the window105. The preferred window density may specify a percentage of the AR view100over which windows105are overlaid. In one embodiment, the preferred window density is for a specified view angle such as a 30 degree angle as measured from the user's eye. Alternatively, the preferred window density may be for the total pixels presented by the AR device120.

The window characteristics203describe presentation characteristics of a window105. The window characteristics203are described in more detail inFIG.2B. The thresholds225include one or more thresholds used to make automatic decisions. The current activity227records the current activity of the user with the AR device120in the environment103.

FIG.2Bis a schematic block diagram illustrating one embodiment of the window characteristics203. The window characteristics203maybe organized as a data structure in a memory. In the depicted embodiment, the window characteristics203include a window size205, a window angle207, a window distance209, a window orientation211, the AR position213, the AR time231, AR people235, an AR context237, a window timestamp215, a surface relationship217, surface characteristics219, a reposition status221, a usage history223, and user activity229. An AR placetime233may comprise the AR position213, AR time231, AR people235, and/or AR context237.

The window size205may record the size of the window105as an angle is viewed from the user's eye and/or in pixels. For example, a window105may have a window size205of 15 degrees and/or 300 pixels. The window angle207may specify an angle of an apparent normal vector from a center of the window to a vector from the user's eye to the center of the window105. The window distance209may be an apparent distance in the environment103from the user's eye to the window105. The window orientation211may record whether the window105is presented in landscape or portrait.

The AR position213specifies the position of the window105within the AR view100. The AR time231may record a time interval when the window105was displayed at the AR position213. The AR people235may record other people in the AR view100when the window105was displayed. The AR context237records the context of usage of the AR device120and/or activity displayed in the AR view100.

The window timestamp215records when the window105was last displayed using the window characteristics203. The surface relationship217records whether the window105was docked to a surface such as a wall. The surface characteristics219record characteristics of any surface that the window105was docked to. The surface characteristics219may include a surface color, a surface uniformity, and object density, and the like. The surface color may record the average color of the surface. The surface uniformity may record an average deviation from the surface color across the surface. The object density may record a percentage of the surface area of the surface that includes objects that form discontinuities in the surface.

The reposition status229records details of the last repositioning of the window105within the AR view100. The usage history223records uses of the window105. The user activity229records other interactions of the user with the AR view100while the window105was active.

FIG.3is a perspective drawing illustrating one embodiment of an AR view100with new window characteristics203. The AR view100of the new environment103bofFIG.1Cis shown. The window105is displayed with new window characteristics203that conform to the new, current environment103b. As a result, the user may view the window105in the new environment103b, giving user access to customary information in the window105and improving the function and efficiency of the AR device120.

FIG.4is a schematic block diagram illustrating one embodiment of the AR device120. In the depicted embodiment, the AR device120includes a processor405, a memory410, communication hardware415, the camera125, and a display130. The memory410may comprise a semiconductor storage device. The memory410stores code. The processor405executes the code. The communication hardware415communicates the camera125, the display130, and/or with other devices such as a network, server, and/or computer. The display130may be disposed in the AR device120and may be an optical engine of the AR device120.

FIG.5is a schematic flow chart diagram illustrating one embodiment of a window display method500. The method500may display the window105with the new window characteristics203. The method500may be performed by the AR device120and/or the processor405.

The method500starts, and in one embodiment, the processor405records501the window characteristics203for the AR view100in the previous environment103a. The processor405may record a plurality of previous window characteristics203for each environment103in which the AR device120is employed.

The processor405further detects503a new AR placetime233using the camera125. The new AR placetime233may be different from the most recent previous AR placetime233. In addition, the new AR placetime233may be different from all previous AR placetimes233. In one embodiment, the new AR placetime233is detected from a new AR position213. For example, if the AR device120is used at a new AR position213, the new AR placetime233may be detected.

In addition, the new AR placetime233may be detected503from a new AR time231. For example, if the AR device120is used at a new AR time231, the new AR placetime233may be detected503. The new AR placetime233may be detected503in response to elapsed time from the window timestamp215exceeding an elapsed time threshold225.

In one embodiment, the new AR placetime233is detected503in response to new AR people235in the environment103. For example, if a new colleague is visible in the AR view100, a new AR placetime233may be detected503.

The new AR placetime233may also be detected503from a change in AR context237. For example, if the AR device120was moving in the previous environment103aand is stationary in the new, current environment103b, the new AR placetime233may be detected503. In one embodiment, the new AR placetime233is detected503from a change in any of the window characteristics203. The new AR placetime233may be automatically detected503.

In response to detecting503the new AR placetime233, the processor405may determine505whether to redeploy the window105within the AR view100. The window105may be redeployed with new window characteristics203. In one embodiment, the processor405queries to reposition the window105. The processor405may query the user as to whether to reposition the window105. For example, the processor405may present a “Reposition Window” option to the user. The processor405may determine505to redeploy the window105in response to an affirmative reply from the user. Thus, the window105may be displayed with the new window characteristics203in response to being directed to reposition the window105.

In one embodiment, the processor405determines505to redeploy the window105within the AR view100if the usage history223and/or user activity229of a previous window characteristic203is similar to the current activity227. For example, if the user was previously performing an inspection user activity229recorded in a previous window characteristic203and the user is now performing an inspection current activity227, the processor405may determine505to redeploy the window105.

In one embodiment, the processor405determines505to redeploy the window105within the AR view100if the current time is within a daily time interval of the AR time231of a previous window characteristic203. As used herein, a daily time interval is a range of times on any time and/or a specific day. For example, 10:00 to 12:00 on Monday may be a daily time interval.

In one embodiment, the processor405automatically determines505to redeploy each window105from a most recent previous environment103within the AR view100. For example, if a first window105was displayed the last time the AR device120was used, the first window105is determined to be redeployed.

In one embodiment, the processor405determines505not to redeploy the window105if the elapsed time from the window timestamp215exceeds a significant time threshold225. If the processor405determines505not to redeploy the window105, the processor405may query513the user for new window characteristics213. For example, the processor405may present a user interface that queries513for the new window characteristics213. The processor405may further receive the new window characteristics213from the user.

If the processor405determines505to redeploy the window105, the processor405may determine507the window preferences201for the window105displayed at one or more previous AR placetimes233. In one embodiment, the window preferences201are calculated to conform to one or more elements of one or more previous window characteristics203. In addition, the window preferences201may be received from the user.

The processor405may calculate509the new window characteristics203for the window105. The new window characteristics203may be calculated509for the new AR placetime233.

In one embodiment, the new window characteristics203are based on one instance of previous window characteristics203. The one instance of previous window characteristics203may be selected based on the current activity227. For example, the one instance of previous window characteristics203may record user activity229that is equivalent to the current activity227of the user.

In a certain embodiment, the new window characteristics203are based on an instance of the previous window characteristics203with the usage history223that conforms to the current activity227. In addition, the new window characteristics203may be based on an instance of the previous window characteristics203with a reposition status221that is most similar to the current activity227. In one embodiment, the new window characteristics203are based on an instance of the previous window characteristics203with surface characteristics219that are most similar to the surface characteristics219of the current, new environment103b.

In one embodiment, the new window characteristics203are calculated509based on an average, median, and/or mean of the plurality of previous window characteristics203. In addition, the new window characteristics203may be calculated509based on an average, median, and/or mean of one or more elements of the plurality of previous window characteristics203.

The processor405may calculate509the new window characteristics203for the window105at a new AR placetime233based on previous window characteristics203. For example, one or more of the window size205, the window angle207, the window distance209, and/or window orientation211may be within a similarity threshold225of the window size205, the window angle207, the window distance209, and/or window orientation211of the previous window characteristics203. The similarity threshold may be plus or minus 5 to 20 percent. In addition, the new window characteristics203may conform to the window preferences201.

The processor405may display511the window105with the new window characteristics203and the method500ends. The window105may be displayed511in the AR view100for the new environment103b.

The embodiments detect the new AR placetime233and calculate new window characteristics for the window105at the new AR placetime233based on previous window characteristics203. The window105is displayed with the new window characteristics203. As a result, the window105and information displayed thereon is automatically available to the user through the AR device120when the user moves between environments103, improving the efficiency and effectiveness of the AR device120in presenting information to the user.