Patent Description:
However, while some of these devices are able to detect the proximity of an electronic pen to a screen of the devices (e.g., when the pen is in proximity, the device becomes aware of the device presence), the position of the device relative to the screen (e.g., right side or left side) if very difficult or impossible to determine. For example, some systems use a received signal magnitude to detect pen presence. But, this signal magnitude is not indicative of the pen position relative to the device. Specifically, because signal magnitude drops slowly, it is hard to determine a position of the pen, particularly if the pen is located on the side of the device. Moreover, the equi-capacitance to the device surfaces where the pen can reside around the device can lead to ambiguity with respect to the pen's actual position, and therefore a desired pen operation.

<CIT> discloses a touch-sensing system which includes a display device having a touch sensor with a matrix of row electrodes and column electrodes. Drive logic drives the row electrodes in a plurality of stylus sync sub-frames. In each, some row electrodes, referred to for that stylus sync sub-frame as sync-driven row electrodes, are driven by the drive logic with synchronization waveforms to synchronize the display device with an active stylus. For each stylus sync sub-frame, the sync-driven row electrodes are differentially driven by the drive logic, in the sense that a synchronization waveform used to drive one of the sync-driven row electrodes is different than a synchronization waveform used to drive another of the sync-driven row electrodes.

Furthermore, <CIT> discloses a movable data entry device which has two output elements separated from each other. Each of the output elements are repeatably powered to send a repeated signal from the device, and power between each of the elements is toggled. As the data entry device is moved along a path on a writing area of a surface, such as a white board, a transmitted signal is alternately sent from the first output element and the second output element, and is received by two or more external receivers. Triangulation methods are used to determine the location of output elements on the movable data entry device, in relation to the writing area.

A method comprises transmitting signals from a plurality of regions of an electronic device. The method further comprises receiving location information from an electronic pen in proximity to the electronic device. The location information is determined by the electronic pen and corresponds to a location of the electronic pen relative to a screen of the electronic device. The location information is determined using one or more of the transmitted signals received by the electronic pen. The method also comprises controlling a display on the screen of the electronic device based at least in part on the received location information.

In the figures, the systems are illustrated as schematic drawings. The drawings may not be to scale.

The computing devices and methods described herein transmit signals from different parts of a screen of an electronic device (e.g. electronic smartboard, such as the Microsoft Surface Hub®) to allow an electronic pen in proximity to the interface surface of the electronic device to detect or determine a position of the electronic pen relative to the electronic device. In some examples, an active pen is thereby able to know a position with respect to an inking device. As a result, an operational behavior of the pen can be change, such as changes in power modes (idle/active) and operation cycles (searching for other devices).

Touch sensors transmit beacons (uplink) signals that can be received by the pens to indicate to the pens the device presence and communication protocol. In some examples, having the beacons transmitted from different parts of the device, either at different times, or simultaneously using different signals, enables the receiving pen to determine a three-dimensional (3D) position of the pen relative to the device. Thus, by transmitting the uplink signal, for example, from only one part of the screen (and toggling over time to the other parts) enables the pen's 3D position relative to the device to be extracted and determined by the pen (e.g., the pen decodes signals received from different parts of the electronic device to determine position). Alternatively, by transmitting different signals (e.g., Tx, Tx', Tx", Tx‴ having different information) from different parts of the screen simultaneously, the same location information can be determined. As a result, a more efficient pen operating state is thereby provided that improves the user experience. In this manner, when a processor is programmed to perform the operations described herein, the processor is used in an unconventional way, and allows for the more efficient user input with the device.

<FIG> illustrates a system <NUM> including an electronic device <NUM> and an associated pen <NUM> according in one example. The electronic device <NUM> and pen (or pen device) <NUM> are associated or linked such that the electronic device <NUM> and the pen <NUM> respond to each other. In one example, the pen <NUM> is uniquely associated with the electronic device <NUM> such that other pens and/or similar devices do not interact or interact differently with the electronic device <NUM>.

The electronic device <NUM> includes a screen <NUM> defining a screen interface and is configured in one example as an electronic interactive smartboard. The screen interface receives input via touch sensor(s), pressure sensor(s), or the like. The input can be in the form of shapes, expressions, symbols, handwriting, etc. In one example, the pen <NUM> is used to touch, contact, depress, or otherwise interact with the screen <NUM> in order to provide input to the electronic device <NUM>.

Additionally, the electronic device <NUM> includes one or more transmitters (not shown in <FIG>) configured to transmit uplink signals, which in some examples are electrodes of the screen interface. In one example as illustrated in <FIG>, the uplink signal is transmitted from only one part of the screen (and toggled over time to the other parts), which enables the 3D position of the pen <NUM> to be determined relative or with respect to electronic device <NUM> (e.g., location in space in front of the screen <NUM>) by the pen itself. In the illustrated example, the uplink signal is transmitted from four different regions <NUM>, each at a different time, such as sequentially from T<NUM> then T<NUM> then T<NUM> then T<NUM>. It should be appreciated that additional or fewer regions <NUM> can be provided and the order and timing of where the uplink signal is transmitted can be varied.

In another example, as shown in <FIG>, different uplink signals (illustrated as Tx, Tx', Tx", Tx‴) are transmitted from the different regions <NUM> of the screen <NUM> simultaneously. For example, each of the signals in some examples have a different frequency or amplitude. However, any types of signals having different information can be used to allow the pen <NUM> to determine position information relative to the electronic device <NUM>. It should be noted that some or all of the transmission schemes illustrated in <FIG> can be combined in some examples. Thus, spatial and/or temporal variations in the transmission of the signals can be implemented in various examples to allow the location of the pen <NUM> to be determined by the pen <NUM>. That is, a pen <NUM>, which is a small object with one spatial receiver, in some examples, is able to determine a location of the pen <NUM> relative to the electronic device <NUM>.

The uplink signals in various examples are employed to detect the presence of the pen <NUM>, as well as to allow the position of the pen <NUM> relative to the screen <NUM> of the electronic device <NUM> (e.g., in proximity to one of the regions <NUM>) to be determined directly by the pen <NUM>. Moreover, in some examples, one or more communication channels are established between the electronic device <NUM> and the pen <NUM>, using communication techniques in the electronic pen technology, to enable uplink and downlink communication for bi-directional data exchange between the electronic device <NUM> and the pen <NUM>. Thus, the uplink signal can be any type of signal transmitted from the electronic device <NUM> and detectable by the pen <NUM>.

The pen <NUM> in one example is an active electronic pen (e.g., an active stylus) that includes electronic components that enable the pen <NUM> to interact with the electronic device <NUM>, a user of the pen <NUM>, other electronic devices, etc. For instance, in some examples, the pen <NUM> includes a wireless interface that enables the pen <NUM> to communicate wirelessly (via Wi-Fi, cellular, BLUETOOTH® short-range wireless communication protocol, other radio frequency communications, etc.) with the electronic device <NUM>, even when the pen <NUM> is not in contact with the electronic device <NUM>. Further, the pen <NUM> includes one or more of buttons, switches, and/or other input interfaces in some examples, which a user of the pen <NUM> uses to interact with the pen <NUM> and/or electronic components of the pen <NUM>. Additionally, or alternatively, the pen <NUM> can include pressure sensors, motion sensors, accelerometers, gyroscopic sensors, or the like that enable the detection of motion, direction, angle, user input, gestures, etc. of the pen <NUM>.

In operation, a proximity and relative position (e.g., XY position) of the pen <NUM> to the electronic device <NUM> is determined. The pen <NUM> is configured to receive one or more signals from the electronic device <NUM> to allow the pen <NUM> to determine a location of the pen <NUM> relative to the electronic device <NUM>. In some examples, a detection of pen proximity to the electronic device <NUM> includes interpreting a signal strength of a wireless signal from the pen <NUM> as an indicator of pen proximity to thereby initiate signals being sent by the electronic device <NUM> to the pen <NUM> for use by the pen <NUM> to determine the location of the pen <NUM> (e.g., XY location of the pen <NUM> in front of the electronic device <NUM>). For instance, the signal strength of the wireless signal is generally stronger when the pen <NUM> is close to a portion of the electronic device <NUM> that is transmitting the signal (e.g., uplink signal) and the signal strength of the wireless signal becomes weaker as the pen <NUM> moves away from the electronic device <NUM>. Alternatively, or additionally, the pen <NUM> can also transmit, to the electronic device <NUM>, information describing pen motion, pen direction, pen angle, etc. that can be used by the electronic device <NUM> to determine information relating to the pen <NUM>.

In one example, when the electronic device <NUM> detects a proximity of the pen <NUM> by, at least in part, detecting a wireless signal from the pen <NUM>, the electronic device <NUM> initiates a pen <NUM> location detection process, wherein signals are sent to the pen <NUM> as described herein to allow the pen <NUM> to determine a location thereof relative to the electronic device <NUM>. In some examples, signal strength of the wireless signal received by the pen <NUM> can be used to determine pen location information.

The pen location detection process is initiated in some examples using a pen proximity threshold defined to include a signal strength threshold such that, when the signal strength threshold is crossed, the electronic device <NUM> determines that the pen <NUM> is within the pen proximity threshold and then begins transmission of the uplink signal(s) to allow the pend <NUM> to determine a position or location of the pen <NUM> relative to the screen <NUM> of the electronic device <NUM>, such as where along or in front of the electronic device <NUM> the pen <NUM> is located. It should be appreciated that the initiation of the pen location determination process can be performed in many different ways, such as based on a user depression of a button on the pen <NUM> when in proximity to the electronic device <NUM>, the electronic device <NUM> periodically transmitting signals therefrom, etc..

<FIG> illustrates a block diagram of the electronic device <NUM> in one example. The electronic device <NUM> includes a user interface <NUM>, an operating system <NUM> that includes a pen location handler <NUM> and pen-compatible control(s) <NUM>, application(s) <NUM>, and a network interface <NUM>. The user interface <NUM> further includes an input interface <NUM> and an output interface <NUM>.

In one example, the user interface <NUM> includes a touch screen, such as a smartboard touch screen. The input interface <NUM> is some examples includes a layer or portion of the touch screen that detects the location of contact, depression, or the like on the touch screen. Contact on the touch screen, whether by a user's finger, the pen <NUM>, other types of stylus, or the like, is detected by the input interface <NUM> and interpreted as input to the user interface <NUM>. The output interface <NUM> in some examples includes a layer or portion of the touch screen that displays, renders, or otherwise outputs information to a user of the electronic device <NUM>. The output interface <NUM> can display colors, shapes, letters, or the like to communicate output information to a user of the electronic device.

Alternatively, or additionally, the input interface <NUM> receives input from a pen device (e.g., the pen <NUM>) linked to the electronic device <NUM> as described herein. The pen <NUM> and electronic device <NUM> are in communication via the network interface <NUM> of the electronic device <NUM>.

The input interface <NUM> can include other interfaces, such as keyboards, mice, switches, buttons, microphones, cameras, motion detection, etc. in some examples. These components of the input interface <NUM> further enable a user to input information into the electronic device <NUM>. For instance, a camera associated with the input interface <NUM> can detect a user's gestures and interpret the gestures as a form of input information. In another example, the camera and input interface <NUM> are associated with an augmented reality device and/or a virtual reality device.

In some examples, the output interface <NUM> further includes speakers, vibration components, projector components, etc. These components of the output interface <NUM> further enable the electronic device <NUM> to communicate output information to a user. For instance, a vibration component of the output interface <NUM> vibrates to provide a notification to the user of the electronic device <NUM>.

The operating system <NUM> in one example is a software component configured to perform core software operations of the electronic device <NUM> and to provide a platform upon which other software components (e.g., application(s) <NUM>, etc.) are executed. It should be understood that the operating system <NUM> functions according to typical methods as understood by a person of ordinary skill in the art of computer science, computer engineering, or the like.

The pen location handler <NUM> includes software that interacts with the user interface <NUM>, including controlling signals sent to the pen device (e.g., the pen <NUM>, etc.) for use in determining the location of the pen <NUM> in space relative to the electronic device <NUM>. The pen location handler <NUM> initially can listen for communications from the user interface <NUM> associated with pen proximity data during use of the electronic device <NUM>. Based on, for example, a received initiation signal, the pen location handler <NUM> triggers a pen event that is sent to and/or received by other software components (e.g., application(s) <NUM>, etc.) of the electronic device <NUM> in some examples. As described in more detail herein, the pen location handler <NUM> controls generation of uplink signals by one or more transmitter(s) <NUM> to allow the pen <NUM> to determine the location of the pen <NUM> relative to the screen <NUM> of the electronic device <NUM>. In some examples, transmitters <NUM> are electrodes in the electronic device that are selectively driven (e.g., radio-frequency (RF) driven) in defined areas corresponding to the regions <NUM> that allow the pen <NUM> to determine a relative location to the electronic device <NUM>. Thus, in some examples, a subset of electrodes are driven that are located in each of the regions <NUM> to provide signals to the pen <NUM> for use in determining the location information.

The pen-compatible control(s) <NUM> are software components associated with the operating system <NUM> that cause the electronic device <NUM> to react to and/or interact with the pen <NUM>. In one example, the pen-compatible control(s) <NUM> cause the user interface <NUM> to provide a user of the electronic device <NUM> with a writing region, information and/or guidance regarding context of the writing region, or the like. For instance, the pen-compatible control(s) <NUM> cause the user interface <NUM> to display a box on the screen containing a writing region and a prompt in or near the box describing the box as an address field, name field, signature field, search term field, or the like. Further the pen-compatible control(s) <NUM> can include check boxes and/or list items that expand to provide additional details based on the proximity of the pen or map controls that zoom in to a geographic area and provide a virtual writing area associated with the geographic area that the user can use to add notes about the geographic area. The location on the screen <NUM> of the displayed information changes in some examples based on the location of the pen <NUM> as determined by the pen <NUM>.

The application(s) <NUM> are software components that can be installed on the electronic device <NUM>. In one example, the application(s) <NUM> use the operating system <NUM> as a platform for executing instructions and/or providing functionality to a user of the electronic device <NUM>. For instance, the application(s) <NUM> can be a word processing application, an email application, a web browser, a messaging application, a game, or the like. It should be understood that the application(s) <NUM> can be of any application type known to a person of ordinary skill in the art without departing from the scope of the description herein.

The application(s) <NUM> in some examples include a pen event handler component that is configured to receive, detect, and/or react to pen events that are triggered by the operating system. The pen event handler of the application(s) <NUM> cause the activation of pen-compatible controls or other operations to be performed.

The application(s) <NUM> in some examples lack application-specific controls and/or functionality to accept and process pen input, and instead access the one or more pen-compatible controls <NUM> from the operating system <NUM> for use within the application <NUM> when pen input is detected. In one example, a messaging application is not configured to handle pen input and, when a pen event is received by the messaging application, the messaging application accesses or requests to use a pen-compatible control provided by the operating system <NUM> to display a writing region and convert pen input into text which the messaging application then uses to communicate to a user's contact in a message. The request can occur via an application programming interface (API) call to the operating system.

Alternatively, the application(s) <NUM> are configured to include pen-compatible controls and handle incoming pen events without requesting the pen-compatible control(s) <NUM> from the operating system <NUM>. For instance, a map application can include the pen-compatible control(s) <NUM> that automatically transform pen input on a map region of the user interface <NUM> into a pinpoint on the map. Further, a user can be prompted as to whether the pinpoint is a new destination, a new location of interest, or the like.

The network interface <NUM> provides an interface by which the electronic device <NUM> communicates with other electronic devices, computing devices, access points, or the like. The network interface <NUM> also provides access to one or more channels of network communication, including wired network connections, wireless network connections, etc. In some examples, components described as being part of the electronic device <NUM> can instead be located outside of the electronic device <NUM> and accessed by the electronic device <NUM> via the network interface <NUM>.

In various examples, the electronic device <NUM> comprises a user interface, at least one processor, and at least one memory comprising computer program code. The computer program code is configured to, with the at least one processor, perform the operations illustrated in the flowcharts.

In operation, in one example, the input interface <NUM> is configured to recognize an input received via touchscreen functionality of the electronic device <NUM>, such as a digitizer panel. The input interface <NUM> operates to detect a presence of the pen <NUM> to thereby initiate transmission of signals to allow the location of the pen <NUM> to be determined by the pen (e.g., of the pen <NUM> within a threshold distance/proximity to the electronic device <NUM>), recognize and resolve input provide via the pen <NUM>, and so on. The input can take a variety of different forms, such as to recognize movement of the pen <NUM> across the screen <NUM> of the electronic device <NUM>, pressing and tapping on the digitizer panel, drawing of a line, and so on. As a result, a user is able to perform a variety of different writing operations, such as to draw a line to mimic a pencil, pen, brush, and so on, as well as to perform erase operations, such as to mimic a rubber eraser.

Thus, in accordance with techniques described herein, the pen location handler <NUM> is configured to recognize the presence of the pen <NUM> and control transmission of signals to allow the pen to resolve a position of the pen <NUM> in space relative to the electronic device <NUM>. In implementations, pen recognition and position are based on uplink signals transmitted to and received by the pen <NUM>. For example, an analysis of uplink signals received from a region <NUM> (shown in <FIG>) by the pen <NUM> can be used to determine the location of the pen <NUM>, which allows mapping to different contexts including different interaction modes, stylus positions, hand positions, user positions, and scenarios, or operational behavior, among others. Accordingly, the pen <NUM> can recognize different uplink signals (such a signals having different magnitudes or frequencies) from the transmitters <NUM> and match the uplink signal (e.g., signal decoding) to corresponding locations of the electronic device <NUM> from where signals were transmitted, or other contexts.

As further illustrated in <FIG>, the pen <NUM> can include different types of circuitry <NUM>. For example, the circuitry <NUM> in the illustrated example includes location-assistance circuitry <NUM> to aid in determining the location (e.g., an XY location) of the pen <NUM> in relation to the screen <NUM>, for example, using signals transmitted by the electronic device <NUM> (e.g., the electronic device <NUM> transmits signals from different areas in a distinguishable way, such as time sharing of different signals). In this example, the circuitry <NUM> also includes a radio <NUM> to support communication with the electronic device <NUM>, such as to communicate data used once the XY location is determined by the pen <NUM>. To power the circuitry <NUM>, the pen <NUM> includes a battery <NUM>.

The pen <NUM> also includes a usage module <NUM>. The usage module <NUM> is representative of functionality of the pen <NUM> to enter different usage modes. For example, the usage module <NUM> in the illustrated example supports an active mode <NUM> in which the circuitry <NUM> of the pen <NUM> is made active and therefore permitted to consume power from the battery <NUM>. Thus, the circuitry <NUM> is available for use, such as to assist in determining the XY location of the pen <NUM> to the electronic device <NUM> and for receiving and processing data conveyed from the electronic device <NUM> to the pen <NUM>.

The usage module <NUM> also supports a battery-conservation mode <NUM> to conserve power of the battery <NUM>, such as to make some of all of the circuitry <NUM>, such as the location-assistance circuitry <NUM>, the radio <NUM>, and so on inactive to minimize consumption of the battery <NUM>. In this way, the usage module <NUM> enters a low power state and conserves resources of the battery <NUM>, yet enables functionality of the circuitry <NUM> at appropriate times. It should be appreciated that many other operational behaviors or characteristics can be similarly controlled, such as the manner in which the pen <NUM> interacts with the electronic device <NUM>.

<FIG> illustrates a diagram of interactions between a pen/device interface (e.g., the user interface <NUM>, etc.) and the pen <NUM> according to an example. The pen/device interface transmits uplink signals to the pen <NUM> at <NUM>. At <NUM>, the pen receives uplink signals, which can be one or more from different regions <NUM> of the electronic device <NUM>, and determines from what region <NUM> the signal was transmitted. It should be noted that in some examples, prior to initiating transmission of the uplink signals, the pen/device interface receives pen proximity data as described herein and determines whether a proximity threshold is crossed or passed based on the pen proximity data, and thereby indicating that a pen location determination process should begin. Additionally, it should be noted that the pen <NUM>, in some examples, only receives uplink signals from the transmitters <NUM> of the electronic device <NUM> when the pen <NUM> is in proximity to the electronic device <NUM>. For example, in some examples, the power of the transmitters <NUM> are set to allow reception by the pen <NUM> only when the pen <NUM> is located within the physical space in proximity to where the transmitters <NUM> are located.

The pen <NUM> then compares information transmitted with the uplink signals or as determined from the transmitted uplink signals (e.g., unique frequency, magnitude or encoding) with data stored in a lookup table at <NUM> (e.g., corresponding location data). For example, the lookup table allows the pen <NUM> to determine from which region <NUM> and/or at what frequency the uplink signals were transmitted. For example, the lookup table allows the pen <NUM> to determine a transmitting region <NUM> of the uplink signals. In some examples as described in more detail herein, triangulation is performed at <NUM> determine the transmitting region <NUM> of the uplink signals, such as based on the magnitude and encoding information within the signals.

It should also be noted that as part of the communication exchange, the pen <NUM> is also configured to determine the particular electronic device <NUM> with which the pen <NUM> is communicating and adjust the operational behavior of the pen <NUM>. Moreover, a specific lookup table can be associated with different electronic devices <NUM> to allow for decoding of different uplink signals from the different electronic devices <NUM>.

In other examples, the location-assistance circuitry <NUM> of the pen <NUM> decouples the received uplink signals from the different regions <NUM> and performs a numerical analysis based on the signals, for example when the uplink signals are transmitted from regions <NUM> as illustrated in <FIG>. In one example, the numerical analysis translates a plurality of measurements from received uplink signals to position the pen <NUM> in space relative to the transmitting device, namely the transmitters <NUM>. The translation in some examples is configured based on machine learning or neural networks (or other deep learning engine) that trains the location-assistance circuitry <NUM> in controlled training sessions (e.g., perform measurements at different locations that are input to a neural network). In some examples, triangulation processes are trained.

The pen <NUM> in some examples can then transmit determined location information to the electronic device <NUM> at <NUM>. For example, based on the information determined from the lookup table or other analysis performed, the pen <NUM> transmits to the electronic device <NUM> the relative location of the pen <NUM>, such as the region <NUM> in proximity to which the pen <NUM> is located. Thus, the electronic device <NUM> receives information from the pen <NUM>, such as regarding the location of the pen <NUM> (e.g., located in an upper left portion of the screen <NUM>, a lower left right portion of the screen <NUM>, etc.).

At <NUM>, the pen/device interface controls the display of the screen <NUM>. For example, the pen/device interface controls the content of data, the active data area, etc. of the screen <NUM> based on the region <NUM> where the pen <NUM> is located (e.g., the pen <NUM> is in front of a particular region <NUM>). In some examples, the screen <NUM> includes a plurality of pixels (electrodes) that are driven by the pen/device interface, which can be selectively controlled based on the determined location of the pen <NUM>.

<FIG> illustrates a flow chart of a method <NUM> for determining a location of an electronic pen. The method <NUM> allows for more accurate and reliable locating of the electronic pen relative to a screen. For example, with methods that use a change in magnitude of a received signal from all portions of the electronic device, the XY location of the pen relative to the screen of the electronic device cannot accurately and reliably determined (e.g., location relative to an electronic interactive whiteboard). It should be noted that the exemplary flow chart illustrates operations of a computing device of the electronic device to determine the location of the electronic pen relative to the screen of the electronic device. The operations illustrated in the flow charts described herein can be performed in a different order than is shown, can include additional or fewer steps and can be modified as desired or needed. Additionally, one or more operations can be performed simultaneously, concurrently or sequentially.

More particularly, the method <NUM> includes transmitting signals from a plurality of regions of an electronic device at <NUM>. For example, one or more uplink signals capable of being received by an electronic pen are transmitted from the electronic device. As described herein, uplink signals are transmitted from different regions of the electronic device. For example, a time division transmission scheme is used in some examples wherein the uplink signals are transmitted from the different regions at different times. For example, over a defined time period, an uplink signal is transmitted from each region, one at a time. The transmission from each region can be a single uplink signal or multiple uplink signals transmitted over a period of time. In some examples, the uplink signals are sequentially transmitted from within each of the regions, which can be repeated multiple times. It should be noted that when using time division transmission, the pen receiving the signals does not have to decouple the transmission.

As another example, the different regions transmit signals having different information at the same time. For example, signals from the different regions can be transmitted at different frequencies or magnitudes simultaneously or concurrently. Different types of signal modulation are used in different examples, such as frequency modulation or amplitude modulation. In other examples, the signals are encoded with data to allow for correlating the signal with a particular region. It should be noted that when transmitting signals with different information, the pen receiving the signal can perform triangulation calculations, such as used in the object locating technology after receiving the signals to determine pen location. It should also be noted that in one example, when the signals are transmitted simultaneously, the signals are orthogonal in some aspects so as to not interfere. The signals can then be decoded by the pen.

The method <NUM> includes receiving one or more of the signals at the electronic pen <NUM>. For example, the transmitted uplink signals are received by the pen when the pen is in proximity to the electronic device. In some examples, as described herein, the electronic device only begins transmission after a proximity detection is satisfied, wherein the pen is determined to be within a threshold distance of a surface of the electronic device (e.g., hovering in front of or over the screen of an interactive whiteboard).

The method <NUM> includes determining, by the electronic pen, a location of the electronic pen relative to a screen of the electronic device using the received one or more signals at <NUM>. For example, the pen can use a lookup table (separate lookups table can be provided for different electronic devices) to identify from which transmitter(s) the signal(s) originated when the signal(s) are transmitted using time division. The signals do not have to be decoupled because the signals are transmitted from each of the regions at different times. If signals are transmitted simultaneously from the multiple regions, the pen in some examples can decode the signals using frequency demodulation and determine the origin of the transmission using triangulation calculations. Thus, from the transmissions from the multiple regions, the pen is able to determine, for example, an XY location relative to a screen of the electronic device.

The method <NUM> also includes in some examples transmitting determined location information from the electronic pen to the electronic device based on the determined location at <NUM>. For example, once the pen determines the region that transmitted the uplink signal(s) and determines the location of the pen in space in front of the electronic device, the pen transmits location information to the electronic device indicating where the pen is located.

Thus, the pen is able to determine a location thereof in space relative to the electronic device. When multiple electronic devices (e.g., electronic whiteboards) are placed side by side, the method facilitates an easier and more accurate determination of the location of the pen (as determined by the pen), particularly when positioned along an end of one of the electronic devices. Moreover, in some examples, the location information is used to control a display on the screen of the electronic device based at least in part on the determined location information.

<FIG> illustrates a flow chart of a method <NUM> for determining a location of an electronic pen. The method includes defining transmission regions of an electronic device at <NUM>. The electronic device is any device that can interact with the electronic pen. In one example, the electronic device is an electronic whiteboard having electrodes driven by RF signals to transmit across the screen of the electronic whiteboard. The transmission regions in some examples are defined by a subset of the electrodes, such that the electrodes in each region are controlled as described herein to determine location information for the electronic pen. For example, the electrodes in each of the regions can be selectively driven or driven together, but at different magnitudes or frequencies by one or more RF drivers.

The method <NUM> includes, in some examples, detecting whether a pen device, such as the electronic pen, approaches the electronic device at <NUM>. For example, a proximity detection can be performed as described herein to detect when the electronic pen is within a transmission range of the electronic device. Detecting the approach of the electronic pen is based on pen proximity data as described herein. For example, detecting the approach can include detecting a wireless signal from the electronic device. If pen approach is not detected, continued monitoring for pen approach is performed at <NUM>. In some examples, a detection process is not performed until the electronic pen is within a threshold distance from the screen of the electronic device.

If pen approach is detected at <NUM>, uplink signals are transmitted from the defined transmission regions at <NUM>. For example, the electrodes in one or more of the transmission regions are driven by one or more RF driving circuits in some examples. The RF driving circuits can be any type of electrode driving circuit for displays, such as for touchscreen or smartboard types of displays. As described herein, different transmission methods or schemes can be used, such as, time division transmissions, frequency modulated transmission, amplitude modulated transmissions, etc..

A determination is then made at <NUM> whether location information is received from the electronic pen. For example, as described herein, the electronic pen transmits determined location information based on calculations or an analysis of the received uplink signals performed by the pen. The location information indicates to the electronic device where the electronic pen is located in space. If location information is not received, then uplink signals continue to be transmitted from the transmission regions at <NUM>. For example, if location information is not received after transmission from one transmission region, transmission from another transmission region is performed. Additionally, if location information is not received after transmission from all the transmission regions, the process is repeated, which occurs for a predetermined number of cycles in some examples. It should be noted that a confirmation signal or proximity can be sent from the pen to the electronic device in some examples when the electronic devices approaches or is in proximity to the electronic device.

If location information is received by the electronic device, the display of the electronic device is controlled based on the location information at <NUM>. For example, a portion of the screen of the electronic device is made active or caused to display certain information. In some examples, the location information allows side by side electronic devices to determine which device(s) should be active when the electronic pen is at an edge between two electronic devices (e.g., at an edge between two electronic whiteboards).

Thus, in some examples, the methods <NUM> and <NUM> allow an electronic pen to determine an electronic pen location in space relative to an electronic device.

The present disclosure is operable with a computing apparatus <NUM> according to an embodiment as a functional block diagram <NUM> in <FIG>. In one example, components of the computing apparatus <NUM> can be implemented as a part of an electronic device according to one or more embodiments described in this specification. The computing apparatus <NUM> comprises one or more processors <NUM> which can be microprocessors, controllers or any other suitable type of processors for processing computer executable instructions to control the operation of the electronic device. Platform software comprising an operating system <NUM> or any other suitable platform software can be provided on the apparatus <NUM> to enable application software <NUM> to be executed on the device. According to an embodiment, location information <NUM> determined by an electronic pen based on signals transmitted from a transmission controller <NUM> (e.g., signal transmissions from different regions) can be accomplished by software.

Computer executable instructions can be provided using any computer-readable media that are accessible by the computing apparatus <NUM>. Computer-readable media can include, for example, computer storage media such as a memory <NUM> and communications media. Computer storage media, such as the memory <NUM>, include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or the like. Computer storage media include, but are not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computing apparatus. In contrast, communication media can embody computer readable instructions, data structures, program modules, or the like in a modulated data signal, such as a carrier wave, or other transport mechanism. As defined herein, computer storage media do not include communication media. Therefore, a computer storage medium should not be interpreted to be a propagating signal per se. Propagated signals per se are not examples of computer storage media. Although the computer storage medium (the memory <NUM>) is shown within the computing apparatus <NUM>, it will be appreciated by a person skilled in the art, that the storage can be distributed or located remotely and accessed via a network or other communication link (e.g. using a communication device <NUM>).

The computing apparatus <NUM> can comprise an input/output controller <NUM> configured to output information to one or more input devices <NUM> and output devices <NUM>, for example a display or a speaker, which can be separate from or integral to the electronic device. The input/output controller <NUM> can also be configured to receive and process an input from the one or more input devices <NUM>, for example, a keyboard, a microphone or a touchpad. In one embodiment, the output device <NUM> can also act as the input device <NUM>. An example of such a device can be a touch sensitive display. The input/output controller <NUM> can also output data to devices other than the output device <NUM>, e.g. a locally connected printing device. In some embodiments, a user can provide input to the input device(s) <NUM> and/or receive output from the output device(s) <NUM>.

In some examples, the computing apparatus <NUM> detects voice input, user gestures or other user actions and provides a natural user interface (NUI). This user input can be used to author electronic ink, view content, select ink controls, play videos with electronic ink overlays and for other purposes. The input/output controller <NUM> outputs data to devices other than a display device in some examples, e.g. a locally connected printing device.

NUI technology enables a user to interact with the computing apparatus <NUM> in a natural manner, free from artificial constraints imposed by input devices such as mice, keyboards, remote controls and the like. Examples of NUI technology that are provided in some examples include but are not limited to those relying on voice and/or speech recognition, touch and/or stylus recognition (touch sensitive displays), gesture recognition both on screen and adjacent to the screen, air gestures, head and eye tracking, voice and speech, vision, touch, gestures, and machine intelligence. Other examples of NUI technology that are used in some examples include intention and goal understanding systems, motion gesture detection systems using depth cameras (such as stereoscopic camera systems, infrared camera systems, red green blue (rgb) camera systems and combinations of these), motion gesture detection using accelerometers/gyroscopes, facial recognition, three dimensional (3D) displays, head, eye and gaze tracking, immersive augmented reality and virtual reality systems and technologies for sensing brain activity using electric field sensing electrodes (electro encephalogram (EEG) and related methods).

The functionality described herein can be performed, at least in part, by one or more hardware logic components. According to an embodiment, the computing apparatus <NUM> is configured by the program code when executed by the processor(s) <NUM> to execute the embodiments of the operations and functionality described. For example, and without limitation, illustrative types of hardware logic components that can be used include FPGAs, ASICs, ASSPs, SOCs, CPLDs, and GPUs.

At least a portion of the functionality of the various elements in the figures can be performed by other elements in the figures, or an entity (e.g., processor, web service, server, application program, computing device, etc.) not shown in the figures.

Examples of well-known computing systems, environments, and/or configurations that can be suitable for use with aspects of the disclosure include, but are not limited to, mobile or portable computing devices (e.g., smartphones), personal computers, server computers, hand-held (e.g., tablet) or laptop devices, multiprocessor systems, gaming consoles or controllers, microprocessor-based systems, set top boxes, programmable consumer electronics, mobile telephones, mobile computing and/or communication devices in wearable or accessory form factors (e.g., watches, glasses, headsets, or earphones), network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. In general, the disclosure is operable with any device with processing capability such that it can execute instructions such as those described herein. Such systems or devices can accept input from the user in any way, including from input devices such as a keyboard or pointing device, via gesture input, proximity input (such as by hovering), and/or via voice input.

Examples of the disclosure are described in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices in software, firmware, hardware, or a combination thereof. The computer-executable instructions can be organized into one or more computer-executable components or modules. Aspects of the disclosure can be implemented with any number and organization of such components or modules. Other examples of the disclosure can include different computer-executable instructions or components having more or less functionality than illustrated and described herein.

According to the present disclosure, there is provided an electronic device, a method, and one or more computer storage media as set out in the independent claims. Embodiments are set out in the dependent claims.

Any range or device value given herein can be extended or altered without losing the effect sought, as will be apparent to the skilled person.

It will be understood that the benefits and advantages described above can relate to one embodiment or can relate to several embodiments.

The embodiments illustrated and described herein as well as embodiments not specifically described herein but within the scope of aspects of the claims constitute exemplary means for training a neural network. The illustrated one or more processors <NUM> together with the computer program code stored in memory <NUM> constitute exemplary processing means for determining the location of an electronic pen.

The terms 'computer', 'computing apparatus', 'mobile device' and the like are used herein to refer to any device with processing capability such that it can execute instructions. Those skilled in the art will realize that such processing capabilities are incorporated into many different devices and therefore the terms 'computer' and 'computing apparatus' each can include PCs, servers, laptop computers, mobile telephones (including smart phones), tablet computers, media players, games consoles, personal digital assistants, and many other devices.

In some examples, the operations illustrated in the figures can be implemented as software instructions encoded on a computer readable medium, in hardware programmed or designed to perform the operations, or both. For example, aspects of the disclosure can be implemented as a system on a chip or other circuitry including a plurality of interconnected, electrically conductive elements.

That is, the operations can be performed in any order, unless otherwise specified, and examples of the disclosure can include additional or fewer operations than those disclosed herein.

The terms "comprising," "including," and "having" are intended to be inclusive and mean that there can be additional elements other than the listed elements.

Claim 1:
An electronic device (<NUM>) comprising:
a screen (<NUM>);
at least one processor (<NUM>); and
at least one memory (<NUM>) comprising computer program code, the at least one memory (<NUM>) and the computer program code configured to, with the at least one processor (<NUM>), cause the electronic device (<NUM>) to at least:
transmit (<NUM>) signals from a plurality of regions (<NUM>) of the electronic device (<NUM>);
receive (<NUM>) location information (<NUM>) from an electronic pen (<NUM>) in proximity to the electronic device (<NUM>), the location information (<NUM>) determined by the electronic pen (<NUM>) and corresponding to a location of the electronic pen (<NUM>) relative to the screen (<NUM>) of the electronic device (<NUM>) using one or more of the transmitted signals received by the electronic pen (<NUM>), wherein the location information (<NUM>) comprises a three-dimensional, 3D, location of the electronic pen (<NUM>) relative to the screen (<NUM>) of the electronic device (<NUM>); and
control (<NUM>) a display on the screen (<NUM>) of the electronic device (<NUM>) based at least in part on the received location information (<NUM>).