Patent Description:
<CIT>, <CIT> and <CIT> may provide useful background information.

Examples are disclosed that relate to methods and computing devices for determining a distance of an input device from a surface of a computing device. In one example, a method comprises receiving a plurality of input device signals via the input device. A portion of the input device signals are used to determine an effective voltage of the input device. Adjusted input device signals are calculated by adjusting another portion of input device signals using the effective voltage of the input device. The method further comprises providing the adjusted input device signals as an input to a to a distance model that is used to calculate the distance of the input device from the surface of the computing device.

Optional features are defined in the appended dependent claims.

Some computing devices receive inputs from input devices and/or touch-sensitive surfaces. For example, a tablet computing device may need to use an input based on a distance between a tip of an electronic pen and a capacitive touch-screen surface. In some examples, electronic ink is displayed on the screen based on the distance between the pen tip and the screen. The ink may be displayed when the computing device determines that the pen tip is placed on the screen.

In some examples, the computing device may determine that an input device is placed on the screen when a pressure sensor of the input device is actuated. For example, an electronic pen may include a pressure sensor in its tip that is actuated when the tip is pressed against a surface. The pressure sensor may actuate when a threshold amount of force, such as <NUM> grams, is applied. However, some users may find it difficult to apply such force, which may contribute to an unintended lack of actuation and electronic ink, which is a less than satisfactory user experience.

In other examples, a computing device may determine a distance between an input device and a surface via capacitive sensing. For example, the computing device may use a distance data model, such as a neural network or other machine learning algorithm, to predict the distance between the input device and the surface from capacitive sensor data. However, machine learning training data collected with one input device may not accurately represent other input devices. For example, a voltage generated at a tip of an active electronic pen may vary based on the pen's power supply and/or other components. In some examples, the voltage generated by different units of the same model pen may vary by ±<NUM>%. Accordingly, a data model trained on a "golden pen" (e.g., a pen having a particular "golden voltage" and utilized to provide ground truth data) may output inaccurate distances when used with inputs from another pen that generates a voltage different from the golden voltage of the golden pen. As result, a data model trained using multiple pens with different voltages may output unreliable distance values.

The effects of different voltages may be mitigated by utilizing algorithms that do not include values of capacitive signals. However, such algorithms may output less accurate position and/or orientation values. Another solution would be to measure relative voltage generated by an input device. For example, a relative voltage generated by an input device may be calculated by positioning the input device at a specific location on a capacitive sensor and comparing the sensor's response to another pen positioned at the same location. However, it may be difficult to position the pen exactly, even during the training process, and it may be undesirable to request such positioning from an end-user.

Accordingly, examples are disclosed that relate to methods and computing devices for determining a distance of an input device from a surface of a computing device. In one example, a method comprises receiving a plurality of input device signals via the input device. A portion of the input device signals are used to determine an effective voltage of the input device. Adjusted input device signals are then calculated by adjusting subsequently received input device signals using the effective voltage of the input device. The method further comprises providing the adjusted input device signals as an input to a to a distance model that is used to calculate the distance of the input device from the surface of the computing device.

With reference now to <FIG>, one example of a system <NUM> is illustrated that includes an input device <NUM> and a computing device <NUM> comprising a surface <NUM>. The computing device <NUM> also comprises a processor <NUM> and a memory <NUM>. The memory <NUM> stores instructions <NUM> executable by the processor <NUM> to determine a distance <NUM> of the input device <NUM> from the surface <NUM> of the computing device <NUM> as described herein. Additional details regarding the components and computing aspects of the computing device <NUM> are described in more detail below with reference to <FIG>.

The input device <NUM> and the computing device <NUM> may take any suitable form. With reference briefly to <FIG>, the input device <NUM> may take the form of an electronic pen <NUM> and the computing device <NUM> may take the form of a tablet computing device <NUM>. The surface <NUM> may take the form of a capacitive touch-screen surface <NUM> on the tablet computing device <NUM>.

With reference again to <FIG>, the computing device <NUM> is configured to receive a plurality of input device signals <NUM> via the input device <NUM>. In some examples, as introduced above, the input device signals <NUM> are provided by a pressure sensor <NUM>. In other examples, and as described in more detail below, the input device signals <NUM> take the form of a current generated in one or more sensors <NUM> in the surface <NUM> of the computing device <NUM> via one or more transmitters <NUM> of the input device <NUM>.

As illustrated by example in <FIG>, an input device in the form of an electronic pen <NUM> includes a tip transmitter <NUM> and a body transmitter <NUM>. The tip transmitter <NUM> is located at the tip <NUM> of the pen <NUM>, with the tip configured to contact the touch-screen surface <NUM>. The body transmitter <NUM> is spaced from the tip <NUM>. In the present example, the body transmitter <NUM> comprises a ring that encircles the body of the electronic pen <NUM> approximately <NUM> above the tip <NUM>. In other examples, the body transmitter <NUM> may utilize different shapes and configurations, and may be spaced at different distances from the tip <NUM>.

Each of the tip transmitter <NUM> and the body transmitter <NUM> is coupled to a power source <NUM>. In some examples, the tip transmitter <NUM> and the body transmitter <NUM> are coupled to different power sources. Using power from the power source, and as described in more detail below, the tip transmitter <NUM> and the body transmitter <NUM> are capacitively linked to the touch-screen surface <NUM>. As illustrated by example in <FIG>, which shows another view of the tablet computing device <NUM> of <FIG>, the touch-screen surface <NUM> includes a plurality of sensors in the form of antennas <NUM> that can detect the tip transmitter <NUM> and the body transmitter <NUM> of the electronic pen <NUM> via capacitive sensing. In the present example, the antennas <NUM> are arranged in an X-Y grid. In some examples, the touch-screen surface <NUM> may include <NUM>-<NUM> antennas extending perpendicular to the X-axis, and another <NUM>-<NUM> antennas extending perpendicular to the Y-axis. In the example of <FIG> and for ease of illustration, the touch-screen surface <NUM> includes <NUM> antennas <NUM> perpendicular to the X-axis, and <NUM> antennas <NUM> perpendicular to the Y-axis. In other examples, any suitable number and configuration of antennas may be utilized. For example, the antennas <NUM> may be arranged in a triangular grid, concentric circles, or any other suitable configuration.

With reference also to <FIG>, a voltage applied to the tip transmitter <NUM> may generate a current in one or more of the antennas <NUM> in the touch-screen surface <NUM> of the tablet computing device <NUM>. The current generated in each of the antennas <NUM> is converted to a digital signal by a digitizer, such as an analog-to-digital converter implemented in firmware of the tablet computing device <NUM>.

In some examples, the tablet computing device <NUM> obtains input device signals from up to nine antennas <NUM> extending in the X-axis direction and up to nine antennas <NUM> extending in the Y-axis direction that each have portions surrounding the location of the tip transmitter <NUM>. The signals may be processed (e.g. by the processor <NUM> of <FIG>) using suitable capacitive sensing techniques to determine a position of the electronic pen <NUM> relative to the touch-screen surface <NUM> in the X- and Y-directions. In some examples, signals received from one or more antennas <NUM> via the body transmitter <NUM> may be processed (e.g. by the processor <NUM> of <FIG>) using suitable capacitive sensing methods to determine a rotational orientation of the electronic pen, including tilt and/or azimuth, relative to the touch-screen surface <NUM>. In some examples, and as described in more detail below with reference to <FIG>, signals received via the tip transmitter <NUM> are used to determine a distance <NUM> of the electronic pen <NUM> from the touch-screen surface <NUM> in the Z-axis direction.

With reference again to <FIG> and as noted above, each of the tip transmitter <NUM> and the body transmitter <NUM> is coupled to power source <NUM>. The power source <NUM> provides a voltage (V) that allows the tip transmitter <NUM> and the body transmitter <NUM> to generate a current in antennas of the touch-screen surface <NUM> that is proportional to the capacitance between the transmitters and each antenna. The power source <NUM> may provide any suitable voltage and any suitable current (e.g. AC or DC). For example, the power source <NUM> may provide <NUM>-20V AC at a frequency of <NUM>-<NUM>. In some examples, the power source <NUM> provides the same frequency and voltage to both the tip transmitter <NUM> and the body transmitter <NUM> (e.g. <NUM> V, <NUM>). In other examples, the power source <NUM> provides different frequencies and/or voltages to the tip transmitter <NUM> and the body transmitter <NUM>.

In some examples, the tip transmitter <NUM> and the body transmitter <NUM> may be activated at the same time. In other examples, the tip transmitter <NUM> and the body transmitter <NUM> may be activated at different times. For example, the tip transmitter <NUM> and the body transmitter <NUM> may each be energized during separate windows of time. In some examples, each window of time is the same length. In other examples, the tip transmitter <NUM> and the body transmitter <NUM> may be energized for different amounts of time. For example, the tip transmitter <NUM> may be energized for <NUM>, and the body transmitter <NUM> may be energized for <NUM>.

As noted above, voltages generated by different electronic pens may vary based on the pen's power supply and/or other components. As the signals used to determine position and/or orientation coordinates of a particular electronic pen are a function of its generated voltage, it may be difficult to obtain reliable coordinates for one electronic pen using a data model that was trained on a different pen that generates a different voltage. Furthermore, a data model trained using multiple electronic pens with different voltages may output unreliable coordinates, such as Z-axis distances of the input device from the surface of the computing device.

Accordingly, and in one potential advantage of the present disclosure, a Z-axis position of an input device relative to a computing device surface may be determined more accurately by first determining an effective voltage of the input device. Briefly, with reference again to <FIG> and as described in more detail below, the computing device <NUM> may calculate the effective voltage <NUM> of the input device <NUM> using a voltage data model <NUM>. The computing device <NUM> may generate adjusted input device signals <NUM> using the effective voltage <NUM> of the input device <NUM>. The computing device <NUM> may then use the adjusted input device signals <NUM> to more accurately determine the distance <NUM> of the input device <NUM> from the surface <NUM> of the computing device <NUM>.

With reference now to <FIG>, a flow diagram is provided depicting an example method <NUM> for determining a distance of an input device from a surface of a computing device using an effective voltage of the input device. The following description of method <NUM> is provided with reference to the software and hardware components described herein and shown in <FIG> and <FIG>. For example, the method <NUM> may be performed by the processor <NUM> of <FIG>, firmware on the computing device <NUM> of <FIG> or the tablet computing device <NUM> of <FIG>, an operating system or other software component of the computing device <NUM> or the tablet computing device <NUM>, or some suitable combination of components described herein. It will be appreciated that method <NUM> also may be performed in other contexts using other suitable components.

As illustrated by example in <FIG>, sensors <NUM> provide to a digitizer <NUM> current <NUM> that is generated via the tip transmitter <NUM> of the electronic pen <NUM>. The sensors <NUM> may comprise the sensors <NUM> of <FIG> or the antennas <NUM> of <FIG>. The digitizer <NUM> is configured to convert the current <NUM> from the sensors <NUM> into digital input device signals <NUM>. In some examples, the digitizer <NUM> is at least partially implemented in firmware on a computing device, such as the computing device <NUM> of <FIG> or the tablet computing device <NUM> of <FIG>. In other examples, the method <NUM> may utilize raw or analog values of the current <NUM> as input device signals.

An effective voltage of the electronic pen <NUM> may be determined as follows. With reference again to <FIG>, a voltage Vt applied to the tip transmitter <NUM> (t) and a current Itk in an antenna <NUM> (k) are related as shown in equation (<NUM>):
<MAT>.

In equation (<NUM>), ω represents an AC voltage frequency (multiplied by <NUM>π) and Ctk represents the capacitance between the tip transmitter <NUM> (t) and an antenna <NUM> (k). Accordingly, the electronic pen <NUM> and another electronic pen manufactured to the same specifications and generating the same voltage, when positioned at the same location relative to antenna <NUM> (k), will create substantially the same Ctk values. However, and as noted above, the voltage Vt generated by an electronic pen power source may vary from one device to another, due to factors such as variations in power sources. It follows that such varying voltages can cause a data model trained on a golden pen having a golden voltage to output inaccurate distances when used with inputs from other pens that generate different voltages. For purposes of the present disclosure, "golden voltage" is defined as the voltage of a golden input device, such as an electronic pen, that is utilized to provide ground truth data in training a voltage data model. In this way it will be appreciated that the golden input device is a particular device against which all later devices are tested and judged. The term "golden" in this context is used to convey that this particular device is used to establish a baseline "golden" voltage. The golden voltage is the output voltage of this particular input device and forms calibration data for subsequent usage of other devices. In some examples, a golden input device may be an idealized device that outputs a precise and/or known golden voltage. Thus and as described in more detail below, the present disclosure provides techniques for determining and utilizing an effective voltage of a given electronic pen to compensate for such varying voltages.

Initially it will be appreciated that the current Itk induced in the antenna <NUM> (k) by the tip transmitter <NUM> (t) is linearly proportional to Vt. As shown in equation (<NUM>), if Vt is changed by some factor a, the current Itk changes by the same factor a.

Using this relationship, an effective voltage of the electronic pen <NUM> may be calculated from the current Itk by utilizing a voltage data model <NUM> that is trained by a golden pen having a golden voltage. For example, and with reference again to <FIG>, the digitizer <NUM> may provide the input device signals <NUM> to voltage data model <NUM> (e.g. the voltage data model <NUM> of <FIG>) to calculate an effective voltage <NUM> of an input device. In some examples, and as described in more detail below with reference to <FIG>, the voltage data model <NUM> is a linear regression model that is trained on a golden pen. It will also be appreciated that the voltage data model <NUM> may comprise any other suitable type of data model, including machine learning models and neural networks. For example, the voltage data model <NUM> may comprise a neural network trained to determine the effective voltage <NUM> at runtime using reinforcement learning.

As described in more detail below, and once an effective voltage has been determined, at <NUM> input device signals <NUM> are adjusted using the effective voltage <NUM> to generate adjusted input device signals <NUM>. The adjusted input device signals <NUM> are then provided to a distance model, such as neural network <NUM>, to determine the distance <NUM> of the electronic pen from the surface of the computing device. At <NUM> additional processing using the distance <NUM> may be performed to generate an input report <NUM>, which may be output to one or more devices or applications as a user input.

Additional description of the voltage data model <NUM> will now be provided. With reference now to <FIG>, a flow diagram is provided depicting one example of a method <NUM> for building a voltage data model <NUM>. In this example, a golden pen is utilized in the form of electronic pen <NUM>' that has the same components and functionality as electronic pen <NUM> described herein. The following description of method <NUM> is provided with reference to the software and hardware components described herein and shown in <FIG> and <FIG>. It will be appreciated that method <NUM> also may be performed in other contexts using other suitable components.

In some examples, the method <NUM> may be performed during manufacturing of the tablet computing device <NUM> or other computing device to produce the voltage data model <NUM>. The resulting trained voltage data model <NUM> may be loaded into the computing devices during manufacturing, upon installation of an operating system, or at other appropriate timeframes. In other examples, portions of the method <NUM> may be performed at runtime on a computing device, or on a combination of one or more manufacturer computing devices and one or more end user computing devices.

As illustrated in <FIG>, electronic pen <NUM>' energizes sensors <NUM> in a computing device, such as tablet computing device <NUM>, to produce current <NUM> in the sensors <NUM>. A digitizer <NUM> converts the current <NUM> into digital input device signals <NUM>. At <NUM>, the method <NUM> includes collecting data in the form of the input device signals <NUM>. Such data is collected while the electronic pen <NUM>' is moved to a plurality of positions and orientations relative to the tablet computing device <NUM>. In some examples, such data also is collected on condition of determining that the electronic pen <NUM>' is contacting the surface of the tablet computing device <NUM>.

For example, the computing device <NUM> may determine that the electronic pen <NUM>' is contacting the surface <NUM> of the computing device by receiving a pressure signal from a pressure sensor <NUM> (e.g. a pressure sensor in the tip of the pen). In other examples, the computing device <NUM> determines that the electronic pen <NUM>' is contacting the surface <NUM> by determining that the pen is moving across the surface. For example, the computing device <NUM> may determine that the electronic pen <NUM>' is contacting the surface at one or more points in the middle of an electronic pen stroke.

In these examples, and given that the data is collected while the pen tip is contacting the surface of the computing device, a capacitance between a transmitter of the electronic pen and the sensors <NUM> may be expressed as a function of the pen's tilt (Θ), azimuth (φ), and position (in x/y coordinates) relative to the sensors <NUM> (e.g. the antennas <NUM> of <FIG>). Accordingly, the current <NUM> (Ik) on a sensor <NUM> (k) may be represented as follows:
<MAT> where ω represents an AC voltage frequency (multiplied by <NUM>π). Modeling this relationship using a function that is linear when signals from each sensor are scaled together and that is not dependent on (x,y,Θ,φ) may simplify calculation of the voltage V. The following equations show one example of such a linear function: <MAT> <MAT>.

In some examples, we put as a requirement that F(ωC<NUM>, ωC<NUM>. , ωCn) is a constant function with a value of C for the golden pen. Accordingly, an effective voltage V for any other pen may be calculated as follows:
<MAT>.

In this manner, calculating the value of F at one point and dividing by C yields the voltage V of the pen.

With reference again to <FIG>, at <NUM>, the method <NUM> includes building a function using the principles outlined above. For example, a function F may comprise a polynomial including a sum of signals from a plurality of sensors (I<NUM>, I<NUM>,. , In) with coefficients (kn):
<MAT>.

To find the coefficients (kn), at <NUM>, the method <NUM> includes optimizing the function F to generate the voltage data model <NUM>. The function may be optimized to find coefficients that make the function F as constant as possible. The coefficients may be optimized using any suitable algorithm. For example, the coefficients may be optimized using linear regression.

However, the polynomial function described above in equation (<NUM>) may continue to fluctuate slightly as a function of (x, y, Θ, φ). Accordingly, a more constant function may be built by considering non-linear features. One example is provided below in equation (<NUM>):
<MAT>.

In equation (<NUM>), ki and kij represent a set of coefficients that may be optimized to make the function more constant.

In some examples, a function that incorporates tilt (Θ) and azimuth (φ) of the pen may be utilized for the voltage data model <NUM>. In these examples, signals from a plurality of transmitters on the pen may be utilized to incorporate tilt (Θ) and azimuth (φ). For example and with reference to <FIG>, a tip signal received via the tip transmitter <NUM> and a body signal received via the body transmitter <NUM> of electronic pen <NUM> may be utilized.

However, in some examples the tip transmitter <NUM> and the body transmitter <NUM> may have different voltages, for example due to each transmitter having different drivers in the electronic pen <NUM>'. To take the tip transmitter <NUM> and the body transmitter <NUM> into account, a "signal moment" M may be used to represent signals received along each axis of the touch-screen surface. In some examples, signal moment M comprises a sum of a plurality of signals along each axis. For example, MTX<NUM> is a sum of signals ITX received via the tip transmitter <NUM> along the X-axis of the touch-screen surface <NUM>:
<MAT>.

An X-axis moment, MTX<NUM>, represents a position of the center of mass for the tip along the X-axis:
<MAT>.

Similarly, MTY<NUM> is a sum of signals ITY received via the tip transmitter <NUM> along the Y-axis of the touch-screen surface <NUM>. A Y-axis moment, MTY<NUM>, represents a position of the center of mass for the tip along the Y-axis.

Higher order moments of order S are a sum of signals with position centered to the center of mass:
<MAT>.

When S is <NUM>, the moment M may represent a width of a bell-shaped curve produced on the antennas <NUM> of <FIG> by the tip transmitter <NUM>. The momentM may represent skew when S is <NUM>. Additionally, when S is greater than zero, the moment MTXS is not a function of voltage. Correspondingly, when S is zero the moments MTX<NUM>, MTY<NUM>, MRX<NUM>, MRY<NUM> are a function of voltage, where MRX<NUM>, MRY<NUM> represent the moments of the body transmitter <NUM> in the X- and Y-axes, respectively.

A set of features f may be built using the following moments: MTX<NUM>, MTY<NUM>, MTX<NUM>, MTY<NUM>, MTX<NUM>, MTY<NUM>, MTX<NUM>, MTY<NUM>, MTX<NUM>, MTY<NUM>, MRX<NUM>, MRY<NUM>, MRX<NUM>, MRY<NUM>. The last <NUM> moments M in this list (MTX<NUM>, MTY<NUM>, MTX<NUM>, MTY<NUM>, MTX<NUM>, MTY<NUM>, MTX<NUM>, MTY<NUM>, MRX<NUM>, MRY<NUM>, MRX<NUM>, MRY<NUM>) may be designated as Mfi. In some examples, the S-order moments for the body transmitter <NUM> may not be used. The first two moments (MTX<NUM>, MTY<NUM>) are a linear function of voltage.

In the following example of a set of features f, variables i and j range from <NUM> to <NUM>, the moments MTX<NUM> or MTY<NUM> are included one time, and all other moments M are included up to a power of <NUM>:
<MAT>
<MAT>
<MAT>
<MAT>
<MAT>
<MAT>.

This feature set includes a total of <NUM> features f. Each feature is linear with respect to the voltage, as it includes MTX<NUM> or MTY<NUM> one time, and the remaining moments are dimensionless on voltage. In this manner, a function F may be defined as a linear sum of these features, with each feature denoted in simplified form as fi, and with coefficients ki:
<MAT>.

Linear regression may be used to find suitable values of ki such that F is as constant as possible across different points. For example, coefficients k<NUM>,. , k<NUM> may be set such that when the function F is trained on a golden pen (e.g. using <NUM>,<NUM> samples), the function outputs a mean value of <NUM> when subsequently evaluated on the golden pen. In some examples, the function F outputs a normal distribution of values with a mean of <NUM> and a standard deviation of approximately <NUM>-<NUM>% on a subset of training data from the golden pen.

Evaluating the function F on another input device, such as a second electronic pen, outputs the effective voltage of that device as a fraction (k) that represents a determined (actual) voltage of the input device Vx divided by a golden voltage (VG) of a golden input device:
<MAT>.

For example, an output of <NUM> indicates that the voltage of the input device is <NUM>% of the voltage of the golden pen. In this manner, a voltage data model <NUM> utilizing the function F can calculate the effective voltage of a given input device.

With reference again to <FIG>, the voltage data model <NUM> may be applied to calculate an effective voltage <NUM> of electronic pen <NUM>. As the output of the voltage data model <NUM> may be subject to some variation, an initial portion of input device signals <NUM> may be collected and used to calculate an average effective voltage before adjusting at <NUM> subsequently received input device signals that comprise another portion of the signals. In some examples, the initial portion includes between <NUM> - <NUM> input device signals. For example, a moving average effective voltage may be calculated using the most recent <NUM> input device signals <NUM>.

In some examples, a portion of input device signals <NUM> are received via the electronic pen <NUM> for at least a threshold period of time before using the input device signals to determine an effective voltage <NUM> of the input device, followed by determining the distance of the input device from the surface of the computing device using the adjusted input device signals. For example, a reliable value of the effective voltage <NUM> may be determined by averaging a set of <NUM> input device signals. In some examples, the input device signals <NUM> are sampled every <NUM>, and <NUM> samples may be accumulated in <NUM> seconds. Accordingly, the threshold period of time may be set to a predetermined value of <NUM> seconds. In some examples, the resulting value of the effective voltage <NUM> may be repeatable within <NUM>% on each run, which may correspond to an accuracy of <NUM>%. In some examples, this initial calculation of the average effective voltage may be performed as part of the out-of-box-experience when an end-user first begins using the electronic pen <NUM> with the tablet computing device <NUM>.

In some examples, and in a similar manner as described above with respect to <FIG>, only input device signals <NUM> that are collected while the electronic pen <NUM> is contacting the surface of the tablet computing device <NUM> are used in calculating the effective voltage. As noted above, the computing device <NUM> may determine that the electronic pen <NUM> is contacting the surface <NUM> of the tablet computing device <NUM> via a pressure signal received from a pressure sensor in the tip of the pen. In other examples, the computing device <NUM> determines that the electronic pen <NUM> is contacting the surface <NUM> by determining that the pen is performing an electronic pen stroke on the surface.

In some examples, and with reference again to <FIG>, the effective voltage <NUM> output by the voltage data model <NUM> may be used to continue training and optimizing the voltage data model <NUM> at <NUM>. In this manner, the voltage data model <NUM> may be at least partially trained at runtime using additional input device signals <NUM> and effective voltages <NUM>, which may improve the accuracy of the voltage data model <NUM> over time.

In other examples, input device signals <NUM> may be collected from a one or more additional user input devices. For example, training data may be collected using one, two, three, or more electronic pens in addition to the golden pen. Each of the input device signals <NUM> from each additional pen may be divided by the effective voltage determined for the golden pen before using it in the training process. In this manner, the voltage data model <NUM> may avoid becoming over-fit and may reflect variation in electrical characteristics and geometries among input devices.

Returning again to the method of <FIG>, and once an effective voltage <NUM> is determined, at <NUM> the method <NUM> includes adjusting input device signals <NUM> using the effective voltage <NUM> to generate adjusted input device signals <NUM>. As described above, the input device signals <NUM> are adjusted by dividing the input device signals by the effective voltage <NUM>. In this manner, the adjusted input device signals <NUM> are scaled to a standardized voltage as described above.

The adjusted input device signals <NUM> are then used to determine a distance <NUM> of the electronic pen <NUM> from the surface of the computing device. As described above, the distance <NUM> corresponds to a Z-axis position of the tip <NUM> of the electronic pen <NUM> relative to the surface of the computing device. In some examples, the distance <NUM> is determined by providing the adjusted input device signals <NUM> to a distance data model configured to output the distance. Some examples of suitable data models include linear or non-linear functions (e.g. optimized using a regression algorithm), neural networks, and other machine learning data models.

In the example of <FIG>, the adjusted input device signals <NUM> are provided to a neural network <NUM> that determines the distance <NUM>. In some examples, the neural network <NUM> comprises a fully connected <NUM>-layer network that receives the adjusted input device signals <NUM> and outputs the distance <NUM>. In other examples, any suitable neural network configuration may be utilized.

In some examples, the neural network <NUM> is built and trained during manufacturing of the computing device. For example, the neural network <NUM> may be trained by collecting input device signals when an input device is positioned on the surface and at varying distances away from the surface, such as <NUM>, <NUM>, <NUM>, etc. Like the voltage data model <NUM>, the trained neural network <NUM> may be loaded onto one or more end-user computing devices during manufacturing of the computing devices. In other examples, at least a portion of the neural network <NUM> may be built and/or further trained at runtime on the computing device, or on a combination of one or more manufacturer computing devices and one or more end user computing devices.

<FIG> shows one example of distance values determined using an effective voltage of an input device that is calibrated according to the method <NUM> of <FIG>. <FIG> shows a determined distance <NUM> of the tip of an input device from a surface of a computing device over time <NUM>. The determined distance <NUM> is plotted in millimeters (mm) and the time <NUM> is plotted in seconds (s), with the distance <NUM> sampled and plotted at <NUM> millisecond (ms) intervals. The input device was in continuous contact with the surface at all times, such that the actual distance was <NUM>.

At <NUM> (time = <NUM> seconds), distance calculation was initiated without adjusting the input device signals based on an effective voltage of the input device. From <NUM> seconds to <NUM> seconds, the distance <NUM> was determined by providing a portion of raw digital input device signals from the digitizer to the neural network <NUM>. The neural network <NUM> output values of the distance <NUM> between approximately <NUM> and approximately <NUM>.

During the initial <NUM> seconds, <NUM> samples of input device signals were collected and used to determine an average effective voltage of the input device. As indicated at <NUM>, after <NUM> seconds the distance <NUM> was determined using the average effective voltage to calibrate these subsequent input device signals as described above regarding <FIG>. In the present example, the effective voltage of the input device was <NUM>.

Without voltage adjustment, the distance <NUM> determined by the neural network <NUM> was in a range of approximately <NUM> to <NUM> microns. Following implementation of the effective voltage at <NUM>, the distance <NUM> determined by the neural network <NUM> on this other portion of input device signals was between approximately <NUM> microns and approximately <NUM> microns. As illustrated by example in <FIG>, using the method <NUM> of <FIG> to determine the distance <NUM> resulted in improved accuracy and precision.

In some examples, it may be desirable to report whether an input device is either contacting a surface of a computing device or within a threshold distance of the surface. For example, as introduced above, the tablet computing device <NUM> of <FIG> may display electronic ink when the tip <NUM> of the electronic pen <NUM> is placed on the touch-screen surface <NUM>. In some examples, the tablet computing device <NUM> may determine that the tip <NUM> is contacting the touch-screen surface <NUM> when the determined distance <NUM> is less than a threshold distance. For example, to provide a natural and realistic user experience, the threshold may be set at a suitable level within a typical user's motor capabilities. For example, a typical user may be able to manipulate the electronic pen <NUM> with a precision of about <NUM>. Accordingly, the threshold distance may be set at <NUM> or less. The tablet computing device <NUM> may additionally or alternatively include hysteresis in determining whether the tip <NUM> of the electronic pen <NUM> is contacting the touch-screen surface <NUM>.

Accordingly, and with reference again to <FIG> and <NUM>, the method <NUM> may include performing additional processing using the distance <NUM> to generate an input report <NUM>. The input report <NUM> may be output to the operating system of the tablet computing device <NUM>, to an application running on the device, or to one or more other devices or applications. In some examples, the processing <NUM> may include one or more of time-series smoothing, clipping, hysteresis processing, state machine, and applying any other suitable algorithms or transformations to the distance value <NUM>.

In other examples, different functionality may be enabled based on the distance <NUM> of the tip <NUM> from the surface <NUM>. For example, the tablet computing device <NUM> of <FIG> may include an "airbrush" mode in which different amounts of electronic ink are displayed based on the distance <NUM>. For example, when the distance is greater than a threshold distance (e.g. <NUM>), the tablet computing device <NUM> may not display electronic ink. When the distance <NUM> is less than the threshold distance, the tablet computing device <NUM> may display more electronic ink as the distance <NUM> decreases.

With reference now to <FIG> and <FIG>, a flow diagram is provided depicting an example method <NUM> for determining a distance of an input device from a surface of a computing device. The following description of method <NUM> is provided with reference to the software and hardware components described herein and shown in <FIG> and <FIG>. For example, the method <NUM> may be performed by the processor <NUM> of <FIG>, firmware on the computing device <NUM> of <FIG> or the tablet computing device <NUM> of <FIG>, an operating system or other software component of the computing device <NUM> or the tablet computing device <NUM>, or some suitable combination of components described herein. It will be appreciated that method <NUM> also may be performed in other contexts using other suitable components.

With reference to <FIG>, at <NUM>, the method <NUM> includes receiving a plurality of input device signals via the input device. At <NUM>, the method <NUM> may include, wherein the plurality of input device signals comprise a tip signal from a tip transmitter of the input device and a body signal from a body transmitter of the input device that is spaced from the tip transmitter.

At <NUM>, the method <NUM> includes using a portion of the input device signals to determine an effective voltage of the input device. At <NUM>, the method <NUM> may include, wherein using the portion of input device signals to determine the effective voltage of the input device comprises providing the portion of input device signals to a voltage data model that calculates the effective voltage. At <NUM>, the method <NUM> may include determining that the input device is contacting the surface of the computing device; and providing the portion of the input device signals to the voltage data model on condition of determining that the input device is contacting the surface of the computing device. At <NUM>, the method <NUM> may include, wherein determining that the input device is contacting the surface of the computing device comprises (a) receiving a pressure signal from a tip of the input device, or (b) determining that the input device is moving across the surface of the computing device. At <NUM>, the method <NUM> may include, wherein the effective voltage of the input device comprises a determined voltage of the input device divided by a voltage of a golden input device, wherein the golden input device is utilized to train a voltage data model that calculates the effective voltage.

With reference now to <FIG>, at <NUM>, the method <NUM> includes generating adjusted input device signals by adjusting another portion of input device signals using the effective voltage of the input device. At <NUM>, the method <NUM> may include, wherein generating the adjusted input device signals comprises dividing the other portion of input device signals by the effective voltage of the input device. At <NUM>, the method <NUM> may include receiving the portion of the input device signals via the input device for at least a threshold period of time before generating the adjusted input device signals by adjusting the another portion of the input device signals.

At <NUM>, the method includes providing the adjusted input device signals as an input to a distance model. At <NUM>, the method <NUM> may include using the adjusted input device signals to train a neural network configured to determine the distance of the input device from the surface of the computing device. At <NUM>, the method <NUM> includes receiving, from the distance model, the distance of the input device from the surface of the computing device. At <NUM>, the method <NUM> includes outputting the distance of the input device from the surface of the computing device.

With reference now to <FIG>, a flow diagram is provided depicting another example method <NUM> for determining a distance of an input device from a surface of a computing device. The following description of method <NUM> is provided with reference to the software and hardware components described herein and shown in <FIG> and <FIG>. For example, the method <NUM> may be performed by the processor <NUM> of <FIG>, firmware on the computing device <NUM> of <FIG> or the tablet computing device <NUM> of <FIG>, an operating system or other software component of the computing device <NUM> or the tablet computing device <NUM>, or some suitable combination of components described herein. It will be appreciated that method <NUM> also may be performed in other contexts using other suitable components.

At <NUM>, the method <NUM> includes determining whether the input device is contacting the surface of the computing device. At <NUM>, the method <NUM> includes, if the input device is contacting the surface of the computing device, then providing a portion of input device signals received via the input device to a voltage data model. At <NUM>, the method <NUM> includes using the voltage data model to calculate an average effective voltage of the input device by averaging effective voltages determined from the portion of input device signals. At <NUM>, the method <NUM> includes generating adjusted input device signals by adjusting another portion of input device signals received via the input device using the average effective voltage of the input device.

At <NUM>, the method <NUM> may include wherein generating the adjusted input device signals comprises dividing the other portion of input device signals by the average effective voltage of the input device. At <NUM>, the method <NUM> includes providing the adjusted input device signals as an input to a distance model configured to determine the distance of the input device from the surface of the computing device. At <NUM>, the method <NUM> includes receiving, from the distance model, the distance of the input device from the surface of the computing device. At <NUM>, the method <NUM> includes outputting the distance of the input device from the surface of the computing device.

Computing system <NUM> may take the form of one or more wearable devices, personal computers, server computers, tablet computers, home-entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smart phone), and/or other computing devices. In the above examples, input device <NUM>, computing device <NUM>, electronic pen <NUM>, and tablet computing device <NUM> may comprise computing system <NUM> or one or more aspects of computing system <NUM>.

Computing system <NUM> includes a logic processor <NUM>, volatile memory <NUM>, and a non-volatile storage device <NUM>. For example, the logic processor may be configured to execute instructions that are part of one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs.

The logic processor <NUM> may include one or more physical processors (hardware) configured to execute software instructions.

The terms "program" and "application" may be used to describe an aspect of computing system <NUM> typically implemented in software by a processor to perform a particular function using portions of volatile memory, which function involves transformative processing that specially configures the processor to perform the function. Thus, a program or application may be instantiated via logic processor <NUM> executing instructions held by non-volatile storage device <NUM>, using portions of volatile memory <NUM>. It will be understood that different programs and/or applications may be instantiated from the same application, service, code block, object, library, routine, API, function, etc. Likewise, the same program and/or application may be instantiated by different applications, services, code blocks, objects, routines, APIs, functions, etc. The terms "program" and "application" may encompass individual or groups of executable files, data files, libraries, drivers, scripts, database records, etc..

When included, input subsystem <NUM> may comprise or interface with the one or more user-input devices such as a keyboard, mouse, touch screen, electronic pen, stylus, or game controller.

Claim 1:
A method for determining a distance (<NUM>) of an input device (<NUM>; <NUM>) from a surface (<NUM>; <NUM>) of a computing device (<NUM>; <NUM>), the method comprising:
receiving at the computing device (<NUM>; <NUM>) a plurality of input device signals (<NUM>; <NUM>) from the input device (<NUM>; <NUM>);
using a portion of the input device signals (<NUM>; <NUM>) to determine an effective voltage (<NUM>; <NUM>) of the input device (<NUM>; <NUM>);
generating adjusted input device signals (<NUM>; <NUM>) by adjusting another portion of the input device signals (<NUM>; <NUM>) using the effective voltage (<NUM>; <NUM>) of the input device;
providing the adjusted input device signals (<NUM>; <NUM>) as an input to a distance model (<NUM>);
receiving, from the distance model (<NUM>), the distance of the input device from the surface (<NUM>; <NUM>) of the computing device; and
outputting the distance of the input device (<NUM>; <NUM>) from the surface (<NUM>; <NUM>) of the computing device.