Control areas on media

In some examples, a controller is to track markings of an input element in a content area defined on a medium without electronic input capability, and track an input made with the input element in a file handling control area defined on the medium, the input made with the pen in the file handling control area comprising information relating to a file and is to cause saving of a representation of the markings made in the content area according to the information relating to the file.

BACKGROUND

A user can make markings on a medium such as a paper, where the markings can include drawings or text written by the user on the paper. To capture such user-entered markings on the paper, a scanner can be used to scan the paper with the markings, which can electronically capture the markings for storage or communication to a target.

DETAILED DESCRIPTION

Traditionally, users can write text, make drawings, or input other markings onto a medium that is without electronic input capability. A medium without electronic input capability is a medium that does not have the capability to electronically detect input made based on touch or proximity to the medium. For example, the medium can be a piece of paper, a transparency foil, or any other substrate on which users can make markings using a pencil, an ink pen, a marker, or any other writing instrument that is able to make a physical marking on the substrate. A medium without electronic input capability is contrasted to an electronic input device, such as a touch-sensitive display device, a touchpad, or a digital pad that has sensing circuitry (e.g., capacitive-based sensors, resistive-based sensors, optical sensors, etc.) to sense touch input made with a user's finger or a digital pen. A medium without sensing circuitry to detect touch input or other user-provided input can also be referred to as a “passive medium,” which is a medium in which markings can be made onto the medium but the medium itself is unable to electronically capture the markings.

A pen can refer to an instrument that is used to enter markings onto a surface of the medium. The markings can be invisible or visible to a user. A marking that is invisible to a user can either be made with ink that is outside the visible light spectrum, or can be a virtual marking that corresponds to the movement (gesture) of the pen but which does not actually make any physical marking on the medium.

A digital pen is a pen that includes a controller, a sensor, and a memory, where the controller can use measurements of the sensor to capture strokes of the digital pen on a medium, and to store information of the strokes in the memory of the digital pen. A “controller” can refer to a hardware processing circuit, such as any or some combination of the following: a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit device, a programmable gate array, or any other type of hardware processing circuit. Alternatively, a controller can refer to a combination of a hardware processing circuit and machine-readable instructions executable on the hardware processing circuit.

Although digital pens can be used to capture gestures made by users on a medium, the medium itself remains devoid of any control features. For example, once a user makes markings on the medium with a digital pen that are captured electronically, such as by a computer, the user would then have to go to the computer to perform control actions with respect to the captured markings, which can be inconvenient.

In the ensuing discussion, reference is made to using a pen to make inputs with respect to a medium. More generally, an input element can be used to make inputs with respect to a medium, where an input element can refer to a pen (either a digital pen or a traditional writing instrument such as an ink pen, a pencil, a marker, etc.), a user's finger, or any other element that can be used to make a marking (either a physical marking or a virtual marking) on the medium.

In accordance with some implementations of the present disclosure, a medium without electronic input capability can include multiple defined areas on the medium, where a first area defined on the medium can include a content area to accept markings made with an input element by a user, and a second area defined on the medium can include a control area in which a user-input selection can be made using the input element to perform a control action with respect to the markings made in the content area.

In some examples, a control area on a medium can include a file handling control area in which an input can be made with an input element, where the input in the file handling control area includes information relating to a file and is to cause saving of a representation of the markings made in the content area according to the information relating to the file. As examples, the information relating to a file can include a name of the file, a type of the file, a location of the file, or any other information associated with the file. As used here, a “file” can refer to any electronic container that is able to store information. For example, a file can be a file that is defined by a file system that provides for a hierarchical arrangement of files and directories. In other examples, a file can include any other type of object in which information can be stored, and where the object can be uniquely identifiable.

In further examples, the control area on the medium can include a layer control area to allow for user selection of any one of multiple layers, so that markings in the content area can be applied to the selected layer.

FIG. 1Ashows an example arrangement that includes a passive medium102and a digital pen104that can be used to make markings in a content area106defined on the medium102. The content area106can be visibly defined using a boundary108that is visible to a user. Alternatively, the content area106is an area on the medium102that is not visibly identified to the user. A control area110is also defined on the medium102, where the control area110can be defined by a boundary112that can be visible to a user, or alternatively, the control area110can be without a boundary that is visible to a user. The control area110can include a file handling control area or a layer control area.

The medium102is also provided with a pattern of location information that can be detected by the digital pen104for use in determining where the digital pen104is relative to the medium102. In some examples, the pattern of location information can include a pattern of dots that can be detected by a sensor114in the digital pen104. The pattern of dots can be printed onto the medium102, and can be either visible or invisible to a user. For example, the pattern of dots can be printed with ink in the visible wavelength spectrum or outside the visible wavelength spectrum.

The pattern of dots on the medium102can encode coordinates on the medium102. The coordinates can be indicated based on different arrangements of dots in different parts of the medium102. In other examples, instead of using a pattern of dots, other patterns of location information (e.g., barcodes, text or numbers, etc.) can be provided on the medium102. When the digital pen104is located over different parts of the medium102, different location information is captured to allow the position of the digital pen104to be derived.

The digital pen104can include various electronic components, including the sensor114, a controller116, and a memory118. The sensor114can be used to capture markings or selections made by a user on the medium102with the digital pen104. The markings made by the digital pen104in the content area106can include drawings, text, or other markings, and the markings can be made with visible ink or invisible ink, or alternatively, the markings are virtual markings where no physical marking is actually made in the content area106, but the gestures made by the user are captured to determine the markings that are being virtually made by the user using the digital pen104. As a user moves the digital pen104in the content area106, the sensor114can sense the gesture movement, and the controller116is able to capture the marking corresponding to the gesture movement.

A control selection made in the control area110can include text or other marking made by the user with the digital pen104in the control area110, or alternatively, a selection can include the selection of a specific position in the control area110, such as a position relating to a control icon or control text printed in the control area110. For example, if a control icon or a control text (more generally referred to as a “control indication”) is printed in the control area110, a control selection can include the digital pen104selecting the control indication in the control area110. The selection of the control indication can include touching the control indication for greater than some specified time duration, or alternatively, the selection of the control indication can include circling the control indication or crossing through control indication with the digital pen104.

The controller116can receive data from the sensor114, where the data can include data representing captured location information of the pattern of location information. In some examples, the controller116can process the captured data from the sensor114, and the controller116is able to determine whether the captured data represents a marking made in the content area106or a selection made in the control area110. If the controller116detects a marking made in the content area106, the controller116can store the captured marking in the memory118. If the controller116detects a selection made in the control area110, the controller116can perform a control action in response to the selection made in the control area110.

In other examples, instead of performing processing of the captured data from the sensor114by the controller116, the controller116can store the captured data along with coordinates associated with the captured data into the memory116, and can communicate the captured data along with the coordinate information to a controller (e.g., a computer) that is separate from the digital pen104. The digital pen104can communicate with the separate controller over a wireless medium or a wired medium. The separate controller can then perform the processing of the captured data to determine whether the captured data includes a marking made in the content area106or a control selection made in the control area110.

The memory110can include either a volatile memory or a non-volatile memory. A volatile memory can include a dynamic random access memory (DRAM), a static random access memory (SRAM), or other memory that loses its stored content when power is removed from the memory. A non-volatile memory can include a flash memory or other type of memory that does not lose its stored content in response to removal of power from the memory.

FIG. 1Bshows another different example arrangement including the passive medium102that has the content area106and the control area110. In the example ofFIG. 1B, the medium102is not provided with a pattern of location information. Instead, the medium102inFIG. 1Bis provided on a digital support structure120that has sensing circuitry to sense a position of a pen122, and gestures made with the pen122on the medium102. The digital support structure120can include a digital board or any other type of support structure. The pen122can include a digital pen or a non-digital pen such as a traditional writing instrument or any other instrument that can be used to make either physical or virtual markings or selections on the medium102. Alternatively, a different input element, such as a user's finger, can be used in the arrangement ofFIG. 1Bto make markings or selections on the medium102.

If the pen122is a digital pen, then the pen122can include an emitter (e.g., an optical emitter, a radio frequency emitter, etc.) to emit signals that can be detected by the sensing circuitry of the digital support structure120. If the pen122is a non-digital pen or other input element, then the sensing circuitry120is able to detect the pen122or other input element when it makes contact with the medium102. For example, in the latter case, the sensing circuitry of the digital support structure120can include capacitive-based sensors, resistive-based sensors, or other types of sensors to sense strokes of the pen122or other input element.

In the examples according toFIG. 1B, the sensed positions and markings made using the pen122on the medium102can be communicated by the digital support structure120to a computer124. The communication can include a wireless communication or a wired communication.

The computer124includes a processor126and a storage medium128that stores passive medium input processing instructions130that are executable on the processor126to process inputs made with the pen122on the medium102, as detected by the digital support structure120. The passive medium input processing instructions130can detect markings made in the content area106, and control selections made in the control area110, and the passive medium input processing instructions130can perform the requested control actions with respect to the markings made in the content area106. In other examples, instead of using the computer124, the digital support structure120can include a controller that can perform tasks similar to the tasks of the passive medium input processing instructions130.

The computer124can be considered a controller in the arrangement shown inFIG. 1B.

More generally, instead of using the digital support structure120on which is placed the medium102, an electronic layer can be placed over or under the medium102, where the electronic layer includes sensing circuitry to sense a position of the pen122or other input element relative to the medium102.

FIG. 1Cshows another example arrangement that includes the passive medium102with the content area106and the control area110. InFIG. 1C, a pen122(digital pen or non-digital pen) can be used to make inputs in the content area106and the control area110. Alternatively, instead of the pen122, a different input element (such as a user's finger) can be used to make an input on the medium102. In addition,FIG. 1Cshows a camera132that can be used to capture an image of the passive medium102and the pen122. The camera130is in communication with the computer124shown inFIG. 1C, where the communication can include a wired communication or a wireless communication.

In the example ofFIG. 1C, passive medium optical input processing instructions134are executable on the processor126of the computer124to process image data captured by the camera132, where the image data includes an image of the passive medium102and the pen122. The captured image can include a still image or images of a video. The passive medium optical input processing instructions134are executable on the processor126to process the image data to determine a relative position of the pen122with respect to the passive medium102, and to determine a gesture made by the pen122in either the content area106or the control area110of the passive medium102. Based on such processing, the passive medium optical input processing instructions134are able to detect markings made in the content area106and control selections made in the control area110and can perform the corresponding control actions with respect to the captured markings.

The control area110and the content area106(or more specifically, the boundaries of the control area110and the content area106) can be pre-printed onto the medium102, or alternatively, can be printed onto the medium102on demand.

In some examples, multiple users collaborating on the same document can print the document at the disparate locations of the users. Each respective user of the multiple users can print a copy of the document onto the medium (102), and the respective user can mark up the same document and each user's markups can be tracked. For example, assuming a corresponding digital pen is used by each respective user, a unique identifier that is assigned to the corresponding digital pen can be used to indicate which user made a specific markup (or specific markups). In examples where multiple layers can be defined, different layers can be associated with the different users, and the unique identifiers of the digital pens of the different users can be associated with respective different layers.

FIG. 2shows an example arrangement that includes a controller200that is able to perform tasks with respect to inputs made by an input element202relative to the passive medium102that includes the content area106and the control area110. In the example ofFIG. 2, it is assumed that the control area110is a file handling control area. The input element202can include a digital pen, a non-digital pen, or a user's finger, as examples. The positions of the input element202relative to the passive medium102, and gestures made by the input element202can be captured using any of the sensing mechanisms depicted inFIGS. 1A-1C, as examples.

The controller200can perform a content area input element tracking task204that tracks markings of the input element202in the content area106defined on the medium102. The controller200further performs a file handling control area input tracking task206that tracks an input made with the pen202in the file handling control area110, where the input that is made in the file handling control area includes information relating to a file and is to cause saving of a representation of the markings made in the content area106according to the information relating to the file.

An example of the file handling control area110is shown inFIG. 3. InFIG. 3, the file handling control area110includes a file name entry area302, where a user can use the input element202to write a name of a file (file name) into the file name entry area302. The file handling control area110also includes several control indications that are selectable by a user using the input device202. A first control indication304when tapped by the input device or otherwise selected by the input device (e.g., such as by drawing a circle around the control indication304, crossed through using the input device, and so forth), provides an indication that the captured markings in the content area106are to be uploaded to a digital pen. A control indication306when selected indicates that the captured markings in the content area106are to be uploaded to a cloud or other storage location. A control indication308when selected indicates that the markings are to be stored in a Portable Document Format (PDF) document.

More generally, information relating to a file that can be entered into the file handling control area110can include a characteristic of the file selected from among a file name, a type of the file (e.g., a PDF file, a word processing file, a spreadsheet file, a drawing file, etc.), and/or a location of the file (e.g., a digital pen, a cloud, a network, etc.). In an example, the control indications may identify locations in which to save files. A location of the file may include a device on which to save the file (e.g., the digital pen, the cloud, the network, etc.), a directory in which to save the file (e.g., commonly used directories, user-specified directories, etc.), both a device and a directory, and so forth.

The file may be saved when a control indication of the location is selected (e.g., control indication304,306, or308) regardless of what additional information has been specified. In an example, the file may be saved when a minimum set of information has been specified, such as a file name and a location. The file may be saved when any information is entered. The file may be updated as additional information is received. For example, the file may be saved in a default location and as a default type when the file name is entered. It can be moved or the type can be changed as more information is entered. Similarly, a default name may be assigned if a control action other than a file name is selected. The default name may be replaced if a file name is later entered. Saving the file when any information is entered may reduce the risk of the user accidentally not saving the file.

Although a specific arrangement of the file handling control area110is shown inFIG. 3, it is noted that in other examples, the file handling control area110can have other arrangements.

In some examples, a user or a program can control (such as with a computer) what control elements are to be included in a control area (such as the control area110). Based on the control elements the user or program has selected for inclusion in the control area110, a copy of the medium102can be printed with the control area110containing the control elements that have been selected for inclusion in the control area110. For example, a user may select control boxes or icons representing common file types, particular directories, etc., for inclusion in the control area110. Similarly, the user may remove control boxes or icons representing file types or storage devices that the user does not use or that are not available. Other types of control elements can be selected by a user or program for inclusion in the control area110in other examples.

FIG. 4is a block diagram of an example of a passive medium400that includes a file handling control area402(similar to the file handling control area shown inFIG. 3, for example), a content area404, and a layer control area406. In other examples, the file handling control area402can be omitted from the medium400.

The layer control area406includes layer indications (numbers 1, 2, 3, 4, and 5 in the example ofFIG. 4) that are used to select a layer from among multiple layers (e.g., layer 1, layer 2, layer 3, layer 4, and layer 5) that correspond to the layer indications. For example, selection of the number 1 in the layer control area406is a selection of layer 1, selection of the number 2 in the layer control area406is a selection of layer 2, and so forth. Selecting a given layer indication can include touching the given layer indication with an input element (e.g., tapping the given layer indication or touching the given layer indication for greater than a specified time duration), circling the given layer indication, crossing through the given layer indication, and so forth.

A “layer” can refer to a logical representation of the medium102in which markings can be made in the layer. Although there is just one physical medium (i.e., the medium102), there can be multiple logical layers defined with respect to the physical medium.

More generally, to select a layer, an input element action relative to a given layer indication (of multiple layer indications on the medium102) is detected, where the detected input element action relative to the given layer indication selects a given layer of the multiple different layers that correspond to respective layer indications. Once the given layer is selected, markings in the content area404are made with respect to the given layer. When a user is finished with the given layer, the markings made for the given layer can be saved and retrieved later. Additionally, the user (or a different user) at a later time can select another layer to enter markings for the other layer.

Examples where layers can be used include a trace mode, where a layer can be defined to accept markings that trace over markings previously made on the medium102or another layer. For example, the markings on the medium102can include a drawing, and a layer can be selected to accept input markings made using an input element to trace over the drawing, such as to make changes to the drawing. The input markings made in the selected layer can be saved electronically, and can be combined with a stored representation of the drawing on the medium102to present (e.g., display, communicate, etc.) the marked up drawing that includes the original drawing and the markings that indicate changes to the drawing.

As another example, the markings on the medium102can include text, and a layer can be selected to accept input markings to make edits to the text.

Multiple layers can be defined to accept respective successive markings that indicate changes to previous versions of the content made on the medium102and/or another layer(s).

In other examples, multiple layers can be used to produce story boards for in a movie production. Story boards can include successive images of a scene in the movie, for example, where the successive images include common features with some changes (e.g., movement of a character, movement of a vehicle, etc.). The multiple layers can represent the story boards.

In some examples, the layer control area406can also include a bookmark indication408that is selectable using an input element to bookmark a layer, which identifies the layer as one which a user may wish to retrieve at a later time. A bookmark representation of the bookmarked layer can be stored and presented (e.g., displayed) on a display device of an electronic device associated with the user. The user can select the bookmark representation to retrieve the bookmarked layer at a later time.

In further examples, the layer control area406can also include a new layer indication410that is selectable using an input element to cause creation of a new layer. Creating a new layer can cause a new layer indication (e.g., the number 6) to be added to the layer control area406. Initially, the new layer indication to be added to the layer control area406can be stored by a system (e.g., a digital pen or a computer). When another copy of the medium400is next printed, either by the same user who has been working on the medium400or by another user, the new layer indication would be included in the layer control area406of the newly printed medium400.

In additional examples, the layer control area406can include a delete layer indication412which when selected using an input element causes deletion of a currently selected layer. The system processing the delete layer indication412can store the indication so that when another copy of the medium400is next printed, the deleted layer would no longer appear in the layer control area406. Alternatively, instead of using the delete layer indication412, a user can use the input element to cross through one of the layer indications (1-5) to cause deletion of the layer. Deleting a layer can include deleting markings captured for the deleted layer.

FIG. 5shows a non-transitory machine-readable storage medium500storing machine-readable instructions that upon execution cause a system to perform various tasks. The machine-readable instructions include layer control area input receiving instructions502to receive input information regarding an input made with respect to a layer control area (e.g.,406inFIG. 4) defined on a passive medium, where the input made with respect to the layer control area selects a first layer of multiple layers. The machine-readable instructions further include content area marking receiving instructions504to receive information of markings made in a content area (e.g.,404inFIG. 4) defined on the medium.

The machine-readable instructions further include marking applying instructions506to apply the markings made in the content area to the first layer in response to the input information.

FIG. 6is a flow diagram of a process according to some examples. The process ofFIG. 6includes tracking (at602), by a controller in a digital pen (e.g.,104inFIG. 1A), markings of the digital pen in a content area defined on a passive medium.

The process further includes tracking (at604), by the controller in the digital pen, an input made with the digital pen in a file handling control area defined on the medium, the input made with the digital pen in the file handling control area including information relating to a file and is to cause saving of a representation of the markings made in the content area according to the information relating to the file, the information relating to the file selected from among a name of the file, a location of the file, and a type of the file.