Encoding hyperlink data in a printed document

A method may include receiving a request from a software application to print an electronic document; in response to the request, parsing the electronic document; based on the parsing, detecting a text element in a portion of the electronic document that is associated in a structure of the electronic document with a network link, the network link being different than the text element; modifying the portion of the electronic document with the text element to include a printable element encoded with the network link; and transmitting the electronic document as modified to a printer driver.

BACKGROUND

When a user creates an electronic document, they may want to indicate where additional information may be found, such as at a webpage. Because Uniform Resource Locators (URL) of a webpage or a file location are often lengthy and potentially incomprehensible, the link may represented in the document using alternative text. For example, a user may write “Further information may be found here” with the underlining of “here” indicating that a user may click on the word and be taken to the link location.

Given that most documents are consumed on electronic devices, there is no loss of information by not spelling out the URL and instead including an activatable link. However, if a user chooses to print a document, the word “here” will still be the word that is printed out with no indication of the underlying web page linked by the author.

DETAILED DESCRIPTION

Throughout this disclosure, electronic actions may be taken by components in response to different variable values (e.g., thresholds, user preferences, etc.). As a matter of convenience, this disclosure does not always detail where the variables are stored or how they are retrieved. In such instances, it may be assumed that the variables are stored on a storage device (e.g., RAM, cache, hard drive) accessible by the component via an API or other program communication method. Similarly, the variables may be assumed to have default values should a specific value not be described. User interfaces may be provided for an end-user or administrator to edit the variable values in some instances.

In various examples described herein, user interfaces are described as being presented to a computing device. Presentation may include transmitting data (e.g., a hypertext markup language file) from a first device (such as a web server) to the computing device for rendering on a display device of the computing device via a rendering engine such as a web browser. Presenting may separately (or in addition to the previous data transmission) include an application (e.g., a stand-alone application) on the computing device generating and rendering the user interface on a display device of the computing device without receiving data from a server.

Furthermore, the user interfaces are often described as having different portions or elements. Although in some examples these portions may be displayed on a screen at the same time, in other examples the portions/elements may be displayed on separate screens such that not all of the portions/elements are displayed simultaneously. Unless indicated as such, the use of “presenting a user interface” does not infer either one of these options.

Additionally, the elements and portions are sometimes described as being configured for a certain purpose. For example, an input element may be described as being configured to receive an input string. In this context, “configured to” may mean presentation of a user interface element that is capable of receiving user input. Thus, the input element may be an empty text box or a drop-down menu, among others. “Configured to” may additionally mean computer executable code processes interactions with the element/portion based on an event handler. Thus, a “search” button element may be configured to pass text received in the input element to a search routine that formats and executes a structured query language (SQL) query with respect to a database.

Many authors add hyperlinks/encoded files in their digital documents (resume, books, research papers, business cards) which work well on digital devices as the receiver may click on the link to navigate to the intended resource. The same scenario is broken when the digital artifact (e.g., the text element of the link) is printed because the printing service ignores the underling link and prints the text element.

This scenario creates at least two technical problems. First, when the hyperlink is not used in the document itself, and instead text such as “Click here” is used, the reader is unable able to view the underlying data (link/file location). This results in a technical problem of data loss when printed. Additionally, it presents particular problems for users that rely on accessibility reading devices, which cannot access the underlying link. Second, even if the full hyperlink is typed out in the document, a user may have to manually type the data into their web browsers. This process is both very inefficient and prone to errors when a hyperlink is particularly long. Thus, the encoded data is human readable but not machine interpretable. This second problem is a technical problem related to the inability of devices to interpret printed text as a hyperlink.

As a solution to the above problems, system and methods are described herein to represent the hyperlink data in a printed artifact that makes a link both human readable and machine interpretable. Two methods are described. The first uses masking of the ‘text to display’ with a pattern in which the pattern contains the encoded data, which can be scanned using a quick response code or other camera application. The second creates a bounding box around the ‘text to display’ region in which the code in the white space is encoded with the data. In this method, the contrast of the printed pattern may have a lower contrast so as to not detract from the legibility of text when printed. Both of the above technical solutions result in a printed document that maintains its formatting (e.g., the location of the words do not change) and prevents information loss when printing.

Additionally, the below discussion focuses on printing to physical media (e.g., paper), but the encoding process may be used in other instances as well. For example, the output may be a PDF file or used in a webpage to allow a user taking a quick picture/screenshot of the webpage to still be able to obtain any underlying link data.

FIG.1is a schematic diagram of components associated with a print service, according to various examples.FIG.1includes print service102, client device104, original electronic document106, text element108, application logic112, processing system114, API116, data store118, user accounts120, document parser122, link encoder124, encoding schemes126, access rights128, printer driver130, printer132, printed document134, and encoded graphical element136.

Print service102is illustrated as set of separate elements (e.g., component, logic, etc.). However, the functionality of multiple, individual elements may be performed by a single element. An element may represent computer program code that is executable by processing system114. The program code may be stored on a storage device (e.g., data store118) and loaded into a memory of the processing system114for execution. Portions of the program code may be executed in a parallel across multiple processing units (e.g., a core of a general purpose computer processor, a graphical processing unit, an application specific integrated circuit, etc.) of processing system114. Execution of the code may be performed on a single device or distributed across multiple devices. In some examples, the program code may be executed on a cloud platform (e.g., MICROSOFT AZURE® and AMAZON EC2®) using shared computing infrastructure.

Client device104may be a computing device which may be, but is not limited to, a smartphone, tablet, laptop, multi-processor system, microprocessor-based or programmable consumer electronics, game console, set-top box, or other device that a user utilizes to communicate over a network. In various examples, a computing device includes a display module (not shown) to display information (e.g., in the form of specially configured user interfaces). In some embodiments, computing devices may comprise one or more of a touch screen, camera, keyboard, microphone, or Global Positioning System (GPS) device.

A user may create an electronic document using an application installed on client device104, such as a word processing application. In other examples, a user may create an electronic document using a web-based application. In some examples, a user may create the electronic document using the installed application and edit the same document using a web-based version of the application.

As part of the electronic document creating process, a user may link a text element (e.g., one or more words) to a network location. The network location may be a website, file path, email address, in various examples. The characters of text element may not match the network location. For example, the application may implement a feature in which a user may link the text element to the network location but use the text element as the displayed content.

In various example, the electronic document adheres to a standardized data schema through use of a markup language such as extensible markup language (XML). Accordingly, the editing application may present the text element, but the underlying electronic document may store additional information such as formatting, network links, etc. For example, consider that a user has written “my link” and linked it to Wikipedia. If a plain text editor were to display a portion of electronic document, an entry may such the below may appear:<w:hyperlink r:id=“rId5” w:history=“1”><w:r w:rsidR=“00AA1899” w:rsidRPr=“00AA1899”><w:rPr><w:rStyle w:val=“Hyperlink”/></w:rPr><w:t>my link</w:t></w:r></w:hyperlink>

As seen in above text, there is a “hyperlink” with an “r:id” of “rId5” that is associated with the text “my link”. Elsewhere in the electronic document an entry such as the below may appear:Target=“http://www.wikipedia.org/” TargetMode=“External”/><Relationship Id=“rId5”

In this case, the same “rId5” identifier appears that indicates the value of the “Target” is the underlying network link that is associated with the text element “my link.” Accordingly, as discussed in further detail below, document parser122may iterate through a document to identify text elements that are associated with a network link (e.g., using tags), and by using a reference identifier, determine the associated network link. In instances where a link is written out, there may not be a “hyperlink” entry. In these instances, regular expressions or other text pattern matching techniques may be used to identify the links. (e.g., any text beginning with http://).

Original electronic document106represents a document with text element108before any modifications by print service102. After processing by print service102, a modified version of the document may be passed to printer driver130for formatting the document into a printer description language understood by printer132for printing of printed document134. As seen printed document134now includes encoded graphical element136, which when viewed by a code reader, may instruct a web browser to open up the underlying network link.

If print service102is not operating locally on client device104, client device104and print service102may communicate via a network (not shown). The network may include local-area networks (LAN), wide-area networks (WAN), wireless networks (e.g., 802.11 or cellular network), the Public Switched Telephone Network (PSTN) Network, ad hoc networks, cellular, personal area networks or peer-to-peer (e.g., Bluetooth®, Wi-Fi Direct), or other combinations or permutations of network protocols and network types. The network may include a single Local Area Network (LAN) or Wide-Area Network (WAN), or combinations of LAN's or WAN's, such as the Internet.

In some examples, the communication may occur using an application programming interface (API) such as API116. An API provides a method for computing processes to exchange data. A web-based API (e.g., API116) may permit communications between two or more computing devices such as a client and a server. The API may define a set of HTTP calls according to Representational State Transfer (RESTful) practices. For examples, A RESTful API may define various GET, PUT, POST, DELETE methods to create, replace, update, and delete data stored in a database (e.g., data store118).

For example, API116may take as input a document identifier and a Bool value signifying that a the document tied to the document identifier should be modified to include a printable element encoded with network link data. User preferences may also be included as elements of the API call.

Print service102may include a web server (not shown) to enable data exchanges with client device104when print service102is operating remotely from print service102. Although generally discussed in the context of delivering webpages via the Hypertext Transfer Protocol (HTTP), other network protocols may be utilized by web server110(e.g., File Transfer Protocol, Telnet, Secure Shell, etc.). A user may enter in a uniform resource identifier (URI) into web client106(e.g., the INTERNET EXPLORER® web browser by Microsoft Corporation or SAFARI® web browser by Apple Inc.) that corresponds to the logical location (e.g., an Internet Protocol address) of web server110. In response, web server110may transmit a web page that is rendered on a display device of a client device (e.g., a mobile phone, desktop computer, etc.).

Additionally, web server110may enable a user to interact with one or more web applications provided in a transmitted web page. A web application may provide user interface (UI) components that are rendered on a display device of client device104. The user may interact (e.g., select, move, enter text into) with the UI components, and, based on the interaction, the web application may update one or more portions of the web page. A web application may be executed in whole, or in part, locally on client device104. The web application may populate the UI components with data from external sources or internal sources (e.g., data store118) in various examples. For example, the web application may include options to set default print preferences, change print permissions, etc.

Print service102may be implemented according to application logic112. Application logic112may use the various elements of print service102to implement the document modification. For example, application logic112may issue API calls to retrieve or store data from data store118and transmit it for display on client device104. Similarly, data entered by a user into a UI component may be transmitted using API116back to a web server. Application logic112may use other elements (e.g., document parser122, link encoder124, encoding schemes126, etc.) of print service102to perform functionality as described further herein.

Data store118may store data that is used by print service102. Data store118is depicted as singular element, but may in actuality be multiple data stores. The specific storage layout and model used in by data store118may take a number of forms—indeed, a data store118may utilize multiple models. Data store118may be, but is not limited to, a relational database (e.g., SQL), non-relational database (NoSQL) a flat file database, object model, document details model, graph database, shared ledger (e.g., blockchain), or a file system hierarchy. Data store118may store data on one or more storage devices (e.g., a hard disk, random access memory (RAM), etc.). The storage devices may be in standalone arrays, part of one or more servers, and may be located in one or more geographic areas.

User accounts120may include user profiles on users of print service102. A user profile may include credential information such as a username and hash of a password. A user may enter in their username and plaintext password to a login page of print service102to view their user profile information or interfaces presented by print service102in, various examples.

A user account may also include preferences of the user. The preferences may include changing access rights with respect to encoded links. For example, a user may indicate that only certain people—as identified by an identifier such as an email address—may print the document with underlying links encoded as printable elements. Other preferences may include a default setting with respect to encoding such as using a letter mask or a bounding box—or if by default no modifications should occur.

Print service102may be invoked in a number of manners such as using a drop-down menu, a shortcut key combination, clicking a toolbar, etc. In various examples, print service102augments an existing print service of an application to add the printable element described herein. For example, the addition of a printable element with an encoded link may be performed automatically with no additional user input needed. In other examples, a user may select an option in a printing dialog user interface to have the printable element added.

Regardless of the way print service102was invoked, document parser122may parse the document that is to be printed to determine if network links have been associated with any text elements in the electronic document. As part of the parsing process, document parser122may iterate through the document for tags within the structure of the document that indicate a hyperlink. These tags may not be visible to the user when editing the document. For example, there may be a tag with a label of “hyperlink”, and within text of that tag there may be a reference identification.

Document parser122may then use the reference identification to obtain the link from another portion of the document in which the reference identification is used. In various examples, if a text element and its associated network link are the same, the text element is not further processed. Thus, a printable element encoded with the network link is not added to the electronic document in these instances. In other examples, hyperlink text is treated the same as a text element such that the characters of the hyperlink or an added bounding box are encoded with the hyperlink location.

For each text element that has been associated with a network link, link encoder124may generate a printable graphical element that encodes the associated network link according to an encoding scheme selected from encoding schemes126. The process of encoding a link is further discussed in the context ofFIG.2andFIG.3.

Access rights128may store permission information on which user accounts are authorized to print using the added printable elements. The permission information may set at a document level, a user level, or a domain/company level, according to various examples. For example, one company may not permit the addition of the printable elements, whereas another company may allow a subset of their employees to use the printable elements. Before document parser122begins, access rights128may be queried with the requesting user's user identifier (e.g., an email address) to determine if the user has the correct permissions to add the printable elements.

In some examples, the access rights are stored as metadata associated with the document. For example, a user creating the document may identify certain users that are able to print the document using the encoding process described herein. Then, even if there is no communication path to access rights128, a printing service may check the metadata and user identifier of the user who is printing to determine if the printing user is on the identified list.

In various examples, access rights128may also define which encoding scheme to use. For example, some encoding schemes may include an encryption level as well to prevent any unauthorized users from being able to decode the printable element using a code reader. For example, the text may be encrypted using asymmetric encryption techniques (e.g., public key infrastructure) or symmetric encryption (e.g., using a single private key). As with permissions above, encoding schemes may be set at a document, user, or domain level.

FIG.2is an illustration of encoding a character with link data, according to various examples.FIG.2includes text element202, letter segmentations204, encoding pattern206, network link characters208, binary representations210, graphical bit representations212, link character214, binary representation216, and graphical bit representation218.

As indicated above, the network link associated with a text element may be encoded in several ways using an encoding scheme from encoding schemes126.FIG.2illustrates a manner of using the text element as a template for encoding the network link by segmenting one or more of the characters into a plurality of segments. An encoding scheme may store the location and sequence number for characters (capitalized and lowercase) that may make up a text element.

With respect toFIG.2, the text element202is “LINK.” Accordingly, segmentation for the letters ‘L’, ‘I’, ‘N’, and ‘K’ may be retrieved from the encoding scheme.FIG.2includes an example of segmentations of the character ‘L’. The sequence, location, and size of letter segmentations204are an example, and others may be used. As illustrated, letter segmentations204has 16 segments with the first segment of the sequence being in the lower right portion. Accordingly, the first character that is to be encoded will be placed in the location sequence number1, then the second character will use the second location, etc.

Encoding pattern206identifies the location and sequence of each bit of a binary encoding. In this case, encoding pattern206is for eight bits. As with letter segmentations204, the order, shape, and number of bits in encoding pattern206are an example and others may be used.

Network link characters208represent a portion of a link that may be associated with text element202. For illustration purposes, the word “CODE” is being used, but complete URLs will likely be used in most cases. Accordingly, in this instance only four segments of letter segmentations204are needed to complete the encoding, but longer network links may use all 16 segments and continue to use segments of the other letters (e.g., ‘I’). Furthermore, although not described, additional information may be encoded in letter segmentations204. For example, error correction bits, an end of message block, total character count, etc., may be used to speed up processing and ensure more robust decoding if a print is not clear.

Binary representations210are character encodings of each letter of the network link in binary representation. The character encoding may be separate from the encoding schemes in encoding schemes126. For example, the American Standard Code for Information Interchange (ASCII) character encoding may be used. Graphical bit representations212may result from using the binary representations210and encoding pattern206. In the illustrated case, a black portion indicates a value of ‘1’ in the binary representation and a value of ‘0’ is white.

For example, link character214is a ‘C’. A capital C in ASCII has a value of 67 in decimal form. 67 is represented as 01000011 in binary form (e.g., binary representation216). The first digit of binary representation216is a zero. Accordingly, the first block of the sequence in encoding pattern206may remain white. The second digital of binary representation216is a one. Thus, the second block of encoding pattern206is black, and so on.

In various examples, encoding schemes126maps characters to graphical bit representation. When a network link is being encoded, the graphical bit representation corresponding to a character of the network link may be retrieved from encoding schemes126—as opposed to generating the representation dynamically when requested.

To decode an encoded network link, the reverse of the process inFIG.2may be used. For example, an image of a printed document may be analyzed by a decoder application running on a client device. The analysis may include using a computer vision process that looks for characters in which letters of a printed document are dissimilar to the others. For example, because the graphical bit representations have white and black space, a letter will inherently look different (e.g., be lighter in color due to the loss of certain printed portions) than one that is printed without the graphical bit representations. In other examples, orientation markers may be used in an outline around the encoded text as discussed inFIG.3. Then, using an encoding scheme from encoding schemes126, the characters may be segmented, and each graphical bit representation may be read in sequence and converted back to the encoded network link.

FIG.3is an illustration of encoding a bounding box with link data, according to various examples.FIG.3includes text element302, bounding box representation304, orientation marker306, encoding pattern308, encoding pattern310, graphical bit representation312, and graphical bit representation314.

In contrast to segmenting characters of the text element as inFIG.2, white space surrounding the text element is segmented in an outline. As seen in bounding box representation304, there are nine segments. Additionally, there are three orientation markers, such as orientation marker306. The orientation markers may be used by the decoder application to determine the bounds of the bounding box. Orientation markers may also be used when encoding the network link into the text elements to allow the decoder application to find the bounds of the encoded text more quickly.

Although not required, encoding patterns may be created in a number of orientations. For example, encoding pattern308is oriented vertically whereas encoding pattern310is oriented horizontally. The vertical orientation encoding bit patterns may be used on the left and right sides of bounding box representation304, and the horizontal encoding patterns may be used on the top and bottom.

Graphical bit representation312may be generated in a similar manner as the graphical bit representations ofFIG.2. For example, assuming the same network link of “CODE,” the 4th letter is ‘E’, which corresponds with segment4of bounding box representation304. Accordingly, a vertical pattern (such as encoding pattern308), may take the ASCII value ‘E’ (69) and apply encoding pattern308to generate graphical bit representation312. With respect to segment1of bounding box representation304, encoding pattern310may be used for the ‘C’ to generate graphical bit representation314.

FIG.4is an illustration of a method of modifying an electronic document, according to various examples. The method is represented as a set of blocks that describe operation402to operation410of flowchart400. The method may be embodied in a set of instructions stored in at least one computer-readable storage device of a computing device(s). A computer-readable storage device excludes transitory signals. In contrast, a signal-bearing medium may include such transitory signals. A machine-readable medium may be a computer-readable storage device or a signal-bearing medium. The computing device(s) may have one or more processors that execute the set of instructions to configure the one or more processors to perform the operations illustrated inFIG.4. The one or more processors may instruct other component of the computing device(s) to carry out the set of instructions. For example, the computing device may instruct a network device to transmit data to another computing device or the computing device may provide data over a display interface to present a user interface. In some examples, performance of the method may be split across multiple computing devices using a shared computing infrastructure.

In various examples, operation402of method400includes receiving a request from a software application to print an electronic document. For example, the request may be received from a printing service such as print service102. The software application may be a word processing application.

In various examples, operation404of method400includes in response to the request, parsing the electronic document. The parsing may be performed using a parser such as document parser122. Parsing may include iterating through the underlying file structure of the electronic document and not just text visible to a user when using the software application.

In various examples, before the parsing, method400may include determining that a user associated with the request has permission to modify the portion of the electronic document associated with the text element based on permission metadata of the electronic document.

In various examples, operation406of method400includes based on the parsing, detecting a text element in a portion of the electronic document that is associated in a structure of the electronic document with a network link. In some examples, the network link is different than the text element. Detecting may include finding a tag that signifies a hyperlink.

In various examples, operation408of method400includes modifying the portion of the electronic document with the text element to include a printable element encoded with the network link. For example, an encoding scheme that maps characters to a graphical bit representation may be retrieved. The encoding scheme may be retrieved from encoding schemes126. Generating the printable element may include encoding the network link using the encoding scheme. Encoding may be performed using link encoder124.

In various examples, the modifying is performed automatically in response to receiving the request whereas in other examples the modifying is performed in response to receiving a selection of a presented user interface element option to modify the electronic document.

In various examples, as part of the received request formatting preferences for the modifying may be received. and the portion of the electronic document associated with the text element may be modified in accordance with the request. For example, a formatting preference may indicate to use a bounding box vs an encoded text element. The preference may also indicate a level of contrast to use for the bounding box. For example, the printable element may be formatted at a lower contrast level than the text element.

The printable element may be added to the electronic document by adding the printable element as an outline (e.g., a bounding box) surrounding the text element. Portions (e.g., segments) of the outline may be encoded with character data corresponding to character data of the network link. An example of an outline surrounding the text element is discussed in the context ofFIG.3.

In various examples, the text element may be removed and replaced with the printable element. The printable element may be an encoded text element in which portions of characters of the encoded text element are encoded with character data corresponding to character data of the network link. An example of an encoded text element is discussed inFIG.2.

In various examples, modifying the portion of the electronic document with the text element to include the printable element encoded with the network link preserves placement of other text elements in the electronic document. Accordingly, in various examples, a portion may be defined as the text element and white space surrounding the text element. Thus, modifying the portion of the electronic document may not introduce additional spacing or alter the location or size of the non-linked text elements.

In various examples, retrieving may include selecting the encoding scheme from a plurality of encoding schemes based on a preference received as part of the request. For example, the request may be associated with a user identifier and the preferences of the user identifier may indicate a specific encoding scheme to use due to the user identifier being part of a company domain.

In various examples, operation410of method400includes transmitting the electronic document as modified to a printer driver. The printer driver may translate the modified electronic document into a printer description language before printing the electronic document.

Examples, as described herein, may include, or may operate on, logic or a number of components, modules, or mechanisms. Modules may be hardware, software, or firmware communicatively coupled to one or more processors in order to carry out the operations described herein. Modules may hardware modules, and as such modules may be considered tangible entities capable of performing specified operations and may be configured or arranged in a certain manner. In an example, circuits may be arranged (e.g., internally or with respect to external entities such as other circuits) in a specified manner as a module. In an example, the whole or part of one or more computer systems (e.g., a standalone, client or server computer system) or one or more hardware processors may be configured by firmware or software (e.g., instructions, an application portion, or an application) as a module that operates to perform specified operations. In an example, the software may reside on a machine-readable medium.

In an example, the software, when executed by the underlying hardware of the module, causes the hardware to perform the specified operations. Accordingly, the term hardware module is understood to encompass a tangible entity, be that an entity that is physically constructed, specifically configured (e.g., hardwired), or temporarily (e.g., transitorily) configured (e.g., programmed) to operate in a specified manner or to perform part or all of any operation described herein. Considering examples in which modules are temporarily configured, each of the modules need not be instantiated at any one moment in time. For example, where the modules comprise a general-purpose hardware processor configured using software; the general-purpose hardware processor may be configured as respective different modules at different times. Software may accordingly configure a hardware processor, for example, to constitute a particular module at one instance of time and to constitute a different module at a different instance of time. Modules may also be software or firmware modules, which operate to perform the methodologies described herein.

Example computer system500includes at least one processor502(e.g., a central processing unit (CPU), a graphics processing unit (GPU) or both, processor cores, compute nodes, etc.), a main memory504and a static memory506, which communicate with each other via a link508(e.g., bus). The computer system500may further include a video display unit510, an input device512(e.g., a keyboard), and a user interface (UI) navigation device514(e.g., a mouse). In one embodiment, the video display unit510, input device512and UI navigation device514are incorporated into a touch screen display. The computer system500may additionally include a storage device516(e.g., a drive unit), a signal generation device518(e.g., a speaker), a network interface device520, and one or more sensors (not shown), such as a global positioning system (GPS) sensor, compass, accelerometer, or other sensor.

The storage device516includes a machine-readable medium522on which is stored one or more sets of data structures and instructions524(e.g., software) embodying or utilized by any one or more of the methodologies or functions described herein. The instructions524may also reside, completely or at least partially, within the main memory504, static memory506, and/or within the processor502during execution thereof by the computer system500, with the main memory504, static memory506, and the at least one processor502also constituting machine-readable media.