Hybrid text and image based encoding

A configuration for encoding and decoding the data is disclosed herein. A server retrieves webpage content to filter and extract text and image data. The text data is encoded using a lossless encoder, whereas the image data is downsampled to a lower resolution and encoded using a lossy encoder. The encoded text and image data is transmitted over a network. Once the encoded data is received on the client device, the text and image data is decoded using an inverse encoding algorithm and resized at a resolution appropriate to the native resolution of the display device.

BACKGROUND

1. Field of Art

The disclosure generally relates to the field of mobile web access.

2. Description of the Related Art

Mobile web access, or accessing web content on a web browser executing on a handheld mobile device has become commonplace. Mobile web access however, suffers from several limitations and usability problems related to mobile devices' small screen size, limited bandwidth and limited processing capability. The small screen size of a handheld mobile device makes it difficult to render text and graphics configured to be viewed on a standard size desktop computer screen. The limited bandwidth available to handheld mobile devices combined with limited processing capabilities pose additional challenges in delivering a satisfactory web browsing experience to user because of the long page load and refresh times. For example, an independently processing web browser executing on a mobile device would take a very long time to process and load content rich webpages (or web pages).

One solution to overcome the above limitations is to use a proxy server to speed up webpage load times. Proxy servers access web content, process the content and deliver the processed content of a smaller data size to the mobile device. Because the web browser functions are processed in the proxy server, the mobile device can retrieve the web content from the proxy server much faster than it can from accessing web content and processing it locally.

However, processing web content on a proxy server poses several issues, including issues with the delivery of the browsing experience to the client user, such as interacting with the webpage. For example, if the user interacts with the webpage, the mobile device must instruct the proxy server to interact with the webpage in the same manner. Subsequently, the proxy server must process the webpage according to the instructions and transmit the processed data to the requesting mobile device. The process therefore greatly increases the latency within the system evident to the user and provides a poor web browsing experience.

DETAILED DESCRIPTION

Configuration Overview

Embodiments of disclosed system, method and computer readable storage medium seek to deliver webpage (or web page) content to a mobile device to enable faster webpage load times and provide a richer user browsing experience. By providing hi-resolution encoded text data and low resolution image data, the system reduces the size of the data sent to the mobile device, enabling faster page loads. Additionally, unlike retrieving text data of different resolution corresponding to different zoom levels, since the encoded text data is locally stored at a hi-resolution, it is rendered much faster at one or more zoom levels within an application executing on a mobile device102.

The data is encoded by retrieving a webpage (or web page), responsive to a request by filtering the text and image data within the webpage content. The identified text data is extracted from the webpage content and rendered as glyph data and string text or as an image of the text data. A lossless encoder is applied to the extracted text data to generate compressed text data with no loss of resolution. Furthermore, image data is identified within webpage content and extracted as an image. The extracted image is downsampled to a lower resolution. A lossy encoder is then applied to the downsampled image to generate encoded image data. The encoded image data and the encoded text data are transmitted over a network.

Additionally, embodiments of disclosed system, method and computer storage medium enable faster page load time and a better user browsing experience on a mobile device by decoding image data and text data to generate a webpage data described herein. The webpage data is rendered on a mobile device by first receiving the encoded text and image data. The encoded text data is decoded to produce full resolution text data and the encoded image data is decoded to produce partial resolution image data. The decoded image and text data are stored locally on the mobile device or remotely. The stored text and imaged data is resized to a first resolution as determined by a first zoom level of an application rendering the webpage on the mobile device. The resized text and image data are displayed on the application executing on the mobile device.

System Architecture

FIG. 1is a high-level block diagram of a computing environment100according to one embodiment. It is noted that for ease of discussion, in most instancesFIG. 1and the other figures use like reference numerals to identify like elements. A letter after a reference numeral, such as “102A,” indicates that the text refers specifically to the element having that particular reference numeral. A reference numeral in the text without a following letter, such as “102,” refers to any or all of the elements in the figures bearing that reference numeral (e.g. “102” in the text refers to reference numerals “102A” and/or “102B” in the figures).

FIG. 1illustrates client devices (or devices)102A-D (generally102), a plurality of encoding servers106, and a plurality of web hosting servers108connected by a network110. At a high level, an application executing on a client device102requests webpage content over the network110. The server106, responsive to the request received through the network110, retrieves the webpage content over the network110from the appropriate webpage hosting server108. Additionally, the server106identifies and extracts text data and image data from the retrieved webpage content, encodes the text and image data separately and transmits the encoded data over the network110. The client device102receives the encoded text and image data, decodes the data and displays the data using an application executing on the client device102.

The network110represents the communication pathways between the client device102, the encoding server106and the webpage hosting server108. The network110can also utilize dedicated or private communications links that are not necessarily part of the Internet. In one embodiment, the network110uses standard communications technologies and/or protocols. Thus, the network110can include links using technologies such as Ethernet, Wi-Fi (802.11), integrated services digital network (ISDN), digital subscriber line (DSL), asynchronous transfer mode (ATM), etc. Similarly, the networking protocols used on the network110can include multiprotocol label switching (MPLS), the transmission control protocol/Internet protocol (TCP/IP), the hypertext transport protocol (HTTP), the simple mail transfer protocol (SMTP), the file transfer protocol (FTP), etc. In one embodiment, at least some of the links use mobile networking technologies, including general packet radio service (GPRS), enhanced data GSM environment (EDGE), code division multiple access 2000 (CDMA 2000), and/or wide-band CDMA (WCDMA). The data exchanged over the network110can be represented using technologies and/or formats including the hypertext markup language (HTML), the extensible markup language (XML), the wireless access protocol (WAP), the short message service (SMS) etc. In addition, all or some of links can be encrypted using conventional encryption technologies such as the secure sockets layer (SSL), Secure HTTP and/or virtual private networks (VPNs). In another embodiment, the entities can use custom and/or dedicated data communications technologies instead of, or in addition to, the ones described above.

The web hosting server108is an entity that provides webpages and/or other electronic documents to client devices102. The web hosting server108can be, for example, a major Internet web site operated by a national media outlet, a personal blog on a web server operated by a lone individual, and/or another distributor of web pages. While only a plurality of web hosting servers108are shown inFIG. 1, embodiments of the environment100can have thousands or millions of different publishers. Only a plurality of web hosting servers108are shown for purposes of clarity. This description uses the term “webpage” to refer to any electronic file, e.g., document, media, and the like, served by a web hosting server108, regardless of whether the document is technically a webpage.

At least some of the webpages served by the web hosting server108have regions that can be characterized as image and/or text. For example, a webpage may contain text, such as a news story and images which can be related to the news story. Additionally, web page may contain text or image ads hosted by the webpage publisher. The images can be embedded within text, displayed in the background of the text, as a banner across the top or the bottom of the page, along the left and/or the right side margin of the page. Collectively, all the data displayed on a webpage may be characterized as image and/or text data.

An encoding server106represents an entity that receives webpage content request from a client device102. The encoding server106is sometimes referred to as the “server.”FIG. 1illustrates a plurality of servers, embodiments however, can have many more participating servers. The encoding server106retrieves webpage content responsive to a user request102. The image data and the text data from the webpage content is extracted and separately encoded by software and/or hardware executing on the server. Subsequently, the encoding server106transmits the encoded image and text data over the network110.

The client device102represents any entity operated by a user that receives webpage data from the encoding server106. The client device102is sometimes referred to as a “mobile device” or a “display device.” In one embodiment, the client device102includes a computer system utilized by an end-user to communicate with other computers on the network110in order to view a webpage. In other embodiments, the client device102includes a network-capable device other than a computer system, such as a personal digital assistant (PDA), a cellular telephone, a smartphone, a pager, a television “set-top box” etc. AlthoughFIG. 1illustrates only four client devices,102A-102D, embodiments of the present invention can have thousands or millions of client devices connected to the network110.

Thus, the system described herein provides a richer webpage browsing experience to an end-user of a client device102by providing fast page load times and by providing fast rendering of text once the user interacts with the loaded webpage. In contrast to the conventional PCM mode operation where a macroblock of a frame is either encoded losslessly using the PCM mode, or in a lossy mode without using the PCM mode for higher compression ratio, the hybrid encoding mode enables the encoding server106to flexibly encode the text data on the webpage losslessly while the image data can be encoded in a lossy mode. As such, the webpage data is rendered on an application executing on the client device102with hi-resolution text data and low resolution image data. The end user is thus able to read hi-resolution text at several different zoom levels without having to wait for the encoding server106to fetch new, higher resolution text data.

Example Computing Machine Architecture

FIG. 2is a high-level block diagram illustrating one embodiment of a typical computing machine200. It is noted that the computing machine200may be a system or part of a system, e.g., two or more machines operating together or one or more machines operating with one or more other devices. The computing entities illustrated in the environment100(e.g.,102A-D,106,108) are structured similarly to the computing machine200and can be further customized for particular functional configurations (e.g., operations) as described further herein.

The example computer machine200includes a processor202(e.g., a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), one or more application specific integrated circuits (ASICs), one or more radio-frequency integrated circuits (RFICs), or any combination of these), a main memory204, and a static memory206, which are configured to communicate with each other via a bus108. The computer system200may further include graphics display unit210(e.g., a plasma display panel (PDP), a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)). The computer system200may also include alphanumeric input device212(e.g., a keyboard), a cursor control device214(e.g., a mouse, a trackball, a joystick, a motion sensor, or other pointing instrument), a storage unit216, a signal generation device218(e.g., a speaker), and a network interface device220, which also are configured to communicate via the bus208.

The storage unit216includes a machine-readable medium222on which is stored instructions224(e.g., software) embodying any one or more of the methodologies or functions described herein. The instructions224(e.g., software) may also reside, completely or at least partially, within the main memory204or within the processor202(e.g., within a processor's cache memory) during execution thereof by the computer system200, the main memory204and the processor202also constituting machine-readable media. The instructions224(e.g., software) may be transmitted or received over the network110via the network interface device220.

Configuration for Encoding Data

Referring now toFIG. 3, it illustrates an embodiment of the encoding server (or system)106. As noted previously, the encoding server106is structurally configured similar to the computing machine200and further configured as noted herein. The encoding server106is illustrated as a high-level block diagram of modules configured to encode text data and image data within webpage content. Upon reviewing the descriptions herein, those of skill in the art will recognize that other embodiments can have different and/or other modules than the ones described here, and that the functionalities can be distributed among the modules in a different manner. The encoding server106includes a webpage retrieval module302, a text/image filtering module304, a text encoding module308, an image downsampling module310, an image encoding module312and a transmission module314. The modules are configured to function and interoperate as described herein.

The encoding server106includes the web page retrieval module302which retrieves webpage content from a web hosting server108. In one embodiment, the webpage retrieval module receives a request to retrieve a webpage from a user. Responsive to the request, the web page retrieval module accesses the location specified by the user request and retrieves webpage content from the appropriate location.

The text/image filtration module304classifies webpage data as text and/or image data. In one embodiment, the filtration module304classifies webpage data as text based on detecting sharp edges in the retrieved webpage data. In another embodiment, other edge detection algorithms, such as search-based edge detection methods, known to those skilled in the art may be used for the determination. In another embodiment, the filtration module304may use other algorithms, for example, content-based analysis, to determine whether to classify data as text or image. Additionally, the filtration module304filters webpage data based on a classification of image and/or text. In one embodiment, once particular data is identified as text data, the filtration module304extracts the text data by copying the text data as image data at a full resolution. This can be accomplished by modifying an html renderer, like Gecko to render a copy of the text data into a separate buffer that is easily identified. The text can be rendered normally into the image buffer as well and then removed in module308. Text pixels can be replaced with a background color to remove it from the image. A good choice is the pixel immediately to the left of the replaced pixel. In another embodiment, the identified text data is rendered as glyph and string text. In another embodiment, the identified image data is rendered as an image in the filtration module304.

In another embodiment, the filtering module304determines data to be text based on metadata associated with the webpage content. In another embodiment, the text/image filtration module304picks two integer values corresponding to the two dominant colors of the webpage image frame, and replaces one value with another one. In this case, the text data is bi-level run data that is a series of runs between replaced and static pixels. Each time the filter of the text/image filtering module304toggles between replacing a pixel and not replacing a pixel, the run between such states is encoded. For example, for a image frame of black text with white background, the text/image filtering module304may pick value 1 for white pixel and value 2 for a black pixel, and replace the black pixels with white pixels after extracting the bitmap of the image frame. As such, after filtering, the white image data can be compressed with a higher compression ratio. It is noted that, in other embodiments, more than two colors (e.g., black and white) can be used to generate bitmap data of the image frame. An example of filtering data is presented in U.S. application Ser. No. 11/929,718, filed on Oct. 30, 2007, which is herein incorporated by reference.

The text encoding module308receives the filtered text data from the filtration module and losslessly encodes the text data. In one embodiment, the text encoding module308encodes the text data using a run-length (RL) algorithm such as unsigned exp-golomb codes to process the data in a scanline order to produce encoded data which is lossless. In another embodiment, a lossless compression scheme such as portable network graphics (PNG) is used to encode the text data. In other embodiments other proprietary lossless encoding schemes may be used to compress the text data. The text encoding module thereby produces encoded text data with no loss of resolution despite having a smaller data size.

The image downsampling module310receives the filtered image data and downsamples the image data, thereby reducing the spatial resolution of the image. In one embodiment, the downsampling module310reduces the spatial resolution of an image from 960×960 pixels to 320×320 pixels, for example. In other embodiments, other downsampling ratios may be used to reduce the spatial resolution of the image. In another embodiment, other methods of downsampling known to those skilled in the art may be used to downsample the image data.

The image encoding module312encodes the downsampled image. In one embodiment, the image encoding module312is a video encoder using the H.264 standard for image compression, wherein the image of a static web page can be compressed as a video frame. For example, in one embodiment, the encoding module312employs the major components used in the H.264 video compression standard. For example, the image encoding module312also uses the flexible macroblock sizes ranging from 16×16 to 4×4. The various macro and sub-macroblock size allow fine-grained tuning of the blacks to the content being encoded. Other H.264 components, such as logarithmic quantizer, may also be used to perform the compression. Those skilled in the art will recognize that H.264 is used only by way of example and that a variety of other encoding and compression schemes may be used. In other embodiment, other image compression methods, such as fractal compression, JPEG, etc., may be used to compress the downsampled image data. An example of compressing an image as a static video frame is provided in U.S. application Ser. No. 11/929,718, filed on Oct. 30, 2007, which is herein incorporated by reference.

It is noted that the filtering module306, the text encoding module308, the image downsampling module310, and the image encoding module312can be implemented in several ways. In one embodiment, downsampling or encoding is handled by dedicated hardware with a very low latency encoder. In another embodiment, image of the text data or the image data may be compressed by a software encoder as a video frame. Other embodiments may implement the encoding modules308,312and downsampling module310in both hardware and/or software. Other embodiments perform different and/or include additional modules than the one described here.

The transmission module314transmits the encoded image and the encoded text data over the network110. The transmission may be performed by sending data over a network configuration such as the World Wide Web, as described above.

FIG. 4is a flow chart illustrating the operation of the encoding server106according to one embodiment. The process starts402and retrieves406data from a webhosting server108. The filtering module304filters408the retrieved data, classifying the retrieved data as text and/or image data. In response to a determination of text data, the filtering unit304extracts410the text data. The extracted text data is subsequently encoded412by the encoding module308and transmitted414over a network110. Additionally, in response to a determination of image data, the filtering unit304extracts416the image data. The extracted image data is subsequently resized418by the image downsampling module310, encoded420by the image encoding module310and transmitted422over a network110by the transmission module314. Those of skill in the art will recognize that other embodiments can perform the steps ofFIG. 4in different orders. Moreover, other embodiments can include different and/or additional steps than the ones described herein.

Initially, the encoding process starts402on the server106, the process retrieves406data stored on a terminal over a network110. The server106retrieves406data by accessing a particular location specified in a URL or by accessing a particular location responsive to an address provided by a client device102. In other embodiments, the process retrieves406only a part of the webpage data based on the identity of client device102or responsive to the local zoom level of an application executing on the client device102. An example of an embodiment retrieving a portion of a webpage is shown inFIGS. 5aand5bas referenced below.

Referring now toFIGS. 5aand5b, they are examples of using one or more portions of a webpage to perform the filtering408and extraction410steps detailed below. As shown inFIG. 5a, in one embodiment, the webpage is divided into one or more thumbnail tiles, T1. . . TN, N representing an integer value. Thus if the process only has to transmit a portion of the webpage, it may only send one or more thumbnail tiles to reduce the amount of data sent over the network to a client device102and to reduce the amount of processing required on the server in regards to the filtering408and extraction410steps.

FIG. 5billustrates desktop tiles D1. . . DN, N representing an integer value. Desktop tiles D1. . . DNare hi-resolution tiles that comprise a portion of a thumbnail tile T1. . . TN. Unlike thumbnail tiles T1. . . TN, which are low resolution, desktop tiles are hi-resolution and therefore contain more data and are larger in data size. Thus, although desktop tiles deliver webpage content containing more detail, they take longer to transmit over a bandwidth limited network because of their larger size. Additionally, desktop tiles require a longer load time because it takes more processing to display a desktop tile as compared to a thumbnail tile. Thus, in one embodiment the server106may retrieve a thumbnail tile or a desktop tile depending on the data requested by the client device. In another embodiment, the server106may deliver a thumbnail tile at an intermediate stage, while processing and delivering one or more desktop tiles.

Continuing with the process400, the retrieved406data is filtered408by classifying the webpage data as text data and/or image data. The retrieved data is classified and filtered408by applying an edge detection algorithm or by using any one of the methods disclosed above in reference to the filtration module304.

Once the retrieved406webpage data is classified and filtered408as text data or image data, the process400extracts410the text data. The text data is extracted410by obtaining a glyph and string text representing the text data or by copying the text data at a hi-resolution. In other embodiments, the text data is extracted using any of the methods disclosed above in reference to the filtering module304. The extracted410text data is encoded412by compressing the text data in a lossless manner. The data can be compressed using a lossless algorithm known in the art or as described above in reference to the text encoding module308.

In addition to processing the extracted text data, the process400also transforms the extracted image data. The process400resizes418the image data using the image downsampling module. For example, an image of native resolution 960×960 may be downsampled to 320×320, or to 640×640 pixels representing the native resolution of the client device. The image is resized418by applying a downsampling algorithm described above or known in the art. Additionally, the image is encoded420by the image encoding module312to reduce the image's data size. In one embodiment, the image is encoded by applying a lossy encoding algorithm described above, such as JPEG, fractal compression, etc. Finally, the process400transmits414,422the encoded image and text data over a network110to a client device102to provide it for further processing, for example, display or storage.

Configuration for Decoding Data

Referring now toFIG. 6, it illustrates an embodiment of a client device102configured to decode webpage data. As noted earlier, the client device102is structurally configured similar to the computing machine200and further configured as noted herein. The client device102configured to decode data is illustrated as a high-level block diagram of modules configured to decode text data and image data to render a webpage. Upon reviewing the descriptions herein, those of skill in the art will recognize that other embodiments can have different and/or other modules than the ones described here, and that the functionalities can be distributed among the modules in a different manner. The client device102configured to decode webpage data includes a receiving module602, a text decoding module604, a text resizing module606, a video decoding module608, an image resizing module610, and a display module. The modules are configured to function and interoperate as described herein.

The client device102configured to decode data includes a receiving module602which receives encoded text and image data from an encoding server106over the network110. In one embodiment, the receiving module receives the encoded text and the encoded image data responsive to a request for the data. In another embodiment, the receiving module602includes a data extractor to identify received data as text data or encoded image data.

The text decoding module604recreates the original full resolution text data as provided on a webpage. In one embodiment, the text decoding module604recreates the original text data using a run-length decoding algorithm. In other embodiments, other algorithms known in the art are used to recreate the encoded text data at a full resolution. In yet another embodiment, the text decoding module604stores the decoded text data locally on a network enabled client device102.

The text resizing module606resizes the text data based on a first zoom level of the client device102. In one embodiment, if the first zoom level of the client device's102display is set to view the full webpage, then the full resolution text data is resized accordingly. In another embodiment, the first zoom level of the client device102is pre-set, thus the text data is resized according to the first zoom level. The text data is therefore rendered to appear at a resolution appropriate for the zoom-level set on an application executing on the client device102and displaying the webpage data. In yet other embodiments, wherein the first zoom level requires that the text to be rendered at a full resolution, the text resizing module606functions to pass the full resolution data to the application executing on the client device to display the webpage.

In addition, the text resizing module606resizes the text data based on a second zoom level of the client device102. In an embodiment where the decoded text data is resized and rendered at a first zoom level, the text resizing module606resizes the text data if a user locally changes the zoom level. For example, if the first zoom level on a client device102is set to display 100% of the webpage content, then the resizing module606resizes the text data to a resolution appropriate to view the full page. Furthermore, if a user changes the local zoom level to zoom-in on a part of the webpage, then the resizing module606retrieves the locally stored, decoded full resolution text data that is stored locally and resizes the text data to display at the second zoom level.

Continuing withFIG. 6, the video decoding module608decodes the received image data. In one embodiment, the video decoding module608recreates the original image data using conventional H.264 decoding procedures, such as inverse quantization, followed by inverse DCT transform. In other embodiments, other algorithms known in the art are used to recreate the image data. In another embodiment, the video decoding module608stores the decoded image data locally on a client device102.

The image resizing module610, resizes the decoded image data based on a first zoom level of the client device102. In one embodiment, the if the first zoom level of the machine102is set to view a full webpage, then the image resizing module, resizes the image data such that an application executing on the client device102can display the full webpage image. In another embodiment, the first zoom level of the client device102is pre-set, thus the image data is resized accordingly.

The image resizing module610, resizes the decoded image to correspond to a second zoom level if the zoom has been changed natively on the client device102. For example, if the zoom level is changed by a user on the client device102, the image resizing module displays a resized image responsive to the second zoom level. It is noted however, if the image resolution to be displayed is greater than the resolution of the decoded image, the image data may appears pixilated and blocky until the server106transmits hi-resolution desktop tile to the client device102.

The display module612renders the resized text data and the resized image data on the display of a client device102. In one embodiment, the display module612enables an application executing on the client device102to display the rendered text and image data on the client device102. In one embodiment wherein local fonts on the client device are used to render the text data, the display module612displays the resized image at the local zoom level, and then renders the text at appropriate parts of the image. For example, if a pixel is replaced by the filtering module304on the encoding server106, the display module612applies an inverse filtering algorithm to replace the background color with the recreated text data. In an embodiment wherein a user changes the local zoom level, the display module displays the resized image corresponding to the second level. In the same embodiment, if the text data is rendered using local fonts, then the display module612displays the resized font data. If the embodiment uses an image of the text data to render text, then the display612renders the appropriate resized text data onto the image.

FIG. 7is a flow chart illustrating the operation of a client device102according to one embodiment. The process starts702and receives704encoded text data and image data from an encoding server106over a network110. The process, decodes706the received text and stores the decoded image locally on an image buffer. The process then uses the text resizing module606to resize the decoded text and render text onto an image. Additionally, the process employs a video decoding module608to decode716the image data. Subsequently the image is stored718locally on the client device and resized720corresponding to the client device's102first zoom level. The display module disclosed above displays622the image data. Those of skill in the art will recognize that other embodiments can perform the steps ofFIG. 4in different orders. Moreover, other embodiments can include different and/or additional steps than the ones described herein.

The process700starts702and receives704encoded text over the network110. In one embodiment, received data is determined to be encoded text data by the presence of a ‘transparent’ color to do a chroma-key operation. In another embodiment, the presence of a compression scheme itself indicates the presence of text data.

The received704text data is decoded706by the text decoding module604to recreate the text data retrieved by the encoding server106. In one embodiment the text data decoded and recreated as a hi-resolution image of the text data. For example, if the text data was retrieved as a full resolution image of the text data and encoded in a lossless manner, the decoded text data is decoded706and recreated at the full resolution. It is noted that in other embodiments, the decoded706text data comprises glyph and string text data. In another embodiment, the process700decodes706the text data by applying a run-length decoder.

The decoded data whether recreated as a glyph and string text data or a full resolution image of the text data is stored708locally on the client device102. In one embodiment the string text data is stored708locally wherein the text data can be rendered onto the decoded image data as described in further detail below. In another embodiment, the decoded text data is first resized710as described below and then stored locally onto the resized image buffer of the client device102.

In one embodiment, the stored text data is resized710according to a first zoom level of the application displaying the webpage on the client device102. For example, a first zoom level may be determined by a pre-set zoom level such as 100% zoom level or it may be determined based on the resolution of the stored708text data. Additionally, the text data may be resized710one or more time based on whether the zoom level is changed locally on the client device102. In one embodiment, at a second zoom level, the text data is resized to a second resolution. An illustration of the embodiment is described in further detail below in reference toFIGS. 8-13.

The resized710text data is rendered712using an application executing on the client device102. In one embodiment, the text is rendered712on top of the resized720image data at the appropriate locations. In another embodiment, if a pixel is replaced by the filtering module304on the encoding server106, an inverse filtering algorithm is applied to replace the background color with the rendered712text data.

Continuing with the process diagram ofFIG. 7, the image data is received714by the data receiving module602. In one embodiment, data is received over the network110and the receiving module determines the presence of image data based on the presence of compressed data or other methods disclosed above. In other embodiments, image data is sent separately over the network110and is separately received714by the data receiving module602.

The received image data is then decoded716using one of the methods described above, such as a conventional H.264 decoding procedure. The decoded data is then stored718locally and resized720according to a first zoom level of the client device102. The resized image is then displayed722on the device at a resolution appropriate for the first zoom level.

Example Configuration

FIGS. 8 and 9illustrate an image of a webpage where no text extraction is performed and the entire webpage data is encoded as image data on the encoding server106and decoded as image data on a client device102.FIGS. 10,11,12and13illustrate an application of the embodiments disclosed herein. The advantages of the present embodiments are also discussed herein with the discussion ofFIGS. 8 to 13.

FIG. 8illustrates an application802executing on a client device102displaying a thumbnail tile of a webpage. A thumbnail tile T1as discussed in reference toFIG. 5is used to get a high-level, zoomed out version of the webpage for easy navigation. It should be noted that unlike the embodiments described herein,FIG. 8shows webpage data that is encoded as image data804and806without performing a filtration step to determine and extract and encode text data on the encoding server106and text decoding step on the client device102, as disclosed above.

FIG. 9illustrates the problem normally associated with treating all the webpage data as image only. To give the user the zoomed in version, the application executing on the client device102will take the thumbnail tile and zoom it up to the appropriate zoom level. However, since the client device102only has the thumbnail tile to draw a more detailed zoomed-in image, it results in the user seeing a blocky, stretched image. All portions of the image looks blocky, including the text data906and the image data904. The entire image remains blocky at this intermediate stage until the server sends the hi-resolution, desktop tiles to the client device.

FIG. 10illustrates an example application of the configuration described herein. The decoded webpage data is displayed on an application1002executing on a client device102. In one embodiment,FIG. 10illustrates webpage data displayed at a first resolution, wherein the image data1004is low resolution image data, encoded at the encoding server106, decoded and resized on the client device102to correspond to the first zoom level. Additionally,FIG. 10illustrates text data1006, wherein the text data is hi-resolution text data resized to correspond to the first zoom level or a rendering of glyph plus string data. It should be noted that at the illustrated first zoom levels inFIG. 8andFIG. 10the two webpage appear visually similar.

FIG. 11illustrates an exemplary embodiment at an intermediate stage as disclosed herein. The illustrated webpage is decoded and displayed at a second zoom level as disclosed above. InFIG. 11, the text data1106is sent as either full resolution image data or as glyph plus string data. Since the text is full resolution, it allows the text data to be perfectly rendered at all zoom levels including the second zoom level ofFIG. 11. However, the image portion of the page is sent at a partial resolution, thus the image data1104appears blocky and stretched out at the zoomed in level, similar to the image data ofFIG. 9. Thus the present embodiment has the advantage of showing hi-resolution text data in an intermediate stage, allowing the user to interact and zoom into a particular part of the webpage without interrupting the reading experience by forcing the user to wait while the hi-resolution file is retrieved and processed on the client device102.

FIG. 11however, represents only an intermediate stage, while the device is waiting for hi-resolution desktop tile data, as described in reference toFIG. 5b.FIG. 12describes the second stage, where once the server fetches the higher resolution data, the user will see all the content of the webpage including image1204and text1206at a hi-resolution. However, such a fetch operation takes a long time because the desktop tile data is generally a large data file and must be processed on the encoding server, and sent over a resource limited network to the client device. If the user in unable to read webpage text data during this slow fetch and rendering operation, it disrupts the continuity of the user reading and browsing experience.

FIG. 13illustrates a portion of the webpage comprising hi-resolution desktop tile1303and another portion of the page showing low resolution thumbnail tile1301. As noted above, the thumbnail tile1301portion of the webpage data appears blocky and pixilated, including text and image data1304. However, the image and text data1306of the webpage represented by the desktop tile1303does not appear blocky or pixilated. Thus, allowing the client device102to provide hi-resolution desktop tiles1303when available, and presenting low-resolution thumbnail tiles1301while the client device102is waiting for the hi-resolution image or when the hi-resolution tile is unavailable.

Thus the illustrated embodiment presents an advantage over prior art. For example, even after the slow hi-resolution desktop tile1303is delivered and loaded on the client device102, showing hi-resolution text and image, if the user scrolls around the page, the user will see the non-desktop tile portion of the webpage at a low-resolution including the text data. This disrupts the browsing/reading experience because in prior art, the text and image data is encoded as low resolution thumbnail tile1301rendering the text data unreadable. The present embodiment has the advantage of showing hi-resolution text data1304even after hi-resolution desktop tile1303is loaded onto the client device102. Thus the present embodiment allows for a richer browsing experience where the user can continue to interact with the displayed webpage without disrupting the reading/browsing experience.

Additional Configuration Considerations

Upon reading this disclosure, those of skill in the art will appreciate still additional alternative structural and functional designs for a system and a process for providing hi-resolution text data to a client device allowing for a rich browsing experience while providing much faster data delivery and load times through the disclosed principles herein. For example, the server encodes the text and image data such that the text data does not lose any resolution and can be rendered perfectly at any zoom level on the client device. Image data is considered less important to the browsing experience, however, as is delivered as a low resolution image data which may appear blocky and stretched out during an intermediate stage after the zoom in and before a hi-resolution image is retrieved and delivered to the client device. Thus, while particular embodiments and applications have been illustrated and described, it is to be understood that the disclosed embodiments are not limited to the precise construction and components disclosed herein. Various modifications, changes and variations, which will be apparent to those skilled in the art, may be made in the arrangement, operation and details of the method and apparatus disclosed herein without departing from the spirit and scope defined in the appended claims.