EXHIBIT 3

Case 2:23-cv-00583-JRG Document 1-3 Filed 12/08/23 Page 1 of 18 PageID #: 51
US008936190B2 

(12) United States Patent (10) Patent No.: US 8,936,190 B2 

Rothschild (45) Date of Patent: *Jan. 20, 2015 

(54) SYSTEMAND METHOD FOR PRESENTING (52) U.S. Cl. 

(71) 

(72) 

(73) 

(*) 

(21) 

(22) 

(65) 

(63) 

(51) 

INFORMATION ABOUT AN OBJECT ON A 

PORTABLE ELECTRONIC DEVICE 

Applicant: Leigh M Rothschild, Sunny Isles Beach, 

FL (US) 

Inventor: Leigh M Rothschild, Sunny Isles Beach, 

FL (US) 

Assignee: Ariel Inventions, LLC, Sunny Isles 

Beach, FL (US) 

Notice: Subject to any disclaimer, the term of this 

patent is extended or adjusted under 35 

U.S.C. 154(b) by 0 days. 

This patent is Subject to a terminal dis 

claimer. 

Appl. No.: 14/181,945 

Filed: Feb. 17, 2014 

Prior Publication Data 

US 2014/O158757 A1 Jun. 12, 2014 

Related U.S. Application Data 

Continuation of application No. 13/868,071, filed on 

Apr. 22, 2013, now Pat. No. 8,651,369, which is a 

continuation of application No. 13/170.810, filed on 

Jun. 28, 2011, now Pat. No. 8,424,752, which is a 

continuation of application No. 12/882,616, filed on 

Sep. 15, 2010, now Pat. No. 7,992,773. 

Int. C. 

G06K 7/10 (2006.01) 

G06O20/10 (2012.01) 

G06O 30/02 (2012.01) 

G06O 50/18 (2012.01) 

G06F 7/30 (2006.01) 

- 20 

Satee 

is 

A stia insistics 

Ter 

Satete 

16. 

sy- Y Corruriatic Network 

CPC G06K 7/10 (2013.01); G06O20/10 (2013.01); 

G06O 30/02 (2013.01); G06Q30/0207 

(2013.01); G06O 50/188 (2013.01); G06F 

17/30 (2013.01) 

USPC ............ 235/375; 235/379; 705/14.1: 705/39; 

7.05780 

(58) Field of Classification Search 

USPC ...................... 235/375, 379; 705/14.1, 39, 80 

See application file for complete search history. 

(56) References Cited 

U.S. PATENT DOCUMENTS 

8,651,369 B2 * 2/2014 Rothschild .................... 235,375 

* cited by examiner 

Primary Examiner — Allyson Trail 

(74) Attorney, Agent, or Firm — Greer, Burns & Crain, Ltd. 

(57) ABSTRACT 

Systems and methods are provided for enabling a portable 

electronic device to retrieve information about an object when the objects symbology, e.g., a barcode, is detected. 

According to one embodiment a method is providing in which 

symbology associated with an object is detected and decoded 

to obtain a decode string. The decode string is sent to one or 

more visual detection applications for processing, wherein 

the one or more visual detection applications reside on the 

portable electronic device, and receiving a first amount of 

information about the object from the one or more visual 

detection applications. The method also includes sending the 

decode string to a remote server for processing and receiving 

a second amount of information about the object from the 

remote server. The first amount of information is combined 

with the second amount of information to obtain cumulative 

information which is displayed on the portable electronic 

device. 

20 Claims, 8 Drawing Sheets 

2 

A Series C 14 } Data is 

- -- 

26 

24 A a k ^ re. ^ 

ificies 

10 

 

Case 2:23-cv-00583-JRG Document 1-3 Filed 12/08/23 Page 2 of 18 PageID #: 52
U.S. Patent Jan. 20, 2015 Sheet 1 of 8 US 8,936,190 B2 

3 TD 14 

Satelite 

i - - - - -r- y --r- Satelite dish/ N- N. --/ \- / 

2. {{ificatio Network 

- A \, 

t N-N- ae --- voie era 

CGinnications :- ? 

wer s A A 

Y s - 

ye; 

 

 

 

Case 2:23-cv-00583-JRG Document 1-3 Filed 12/08/23 Page 3 of 18 PageID #: 53
U.S. Patent Jan. 20, 2015 Sheet 2 of 8 US 8,936,190 B2 

AN 

58 48 

 

Case 2:23-cv-00583-JRG Document 1-3 Filed 12/08/23 Page 4 of 18 PageID #: 54
Case 2:23-cv-00583-JRG Document 1-3 Filed 12/08/23 Page 5 of 18 PageID #: 55
U.S. Patent Jan. 20, 2015 Sheet 4 of 8 US 8,936,190 B2 

{{ y a - 2 

{{f SR 

YR). SS is 

iC3) is \" 

ENCRY 3.N. 

f{}}K 

r Sir AmeM AA3AS 

SRSF & iANA&E N 

\ / C 

{OSSC 

\SYNS 

iC. 

RANS, SSCN 

{}}- 

- 88 

coMMUNICATION/mN EMEN 

E}} 

 

 

 

 

 

 

 

 

 

 

 

 

 

Case 2:23-cv-00583-JRG Document 1-3 Filed 12/08/23 Page 6 of 18 PageID #: 56
U.S. Patent Jan. 20, 2015 Sheet 5 of 8 US 8,936,190 B2 

SY3{{OY 

ifa A{E^ 

fit 

AGE 

(CARE 

AGAON 

{-Er WSA 

ESCCN 

AOACNS 

S{CANN\f 

ACAN 

rCW SCK SERNG 

88, A CRASE 

{RCNC ECS 

122 iDEN FY THE OBJECTS) , ASS{{A} f : 

ECCOE SriNG 

24. 

4. RREW NCAC 

ABO OBJECTS) 

28 SEN). NiAOM 

". C - CRA3 

CRCNC Ey. 

 

 

 

 

 

 

 

 

Case 2:23-cv-00583-JRG Document 1-3 Filed 12/08/23 Page 7 of 18 PageID #: 57
U.S. Patent Jan. 20, 2015 Sheet 6 of 8 US 8,936,190 B2 

ALLOW USER TO SELECT 

rNES 

3 

\ ... RN WSA. - EEC&N 

is . . . . SYSES - 3ASKGirij\; 

34 - 

- 

-1)s TREGGER st 

YES - FROM visuAL DETECTIONs 

s SYSTEM NITATED BY - 

st w W USER - - 

Yal -- 

138 o 

--st 

- DECODABLEN 

< SYMBOOGY ATO, ) spETECTED2 a- --- 

ra 

ra 

NO 

Y--- 

138 ALERT USER THAT IMAGE - ... CONANS SY is COGY 

140 -Y- x. ---------- - s 

- DOES USERs -r- <WISH THAT SYMBOLOGY)-N-(A) YS DECODE32 - N-- - --- N Wr 

YES 

-s a. 

1 HAS A . ?\ YES - VISUALDETECTION - ( B --- APPLICATION BEEN C 

N- NPRE-SELECTED2- N 

 

 

 

 

 

Case 2:23-cv-00583-JRG Document 1-3 Filed 12/08/23 Page 8 of 18 PageID #: 58
U.S. Patent Jan. 20, 2015 Sheet 7 of 8 US 8,936,190 B2 

SEN CE SYS38 CGY 

RE SEECE SA, 

DETECTION APPLICATONS) 144 

Or EC{\{G ANR 

No RAON REREWA 

f D) 

RECEVE AN SORE 146 

NCRA(\; Civil WSA * 

DETECTION APPLICAON(S) ... 8 148 N : 

SEN SEC). SRNG CySNE NFORMA(\ 

Ry{E SERVER RC fSA. ON 

APP. CATIONS AND 

Ry: SERVER 

RECEW NFORMAN 

Ci - Evil SW 

150 - 

SAY NOR.A.C. 

56 

Y. ENA3 SOKA&E {r 

NRAON - SSE3 

158 RN or CASE ACACN 

Y. O E-ASE {ASE 

OF OBJECTS) F DES RED - (A)- -- \ v 

 

 

 

 

 

 

 

 

 

Case 2:23-cv-00583-JRG Document 1-3 Filed 12/08/23 Page 9 of 18 PageID #: 59
U.S. Patent Jan. 20, 2015 Sheet 8 of 8 US 8,936,190 B2 

C{}}E SYS3}}{SY 160 

C. Riv 

CAE SORY SSC 

DETERMINE APPROPRIATE 162 

fSA COM f 

ARiiOATIONES BASED ON 

CAE rY OF O3. C 

84 

- - - - - - - - - Ys 88 Y - Y- s 

- WERE MULTIPLE is EAESERT 

< APPLICATIONS DEEMED TO > i&R FE 

Y- sic - BEAPPROPRIATE2- A CACNS 

s - 

66 ... o 

SEND COESRING TO SEN ECO 

ARCRA ACACN SRNG O SEC is 170 

NCAA CN REWA APFL CATIONS FOR 

NCR.A.N. 

RREWA 

 

 

 

 

 

 

Case 2:23-cv-00583-JRG Document 1-3 Filed 12/08/23 Page 10 of 18 PageID #: 60
US 8,936,190 B2 

1. 

SYSTEMAND METHOD FOR PRESENTING 

INFORMATION ABOUT AN OBJECT ON A 

PORTABLE ELECTRONIC DEVICE 

CROSS-REFERENCE TO RELATED 

APPLICATIONS 

The present application is a continuation patent application 

and claims priority under the laws and rules of the United 

States, including 35 USC S 120, to U.S. patent application Ser. 

No. 13/868,071 filed on Apr. 22, 2013 which is a continuation 

patent application and claims priority under the laws and rules 

of the United States, including 35 USC S120, to U.S. patent application Ser. No. 13/170,810 filed on Jun. 28, 2011 result 

ing in U.S. Pat. No. 8,424.752 issued on Apr. 23, 2013 which 

is a continuation patent application and claims priority under 

the laws and rules of the United States, including 35 USC 

S120, to U.S. patent application Ser. No. 12/882,616 filed on 

Sep. 15, 2010 resulting in U.S. Pat. No. 7,992,773 issued on 

Aug. 9, 2011. The contents of U.S. patent application Ser. No. 

137868,071, U.S. patent application Ser. No. 13/170,810, 

U.S. Pat. No. 8,424,752, U.S. patent application Ser. No. 

12/882,616 and U.S. Pat. No. 7,992,773 are herein incorpo 

rated by reference in their entireties. 

BACKGROUND 

1. Field 

The present disclosure relates generally to portable elec 

tronic devices, and more particularly, to systems and methods 

for enabling portable electronic devices to present informa 

tion about a selected object. 2. Description of the Related Art 

It is increasingly common for individuals to own and carry portable electronic devices, such as mobile phones, personal 

digital assistants (PDAs), etc. Many of these devices are 

capable of providing fully functional computer processing 

incorporating cellular communication and network datacom 

munication. Examples include various WindowsTM Pocket 

PC devices, which have an operating system created by 

Microsoft Corporation of Redmond, Wash. Other examples 

include various mobile phone devices, some of which have 

built-in cameras, Scanning devices, and other types of imag ing devices. Still other portable digital devices include por 

table media players from Creative Media of Singapore, Sam 

Sung of South Korea, Archos, Inc. of Irvine, Calif., and Apple Computers, Inc. of Cupertino, Calif. Additionally, Sony of 

New York and Japan manufacture a portable game playing 

device called the Sony PSPTM (PlayStation Portable). 

Increasingly, many of these portable electronic devices 

include means of transmitting and receiving digital data. Vari 

ous communications technologies are frequently built into 

these devices. Examples of portable communicating devices 

include mobile phones which employ GSM, CDMA, 

W-CDMA, and FOMA technology, among other standards, 

to send and receive data in addition to handling Voice com 

munications. Many of the portable media devices also include 

transmission capabilities including Wi-Fi (IEEE 802.11a, b, g, 

and X among others), BluetoothTM, infrared, etc. to allow the 

user to transmit and receive digital data over relatively short 

distances. The Sony PSPTM device, for example, includes 

Wi-Fi (i.e., IEEE 802.11b) to transmit and receive digital 

data. 

Many portable electronic devices are configured to contain 

various applications, which may be installed during manu 

facture and/or may be downloaded by the user. Examples of 

these applications include e-commerce applications, word 

10 

15 

25 

30 

35 

40 

45 

50 

55 

60 

65 

2 

processing, movie theater locating applications, games, 

e-mail programs, GPS navigation software, medical informa 

tion, weatherapplications, etc. Examples of the portable elec 

tronic devices that contain both applications and imaging 

systems include Apple Computer's IPhone, Google's Droid, 

and various mobile devices from Motorola. 

SUMMARY 

Systems and methods are provided for allowing a user to 

utilize a portable electronic device to retrieve information 

about an object in response to the portable electronic device 

detecting symbology, e.g., a barcode, associated with the 

object. Among various implementations, one embodiment of 

a method is provided in which symbology associated with an 

object is detected and the symbology is decoded to obtain a 

decode string. The method includes sending the decode string 

to one or more visual detection applications for processing, 

where the one or more visual detection applications reside on 

the portable electronic device, and receiving a first amount of 

information about the object from the one or more visual 

detection applications. The method also includes sending the 

decode string to a remote server for processing and receiving 

a second amount of information about the object from the 

remote server. The first amount of information is combined 

with the second amount of information to obtain cumulative 

information and the cumulative information is displayed on a 

display device associated with the portable electronic device. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The above and other aspects, features, and advantages of 

the present disclosure will become more apparent in light of 

the following detailed description when taken in conjunction with the accompanying drawings in which: 

FIG. 1 is a diagram of a system enabling communication 

among a number of components; 

FIG. 2A is a front view of the portable electronic device 

illustrated in FIG.1, according to various embodiments of the 

present disclosure; 

FIG. 2B is a rear view of the portable electronic device of 

FIG. 2A; 

FIG. 3 is a block diagram of various modules included in 

the portable electronic device illustrated in FIGS. 1, 2A, and 

2B, according to various embodiments of the present disclo 

Sure; 

FIG. 4 is a block diagram of various modules included in 

the server illustrated in FIG. 1, according to various embodi 

ments of the present disclosure; 

FIG. 5 is a block diagram of various modules included in 

the symbology management module illustrated in FIG. 3, 

according to various embodiments of the present disclosure; 

FIG. 6 is a flow diagram illustrating a method of a server for 

retrieving information about an object; 

FIGS. 7A through 7C collectively illustrate a flow diagram 

of a method of a portable electronic device for presenting 

information of an object, according to various embodiments 

of the present disclosure. 

DETAILED DESCRIPTION 

Preferred embodiments of the present disclosure will be 

described hereinbelow with reference to the accompanying drawings. In the following description, well-known functions 

or constructions are not described in detail to avoid obscuring the present disclosure in unnecessary detail. 

Case 2:23-cv-00583-JRG Document 1-3 Filed 12/08/23 Page 11 of 18 PageID #: 61
US 8,936,190 B2 

3 

Systems and methods are provided for enabling portable 

electronic devices to gather information about a selected 

object and present the information on a display device. Using 

any applicable visual detection device (e.g., a camera, Scan 

ner, or other device) on the portable electronic device, the user 

may select an object by scanning or capturing an image of 

symbology (e.g., barcodes) associated with the object. The 

object may be an article of commerce, product, service, or any item associated with various types of symbology. 

Many types of detection systems are available to enable a 

user to select an object. One Such system may be an image 

capture device containing a charge couple device (CCD) or 

camera, where the user may scan or take a picture of symbol 

ogy (e.g., a barcode) placed on or associated with an object. 

Portable devices (e.g., IPhone from Apple, Android from 

HTC, etc.) may include decoding software to be used to 

decode the scanned barcode symbology. In a further embodi 

ment, the object may contain Radio Frequency Identification 

(RFID) tags used to identify the object to a reader device. 

Furthermore, if the portable electronic device contains an 

image capture device (e.g., a camera) and character recogni 

tion Software, the user may take a picture of the object and the 

character recognition Software may be used to decode the 

image and/or name of the object. 

When symbology has been detected according to the 

embodiments of the present disclosure, the portable elec 

tronic devices are configured to send the symbology informa 

tion to the appropriate applications residing on the portable 

electronic device. These applications are instructed to decode 

the symbology, if not already decoded by other applications 

on the portable electronic device. The decode string repre 

senting the decoded symbology may be used to determine the 

object associated with the symbology and then gather infor 

mation about that object. Furthermore, the decode string is 

sent to a server that is configured to decode symbology (if 

necessary), identify one or more objects associated with the 

symbology, retrieve information about the one or more 

objects, and send the information to the portable electronic 

device. The portable electronic device may then combine the 

information from the different sources and display the infor 

mation to the user. 

Some applications that may be downloaded to portable 

electronic devices include symbology scanning and/or 

decoding programs. Examples of applications that allow 

scanning include Neomedia's Neo Reader, Microsoft's Smart 

Tags, Android's ShopSavvy, Red Laser, ScanBuy, etc. How 

ever, when a user wishes to scan an object, the user must then 

select an application on the portable electronic device that is 

capable of accomplishing the desired functions. Since a user 

may have dozens of applications loaded on his or her portable 

electronic device, it may be difficult to select the appropriate application for executing the scanning functions. The present 

disclosure provides for the automatic selection of scanning application upon recognition of applicable symbology. The 

embodiments of the present disclosure also allows for the 

selection of the best application or applications for scanning 

a particular symbology when multiple Scanning applications 

reside on the device. In some embodiments, the symbology managing application may be configured to recognize decod 

able symbology when the user is using the portable electronic 

device for a simple image capture process. In this sense, the 

device may allow the user to select whether or not the sym bology is to be decoded. 

According to various embodiments of the present disclo 

Sure, various portable devices already owned by many cus 

tomers can be enhanced with Software applications to enable 

information retrieval procedures as described in the present 

5 

10 

15 

25 

30 

35 

40 

45 

50 

55 

60 

65 

4 

disclosure. The portable electronic device may use any Suit 

able communications protocols including but not limited to 

WiFi (a, b, c d, n., x), cellular communications such as GSM, 

CDMA, FOMA, etc., infrared communications, cabled com 

munications, Bluetooth communications, satellite communi 

cations, etc. 

FIG. 1 is a diagram illustrating an embodiment of a com 

munication system 10 in accordance with various implemen 

tations of the present disclosure. The communication system 

10 in this embodiment includes a server 12, a storage medium 

14, a portable electronic device 16, one or more satellites 18, 

one or more satellite dishes 20, one or more mobile terminal 

communications towers 22, one or more Wi-Fi towers 24, 

each of which may be configured to transmit and/or receive 

signals to the other elements of the communication system 10 

via a communication network 26. For example, the portable 

electronic device 16 may be configured to communicate with 

the server 12 via the communication network 26 to transfer 

cellular voice signals, data signals, or other types of signals. 

The portable electronic device 16 and server 12 may be 

connected to the communication network 26 (e.g., the Inter 

net) by any suitable means, such as, for example, hardwired 

and/or wireless connections, such as dial-up, hardwired, 

cable, DSL, satellite, cellular, PCS, wireless transmission 

(e.g., 802.11a/b/g), etc. It is to be appreciated that the com 

munication network 26 may be a local area network (LAN), 

wide area network (WAN), the Internet, or any other type of 

network that couples a plurality of computers to enable vari 

ous modes of communication via network messages. Further 

more, the server 12 may communicate using various proto 

cols, such as Transmission Control Protocol/Internet Protocol 

(TCP/IP), File Transfer Protocol (FTP), Hypertext Transfer 

Protocol (HTTP), etc. and secure protocols such as Internet 

Protocol Security Protocol (IPSec), Point-to-Point Tunneling 

Protocol (PPTP), Secure Sockets Layer (SSL) Protocol, etc. 

The server 12 may access the storage medium 14, which is 

configured to store a database containing information about a 

number of objects, such as tables identifying objects by their 

respective decoded symbology, information about the speci 

fications, cost, features, and other details about the objects, 

and other information and data. 

According to some embodiments, the objects of interest 

may refer to products and/or services sold at various stores. 

As will be appreciated from the description below, a unique 

identification code is associated with each product and/or 

service and may take many forms, such as a barcode number, 

a UPC number, an alphanumeric number assigned to the 

product, a name of the article (e.g., “Batman Begins DVD'), 

etc. The identification codes may be attached to or printed on 

the products or in some other way associated with the prod 

ucts. According to other embodiments, the products or items 

may refer to any type of object for which the user wishes to 

receive information, identifiable by a unique identification 

code. 

The user may utilize the portable electronic device 16 to 

obtain a visual representation of an object 28 or portion of the 

object 28. For example, if the object 28 includes a barcode 30, 

the portable electronic device 16 may be configured to scan 

the barcode 30 and decode the barcode to acquire an identi 

fication code of the object. In some embodiments, the por table electronic device 16 may be configured to capture an 

image of the barcode 30 and decode the barcode to acquire the 

associated identification code of the object. Using standard 

and existing computer processing power and Software solu 

tions such as Attrasoft Image recognition software, or optical 

character recognition software (OCR software such as 

Case 2:23-cv-00583-JRG Document 1-3 Filed 12/08/23 Page 12 of 18 PageID #: 62
US 8,936,190 B2 

5 

OmniPage or ReadIris), the image or name of the object may 

be decoded to determine the identity of the object. 

In another embodiment, the user may enter an identifica 

tion code of the object or the name of the object via a user 

interface, if the portable electronic device 16 contains a user 

interface allowing user keyboard input. In some implemen 

tations, the user may speak the name of the product into the 

portable electronic device 16 using a microphone and the 

audio signals may in turn be processed into digital form via 

speech recognition software. 

In a further embodiment, some objects may contain Radio 

Frequency Identification (RFID) tags. The portable digital 

device 16 may contain an RFID reader to resolve the identity 

of the product. In another embodiment, the name or identifi 

cation code of an object may be broadcast via the Wi-Fi tower 

24 over a predetermined range. Furthermore, the name or 

identification code of an object may be broadcast from 

another portable electronic device (e.g., a gaming device, 

mobile phone, etc.). Here, a user may approach the other 

portable electronic device having an application stored in 

memory or currently being used. The other portable elec 

tronic device may be configured to automatically broadcast 

the identification code of the object (i.e., application) or the 

user of the other portable electronic device may initiate a 

transfer of the identification code, e.g., via an infrared port or 

other wireless port, to the first user. 

The portable electronic device 16 may use the resources 

with the device itself to obtain information about the identity 

of the object and other details of the object. Also, once the 

product is identified, the portable electronic device 16 may 

use communications protocols including but not limited to 

Wi-Fi (a,b,c d, n., x), mobile phone communications such as 

GSM, CDMA, FOMA, etc., infrared communications, 

cabled communications, Bluetooth communications, satellite 

communications, to transmit the identification code to the 

server 12 on the global computer network (GCN), e.g., the 

Internet. It is to be appreciated that the portable electronic 

device 16 may use any one or a combination of protocols to 

transmit the identification code to the server 12. For example, the portable electronic device 16 may employ a telecommu 

nication protocol to transmit the identification code via the 

communication module 26. In another embodiment, the por 

table electronic device 16 may communicate to the server 12 

via the same Wi-Fi tower 24 that broadcasts a list of objects 

for which information is available, e.g., a hot spot in a retail 

location. 

The server 12 may be maintained by a vendor associated 

with the object. For example, in the case of seeking informa 

tion about a Warner BrothersTM DVD movie, the server 12 

may be a Warner Brothers server connected to a Warner 

Brothers website. However, the server 12 may also be a ven 

dor not associated or even competing with the object. For 

example, if the product is a Warner BrothersTM DVD movie, 

the server 12 may be an e-commerce site such as 

Amazon.com that offers DVD movies from many companies including Warner Brothers. 

FIG. 2A shows an embodiment of the portable electronic 

device 16 shown in FIG.1. According to various implemen tations, the portable electronic device 16 may be configured 

to transmit and receive wireless communication signals. As 

illustrated in this embodiment, the portable electronic device 

16 includes various electrical components disposed within a 

generally rectangular housing 40. A display device 42 is 

provided for displaying text, images, video, and other visual 

content, such as movies, animation, etc. A speaker 44 is 

provided for producing audio (e.g., voice, music, Soundtracks 

associated with a video, or other types of audio content). An 

10 

15 

25 

30 

35 

40 

45 

50 

55 

60 

65 

6 

audio port 46 may be configured to receive a plug or connec 

tor from a headphone, speaker system, stereo system, etc., to 

stream the audio to an external device. It is to be appreciated 

that when the external device is connected to the audio port 46, the speaker 44 may be disabled. Input means 48 may 

include a plurality of buttons 50 for inputting data and navi 

gating through a plurality of menus. A touch screen overlaid 

upon the display device 42 may also be coupled to the input 

module for facilitating user input. The portable electronic 

device 16 further includes a memory port 52 for storing a 

plurality of content and a transmission system (not shown) for 

transmitting/receiving data and/or content to another device, 

e.g., a personal computer, a personal digital assistant (PDA), 

a server residing on the Internet, etc. Optionally, the portable 

electronic device 16 may include a microphone 54 for acquir 

ing audio from the user of the device to input data. In some 

embodiments, the portable electronic device 16 may include 

a scanning device 56. An image capture device 58 may be 

located on a back side of the portable electronic device 16 as 

shown in FIG. 2B. 

Referring to FIG. 3, various components of the portable 

electronic device 16 of FIG. 2 are described. In FIG. 3, the 

portable electronic device 16 contains a bus 60 that intercon 

nects a display module 62 (related to the display device 42), 

an input module 64 (related to input means 48), a storage 

module 66 (related to memory port 52), a transmission mod 

ule 68, a computer processing module 70, a capture module 

72 (related to image capture device58), a scanning module 74 

(related to scanning device 56), other visual detection mod 

ules 76, a communication module 78, a symbology manage 

ment module 80, and an encryption module 82. 

The computer processing module 70 (e.g., a microproces 

sor) may use computer Software instructions and conven 

tional computer processing power to interact and organize the 

traffic flow between the various other modules. It is to be 

understood that the present disclosure may be implemented in 

various forms of hardware, Software, firmware, special pur 

pose processors, or a combination thereof. The bus 60 couples 

the various components shown in FIG.3 and may be any of 

several types of bus structures (e.g., a memory bus, memory 

controller, a peripheral bus, a local bus, etc.) using any of a 

variety of bus architectures. The portable electronic device 16 

also includes an operating system and micro instruction code 

preferably residing in read only memory (ROM) of the stor 

age module 66. The various processes and functions 

described herein may either be part of the micro instruction 

code or part of an application program (or a combination 

thereof) which is executed via the operating system. Exem 

plary operating systems include but are not limited to Sym 

bianOS, Windows Mobile/Windows CE, Palm OS, Linux, 

Blackberry OS, BREW, etc., which have been developed for 

mobile computing applications and can handle both data 

computing and communication applications, e.g., Voice com 

munications. 

It is to be further understood that because some of the 

constituent device components and method steps depicted in 

the accompanying figures may be implemented in Software, 

the actual connections between the device components (or the 

process steps) may differ depending upon the manner in 

which the present disclosure is programmed. Given the teach 

ings of the present disclosure provided herein, one of ordinary 

skill in the related art will be able to contemplate these and 

similar implementations or configurations of the present dis 

closure. 

The computer processing module 70 may further include, in addition to a microprocessor, a digital signal processor 

(DSP) for decoding stored audio, video and photo files to be 

Case 2:23-cv-00583-JRG Document 1-3 Filed 12/08/23 Page 13 of 18 PageID #: 63
US 8,936,190 B2 

7 

played on the portable electronic device 16. As is known in 

the art, the DSP may include several known decompression algorithms for decompressing Stored media content, e.g., a 

MP3 file. The device 16 of the present disclosure is config ured to Support various file types including but not limited to 

Microsoft Windows Media Video files (.wmv), Microsoft 

Photo Story files (.asf), Microsoft Windows Media Audio 

files (.wma), MP3 audio files (mp3), JPEG image files 

(.jpg, jpeg, jpe, jfif), MPEG movie files (.mpeg. mpg, .mpe, 

m1V, .mp3v .mpeg2), Microsoft Recorded TV Show files 

(.dvr-ms), Microsoft Windows Video files (.avi) and 

Microsoft Windows Audio files (wav). The display module 62 is configured to display digital 

information Such as video files, image files, text files, etc. The 

display module 62 (and any corresponding display devices) 

may be configured in any suitable form, including, for 

example, Liquid Crystal Displays (LCD), Light emitting 

diode displays (LED), Cathode Ray Tube Displays (CRT) or 

any other type of display currently existing or existing in the 

future. The display module 62 may also include an audio 

output device, e.g., speaker 44, audio port 46, etc., allowing 

the user to also hear audio output, e.g., audio associated with 

a video, a MP3 file, etc. 

The input module 64 is configured to either receive user 

instructions via text input by the way of buttons 50, a standard 

keyboard interface coupled to the portable electronic device 

16, or a character recognition capture device which translates 

user text input into alphanumeric characters. Preferably, the 

character recognition device may be a touch screen which 

overlays the display device 42 and text may be entered via a 

pen-like Stylus. Such input devices are standard and currently 

available on many electronic devices including portable digi 

tal assistants (PDAs) and cellular telephones. Optionally, microphone 54 may be further coupled to the input module 64 

for capturing any audio information spoken by the user and 

the input module 64 may further include an analog-to-digital 

(A/D) converter for converting the spoken audio information 

into a digital format. Furthermore, the input module 64 may include a Voice recognition processor that translates the digi 

tal human Voice into alpha numeric characters for user input. 

The user may utilize the input module 64 to enter data, for 

example, to request information about merchandise, initiate a 

purchasing application, etc. 

The storage module 66 includes internal storage memory 

(e.g., random access memory (RAM)) or removable memory (e.g., magnetic storage memory), optical storage memory 

(e.g., various types of CD and DVD media), solid-state stor 

age memory (e.g., a CompactFlash card, a Memory Stick, 

SmartMedia card, MultiMediaCard (MMC), SD (Secure Digital) memory, etc.), or any other memory storage that 

exists currently or will exist in the future. The storage module 

66 may store applications that may be run on the portable 

electronic device 16. For example, Some applications may include one or more image capture applications, one or more 

scanning applications, and other detection (e.g., visual detec 

tion) applications that are configured to detect symbology. As 

described with respect to FIG. 5, these applications may be 

managed by the symbology management module 80. 

The transmission module 68 enables the portable elec 

tronic device 16 to transmit or transfer information to other 

computing devices and to receive information from other 

computing devices, e.g., digital media files, codes to unlock 

downloaded media, encryption/decryption keys, etc. The 

transmission module 68 may perform its functionality by 

hardwired and/or wireless connectivity. The hardwire con 

nection may include but is not limited to hard wire cabling, 

e.g., parallel or serial cables, USB cable, Firewire (1394 

10 

15 

25 

30 

35 

40 

45 

50 

55 

60 

65 

8 

connectivity) cables, etc., and the appropriate ports. Wireless 

connections may operate under any various wireless proto 

cols including but not limited to BluetoothTM interconnectiv 

ity, infrared connectivity, radio transmission connectivity including computer digital signal broadcasting and reception 

commonly referred to as Wi-Fi or 802.11.X (where x denotes 

the type of transmission), satellite transmission or any other 

type of communication protocols or systems currently exist 

ing or to be developed in the future for wirelessly transmitting 

data. The transmission module 68 may compress and encode 

the encrypted information for transmission using any Suitable 

wireless communication technology. In one embodiment, 

antenna ANT is coupled to the transmission module 68 for 

extending the wireless transmission range of the portable 

electronic device 16. 

The capture module 72 may be associated with the image 

capture device 58 to capture an image desired by the user in 

digital form, e.g., an image of a product, a barcode, etc. The 

capture module 72 may include an image sensor, an analog 

to-digital (A/D) converter and a digital signal processor 

(DSP). Referring to FIG.2B, when a user desires to capture an 

image, a lens of the image capture device 56 may be aimed at 

a Subject of the image and is used in conjunction with display 

module 62 for positioning a subject of the image in lieu of a 

viewfinder. Light is allowed to enter through the lens and 

shine on the image sensor, e.g., a charge-coupled device 

(CCD) or complimentary metal-oxide semiconductor 

(CMOS). The capture module 72 may include millions of 

photo-sensors, e.g., pixels, wherein each pixel absorbs the 

light and transforms the light into an electric charge propor 

tional to the intensity of light. Each charge is transmitted to an 

A/D converter where the charge is converted into a digital 

value representing the color the pixel will be, e.g., represent 

ing different intensities of red, green and blue. The digital 

values are then passed to the digital signal processor which 

enhances the image, compresses it and then stores it in a 

digital file format in the storage module 66. 

In other embodiments, the portable electronic device 16 

further includes a scanning module 74 for scanning symbol 

ogy (e.g., a barcode) on an object or product. The scanning 

module 74 in conjunction with the scanning device 56 may 

contain a light source, e.g., LED, and a photocell coupled to 

the computer processing module 70, or alternatively, includes 

a separate decoder engine that decodes the data received by the photocell before sending it to the computer processing 

module 70. Knowledge of the art reveals that many different 

types of scanners currently exist and the inventor realizes that 

the type of scanner would depend upon the type of symbology 

that is utilized for the particular objects. The symbology may 

be in any form currently practiced in the art including bar 

codes (e.g., UPC, EAN, PDF417, etc.), photosymbols, stan 

dard or specialized text, etc., or any future type of symbology. 

It is to be appreciated that the capture module 72 may also 

be used in conjunction with the Scanning module 74 to read 

symbology associated with an object or product. Here, the 

capture module 72 acquires an image of the symbology and 

the Scanning module 74 may further include a digital signal 

processor executing an algorithm for deciphering or decoding 

the symbology from the captured image. The use of an image 

sensor to read symbology, e.g., a barcode, is known in the art 

and systems employing Such technology are commercially 

available from Symbol Technologies of New York. The por 

table electronic device 16 may also include other visual detec 

tion modules 76, Such as additional cameras, additional scan 

ners, video cameras, infrared sensors, etc. In some 

Case 2:23-cv-00583-JRG Document 1-3 Filed 12/08/23 Page 14 of 18 PageID #: 64
US 8,936,190 B2 

embodiments, the other visual detection devices 76 may include sensors that sense light outside the visual spectrum, 

e.g., RF sensors, etc. 

The symbology management module 80 is configured to 

obtain data from any suitable visual detection system incor 

porated in the portable electronic device 16. If the portable electronic device 16 includes an image capture system con 

taining an image capture device and image capture Software 

applications, the symbology management module 80 may be 

configured to recognize symbology within the captured 

image. If the portable electronic device 16 includes a scan 

ning System containing a scanning device and scanning Soft 

ware applications, the symbology management module 80 

may be configured to detect symbology of the Scanned object. 

Other visual detection systems contained in the portable elec 

tronic device 16 may be configured to visually detect sym bology, which may be processed by the symbology manage 

ment module 80. A description of the symbology 

management module 80 with respect to symbology detection 

applications is provided below with respect to FIG. 5. 

The portable electronic device 16 may also include an 

encryption module 82. The encryption module 82 may be 

configured to use conventional code encryption algorithms 

currently in use or that will be in use in the future such as 

symmetric-key algorithms, e.g., DES, Triple-DES, Blowfish, 

RC2, RC4, RC5, etc., and asymmetric-key algorithms, e.g., 

Diffie-Hellman, RSA, ElGamal, etc., to decrypt locked digi 

tal media content files that are stored in the storage module 66. 

The encryption module 82 may also encrypt user information 

before it is transmitted to another device or server. 

In one embodiment, the portable electronic device 16 

according to the principles of the present disclosure may be 

embodied as a mobile phone including the modules and archi 

tecture illustrated in FIGS. 2 and 3. In this embodiment, 

microphone 54 is further coupled to the communication mod 

ule 78 for encoding a user's speech to be transmitted via 

antenna ANT using CDMA, PCS, GSM or any other known 

wireless communication technology. The user may enter 

phone numbers to be dialed via the touch screen, or alterna 

tively, as is known in the mobile phone art, the input means 48 

of the portable electronic device 16 may include a full 

QWERTY keyboard as an input module to enter text infor 

mation. In addition to producing audio from audio or multi 

media content, speaker 44 may be coupled to the antenna 

ANT and a decoder for receiving and decoding Voice com 

munication from another mobile phone. 

It is to be appreciated that the communication module 78 

may include a single integrated circuit chip to perform data 

transfer and Voice communications or a single module includ 

ing a separate data transfer chip, e.g., a WiFi transceiver, and 

a separate Voice communication chip, e.g., a CDMA chip. In 

one embodiment, the communication module 78 operates on 

the wireless GPRS (General Packet Radio Service) data pro tocol, a 3G protocol such as W-CDMA, CDMA2000 and 

TD-SCDMA, a 4G protocol or other ultra-broadband access 

protocol LTE Advanced standards for the 3GPP family, IEEE 

802.16m standard (i.e., WiMAX), etc. The GPRS, 3G, and 4G 

protocols have the ability to carry both voice and data over the same service. 

It is to be appreciated that the portable electronic device 16 

described above is an exemplary device and may include all 

or a portion of the modules described above. Furthermore, it 

is to be appreciated that various types of portable electronic 

devices (e.g., mobile phones, personal digital assistant 

(PDA), digital media playback device, etc.) may be used in 

accordance with the principles of the systems and methods of 

the present disclosure described herein. 

10 

15 

25 

30 

35 

40 

45 

50 

55 

60 

65 

10 

FIG. 4 is a block diagram illustrating an embodiment of 

various modules associated with the server 12 shown in FIG. 

1. In this embodiment, the server 12 includes a computer processing module 84, a storage module 86, a transmission 

module 88, a communication module 90, an encryption mod 

ule 92, a database management module 94, an object identi 

fying module 96, and an information retrieving module 98. 

each interconnected via a bus 100. 

The computer processing module 84 (e.g., a microproces 

sor) may use computer Software instructions and conven 

tional computer processing power to interact and organize the 

traffic flow between the various other modules. It is to be 

understood that the modules of the server 12 may be imple 

mented in various forms of hardware, software, firmware, 

special purpose processors, or a combination thereof. The 

computer processing module 70 may further include, in addi 

tion to a microprocessor, a digital signal processor (DSP), 

which may include several known decompression algorithms 

for decompressing stored media content, e.g., a MP3 file. The 

server 12 of the present disclosure may be configured to 

Support various file types, such as those used by the portable 

electronic device 16 or other devices of the system 10. 

The storage module 86 may include internal storage 

memory (e.g., RAM), non-volatile memory (e.g., ROM), 

removable memory (e.g., magnetic storage memory), optical 

storage memory (e.g., various types of CD and DVD media), Solid-state storage memory (e.g., a CompactFlash, Memory 

Stick, SmartMedia, MMC, SD memory, etc.), or any other 

memory storage that exists currently or will existin the future. 

The transmission module 88 enables the server 12 to trans 

mit or transfer information to other computing devices and to 

receive information from other computing devices, e.g., digi 

tal media files, codes to unlock downloaded media, encryp tion/decryption keys, etc. The transmission module 88 may 

perform its functionality by hardwired and/or wireless con 

nectivity. 

The communication module 90 may include a single inte 

grated circuit chip to perform data transfer and Voice com 

munications or a single module including a separate data 

transfer chip and a separate Voice communication chip. In 

some embodiments, the communication module 90 may 

operate on the wireless GPRS data protocol, a 3G protocol, a 

4G protocol, or other protocol having the ability to carry both 

Voice and data over the same service. 

The encryption module 92 may be configured to use con 

ventional code encryption algorithms currently in use or that 

will be in use in the future, such as symmetric-key algorithms and asymmetric-key algorithms to decrypt locked digital 

media content files that are stored in the storage module 86. 

The encryption module 92 may also encrypt user information 

transmitted from a user device or other server. 

The database management module 94 may be configured 

to manage the contents of the storage medium 14. Particu 

larly, the database management module 94 may retrieve data 

from storage medium 14 or other databases related to infor 

mation about a variety of objects. In response to an informa 

tion request from a user device (e.g., portable electronic 

device 16), the server 12 may utilize the database manage 

ment module 94 to fulfill the information request by retriev 

ing information about the specific object or objects. This 

information may then be transmitted to the user device. 

The object identifying module 96 is configured to receive 

symbology or a decode string from decoded symbology. 

When this is received, the object identifying module 96 is 

configured to identify the object or objects that are associated 

with the symbology or decode string. The object identifying 

module 96 may be configured to utilize the database manage 

Case 2:23-cv-00583-JRG Document 1-3 Filed 12/08/23 Page 15 of 18 PageID #: 65
US 8,936,190 B2 

11 

ment module 94 to lookup information in a database (e.g., 

storage medium 14) that contains the relationships between 

objects (e.g., products) and corresponding symbology or 

character strings that may be obtained by decoding the sym bology. 

After the object of interest is identified by the object iden 

tifying module 96, the information retrieving module 98 is 

configured to retrieve information about the specific object or 

objects. The information retrieving module 98 may also uti 

lize the database management module 94 to retrieve informa 

tion from one or more databases. The information retrieving 

module 98 may access various resources available via the 

communication network 26 to obtain the needed information 

pertaining to the object or objects. 

FIG. 5 is a block diagram of an embodiment of a portion of 

the storage module 66 shown in FIG. 3. For example, several 

applications may be stored in the storage module 66, includ 

ing, among others, the symbology management module 80 

shown in FIG.3, an image capture application 110, a scanning 

application 112, and other visual detection applications 114. 

According to Some embodiments, the other visual detection 

applications 114 may include detection applications that 

detect in a way other than visually. The image capture application 110 may be configured to 

operate with an image capture device (e.g., camera). In some 

embodiments, the image capture application 110 may run in 

the background while other applications are running on the 

portable electronic device 16. While running in the back 

ground, the image capture application 110 may be configured 

to automatically search for recognizable symbology. For 

example, the image capture application 110 may be config 

ured to search for bar codes. The symbology management 

module 80 may control these operations such that when sym bology is detected automatically, the symbology manage 

ment module 80 may prompt the user to determine if he or she 

wishes that the portable electronic device 16 proceeds with 

the decoding of the symbology and retrieving information 

about an object associated with the symbology. According to various implementations, the image capture 

application 110 may be configured to respond to a user's 

prompt or trigger to capture a single image. The symbology 

management module 80 may be configured in the case of this 

typical image capture process to automatically search for and 

decode any symbology that may exist in the captured image. Again, the symbology management module 80 may prompt 

the user as to whether he or she wishes to pursue the decoding of the symbology, if it exists, and retrieving information about 

an object associated with the symbology. The scanning application 112 may operate in a way that is 

similar to that of the image capture application 110 except 

that the scanning application 112 responds to images 

obtained by a scanning procedure instead of an image capture 

procedure. The symbology management module 80 may con 

tinue analyzing scanned data to determine if symbology is 

detected, either during a user-initiated trigger or when 

detected automatically while the scanning application 112 is 

running in the background. The symbology management 

module 80 may also operate in a similar way with respect to 

the other visual detection applications 114 based at least on 

the particular formats, protocols, functions, etc. of the other 

types of detection devices, which may be incorporated in the 

portable electronic device 16 and which may be capable of 

sensing any type of symbology. 

FIG. 6 is a flow diagram showing an embodiment of a 

method that may be performed by a server, such as the server 

12 described with respect to FIGS. 1 and 4. The method 

includes receiving a decode string from a portable electronic 

10 

15 

25 

30 

35 

40 

45 

50 

55 

60 

65 

12 

device (e.g., device 16), as indicated in block 120. In some 

embodiments, the method may include receiving encoded 

symbology information and then decoding this information to 

obtain the decode string. As indicated in block 122, the 

method includes identifying the object or objects associated 

with the decode string. For example, the object may be an 

article or commerce. From the identity of the object(s), the 

method includes retrieving information about the object(s), as 

indicated in block 124. For example, information about vari 

ous objects may be obtained from any database or resource 

available. When the information has been retrieved, the infor 

mation is sent to the portable electronic device (block 126). 

FIGS. 7A, 7B, and 7C collectively form a flow diagram 

showing an embodiment of a method that may be performed 

by a portable electronic device, such as the portable electronic 

device 16 described with respect to FIGS. 1-3 and 5. In this 

embodiment, the method includes allowing a user to select 

preferences as indicated in block 130. Default presets may 

initially be provided, but the user may choose to select other 

preferences at any time as needed. As indicated in block 132, 

one or more visual detection systems are run in the back 

ground of the device, where other programs or systems are 

given priority with respect to processing steps over the one or 

more visual detection systems, unless of course one of the 

visual detection systems is currently being used. The back 

ground systems may include image capture systems (e.g., 

camera systems), Scanning systems, or other detection sys 

tems capable of detecting symbology. 

In decision block 134, the method includes determining 

whethera trigger from a visual detection system is initiated by 

the user. For example, if a detection system is already run 

ning, the user may initiate an image capture, Scan, or other 

input operation (e.g., by pressing a button to take a picture 

with a camera on the portable electronic device). If the system 

does not receive a trigger, the method proceeds to decision 

block 136, which determines whether or not decodable sym 

bology is automatically detected. In some embodiments, the 

automatic detection of decodable symbology may involve 

detection by any visual detection system running in the back 

ground. If no user-initiated trigger is detected in block 134 

and no decodable symbology is detected in block 136, the 

method loops back to decision block 134 until an applicable image is detected. 

If a background application detects decodable symbology 

according to decision block 136, the method proceeds to 

block 138. In block 138, the user is alerted that an automati 

cally detected image contains some type of symbology. As 

indicated in decision block 140, it is determined whether or 

not the user wishes that the symbology is decoded. If not, the 

method skips the remaining steps and comes to an end. If 

symbology is to be decoded, the method goes to decision 

block 142, which indicates that it is determined whether or 

not a visual detection application has been pre-selected. If so, 

the method proceeds to block 144 (FIG. 7B). Otherwise, the 

method goes to block 160 (FIG. 7C). 

As indicated by block 144, shown in FIG. 7B, the detected 

symbology is sent to one or more pre-selected visual detec 

tion applications for decoding and information retrieval. The 

pre-selected visual detection applications may have been 

selected based on the selected preferences made with respect 

to block 130. The visual detection applications and/or related 

decoding Software may be configured to decode the detected 

symbology to obtain a decode string. Also, the applications 

may be configured to obtain information about one or more 

objects associated with the decode string decoded from the 

Case 2:23-cv-00583-JRG Document 1-3 Filed 12/08/23 Page 16 of 18 PageID #: 66
US 8,936,190 B2 

13 

detected symbology. As indicated in block 146, the method 

includes receiving and storing information from the visual 

detection application(s). 

According to block 148, the decode string is also sent to a 

remote server (e.g., server 12). Later, the method receives 

information from the remote server as indicated in block 150. 

The information from the server may be related to informa 

tion about one or more objects identifiable by the decode 

string. As indicated in block 152, the information from the 

visual detection application(s) is combined with information 

from the remote server according to various implementations. 

At this point, the method includes displaying the information 

on a display screen of the portable electronic device. 

While or after the information is displayed, the method also 

includes enabling the user to store the information if desired, 

as defined in block 156. In addition, the user is given an option 

to run a purchasing application to enable the purchase of the 

object or objects if desired. This may also be performed while 

or after the information is displayed. Therefore, one advan 

tage of displaying information for the user is to provide appropriate data to inform the user of objects (e.g., products) 

that the user may consider purchasing. With the information, 

the user may be able to make a more informed purchasing 

decision. After the information is displayed and stored (if 

desired) and one or more objects are purchased (if desired), 

the method comes to an end. 

When visual detection applications have not been pre 

selected by the user, as determined in decision block 142, the 

method jumps ahead to block 160 as shown in FIG. 7C. In 

block 160, the symbology is decoded to determine a category 

of the object. As indicated in block 162, the method includes 

determining the appropriate visual detection application(s) 

based on the objects category. According to decision block 

164, it is determined whether or not multiple applications 

were deemed to be appropriate. If not, the method goes to 

block 166, which indicates that the decode string is sent to the 

one appropriate application for retrieving the information. If 

it is determined in decision block 164 that multiple applica 

tions are appropriate, the method goes to block 168. AS indi 

cated in block 168, the user is enabled to select one or more of 

the applications. In block 170, the decode string is sent to the 

selected application(s) for information retrieval. After the 

decode strings are sent off for information retrieval according 

to blocks 166 and 170, the method returns to block 146, which 

involves receiving and storing the information from the visual 

detection application(s) and the following processes as dis 

cussed above. 

While the disclosure has been shown and described with 

reference to certain preferred embodiments thereof, it will be 

understood by those skilled in the art that various changes in 

form and detail may be made therein without departing from 

the spirit and scope of the disclosure as defined by the 

appended claims. 

What is claimed is: 

1. A method comprising: capturing a digital image using a digital image capturing 

device that is part of an electronic device; 

detecting symbology associated with the digital image 

using the electronic device; 

decoding the symbology to obtain a decode string using 

one or more visual detection applications residing on the 

electronic device; 

sending the decodestring to a remote server for processing: 

receiving information about the digital image from the 

remote server wherein the information is based on the 

decode String; 

10 

15 

25 

30 

35 

40 

45 

50 

55 

60 

65 

14 

displaying the information on a display device associated 

with the electronic device. 

2. The method of claim 1, further comprising enabling a 

user of the electronic device to select one or more presets 

before the symbology is detected. 

3. The method of claim 2, wherein the presets include at 

least one of a selection of one or more visual detection appli cations to handle Scanning operations, a selection of one or 

more visual detection applications to handle image capture 

operations, a maximum number of applications configured to 

receive a decode string, an amount of information to be dis 

played, the type of information to be displayed, and e-com 

merce options. 

4. The method of claim 1, wherein one or more visual 

detection systems are configured to run in the background 

with respect to other systems associated with the electronic 

device, the visual detection systems comprising the one or 

more visual detection applications and one or more visual 

detection devices, the one or more visual detection devices 

configured to detect the symbology associated with the 

object. 

5. The method of claim 1, wherein the one or more visual 

detection systems are configured to automatically detect the 

symbology. 

6. The method of claim 5, further comprising: alerting the user when an image containing symbology has 

been detected; 

asking the user if decoding of the symbology is desired; 

and 

receiving a reply from the user. 

7. The method of claim 1, wherein detecting the symbology comprises detecting the symbology in response to a user 

initiated trigger of a visual detection device. 

8. The method of claim 1, wherein sending the decode 

string to one or more visual detection applications for pro 

cessing and sending the decode String to a remote server for 

processing further comprises: 

instructing the one or more visual detection applications 

and remote server to identify the object associated with 

the decode string; and 

instructing the one or more visual detection applications 

and remote server to retrieve information about the 

object. 

9. The method of claim 1, wherein the method further 

comprises: 

analyzing the decode string to determine a category of the 

object; and 

selecting one or more appropriate applications to process 

the decode string when the one or more visual detection 

applications are not pre-selected, the selection based in 

part on the category of the object. 

10. The method of claim 9, wherein the method further 

comprises: 

determining if multiple applications are deemed to be 

appropriate to process the decode string; and 

enabling the user to select one or more applications when 

multiple applications are deemed to be appropriate. 

11. The method of claim 1, further comprising enabling the user to store the information. 

12. The method of claim 1, further comprising providing 

e-commerce options to allow the user to proceed to a purchas ing process for purchasing the object, wherein the object is an 

article of commerce. 

13. The method of claim 1, further comprising displaying 

the information about the object with the digital image. 

Case 2:23-cv-00583-JRG Document 1-3 Filed 12/08/23 Page 17 of 18 PageID #: 67
US 8,936,190 B2 

15 

14. The method of claim 1, further comprising overlaying the information about the object while displaying the digital image. 

15. The method of claim 1, wherein visual search technol 

ogy is used to decode the symbology. 

16. The method of claim 1, wherein both the digital image and the information associated with the digital image are 

displayed on the electronic device. 

17. A computer application stored on a computer-readable medium and executable by a processing device incorporated in an electronic device, the computer application comprising: logic adapted to capture a digital image using a digital 

image capturing device that is part of the electronic 

device; 

logic adapted to detect symbology associated with an 

object using one or more visual detection applications; logic adapted to decode the symbology to obtain a decode 

String using the one or more visual detection applica 

tions; 

logic adapted to send the decode string to a remote server 

for processing: 

logic adapted to receive a second amount of information 

about the object based on the decode string from the 

remote server; 

logic adapted to display the information on a display device 

associated with the electronic device. 

18. The computer application of claim 17, further compris 

1ng: 

logic adapted to enable a user of the electronic device to 

Select one or more presets before the symbology is 

detected, the presets including at least one of a selection 

of one or more visual detection applications to handle 

10 

15 

25 

30 

16 

Scanning operations, a selection of one or more visual 

detection applications to handle image capture opera 

tions, a maximum number of applications configured to 

receive a decode string, an amount of information to be 

displayed, the type of information to be displayed, and 

e-commerce options. 

19. The computer application of claim 17, wherein the one 

or more visual detection systems are configured to automati 

cally detect the symbology, the computer application further 

comprising: 

logic adapted to alert the user when an image containing 

symbology has been detected; 

logic adapted to ask the user if decoding of the symbology 

is desired; and 

logic adapted to receive a reply from the user. 

20. A symbology management application comprising: receiving a digital image from a digital image capturing 

device; 

a symbology detecting module configured to detect sym 

bology associated with an object; 

a decoding module configured to decode the symbology to 

obtain a decode string: 

a transferring module configured to send the decode string 

to one or more visual detection applications for process ing, the one or more visual detection applications resid 

ing on an electronic device; and 

an information storage module configured to receive infor 

mation about the object from the remote server, and 

provide the information to a display device for display of 

the information. 

Case 2:23-cv-00583-JRG Document 1-3 Filed 12/08/23 Page 18 of 18 PageID #: 68