Source: https://docs.justia.com/cases/federal/district-courts/california/cacdce/2:2018cv01844/703149/1/3.html
Timestamp: 2018-10-18 03:41:42
Document Index: 276847112

Matched Legal Cases: ['application No. 1', 'application No. 07107866', 'application No. 07107866', 'application No. 200810144678', 'application No. 2008101446781', 'application No. 2', 'application No. 2', 'application No. 200810144678', 'application No. 200810144678', 'application No. 07107866', 'application No. 08170532', 'application No. 08170532', 'application No. 07107866', 'application No. 2008', 'art 602']

Justia Dockets & Filings Ninth Circuit California California Central District Court BlackBerry Limited v. Facebook, Inc. et al Filing 1 Attachment 3
EXHIBIT C EXHIBIT C US008279173B2 (12) United States Patent (10) Patent No.: BrOWn et al. (54) USER INTERFACE FOR SELECTINGA PHOTO TAG (73) Assignee: Research In Motion Limited, Waterloo, Ontario 7,693,906 B1 7,735,018 B2 2004/0039988 A1 2004/02521 19 A1 2005/OO30588 A1 2005, OO39108 A1 2005/0057576 A1 CN FOREIGN PATENT DOCUMENTS 1193632 A 1, 2002 (Continued) Second Office Action mailed Jun. 22, 2011. In Chinese patent appli tion No. 200810144678.1. CaO NO Prior Publication Data Continued (Continued) Jan. 26, 2012 Related U.S. Application Data (63) Continuation of application No. 1 1/746,285, filed on May 9, 2007, now Pat. No. 8,031,170. (51) Ea's/00 2/2005 Reese et al. 2/2005 Hudson 3/2005 Shen et al. OTHER PUBLICATIONS Oct. 4, 2011 US 2012/0023436A1 2/2004 Lee et al. 12/2004 Hunleth et al. (Continued) (21) Appl. No.: 13/252,807 (65) 4/2010 Amidon et al. 6, 2010 Bakhash 7,840,907 B2 11/2010 Kikuchi et al. 8,031,170 B2 * 10/2011 Brown et al. ................. 345,156 2003/0088582 A1 5/2003 Pflug Subject to any disclaimer, the term of this patent is extended or adjusted under 35 U.S.C. 154(b) by 0 days. This patent is Subject to a terminal disclaimer. (22) Filed: *Oct. 2, 2012 6,650,889 B1 * 1 1/2003 Evans et al. ................ 455,412.1 7,636,450 B1 12/2009 Bourdev (75) Inventors: Michael S. Brown, Kilchanar (CA); Gerhard D. Klassen, Waterloo (CA); Terrill Dent, Waterloo (CA) (*) Notice: US 8.279,173 B2 (45) Date of Patent: (2006.01) Primary Examiner — Nitin Patel (74) Attorney, Agent, or Firm — Novak Druce + Quigg LLP (57) ABSTRACT There is disclosed a user interface for selecting a photo tag. In . E. c - - - -ificati- - - - - -s so 34554 E. (58) Field of Classification Search .................. s S lication file f i.838, 5/412.1 an embodiment, the user interface embodies a method of selecting a photo tag for a tagged photo, comprising: provid ing a tag entry field for entering a photo tag; in dependence upon a string entered by a user, displaying in a matching tag list any tags from one or more selected tag sources matching (56) the entered String. The method may further comprise display ing a tag type for each tag appearing in the matching tag list. The method may further comprise allowing user selection of ee appl1cauon Ille Ior complete searcn n1Story. References Cited 5,479,602 s EN points aecker et al. 6,002,401 A 6,317,142 B1 a tag in the matching tag list to complete thetag entry field. 12, 1999 Baker 1 1/2001 Decoste et al. 20 Claims, 10 Drawing Sheets 410 404 408 404 ext i typed before 406 412 EXHIBIT C Page 149 Thee Boukeluth Tim Jackson US 8,279,173 B2 Page 2 U.S. PATENT DOCUMENTS 2005. O19301.0 2006/0173918 2006/02621 16 2008/002 1876 2008/0106594 2008, 0215583 2008/0282177 2008/0306921 A1 A1 A1 A1 A1 A1 A1 A1 2008/0309617 A1 2009/0225.178 A1 9, 2005 8, 2006 11/2006 1/2008 5, 2008 9/2008 11, 2008 12/2008 DeShan et al. Nakase et al. Moshiri et al. Ahern et al. ...................... 707/3 Thrun Gunawardena et al. Brown et al. Rothmuller et al. 12/2008 Kong et al. 9, 2009 Nakase et al. 1. FOREIGN PATENT DOCUMENTS CN WO 1933643 A 2006102656 A the internet (on Aug. 17, 2007): URL:http://ieeexplore.ieee.org/ie15/ 7756/33539/01593574.pdf?tp=&isnumber=arnumber=1593574. Ahern et al., Zone tag: Designing context-aware mobile media cap ture to increase participation, yahoo! Research Berkeley, Sep. 6. 2006. http://groups.ischool.berkeley.edu/pics/papers Ahern et al Zonetag pics06.pdf. Communication Pursuant toArticle 94(3) EPC mailed Jan. 30, 2008. In European patent application No. 07107866.1. Summons to Attend Oral Proceedings Pursuant to Rule 115(1) EPC mailed Aug. 27, 2008. In European patent application No. 07107866. 3, 2007 9, 2006 OTHER PUBLICATIONS English translation on Second Office Action mailed Jun. 11, 2011. In Chinese patent application No. 200810144678.1. Third Office Action mailed Feb. 29, 2012. In Chinese patent appli cation No. 200810144678.1. English translation on Third Office Action mailed Feb. 29, 2012. In Chinese patent application No. 2008101446781. Office Action mailed Jan. 10, 2012. In Canadian patent application No. 2,630,947. Office Action mailed Nov. 18, 2010. In Canadian patent application No. 2,630,947. First Office Action mailed May 5, 2010. In corresponding Chinese patent application No. 200810144678.1. English translation on First Office Action mailed May 5, 2010. In corresponding Chinese patent application No. 200810144678.1. Ballagas, R. et al.; The Smart phone: A ubiquitous input device, pervasive computing, IEEE (online) vol. 5, No. 1, Jan. 2006,pp. 70-77, XP002447 195, ISSN: 1536-1268: Retrieved (by EPO) from Decision of the Examining Division mailed Dec. 12, 2008. In Euro pean patent application No. 07107866.1. Extended European Search Reportmailed Febuary 23, 2009. In Euro pean patent application No. 08170532.9. Communication Pursuant toArticle 94(3) EPC mailed Jan. 11, 2010. In European patent application No. 08170532.9. Wiseman, Josh; "iPhoto, meet Facebook'; Mar. 15, 2007; http://blog. facebook.com/blog.php?post–22536571. Extended European Search Report mailed Aug. 28, 2007. In Euro pean patent application No. 07107866.1. First Office Action mailed May 2010. In corresponding Chinese patent application No. 2008.10144678.1. Ballagas Retal. The Smartphone: A ubiquitous input device, perva sive computing, IEEE (online) vol. 5, No. 1, Jan. 2006,pp. 70-77. XPO02447195, ISSN: 1536-1268: Retrieved (by EPO) from the internet (on Aug. 17, 2007): URL:http://ieeexplore.ieee.org/ie15/ 7756/33539/01593574.pdf?tp=&isnumber=arnumber=1593574. Ahern etal. Zone tag: Designing context-aware mobile media capture to increase participation, yahoo! Research Berkeley, Sep. 6, 2006, http://groups. iSchool.berkeley.edu/pics/papers Ahern et al Zonetag pics06.pdf. * cited by examiner EXHIBIT C Page 150 U.S. Patent Oct. 2, 2012 126 Sheet 1 of 10 128 132 SIMIRUIM US 8,279,173 B2 W+ 130 Battery 110 Display 134 Auxiliary I/O 108 136 112 Data Port Flash Memory 106 Application 114 Keyboard Image 116 104 Applications Trackball Communication 117 Subsystem Main PrOCeSSOr 148 146 102 GPS Subsystem 126 Short-Range Communications 122 FIG. 1 EXHIBIT C Page 151 N 100 U.S. Patent Oct. 2, 2012 Sheet 2 of 10 127 N. FIG.2 EXHIBIT C Page 152 US 8,279,173 B2 U.S. Patent Oct. 2, 2012 Sheet 3 of 10 302 301 US 8,279,173 B2 304 & SSS Š Š&isi 302 308 arrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrra sinsists s ss: SSS SSS FIG. 3B N-3s EXHIBIT C Page 153 : 306 U.S. Patent Oct. 2, 2012 Sheet 4 of 10 US 8,279,173 B2 302 S :S SS Sas SSSSSSSSS s ŠxŠišš: Š& S. 310 312 302 s Y S & Ess: S S&SSSSS 3. s , s&SS: As Š Šiš SSSSSSSSSSS& 310 SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS EXHIBIT C Page 154 U.S. Patent Oct. 2, 2012 Sheet 5 of 10 US 8,279,173 B2 316 & SSSSSSSSSSSSSSS sT. T. T. T.....................Nix ises: is is is is: s &s Š SSS 314 EXHIBIT C Page 155 U.S. Patent Oct. 2, 2012 SSSSS SS & Siss Sheet 6 of 10 :S US 8,279,173 B2 SSSSSSSSSSSSS 404 acebook :S SSSSSSS 8. SSSSSSSSSSSSSSSSSS :::::: ::::::::: i: :::::: Š & s s Ssssssssssssssssssssssssssssssssssssssssssss & : & 406 S.& N 400A FIG. 4A 410 S$ y SSSSSSSSSSSSSS Š :S & s: O aramie Terrill Deft text i typed before 412 404 406 S. Tee EOluket in. FIG. 4B EXHIBIT C Page 156 ge U.S. Patent Oct. 2, 2012 Sheet 7 of 10 US 8,279,173 B2 OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOY YO & & S. S SSSSSSSSSS 410 O SSSSSSSSSSSS an Inters. S Šss SSS 404 SSSSSSSSSSSSS k's text i typed before 412a & 406 '''''''''''' &s 412b ' Sac FIG. 4C X. 410 SSS SSSSSSSSS S. irrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr s 412a is a ‘S’ -ss Š SSSSSSSSSSSSSSSSSSSS 406 &S 412b 412C ser 404 N $: ...: S. Š FIG. 4D N-op EXHIBIT C Page 157 EXHIBIT C Page 158 U.S. Patent Oct. 2, 2012 Sheet 9 of 10 US 8,279,173 B2 rt Display a photo in a display 504 Provide a photo tagging mode for editing tags for the displayed photo For each tag used to tag a Subjector object in the photo, identify the tag type and aSSOciate a unique pointer for each tag type to highlight the Corresponding tagged Subjector object 506 508 Display a tag list with the photo 510 Identify the tag type of each tag in the tag list using a visual identifier Upon user Selection of a tag in the tag list, highlight theaSSOciated tagged Subjector object in the photo using the 512 unique pointer Upon user Selection of a tagged Subjector object in the photo, highlight the aSSOciated tag in the tag list 500 514 516 Displaying Context dataaSSOciated with the tag In the photo tagging mode, Constrain the navigation device pointer to be navigable only within the boundaries of the photo End EXHIBIT C Page 159 F G. 5 518 U.S. Patent Oct. 2, 2012 Sheet 10 of 10 US 8,279,173 B2 Start 602 Providing a tag entry field for entering a photo tag Independence upon a string entered by the user, displaying in a matching tag list any tags from One or more selected tag sources matching the entered string 604 606 Displaying a tag type for each tag appearing in the matching tag list 608 Orderingtags in the in the tag list by tag type Allowing user Selection of a tag in the matching tag list to complete the tag entry field If there are no tags remaining in the matching tag list, adding a new free fom text string to a free-form text 610 612 Cache End 600 FIG. 6 EXHIBIT C Page 160 US 8,279,173 B2 1. USER INTERFACE FOR SELECTINGA PHOTO TAG CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. application Ser. No. 1 1/746,285 entitled “USER INTERFACE FOR SELECTING A PHOTO TAG” and filed on May 9, 2007 now U.S. Pat. No. 8,031,170. U.S. application Ser. No. 1 1/746.285 is fully incorporated by reference herein. 10 FIELD OF THE INVENTION network 200, and device-resident functions such as a calcu The present invention relates generally to a user interface for selecting a photo tag. 15 BACKGROUND Identifying people or objects in photographs is popular in many online contexts, such as photo sharing, Social network ing, etc. Selecting a “tag” to associate with an identified point in a photograph can be a complicated task if there are many potential tags to choose from. In addition, wireless mobile 25 communication device where there are constraints on the size of the display and the flexibility of the input method, some of these common techniques used on desktops and laptops with full sized screens do not work as well. What is needed is an improved user interface for selecting tags in a smaller display, such as may be found ona wireless 30 mobile communication device. BRIEF DESCRIPTION OF THE DRAWINGS 35 FIG. 1 is an illustration of a device in accordance with an embodiment; FIG. 2 is a schematic block diagram ofa communication subsystem component in the device of FIG. 1; 40 FIGS. 3A to 3E are illustrative user interface screens for editing photo tags in accordance with an embodiment; FIGS. 4A to 4F are illustrative user interface screens for selecting a photo tag in accordance with an embodiment; FIG. 5 is an illustrative flowchart of a method for editing photo tags in accordance with an embodiment; and FIG. 6 is an illustrative flowchart of a method for selecting a photo tag in accordance with an embodiment. DETAILED DESCRIPTION 45 50 As noted above, the present invention relates to a user interface for selecting a photo tag, particularly within a Smaller display, Such as may be found on a wireless mobile communication device. 55 In an illustrative embodiment, the invention may be prac Once captured by the CCD or CMOS sensor array, the image may then be processed by camera logic 128 into a Suitable digital image file format such as Joint Photographic Experts Group (JPEG), Tagged-Image File Format (TIFF), Bit Mapping (BMP), different variations on these standard image file formats, ora vendor proprietary RAW image for mat. The image file format may allow for the addition of image metadata to an image file in an industry standards exchangeable image file format (EXIF), or in some vendor proprietary meta-data format. The image file may then be stored in available device storage such as RAM 106 or flash memory 108, and displayed on display 110. Still referring to FIG. 1, operating system software used by the main processor 102 is typically stored in a persistent store such as flash memory 108. Those skilled in the art will appre ciate that the operating system, specific device applications, or parts thereof, may be temporarily loaded into a volatile store, such as the RAM 106, for processing by main processor 102. The communication device 100 may send and receive com munication signals over the wireless network 200 after required network registration or activation procedures have been completed. Network access may be associated with a ticed with a handheld mobile communication device in a wireless operating environment. Shown in FIG. 1 is a sche matic block diagram of an illustrative handheld mobile com munication device 100. The communication device 100 may comprise a number of components, including a main proces sor 102 which controls the overall operation of communica tion device 100. Communication functions, including data and Voice communications, may be performed through a communication Subsystem 104. The communication Sub system 104 may receive messagesfrom and send messages to lator or task list. The trackball 117 may be used for various navigation functions, such as navigating through a graphical user interface (GUI) menu displayed on display 110. The trackball 117 may also be configured with a secondary actua tion feature, Such as allowing a user to depress the trackball, to allow selection of a highlighted item. The camera module 126 may be adapted to capture an image through a lens onto a light sensitive image sensor Such as a charge coupled device (CCD) sensor array or a comple mentary metal oxide semiconductor (CMOS) sensor array. The camera lens may bea fixed focus lens, or a variable focus lens with or without Zoom features and controlled by camera logic 128 to focus an image onto the CCD or CMOS sensor array. The size and pixel density of the CCD or CMOS sensor array may be Suitably selected for the image resolution required for a particular application. Camera logic 128 may also control the camera lens aperture and/or shutter speed by incorporating a suitable light exposure meter. Image capture using camera module 126 may initiated by a user controlling a dedicated camera shutter, or a context dependent program mable button or key (on keyboard 116, for example) that may act as a camera shutter button. In the figures which illustrate exemplary embodiments: a wireless network 200. 2 The main processor 102 may also interact with additional Subsystems such as a random access memory (RAM) 106, a flash memory 108, a display 110, an auxiliary input/output (I/O) subsystem 112, a data port 114, a keyboard 116, a trackball 117, a speaker 118, a microphone 120, short-range communications 122, a GPS Subsystem 124, a camera mod ule 126, and associated camera logic 128. Some of the subsystems of the communication device 100 may perform communication-related functions, whereas other subsystems may provide “resident’ or on-device func tions. By way of example, the display 110 and the keyboard 116 may be used for both communication-related functions, Such as entering a text message for transmission over the 60 subscriber or user of the communication device 100. The communication device 100 may be a battery-powered device and may include a battery interface 132 for receiving one or more rechargeable batteries 130. In some embodi ments, the battery 130 may be a smart battery with an embed 65 ded microprocessor. The battery interface 132 is coupled to a regulator (not shown), which assists the battery 130 in pro viding power V+ to the communication device 100. The bat EXHIBIT C Page 161 US 8,279,173 B2 3 tery 130 may beused to power all components and modules in the communication device 100, including the camera module 126 and associated camera logic 128. The main processor 102, in addition to its operating system functions, enablesexecution of various Software applications 134 on the communication device 100. A subset of software applications 134 that control basic device operations, includ ing data and Voice communication applications, will nor mally be installed on the communication device 100 during its manufacture. 10 The Software applications 134 may include a messaging application 136. The messaging application 136 can be any suitable software program that allows a subscriber or user of the communication device 100 to send and receive wireless text communications. Various alternatives exist for the mes 15 saging application 136 as is well known to those skilled in the art. Messages that have been sent or received by the user are typically stored in local storage such as flash memory 108 of the communication device 100, or in some other suitable storage element in the communication device 100. In an alter native embodiment, some of the sent and received messages may be stored remotely from the communication device 100 Such as in a data store ofan associated host system that the communication device 100 communicates with. In an embodiment, the messaging application 136 may include a Message List user interface that is configured to allow a user to see a list of message objects (i.e. email messages) in a 25 convenient list form. This will be described in detail further below. Still referring to FIG. 1, communication device 100 may execute an Image Applications Module 148 that may be operatively integrated with camera module 126, camera logic 128, main processor 102, RAM 106, display 110 and various other modules and components to provide various image application functions for the images captured by the camera module 126. Image Applications Module 148 may include various Sub moduleswhich may interact with each other, and with other application modules such as the messaging appli 30 35 cation 136, Internet browser module 138,address book mod ule 142, etc. in order to perform various functions. In an embodiment, one of the Sub modules of the Image Applications Module 148 may be a photo tagging module 148A configured to allow a user to identify various subjects and objects within a photo. Photo tagging module 148A will 40 be discussed in more detail further below. 45 In an embodiment, another of the submodules of the Image Applications Module 148 may be a photo tagging module 148A configured to allow a user to select a photo tag from a list of tags associated with various Subjects and objects within a photo. Photo tag selection module 148B will be discussed in 50 more detail further below. The communication device 100 may further include a device state module 140, an address book 142, a Personal 4 image applications 148. Display 110 may also be configured to provide a touch-sensitive screen input in response to a prompt or query displayed on display 110. The communica tion device 100 may further include a camera lens that may be used to capture an image as described above with reference to FIG. 1. In an embodiment, the integrated camera 126 may provide a camera lens 127 on the back of the communication device 100, such that a user may use the display 110 as a camera viewfinder for framing an image. Now referring to FIG. 3A, shown is an illustrative user interface screen 300A in which photo tagging module 148A may be configured for tagging a photograph in accordance with an embodiment. As shown, a photo 301 of a subject 302 is displayed within the boundaries of the user interface. With this user interface, a tag list 304 may include various tags associated subject 302 or other subjects or objects within the photo 301. The user may click an “Add” button 306 in order to enter a photo tagging mode as described below. Now referring to FIG. 3B, as shown in screen 300B, once in the photo tagging mode, the photo tagging module 148A may be configured to display to the user a pointer, such as cross-hair pointer 308 on the photo 301. The user may move the cross-hair pointer 308 around the photo 301, but unlike common web browser interfaces, the crosshair pointer 308 may be constrained by the photo tagging module 148A to be navigable only within the boundaries of the photo 301. The user does not have the ability to move the cross-hair pointer around the rest of the screen and perform other tasks, and the navigation device (e.g. trackball 117) is thus dedicated tothis photo tagging function until the user exits tagging mode. As will be appreciated, this may allow users to avoid inadvert ently clicking on some button or menu option just outside of the borders photo 301 when attempting to tag a subject or object near an edge of the photo. This may be particularly useful on a smaller display, where greater dexterity may be required in order to navigate within a small area using a small navigation device. When in tagging mode, the user may cancel tagging mode using the escape key and return to simply viewing the existing tag information. Alternatively, the user may choose to add a tag at the current location of the crosshairpointer 308 using an enter key or clicking on the navigation device (e.g. trackball 117). If the user chooses to add a tag, the user may be presented with another user interface to select the tag type and the context information associated with the tag. As an illustrative example, the tag types could include a free-form alphanu meric string, FacebookTM friends, address book entries (in address book 142), browser bookmarks (in Internet browser module 138), etc. Now referring to FIG. 3C, as shown by way of illustration in screen 300C, when a tag has been added by a user using photo tagging module 148A, the added tag 310 associated with subject 302 may appear in the tag list 304 near the photo Information Manager (PIM) 144, and various other modules 150. Additional software applications may also be loaded 55 301. onto the communication device 100 through at least one of the In an embodiment, thetag 310 could also include an icon, wireless network 200, the auxiliary I/O subsystem 112, the or some other type of visual indicator adjacent the tag 310, data port 114, the short-range communications Subsystem indicating what type oftag it is. Thus, many different types of tags may be used to tag subjects or objects in the photo 301. 122, or otherdevice subsystem 124. Now referring to FIG. 2, shown is an illustrative front view 60 Now referring to FIG. 3D, as shown by illustration in of a handheld mobile communication device 100 that may screen 300D, when the user scrolls over the tag 310 in the tag provide a Suitable operating environment. As shown, the com list 304, the corresponding subject 302 in the photo may be munication device 100 may include a display 110, a keyboard highlighted by the photo tagging module 148A in Some way. 116, and other input or navigation means such as a trackball This highlighting could be different, depending on what type 117. The display 110 may be configured to display various 65 of tag it is. For example, for a FacebookTM friend the pointer screens allowing the user of device 100 to view screen outputs could be a grey square pointer 312 (e.g. like Facebook uses from the various software applications 134, including the online). As another example shown in screen 300E of FIG. EXHIBIT C Page 162 US 8,279,173 B2 5 6 entry field 406. The matchingtag list 412 then disappears, and 3E, for a free-form text tag 314, the tagged point in the photo the selected tag may appear beside the photo (e.g. tag 310 as 301 could be indicated with a circle pointer 316. When the user scrolls over a tag 310 in the tag list 304, a shown in FIG. 3C associated with subject 302 in the photo menu may be activated with options associated with the tag. 301). For example, there may be menu items to edit or delete the 5 Significantly, as the matching tag list 412 includespossible tag. There may also be context sensitive menu options asso tags that may be used from various selected tag sources (such ciated with the specific tag type. For example, for a Facebook as the user's Facebook friends, the user's address book 142, a friend there may be an item to view the friend's Facebook list of the user's browser bookmarks from Internet browser profile. For an address book entry, there may be an item to 138, a cache of the recent view the user's address card. For a browser bookmark, there 10 is provided with a simple free-form text entries, etc.), the user way to associate Subjects or objects may be an item to visit that website. in a photo with a predefined “tag” from one of a number of In another embodiment, once Subjects or objects have been tagged in the photo 301, photo tagging module 148A may be selected tag sources, as may be defined by the user. Thus, the free-form text cache would just be one of many possible tag configured such that upon user selection of a tag310 in thetag list 304 the corresponding tagged subject or object in the 15 Sources, all of which contribute matching tag entries to the photo 301 may be highlighted using the corresponding matching tag list 412. Now referring to FIG. 4F, once a tag has been entered into pointer identifying the tag type. In another embodiment, once subjects (e.g. subject 302) or the tag entry field 406, photo tag selection module 148B may objects have been tagged in the photo 301, photo tagging be configured to allow the user to choose to view some con module 148A may be configured such that upon user selec text data associated with the tag (e.g. an address card if the tag tion ofor scrolling over a tagged Subject or object in the photo identifies a contact in the user's address book 142). Finally, 301, the corresponding tag 310 may be highlighted in the tag photo tag selection module 148B may be configured to allow list 304. As well, context data associated with the tag may be the user to accept the new tag and return to the photo tagging displayed (e.g. in a pop-up window) upon user selection, or user interface (described above with respect to FIGS. 3A to 25 3E), or cancel the tag selection and return to the photo tagging automatically if so configured. Thus, using the above described user interface, a plurality user interface. oftag types may be used to tag subjects and objects in a photo, In an embodiment, in addition to the tag sources mentioned and a type-specific data may be associated with each tag— above, another type of tag source may be landmark tags with Such as the visual indicator or symbol used to highlight the associated geographic location information. For example, ifa tagged subject or object in the photo, the custom actions 30 photo contains a number of distinctive landmarks, it may be possible for each landmark to be tagged with a unique geo available for the tag, etc. Now referring to FIG. 4A, shown in screen 400A is an graphic location tag (e.g. specific latitude and longitude coor illustrative tag selection user interface 404 for displaying a dinates for each landmark). Such a list of geographic location tagsearch facility as may be presented by photo tag selection tags may be obtained, for example, as a user visits each of the module 148B. As shown in FIG. 4A, the user is initially 35 landmarks identified in the photo. presented with a tag entry field 406 indicating that he should For example, an aerial photo of the National Mall in Wash start typing a tag. Upon completion of typing, the user may ington D.C. may show a number of famous landmarks such as the Lincoln Memorial, Vietnam Veterans Memorial, and The click “OK” 408 to select the tag. In an embodiment, as the user begins to type, photo tag Washington Monument in the same photo. A user who has selection module 148B may be configured to search one or 40 visited each of these landmarks, provided appropriate names, more selected “tag sources' for tags that match the currently and recorded geographic location information at each land entered text. As shown by way of illustration in screen 400B mark location may then Subsequently select a landmark tag of FIG. 4B, thesetag sources could include, for example,a list by name from the prepared landmark tag list in order to tagthe of friends from an online service like FacebookTM, a list of appropriate points in the photo at which each of the landmarks contacts from the user's address book 142, a list of the user's 45 appear. Once a user tags each landmark appearing in the browser bookmarks (in Internet browser 138), a cache of photo using the appropriate landmark tag, the corresponding recent free-form text entries, etc. geographic coordinates also become available as context based information accessible through the tag. As shown in screen 400C of FIG. 4C,and 400D of FIG.4D, photo tag selection module 148B may be configured to dis In an alternative embodiment, a list of famous landmarks play any matching tags (e.g. 412a, 412b, 412c) from one of 50 for various cities may be prepared by a third party Such that a the tag sources to the tag being typed by the user in the tag user need not be at each location to record the geographic entry field 406 in a matching tag list 412. Each tag may have coordinates. In this case a landmark tag may be selected by a an icon or some other visual identifier associated with it that user froma prepared landmark list that may be selected as one clearly indicates its type, and allows the user to quickly dis of the tag sources. 55 tinguish between different types of tags. In the tag selection interface described above, each tag may If the usertypes text that does not match any tag from the have some context associated with it, depending on the tag tag sources in the matching tag list 412, the photo tag selec type. As will be appreciated, this associated context may tion module 148B may create a new free-form tag entry and allow the user to visually identify and differentiate between add it to a free-form text cache as a new tag entry. The the tag types in the tag selection user interface 404, and free-form text cache may then become one of thetag sources 60 present other context sensitive menu options. Such as viewing for any Subsequent tag selection by the user. associated address cards, Facebook profiles, websites, geo As the user continues to type, if a tag that the user wishes to graphic location information for landmarks, etc. select appears in the matching tag list 412, the user can scroll Now referring to FIG. 5, shown is a flowchart of an illus to the tag in the matching tag list 412and select it by pressing trative method 500 in accordance with an embodiment. As enter or clicking on the navigation device (e.g. trackball 117). 65 shown, method 500 begins and at block 502 where method For example, as shown in screen 400E of FIG. 4E, the user 500 displays a photo in a display (e.g. display 110) to be tagged. may select a tag 412c which may then be placed into the tag EXHIBIT C Page 163 US 8,279,173 B2 7 Method 500 then proceeds to block504, where method 500 may configure photo tagging module 148A to provide a photo tagging mode for editing tags for the displayed photo. Method 500 may then proceed to block 506, where for each tag used to tag a subject or object in the photo, method 500 may configure photo tagging module 148A to identify thetag type and associate a unique pointer for each tag type to high light the corresponding tagged Subjector object. For example, as described earlier, a grey Square could be used for Facebook friends, and another shape and/or color may be used for address book tag types. Method 500 may then proceed to block 508, where method 500 may configure photo tagging module 148A to display a tag list with the displayed photo. Method 500 may then proceed to block510, where method 500 may configure photo tagging module 148A toidentify the tag typeof each tag in thetag list using a visual identifier. For example, the visual identifier may be an icon, or some Suitable graphic appearing beside a tag name. Method 500 may then proceed to block 512, where upon user selection of a tag in thetag list, method 500 may config ure photo tagging module 148A to highlight the associated tagged Subject or object in the photo using the unique pointer. Method 500 may then proceed to block 514, where upon user selection of a tag in thetag list, method 500 may config ure photo tagging module 148A to highlight the associated tag in the tag list. Method 500 may also proceed to block 516, where method 500 may configure photo tagging module 148A to display context data associated with a tag. For example, context data associated with an address book type may be the email 10 15 25 form text. 30 address, name, and other information stored in the address book module 138 for the corresponding address book entry. In an embodiment, the context data associated with the tag could be displayed within a menu that may be accessed by a suitable menu selection associated with each tag. Method 500 35 then ends. Method 500 may also proceed to block 518, where method 500 may configure photo tagging module 148A to constrain the navigation device pointer to be navigable only within the boundaries of the photo. Now referring to FIG. 6, shown is a flowchart of an illus 40 trative method 600 in accordance with an embodiment. As 45 shown, method 600 begins and at block 602 where method 600 provides a tag entry field for entering a photo tag. Method 600 may then proceed to block 604, where in dependence upon a string entered by the user in thetag entry field, method 600 may configure photo tag selection module 148B to display in a matching tag list 412 any tags (412.a. 412b, 412c) matching the entered string. Method 600 may then proceed to block 606, where method 600 may configure photo tag selection module 148B to dis play a tag type for each tag appearing in the matching tag list 412. For example, the visual identification may be an icon, or other Suitable graphic identification appearing with the tag aC. 8 Method 600 may then proceed to block 612, where if there are not matching tags appearing in the matching tag list 412, then method 600 may add the new free-from text tag to a free-form text tag cache. Method 600 may then proceed to block 614, where method 600 may configure photo tag selection module 148B to select the free-form text tag cache as one of thetag sources for any subsequent tag selection by the user. Method 600 then ends. Thus, in accordance with an embodiment, there is provided a method of selecting a photo tag for a tagged photo, com prising: providing a tag entry field for entering a photo tag; in dependence upon a string entered by a user, displaying in a matching tag list any tags from one or more selected tag Sources matching the entered string. In an embodiment, the method further comprises display ing a tag type for each tag appearing in the matching tag list. In another embodiment, the method further comprises ordering tags in the matching tag list by tag type. In another embodiment, the method further comprises allowing user selection of a tag in the matching tag list to complete the tag entry field. In another embodiment, the method further comprises add ing a new free-form text string to a free-form text cache if there are no tags remaining in the matching tag list. In an embodiment, the method further comprises selecting as a tag source one or more ofan online network profile, an address book, browser bookmarks, landmark tags, and free In another aspect of the invention, there is provided a system for selecting aphoto tag for a tagged photo, compris ing: means for providing a tag entry field for entering a photo tag; means for displaying in a matching tag list, in depen dence upon a string entered by a user, any tags from one or more selected tag sources matching the entered string. In an embodiment, the system further comprises means for displaying a tag type for each tag appearing in the matching tag list. In another embodiment, the system further comprises means for ordering tags in the matching tag list by tag type. In another embodiment, the system further comprises means for allowing user selection ofa tag in the matching tag list to complete the tag entry field. In another embodiment, the system further comprises means for adding a new free-form text string to a free-form text cache if there are no tags remaining in the matching tag list. 50 55 In another embodiment, the system further comprises means for selecting as a tag source one or more ofan online network profile, an address book, browser bookmarks, land mark tags, and free-form text. In another aspect of the invention, there is provided a data processor readable medium storing data processor code that when loaded into a device adapts the device to select aphoto tag for a tagged photo, the data processor readable medium comprising: code for providing a tag entry field for entering a photo tag; code for displaying in amatching tag list, indepen dence upon a string entered by a user, any tags from one or more selected tag sources matching the entered string. In an embodiment, the data processor medium further com prises code for displaying a tag type for each tag appearing in the matching tag list. In another embodiment, the data processor medium further comprises code for ordering tags in the matching tag list by Method 600 may then proceed to block 608, where method 60 600 may configure photo tag selection module 148B to order the matching tags in the tag list by tag type in order to make is easier for the user to select a Suitable tag by tag type. Method 600 may also proceed to block 610, where method tag type. 600 may configure photo tag selection module 148B to allow 65 In another embodiment, the data processor medium further user selection of a tag in the matching tag list to complete the comprises code for allowing user selection of a tag in the matching tag list to complete thetag entry field. tag entry field 406. EXHIBIT C Page 164 US 8,279,173 B2 In another embodiment, the data processor medium further comprises code for adding a new free-form text string to a free-form text cache if there are no tags remaining in the matching tag list. In another embodiment, the data processor medium further comprises code for selecting as a tag source one or more of an online network profile, an address book, browser bookmarks, landmark tags, and free-form text. 5 While illustrative embodiments have been described above, it will be appreciated that various changes and modi fications may be made. More generally, the scope of the invention is defined by the following claims. 10 We claim: 1. A method of selecting a photo tag for a tagged photo, comprising: displaying a tag list including tags from one or more tag Sources matching a search String; displaying a tag type indicator for each tag appearing in the tag list, said tag type being indicative of a tag source associated with the tag. 2. The method of claim 1, further comprising providing a tag entry field for entering the search String. 3. The method of claim 1, further comprising ordering tags in thetag list by tag type. 4. The method of claim 1, further comprising providing a graphical user interface to select a tag in the tag list. 5. The method of claim 1, further comprising: adding a new free-form text string to a free-form text cache based upon the search String. 6. The method of claim 1, further comprising selecting as a tag source one or more of an online network profile, an address book, browser bookmarks, landmark tags, and free 15 25 30 form text. 7. A system for selecting a photo tag for a tagged photo, comprising: an apparatus operable to display a tag list including tags from one or more tag sources matching a search string; an apparatus operable to display a tag type indicator for each tag appearing in the tag list, said tag type being indicative of a tag source associated with the tag. 8. The system of claim 7, further comprising an apparatus operable to provide a tag entry field for entering a photo tag. 35 10 9. The system of claim 7, further comprising an apparatus operable to order tags in thetag list by tag type. 10. The system of claim 7, further comprising an apparatus operable to allow selection of a tag in thetag list to complete the tag entry field. 11. The system of claim 1, further comprising: an apparatus operable to adda new free-form text string to a free-form text cache based upon the search String. 12. The system of claim 1, further comprising an apparatus operable to select as a tag source one or more ofan online network profile, an address book, browser bookmarks, land mark tags, and free-form text. 13. A computer readable medium storing computer code that when loaded into a device adapts the device to select a photo tag for a tagged photo, the computer readable medium comprising: code for displaying a tag list including tags from one or more tag sources matching a search String; code for displaying a tag type indicator for each tag appear ing in the tag list, said tag type being indicative of a tag Source associated with the tag. 14. The computer readable medium of claim 13, further comprising code for providing a tag entry field for entering the search string. 15. The computer readable medium of claim 13, further comprising code for ordering tags in the tag list by tag type. 16. The computer medium of claim 13, further comprising code for providing a graphical user interface to select a tag in the tag list to complete the tag entry field. 17. The computer medium of claim 13, further comprising: code for adding a new free-form text string to a free-form text cachebased upon the search string. 18. The computer readable medium of claim 13, further comprising code for selecting as a tag source one or more of an online network profile, an address book, browser book marks, landmark tags, and free-form text. 19. The computer readable medium of claim 13, further comprising code for displaying context data with the photo tag. 40 20. The computer readable medium of claim 19, wherein the context data is based on information in an address book. EXHIBIT C Page 165 k k k k k