Source: http://patents.com/us-9123086.html
Timestamp: 2019-02-23 07:11:28
Document Index: 482366766

Matched Legal Cases: ['Application No. 14159447', 'Application No. 14159464', 'application No. 35215130', 'Application No. 35015130', 'Application No. 14159464', 'Application No. 628495', 'Application No. 622501', 'Application No. 622501', 'Application No. 14159447', 'Application No. 14159464', 'Application No. 1413935', 'Application No. 61']

US Patent # 9,123,086. Automatically generating event objects from images - Patents.com
United States Patent 9,123,086
Freeland , et al. September 1, 2015
Freeland; Carl (Mountain View, CA), Scott; Jacob (Mountain View, CA), Sadur; Eric (San Francisco, CA), Ronan; Timothy (London, GB), Kross; Michael (Palo Alto, CA), Kwik; Huey (San Francisco, CA)
Palantir Technologies, Inc. (Palo Alto, CA)
Family ID: 1000000346763
13/838,815
61759283 Jan 31, 2013
Current CPC Class: G06T 1/00 (20130101); Y10S 707/915 (20130101)
Current International Class: G06K 9/46 (20060101); G06T 1/00 (20060101)
5555503 September 1996 Kyrtsos et al.
5631981 May 1997 Rao et al.
6141659 October 2000 Barker et al.
6189003 February 2001 Leal
6272489 August 2001 Rauch et al.
7171427 January 2007 Witkowski
7188100 March 2007 De Bellis et al.
7383053 June 2008 Kent et al.
7523100 April 2009 Bionda et al.
7652622 January 2010 Hansen et al.
7818291 October 2010 Ferguson et al.
8028894 October 2011 Lapstun et al.
8042110 October 2011 Kawahara et al.
8402047 March 2013 Mangini et al.
8477994 July 2013 Noshadi
8521135 August 2013 Cryderman
8739059 May 2014 Rabenold et al.
8762870 June 2014 Robotham et al.
9037407 May 2015 Thompson
2003/0152277 August 2003 Hall et al.
2003/0227746 December 2003 Sato
2005/0125715 June 2005 Franco et al.
2006/0206235 September 2006 Shakes et al.
2006/0250764 November 2006 Howarth et al.
2007/0043744 February 2007 Carro
2007/0088596 April 2007 Berkelhamer
2007/0130541 June 2007 Louch et al.
2007/0250491 October 2007 Olszak et al.
2008/0007618 January 2008 Yuasa
2008/0162616 July 2008 Gross et al.
2008/0301559 December 2008 Martinsen et al.
2008/0313281 December 2008 Scheidl et al.
2009/0037912 February 2009 Stoitsev et al.
2009/0125369 May 2009 Kloostra et al.
2009/0138790 May 2009 Larcheveque et al.
2009/0156231 June 2009 Versteeg et al.
2009/0265105 October 2009 Davis et al.
2009/0315679 December 2009 Bauchot et al.
2010/0004857 January 2010 Pereira et al.
2010/0073315 March 2010 Lee et al.
2010/0082842 April 2010 Lavrov et al.
2010/0214117 August 2010 Hazzani
2010/0223543 September 2010 Marston
2011/0022312 January 2011 McDonough et al.
2011/0158469 June 2011 Mastykarz
2011/0202557 August 2011 Atsmon et al.
2011/0276423 November 2011 Davidson
2012/0010812 January 2012 Thompson
2012/0014560 January 2012 Obrador et al.
2012/0032975 February 2012 Koch
2012/0121512 May 2012 Grompe et al.
2012/0166929 June 2012 Henderson et al.
2012/0216106 August 2012 Casey
2012/0277914 November 2012 Crow et al.
2013/0013642 January 2013 Klein et al.
2013/0196614 August 2013 Pahlevani
2013/0262497 October 2013 Case et al.
2013/0262528 October 2013 Folt
2013/0288719 October 2013 Alonzo
2014/0079340 March 2014 Kawano
2014/0081685 March 2014 Thacker et al.
2014/0093174 April 2014 Zhang et al.
2014/0176606 June 2014 Narayan et al.
2014/0258827 September 2014 Gormish et al.
2014/0279865 September 2014 Tamayo
2014/0302783 October 2014 Aiuto et al.
2014/0357299 December 2014 Xu et al.
2015/0080012 March 2015 Sprague et al.
2400448 Dec 2011 EP
2816513 Dec 2014 EP
WO2005104736 Nov 2005 WO
WO2009061501 May 2009 WO
WO2010000014 Jan 2010 WO
WO2010030913 Mar 2010 WO
Official Communication for European Patent Application No. 14159447.3 dated Nov. 25, 2014. cited by applicant .
Official Communication for European Patent Application No. 14159464.8 dated Oct. 8, 2014. cited by applicant .
New Zealand Intellectual Property Office, "First Examination Report" in application No. 35215130/AJS, dated Apr. 1, 2014, 2 pages. cited by applicant .
Claims in New Zealand Application No. 35015130/AJS, dated Apr. 2014, 4 pages. cited by applicant .
Official Communication for European Patent Application No. 14159464.8 dated Sep. 22, 2014. cited by applicant .
Official Communication in New Zealand Application No. 628495 dated Aug. 19, 2014. cited by applicant .
Official Communication for New Zealand Patent Application No. 622501 dated Apr. 1, 2014. cited by applicant .
Official Communication for New Zealand Patent Application No. 622501 dated Jun. 5, 2014. cited by applicant .
"A First Look: Predicting Market Demand for Food Retail Using a Huff Analysis", TRF Policy Solutions, dated Jul. 2012, 30 pages. cited by applicant .
"A Quick Guide to UniProtKB Swiss-Prot & TrEMBL", dated Sep. 2011, 2 pages. cited by applicant .
Acklen, Laura, "Absolute Beginner's Guide to Microsoft Word 2003", dated Dec. 24, 2003, pp. 15-18, 34-41, 308-316. cited by applicant .
Ananiev et al., "The New Modality API", http://web.archive.org/web20061211011958/http://java.sun.com/developer/te- chnicalarticles/J2SE/Desktop/javase6/modality/ dated Jan. 21, 2006, 8 pages. cited by applicant .
Bluttman et al., "Excel Formulas and Functions for Dummies", dated 2005, Wiley Publishing, Inc. pp. 280-284-286. cited by applicant .
Bugzilla@Mozilla, "Bug 18726--[feature] Long-click means of invoking contextual menus not supported", http://bugzilla, Mozilla.org/show.sub.--bug.cgi?id=18726 printed Jun. 13, 2013, 11 pages. cited by applicant .
Canese et al., "Chapter 2: PubMed: The Bibliographic Database", The NCBI Handbook, dated Oct. 2002, pp. 1-10. cited by applicant .
Chen et al., "Bringing Order to the Web: Automatically Categorizing Search Results", CHI 2000, Proceedings of SIGCHI Conference on Human Factors in Computing Systems, dates Apr. 1-6, 2000, pp. 145-152. cited by applicant .
Conner, Nancy, "Google Apps: The Missing Manual", dated May 1, 2008, 15 pages. cited by applicant .
Dramowicz, Ela, "Retail Trade Area Analysis Using the Huff Model", Directions Magazine, dated Jul. 2, 2005, 10 pages. http://www.directionsmag.com/articles/retail-trade-area-analysis-using-th- e-huff-model/123411. cited by applicant .
Delcher, et al., "Identifying Bacterial Genes and Endosymbiont DNA with Glimmer", BioInformatics, vol. 23, No. 6, dated 2007, pp. 673-679. cited by applicant .
GIS-NET 3 Public-Department of Regional Planning. Planning & Zoning Information for Unincorporated LA County., Retrieved Oct. 2, 2013 from http://gis.planning.lacounty.gov.GIS-NET3.sub.--Public/Viewer.html. cited by applicant .
Keylines.com, "An Introduction to KeyLines and Network Visualization," Mar. 2014, <http://keylines.com/wp-content/uploads/2014/03/KeyLines-White-Paper.p- df> downloaded May 12, 2014 in 8 pages. cited by applicant .
Keylines.com, "KeyLines Datasheet," Mar. 2014, <http://keylines.com/wp-content/uploads/2014/03/KeyLines-datasheet.pdf- > downloaded May 12, 2014 in 2 pages. cited by applicant .
Keylines.com, "Visualizing Threats: Improved Cyber Security Through Network Visualization," Apr. 2014, <http://keylines.com/wp-content/uploads/2014/04/Visualizing-Threats1.p- df> downloaded May 12, 2014 in 10 pages. cited by applicant .
Manske, "File Saving Dialogs," <http://www.mozilla.org/editor/ui.sub.--specs/FileSaveDialogs.html>- , Jan. 20, 1999, pp. 7. cited by applicant .
Microsoft--Developer Network, "Getting Started with VBA in Word 2010," Apr. 2010, <http://msdn.microsoft.com/en-us/library/ff604039%028v=office.14%29.as- px> as printed Apr. 4, 2014 in 17 pages. cited by applicant .
Sigrist, et al., "PROSITE, a Protein Domain Database for Functional Characterization and Annotation," Nucleic Acids Research, 2010, vol. 38, pp. D161-D166. cited by applicant .
Hansen, Derek "Analyzing Social Media Networks with NodeXL: Insights from a Connected World". cited by applicant .
Kahan et al., "Annotea: an open RDF infrastructure for shared WEB annotations". cited by applicant .
Olanoff, Drew, "Deep Dive with the New Google Maps for Desktop with Google Earth Integration, It's More than Just a Utility," May 15, 2013, pp. 1-6, retrieved from the internet: http://web.archive.org/web/20130515230641/http://techcrunch.com/2013/05/1- 5/deep-dive-with-the-new-google-maps-for-desktop-with-google-earth-integra- tion-its-more-than-just-a-utility/. cited by applicant .
Hogue et al., "Thresher: Automating the Unwrapping of Semantic Content from the World Wide Web," 14.sup.th International Conference on World Wide Web, WWW 2005: Chiba, Japan, May 10-14, 2005, pp. 86-95. cited by applicant .
Definition "Overlay" downloaded Jan. 22, 2015, 1 page. cited by applicant .
Definition "Identify" downloaded Jan. 22, 2015, 1 page. cited by applicant .
Yang et al., "HTML Page Analysis Based on Visual Cues," 2001, pp. 859-864. cited by applicant .
Nierman, "Evaluating Structural Similarity in XML Documents," 2002, 6 pages. cited by applicant .
Official Communication for European Patent Application No. 14159447.3 dated Jan. 8, 2015. cited by applicant .
Official Communication for European Patent Application No. 14159464.8 dated Aug. 20, 2014. cited by applicant .
Official Communication for Great Britain Patent Application No. 1413935.6 dated Jan. 27, 2015. cited by applicant.
Primary Examiner: Lu; Li
This application claims the benefit of U.S. Provisional Application No. 61/759,283, entitled "Automatically Generating Event Objects From Images" and filed Jan. 31, 2013, the entire contents of which is hereby incorporated by reference as if fully set forth herein, under 35 U.S.C. .sctn.119(e).
1. A computer-implemented method, comprising: receiving a first digital image comprising first image metadata, wherein the first image metadata includes one or more image properties each having an image property value; transforming one or more of the image property values of the one or more image properties into one or more particular values; populating one or more data object property values of a first data object of an object-centric data model with the one or more particular values; generating one or more links to one or more other existing data objects of a data object graph stored in a data repository; populating one or more data object property values of the first data object with the one or more links to the one or more other existing data objects; populating one or more data object property values of the one or more other existing data objects with one or more links to the first data object; storing the first data object with the one or more particular values in the data repository; wherein the method is performed by one or more computing devices.
3. The method of claim 1, wherein the one or more links are generated in response to receiving data from a mobile device indicating the one or more links.
4. The method of claim 1, further comprising: receiving user input indicating a data object type of the data object; storing a value indicating the data object type in association with the data object.
5. The method of claim 4, wherein the data object type is one of an event, a person, an organization, a location, an entity, or an item.
6. The method of claim 1, wherein the one or more image properties include one or more of: date information, time of day information, location information, camera settings information, user information.
7. The method of claim 1, further comprising populating the one or more data object property values of the data object in response to user selection of the digital image.
8. The method of claim 1, further comprising: receiving a second digital image comprising second image metadata; and wherein transforming one or more of the image property values of the one or more image properties into one or more particular values includes generating summary information based on the first image metadata and the second image metadata.
9. The method of claim 8, wherein the summary information includes a particular location value, the particular location value derived from an average of a first location value associated with the first image metadata and a second location value associated with the second image metadata.
10. A non-transitory computer-readable medium storing one or more sequences of instructions, which when executed by one or more processors, cause the one or more processors to carry out the steps of: receiving a first digital image comprising first image metadata, wherein the first image metadata includes one or more image properties each having an image property value; transforming one or more of the image property values of the one or more image properties into one or more particular values; populating one or more data object property values of a first data object of an object-centric data model with the one or more particular values; generating one or more links to one or more other existing data objects of a data object graph stored in a data repository; populating one or more data object property values of the first data object with the one or more links to the one or more other existing data objects; populating one or more data object property values of the one or more other existing data objects with one or more links to the first data object; storing the first data object with the one or more particular values in the data repository.
11. The non-transitory computer readable medium of claim 10, wherein the one or more instructions further comprise instructions which, when executed by one or more processors, cause the one or more processors to perform receiving the first digital image from a mobile device via a wireless signal.
12. The non-transitory computer readable medium of claim 10, wherein the one or more links are generated in response to receiving data from a mobile device indicating the one or more links.
13. The non-transitory computer readable medium of claim 10, wherein the one or more instructions further comprise instructions which, when executed by one or more processors, cause the one or more processors to perform: receiving user input indicating a data object type of the data object; storing a value indicating the data object type in association with the data object.
14. The non-transitory computer readable medium of claim 13, wherein the data object type is one of an event, a person, an organization, a location, an entity, or an item.
15. The non-transitory computer readable medium of claim 10, wherein the one or more image properties include one or more of: date information, time of day information, location information, camera settings information, user information.
16. The non-transitory computer readable medium of claim 10, wherein the one or more instructions further comprise instructions which, when executed by one or more processors, cause the one or more processors to perform populating the one or more data object property values of the data object in response to user selection of the digital image.
17. The non-transitory computer readable medium of claim 10, wherein the one or more instructions further comprise instructions which, when executed by one or more processors, cause the one or more processors to perform: receiving a second digital image comprising second image metadata; and wherein transforming one or more of the image property values of the one or more image properties into one or more particular values includes generating summary information based on the first image metadata and the second image metadata.
18. The non-transitory computer readable medium of claim 17, wherein the summary information includes a particular location value, the particular location value derived from an average of a first location value associated with the first image metadata and a second location value associated with the second image metadata.
19. A computer-implemented method, comprising: receiving, from a mobile device via a wireless signal, a first digital image comprising first image metadata, wherein the first image metadata includes one or more image properties each having an image property value; wherein the one or more image properties include one or more of: date information, time of day information, location information, camera settings information, user information; receiving a second digital image comprising second image metadata; displaying, on the mobile device, a first data object of an object-centric data model with one or more selectable images; wherein the one or more selectable images includes the first image and the second image; receiving, from the mobile device, a selection of the first image and the second image to be associated with the data object; transforming one or more of the image property values of the one or more image properties into one or more particular values; wherein transforming one or more of the image property of the one or more image properties into one or more particular values includes generating summary information based on the first image metadata and the second image metadata; wherein the summary information includes a particular value, the particular location value derived from an average of a first location value associated with the first image metadata and a second location value associated with the second image metadata; populating one or more data object property values of the first data object with the one or more particular values in response to receiving the selection of the first image to be associated with the data object; receiving data from a mobile device indicating one or more links to one or more other existing data objects of a data object graph stored in a data repository; generating one or more links to one or more other existing data objects of a data object graph stored in the data repository; populating one or more data object property values of the first data object with the one or more links to the one or more other existing data objects; populating one or more data object property values of the one or more other existing data objects with one or more links to the first data object; storing the first data object with the one or more particular values in the data repository; receiving user input indicating a data object type of the data object; storing a value indicating the data object type in association with the data object; wherein the data object type is one of an event, a person, an organization, a location, an entity, or an item; wherein the method is performed by one or more computing devices.
3.0 Architectural and Functional Overview 3.1 Mobile Device System Architecture 3.2 Operations Center System Architecture 3.3 Configuring Mobile Device Teams 3.4 Mobile Device Tracking 3.5 Mobile Device Messaging 3.6 Creating Data Objects from Images 3.7 Mobile Device Search
In an embodiment, geosearch logic 310 provides processes for handling geosearch requests sent from a mobile device and analyst workstation users. In general, a geosearch request is a search request for data objects or other information that is associated with one or more specified geographic locations or areas. Examples of processing geo search requests are described in a separate section herein.
In Step 402, one or more mobile device user accounts are generated. For example, a separate mobile device user account may be generated for each field analyst in an organization using an approved mobile device in the mobile data analysis system. In an embodiment, an input mechanism is provided for a system administrator or other user to enter commands for the purposes of generating mobile device user accounts. Here, the term "input mechanism" includes either a command line interaction mechanism or a graphical user interface based interaction mechanism, or a combination of the preceding two. For example, a command may be issued by a user at an analyst work station 114 and received by application server 116 and, in response, application server 116 may generate and store the one or more mobile device user accounts in a repository, such as data repository 118. In another embodiment, mobile device user account information for one or more mobile device user accounts may be received by an application server in the form of an account configuration file, for example, in an XML file or other structured document format.
In Step 404, a first mobile device team and a second mobile device team are generated. For example, an authorized user using an analyst workstation may issue one or more additional commands to create two new mobile device teams. The user may associate a label for each of the new teams, for example, the first mobile device team may be labeled the "Green" team and the second mobile device team may be labeled the "Blue" team. The mobile device team labels may be used for the purposes of identifying a mobile device team in other graphical user interfaces of the mobile device system. For the purposes of illustrating a clear example, only two mobile device teams are generated; however, in other embodiments any number of mobile device teams may be generated.
In an embodiment, information panel 502 comprises information related a mobile device user account associated with a currently selected mobile device in map display 506. In the current example, information panel 502 displays information associated with a mobile device user "Brown." The information about mobile device user Brown may be displayed, for instance, in response to a user indicating input selecting user location icon 508 on map display 506 representing mobile device user Brown, selecting the mobile device user's name from a list, typing the mobile device user name into a search box, inputting a voice command, or otherwise indicating a selection of the mobile device user Brown. For example, a different mobile device and mobile device user account may be selected by indicating input selecting another user location icon that is visible on map display 506, such as user location icon 504 representing mobile device user Smith.
Information panel 502 includes additional information about mobile device user Brown, including "last update" information indicating that mobile device user Brown is assigned to a mobile device team identified by the team name "Red." Information panel 502 further provides information indicating how recently location data was received by a mobile device server from the mobile device associated with mobile device user Brown. In this manner, the last update information may provide an indication of the accuracy of the displayed location for a selected mobile device.
Data object information panel 1104 includes information associated with a currently selected data object from data object list 1102. As depicted, data object information panel 1104 displays information for an Event data object labeled "Suspect sighting." Information displayed in data object information panel 1104 includes a display of a digital image associated with the selected data object, information about other related data objects, information about a user that created the data object, when the data object was created, and a location associated with the data object.
In one embodiment, mobile device users may specify geosearch requests using a mobile device. In general, a geosearch request is a search request for data objects that are associated with a specified geographic location or area. FIG. 12 illustrates an example GUI 1200 of a mobile device that is configured to enable users to specify a geo search request.
In an embodiment, GUI 1200 comprises search button 1202 which, when selected, may be configured to enable a user to specify a geosearch request. In one embodiment, mobile device users may specify a geosearch request using map display 1204. For example, a user may specify on map display 1204 a geographic bounding area of interest for the geo search request. In the current example, bounding area 1206 illustrates an example bounding area for a geosearch request. In the example, the bounding area is depicted as a circle, but other bounding area shapes may be used. The specified geographic bounding area may be used to transmit as part of the geosearch request a set of geographic coordinates or other information that enables application server 116 to determine a geographic area within to search. A geo search request may include other parameters including a time span desired for result data objects. For example, a user may desire that only Event data objects associated with an event that occurred in the past week be returned.
In response to receiving a geo search request, application server 116 may determine one or more result data objects in data repository 118 that include location data specifying one or more locations that are within the bounding area specified by the geosearch request. Application server 116 may send the result data objects to the requesting mobile device and the mobile device may display the result data objects on map display 1204. For example, event data object icon 1208 illustrates an example geosearch result data object for a geosearch request specified by bounding area 1206. Event data object icon 1208 may be displayed at a location on the map based on location data stored in association with the data object result corresponding to the displayed icon. In an embodiment, a user may select one or more of the data object result icons on map display 1204 in order to view additional information associated with the data object.
The term "storage media" as used herein refers to any non-transitory media that store data and/or instructions that cause a machine to operate in a specific fashion. Such storage media may comprise non-volatile media and/or volatile media. Non-volatile media includes, for example, optical disks, magnetic disks, or solid-state drives, such as storage device 1310. Volatile media includes dynamic memory, such as main memory 1306. Common forms of storage media include, for example, a floppy disk, a flexible disk, hard disk, solid-state drive, magnetic tape, or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, NVRAM, any other memory chip or cartridge.
Network link 1320 typically provides data communication through one or more networks to other data devices. For example, network link 1320 may provide a connection through local network 1322 to a host computer 1324 or to data equipment operated by an Internet Service Provider (ISP) 1326. ISP 1326 in turn provides data communication services through the world wide packet data communication network now commonly referred to as the "Internet" 1328. Local network 1322 and Internet 1328 both use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 1320 and through communication interface 1318, which carry the digital data to and from computer system 1300, are example forms of transmission media.
Previous Patent US 9,123,085 | Next Patent US 9,123,087