Abstract:
Disclosed is a system and method for extending search capabilities of contentment management systems, such as SharePoint 2013®, to enable geographic and name entity based searches. Geographic and named entity searches are enabled by a content enrichment web service. The content enrichment web service calls a geotagging or a named entity tagger web service application to tag crawled managed properties as input and return geographically or entity modified managed properties as output. The system associates one or more geographically and named entity modified managed properties with content and stores this information as metadata in a SharePoint 2013® search index. Thus, the search system allows users to identify a particular geographic entity the user is interested in finding, and to receive search results directly related to that geographic entity on SharePoint 2013®.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     This non-provisional patent application claims the benefit of U.S. Provisional Patent Application Ser. No. 61/947,652, entitled “Method For Entity Enrichment Of Digital Content To Enable Advanced Search Functionality In Content Management Systems,” filed Mar. 4, 2014, which hereby incorporated in its entirety herein. 
     This application is related to U.S. patent application Ser. No. 14/557,794, entitled “Method for Disambiguating Features in Unstructured Text,” filed Dec. 2, 2014; U.S. patent application Ser. No. 14/558,300, entitled “Event Detection Through Text Analysis Using Trained Event Template Models,” filed Dec. 2, 2014; and U.S. patent application Ser. No. 14/557,807, entitled “Method for Facet Searching and Search Suggestions,” filed Dec. 2, 2014; each of which are incorporated herein by reference in their entirety. 
    
    
     TECHNICAL FIELD 
     The invention generally refers to search engines and content management. More specifically, extending a content management system&#39;s search engine technology to enable geotagging and named entities enrichment of digital content. 
     BACKGROUND 
     Content management and document management systems for document versioning and collaborative project management are known. One non-limiting example may be Microsoft&#39;s Sharepoint 2013® software and application suite of tools. Microsoft SharePoint 2013® is a family of software products developed by Microsoft Corporation for collaboration, file sharing and web publishing. SharePoint 2013® may provide a user with a vast amount of content or information and it may become difficult for a user to find the most relevant information for a particular circumstance. To mitigate these issues SharePoint 2013® provides a search engine in order to assist users in finding the content that they need. A user may enter a keyword based search query and the search engine in SharePoint 2013® may return to the user a list of the most relevant results found within the context of the SharePoint 2013® platform once the content has been indexed. 
     At times a user may desire to find content related to geographic entities in SharePoint 2013® or other type of entity such as organizations or people referred to within a document. SharePoint 2013® does not provide out of the box functionality to automatically extract entities from documents. Particularly, it does not support geotagging content to extract geographic entities and resolve them to a geographic location. Also, SharePoint 2013 does not support entity tagging in order to identify, disambiguate and extract named entities, such as, organizations or people in a document. However, SharePoint 2013® search may be extended to enable effective geographic searches and other entity related searches, including entity-based search facets. Previous versions of SharePoint 2013® included “FAST Search” for SharePoint, from which it was possible to extend the content processing pipeline through sandboxed applications, but this was both slow and limited in the information it could access. 
     SharePoint 2013® introduces a much more open API which makes it possible to add specialized linguistics such as concept extraction, relationship extraction, geotagging, summarization and as well as sophisticated text analytics. Thus, an opportunity exists to extend the capabilities of SharePoint 2013® search engine to enable geographic and other entity based searches. 
     SUMMARY 
     Disclosed herein are systems and methods for enabling geographic entity-based searches in content management systems, like Microsoft&#39;s SharePoint 2013®. Embodiments described The method involves extending the SharePoint 2013® search architecture by adding a geographic tagging web service. The system includes a computer processor operatively associated with a computer memory and one or more I/O device, in which the processor and memory are configured to operate one or more SharePoint 2013® processes. The system also includes another computer processor operatively associated with a computer memory and one or more I/O devices, in which the processor and memory are configured to host and provide processing for a geotagging web service. The SharePoint 2013® system may include a crawling component, a content processing component and a search indexing component in order to enable search of content. The content processing component in SharePoint 2013® search may extend its functionality by using the Content Enrichment Web Service (CEWS) feature. 
     The method involves crawling content from the different sources in order to obtain an array of crawled properties that are sent for content processing. During content processing, a trigger condition may determine if crawled properties may benefit from additional processing in order to enrich the original content with additional geographic metadata properties. If the crawled properties don&#39;t benefit from additional processing the crawled properties may be mapped to managed processing and sent to a search index. If the crawled properties benefit from external web services processing, the CEWS may make a simple object access protocol (SOAP) request to a configurable endpoint using hypertext transfer protocol (HTTP) or any other web service call method. An entity enrichment service may determine the type of content. If the content is in an image format, its metadata such as file location may be sent to an optical character recognition (OCR) engine so that the original document can be retrieved and processed asynchronously to convert to text and sent back to the crawl component to be re-crawled in text format. If the content is in text format the geotagging web service may identify geographic metadata and associate it with the content as managed properties. After the content has been geotagged, it may be sent to the indexing component. 
     An additional search user interface (UI) may be added using either SharePoint 2013® web parts or by modifying the standard layout of SharePoint 2013® search with standard web development tools such as HTML, HTML 5, JavaScript and CSS among others. The search UI may assist a user in performing geographic search queries or displaying geographic search results using digital geographic features such as for example and without limitation, digital maps. The search UI can also be enhanced to perform faceted search using the additional enriched entities or their associated metadata. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The present disclosure can be better understood by referring to the following figures. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the disclosure. In the figures, reference numerals designate corresponding parts throughout the different views. 
       Content capture is essential for searches. SharePoint 2013® search architecture involves a step of crawling and content processing in which components may be 
         FIG. 1  is a system architecture for tagging and entity enrichment of content in a content management system. 
         FIG. 2  is a process by which content is tagged and indexed for named and geographic entity searches. 
     
    
    
     The present disclosure is here described in detail with reference to embodiments illustrated in the drawings, which form a part here. Other embodiments may be used and/or other changes may be made without departing from the spirit or scope of the present disclosure. The illustrative embodiments described in the detailed description are not meant to be limiting of the subject matter presented here. 
     DEFINITIONS 
     As used herein, the following terms have the following definitions: 
     “Geotagging” refers to the process of extracting geographic entities from unstructured text files. Geotagging may include disambiguating the entity to a specific geographic place and appending geographic metadata such as geographic coordinates, geographic feature type and other metadata. 
     “Entity Tagging” refers to the process of extracting named entities from unstructured text. Entity Tagging may include entity disambiguation, entity name normalization and appending entity metadata. 
     “Named Entity” refers to a person, organization or topic. 
     “Geographic Entity” refers to geographic location or geographic places. 
     “Crawled Properties” refers to content management system metadata obtained from inspecting documents during crawls. 
     DETAILED DESCRIPTION 
     Reference will now be made to the exemplary embodiments illustrated in the drawings, and specific language will be used here to describe the same. It will nevertheless be understood that no limitation of the scope of the invention is thereby intended. Alterations and further modifications of the inventive features illustrated here, and additional applications of the principles of the inventions as illustrated here, which would occur to one skilled in the relevant art and having possession of this disclosure, are to be considered within the scope of the invention. 
       FIG. 1  is a system architecture  100  for geotagging content in SharePoint 2013®. A Search index  124  is one of a number of key components in order to enable search in SharePoint  102 . Another key part of enabling search in SharePoint 2013®  102  may be content capturing in order to index the content. SharePoint  102  includes a crawler  104  component in order to enable content capturing. 
     Crawler  104  may crawl through different content sources  106  adding a list of metadata properties to each content. Examples of content sources may include without limitation, SharePoint content, network file-share or user or intranet content. Crawler  104  may be configured perform the functions of connecting securely to a content source  106 , associating document from the sources to their metadata as crawled properties. The crawler  104  may be configured to full or incremental crawls to content. Examples of crawled properties may include for example and without limitation author, title, creation date among others. 
     SharePoint 2013® includes a content processing  108  component. The content processing  108  component takes content from the crawler  104  and prepares it for indexing. Content processing  108  may involve stages of linguistic processing (language detection), parsing, entity extraction management, content-based file format detection, content processing error reporting, natural language processing and mapping crawled properties to managed properties among others. 
     Content processing  108  may be extended by means of a content enrichment web service (CEWS  110 ). CEWS  110  may enable the enrichment of content processing  108  by allowing a web service callout  112  to call external web service to perform additional actions and enrich the crawled data properties. Web service callout  112  may be a standard simple object access protocol (SOAP) request or any other web service call method used to exchange structured information of the crawled data with an entity enrichment service  114 . Web service callout  112  may include trigger conditions configured in the content enrichment configuration object that control when to call an external web service for enrichment processing. Entity enrichment service  114  may also determine the document type of the crawled data in order to determine content that may come in the form of an image (scanned documents, pictures, etc.). Whenever content in the form of an image is found the entity enrichment service  114  may send the location of the crawled document to an OCR processing engine  116  such as for example and without limitation an optical character recognition component or other image processing component. OCR processing engine  116  may then retrieve and process the image files and convert them to text files asynchronously. The OCR&#39;d processed files  118  may subsequently be re-fed to crawler  104  in order to be crawled as text files and sent back to content processing  108  and proceed with the rest of the workflow. 
     System architecture  100  may include an external geotagger web service  120  and a named entity tagger service  122 . Both geotagger web service  120  and named entity tagger service  122  may be a software module configured to function as a web service application provider and to respond to web service callout  112 . Geotagger web service  120  may use natural language processing entity extraction techniques, machine learning models and other techniques in order to identify and disambiguate geographic entities from crawled content. For example, geotagger web service  120  may disambiguate geographic entities by analyzing statistical co-occurrence of entities found in a gazetteer. Geotagger web service  120  may include a database of statistical co-occurring entities which may be linked against content found by crawler  104 . Following the same technique, named entity tagger service  122  may be used to extract additional entities or text features such as organizations, people or topics. 
     Geotagger web service  120  may analyze an array of managed properties sent as input properties by CEWS  110  and identify any geographic entities referred in text. Non-limiting examples of input properties may include: FileType, IsDocument, OriginalPath and body among others. Geotagger web service  120  may then geotag the text by creating or modifying managed properties with reference to each geographic entity found. Geotagger web service  120  may send modified or new managed properties to the entity enrichment service  114  where a conversion is made that maps the modified managed properties and returns them as output properties back to CEWS  110 . The same process may be used to interact with the named entity tagger service  122  for the extraction and entity tagging of other entities or text features such as organizations, people or topics. 
     After the augmented managed properties are returned by the entity enrichment service  114  the properties are merged with the crawled file managed properties and sent to a search index  124 . 
     Once geographic and other entity tags have been associated with content and indexed, search queries may also be performed using geographic or named entity features. A search UI  126  in SharePoint 2013® may include specific displays that may assist a user in performing a geographic based search as well as support enhanced displays of faceted search results. The search UI  126  may be a custom web part or may also be done by modifying the standard layout of SharePoint 2013® search with standard tools such as HTML, HTML 5, JavaScript and CSS. 
       FIG. 2  is a flow chart  200  illustrating the process steps for tagging content for SharePoint 2013® search. The process may begin when the crawler component in SharePoint 2013® performs a crawl for content (step  202 ). In one embodiment the crawl may be a full crawl, wherein in another embodiment the crawl may be an incremental crawl. The crawler component may then feed crawled properties and metadata to the content processing (step  204 ). A determination is made to verify if the crawled content may include geographic or named entities. For example and without limitation a trigger condition may be used. The trigger condition may contain a set of programmatic logic or rules which may determine if content may benefit from geotagging or entity tagging. If the trigger condition evaluates to false crawled content may be associated with managed properties (step  206 ) and passed to the search index component (step  208 ). If the trigger condition evaluates to true the CEWS may send a web service callout (step  210 ) to an entity enrichment service. The entity enrichment service may analyze the content sent in order to determine if the content may be in an image format (scanned documents, pictures, etc.). Content found in an image format may be processed asynchronously by an OCR engine and sent back to be re-crawled by the crawling component as text files (step  212 ). If the content is not in image format, the content may be processed by a geotagging web service or a name entity tagger service (step  214 ). The web service may extract and disambiguate geographic or named entities referred in the content and enrich them with entity metadata. The identified entities and their metadata may be sent back as managed properties to the content processing component and associated with the content (step  216 ). The associated metadata may then be sent to the search index component (step  206 ). 
     The foregoing method descriptions and the process flow diagrams are provided merely as illustrative examples and are not intended to require or imply that the steps of the various embodiments must be performed in the order presented. As will be appreciated by one of skill in the art the steps in the foregoing embodiments may be performed in any order. Words such as “then,” “next,” etc. are not intended to limit the order of the steps; these words are simply used to guide the reader through the description of the methods. Although process flow diagrams may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be re-arranged. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to the calling function or the main function. 
     The various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed here may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention. 
     Embodiments implemented in computer software may be implemented in software, firmware, middleware, microcode, hardware description languages, or any combination thereof. A code segment or machine-executable instructions may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc. 
     The actual software code or specialized control hardware used to implement these systems and methods is not limiting of the invention. Thus, the operation and behavior of the systems and methods were described without reference to the specific software code being understood that software and control hardware can be designed to implement the systems and methods based on the description here. 
     When implemented in software, the functions may be stored as one or more instructions or code on a non-transitory computer-readable or processor-readable storage medium. The steps of a method or algorithm disclosed here may be embodied in a processor-executable software module which may reside on a computer-readable or processor-readable storage medium. A non-transitory computer-readable or processor-readable media includes both computer storage media and tangible storage media that facilitate transfer of a computer program from one place to another. A non-transitory processor-readable storage media may be any available media that may be accessed by a computer. By way of example, and not limitation, such non-transitory processor-readable media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other tangible storage medium that may be used to store desired program code in the form of instructions or data structures and that may be accessed by a computer or processor. Disk and disc, as used here, include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media. Additionally, the operations of a method or algorithm may reside as one or any combination or set of codes and/or instructions on a non-transitory processor-readable medium and/or computer-readable medium, which may be incorporated into a computer program product. 
     When implemented in hardware, the functionality may be implemented within circuitry of a wireless signal processing circuit that may be suitable for use in a wireless receiver or mobile device. Such a wireless signal processing circuit may include circuits for accomplishing the signal measuring and calculating steps described in the various embodiments. 
     The hardware used to implement the various illustrative logics, logical blocks, modules, and circuits described in connection with the aspects disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but, in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Alternatively, some steps or methods may be performed by circuitry that is specific to a given function. 
     Any reference to claim elements in the singular, for example, using the articles “a,” “an” or “the,” is not to be construed as limiting the element to the singular. 
     The preceding description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined here may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown here but is to be accorded the widest scope consistent with the following claims and the principles and novel features disclosed here.