Team Formation

A method and apparatus are provided for grouping individuals into one or more teams by generating vector representations of individual concept sets (containing individual concepts associated with a corresponding individual) and a project concept set (containing project concepts corresponding to a specified project), and then performing comparison analysis (e.g., a natural language processing (NLP) analysis comparison) of the vector representation of each individual concept set to a vector representation of each project concept set to determine a similarity measure between each individual concept set and each project concept set so that one or more of the plurality of individuals may be selected for grouping into a first team by using the similarity measure to determine which individual concepts associated with a corresponding individual are similar to the project concept set.

BACKGROUND OF THE INVENTION

In the field of artificially intelligent computer systems capable of answering questions posed in natural language, cognitive question answering (QA) systems (such as the IBM Watson™ artificially intelligent computer system or and other natural language question answering systems) process questions posed in natural language to determine answers and associated confidence scores based on knowledge acquired by the QA system. In operation, users submit one or more questions through a front-end application user interface (UI) or application programming interface (API) to the QA system where the questions are processed to generate answers that are returned to the user(s). The QA system generates answers from an ingested knowledge base corpus, including publicly available information and/or proprietary information stored on one or more servers, Internet forums, message boards, or other online discussion sites. Using the ingested information, the QA system can formulate answers using artificial intelligence (AI) and natural language processing (NLP) techniques to provide answers with associated evidence and confidence measures. However, the quality of the answer depends on the ability of the QA system to identify and process information contained in the knowledge base corpus.

With some traditional QA systems, there are mechanisms provided for processing information in a knowledge base by using vectors to represent words to provide a distributed representation of the words in a language. Such mechanisms include “brute force” learning by various types of Neural Networks (NNs), learning by log-linear classifiers, or various matrix formulations. Lately, word2vec, that uses classifiers, has gained prominence as a machine learning technique which is used in the natural language processing and machine translation domains to produce vectors which capture syntactic as well semantic properties of words. Matrix based techniques that first extract a matrix from the text and then optimize a function over the matrix have recently achieved similar functionality to that of word2vec in producing vectors. However, there is no mechanism in place to identify and/or process concepts in an ingested corpus which are more than merely a sequence of words. Nor are traditional QA systems able to identify and process concept attributes in relation to other concept attributes. Instead, existing attempts to deal with concepts generate vector representations of words that carry various probability distributions derived from simple text in a corpus, and therefore provide only limited capabilities for applications, such as NLP parsing, identification of analogies, and machine translation. As a result, the existing solutions for efficiently identifying and applying concepts contained in a corpus are extremely difficult at a practical level.

SUMMARY

Broadly speaking, selected embodiments of the present disclosure provide a system, method, and apparatus for processing of inquiries to an information handling system capable of answering questions by using the cognitive power of the information handling system to generate or extract a sequence of concepts, to extract or compute therefrom a distributed representation of the concept(s) (i.e., concept vectors), and to process the distributed representation (the concept vectors) to carry out useful tasks in the domain of concepts and user-concept interaction, including team formation applications that generate recommendations for groupings of persons based on their skills/knowledge being related to a set of concepts. In selected embodiments, the information handling system may be embodied as a question answering (QA) system which has access to structured, semi-structured, and/or unstructured content contained or stored in one or more large knowledge databases (a.k.a., “corpus”), and which extracts therefrom a sequence of concepts from annotated text (e.g., hypertext with concept links highlighted), from graph representations of concepts and their inter-relations, from tracking the navigation behavior of users, or a combination thereof. In other embodiments, concept vectors may also be used in a “discovery advisor” context where users would be interested in seeing directly the concept-concept relations, and/or use query concepts to retrieve and relate relevant documents from a corpus. To compute the concept vector(s), the QA system may process statistics of associations in the concept sequences using vector embedding methods. However generated, the concept vectors may be processed to identify teams or groups of individuals by providing the ability to generate vectors representing different individuals and to compare the “individual” vectors to one or more concept vectors on the basis of similarity metric distances therebetween to identify a “team” of individuals having skills or knowledge that is related to the set of concepts.

DETAILED DESCRIPTION

The present invention may be a system, a method, and/or a computer program product. In addition, selected aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and/or hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, aspects of the present invention may take the form of computer program product embodied in a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.

FIG. 1depicts a schematic diagram of one illustrative embodiment of a question/answer (QA) system100connected to a computer network102in which the QA system100uses a vector concept engine11to extract concept vectors from a knowledge database108and uses a vector processing application14to generate team recommendations from the extracted concept vectors. The QA system100may include one or more QA system pipelines100A,100B, each of which includes a knowledge manager computing device104(comprising one or more processors and one or more memories, and potentially any other computing device elements generally known in the art including buses, storage devices, communication interfaces, and the like) for processing questions received over the network102from one or more users at computing devices (e.g.,110,120,130). Over the network102, the computing devices communicate with each other and with other devices or components via one or more wired and/or wireless data communication links, where each communication link may comprise one or more of wires, routers, switches, transmitters, receivers, or the like. In this networked arrangement, the QA system100and network102may enable question/answer (QA) generation functionality for one or more content users. Other embodiments of QA system100may be used with components, systems, sub-systems, and/or devices other than those that are depicted herein.

In the QA system100, the knowledge manager104may be configured to receive inputs from various sources. For example, knowledge manager104may receive input from the network102, one or more knowledge bases or corpora of electronic documents108which stores electronic documents103, semantic data105, or other possible sources of data input. In selected embodiments, the knowledge database108may include structured, semi-structured, and/or unstructured content in a plurality of documents that are contained in one or more large knowledge databases or corpora. The various computing devices (e.g.,110,120,130) on the network102may include access points for content creators and content users. Some of the computing devices may include devices for a database storing the corpus of data as the body of information used by the knowledge manager104to generate answers to questions. The network102may include local network connections and remote connections in various embodiments, such that knowledge manager104may operate in environments of any size, including local and global, e.g., the Internet. Additionally, knowledge manager104serves as a front-end system that can make available a variety of knowledge extracted from or represented in documents, network-accessible sources and/or structured data sources. In this manner, some processes populate the knowledge manager, with the knowledge manager also including input interfaces to receive knowledge requests and respond accordingly.

In one embodiment, the content creator creates content in electronic documents103for use as part of a corpus of data with knowledge manager104. Content may also be created and hosted as information in one or more external sources17-19, whether stored as part of the knowledge database108or separately from the QA system100A. Wherever stored, the content may include any file, text, article, or source of data (e.g., scholarly articles, dictionary definitions, encyclopedia references, and the like) for use in knowledge manager104. Content users may access knowledge manager104via a network connection or an Internet connection to the network102, and may input questions to knowledge manager104that may be answered by the content in the corpus of data. As further described below, when a process evaluates a given section of a document for semantic content105, the process can use a variety of conventions to query it from the knowledge manager. One convention is to send a question10. Semantic content is content based on the relation between signifiers, such as words, phrases, signs, and symbols, and what they stand for, their denotation, or connotation. In other words, semantic content is content that interprets an expression, such as by using Natural Language (NL) Processing. In one embodiment, the process sends well-formed questions10(e.g., natural language questions, etc.) to the knowledge manager104. Knowledge manager104may interpret the question and provide a response to the content user containing one or more answers20to the question10. In some embodiments, knowledge manager104may provide a response to users in a ranked list of answers20.

In some illustrative embodiments, QA system100may be the IBM Watson™ QA system available from International Business Machines Corporation of Armonk, N.Y., which is augmented with the mechanisms of the illustrative embodiments described hereafter for identifying and processing concept vectors which may aid in the process of answering questions. The IBM Watson™ knowledge manager system may receive an input question10which it then parses to extract the major features of the question, that in turn are used to formulate queries that are applied to the corpus of data stored in the knowledge base108. Based on the application of the queries to the corpus of data, a set of hypotheses, or candidate answers to the input question, are generated by looking across the corpus of data for portions of the corpus of data that have some potential for containing a valuable response to the input question.

In particular, a received question10may be processed by the IBM Watson™ QA system100which performs deep analysis on the language of the input question10and the language used in each of the portions of the corpus of data found during the application of the queries using a variety of reasoning algorithms. There may be hundreds or even thousands of reasoning algorithms applied, each of which performs different analysis, e.g., comparisons, and generates a score. For example, some reasoning algorithms may look at the matching of terms and synonyms within the language of the input question and the found portions of the corpus of data. Other reasoning algorithms may look at temporal or spatial features in the language, while others may evaluate the source of the portion of the corpus of data and evaluate its veracity.

The scores obtained from the various reasoning algorithms indicate the extent to which the potential response is inferred by the input question based on the specific area of focus of that reasoning algorithm. Each resulting score is then weighted against a statistical model. The statistical model captures how well the reasoning algorithm performed at establishing the inference between two similar passages for a particular domain during the training period of the IBM Watson™ QA system. The statistical model may then be used to summarize a level of confidence that the IBM Watson™ QA system has regarding the evidence that the potential response, i.e., candidate answer, is inferred by the question. This process may be repeated for each of the candidate answers until the IBM Watson™ QA system identifies candidate answers that surface as being significantly stronger than others and thus, generates a final answer, or ranked set of answers, for the input question. The QA system100then generates an output response or answer20with the final answer and associated confidence and supporting evidence. More information about the IBM Watson™ QA system may be obtained, for example, from the IBM Corporation website, IBM Redbooks, and the like. For example, information about the IBM Watson™ QA system can be found in Yuan et al., “Watson and Healthcare,” IBM developerWorks, 2011 and “The Era of Cognitive Systems: An Inside Look at IBM Watson and How it Works” by Rob High, IBM Redbooks, 2012.

To improve the quality of answers provided by the QA system100, the concept vector engine11may be embodied as part of a QA information handling system16in the knowledge manager104, or as a separate information handling system, to execute a concept vector identification process that extracts a sequence of concepts from annotated text sources17(e.g., sources specializing in concepts, such as Wikipedia pages with concepts highlighted or hyperlinked), from graph representations18of concepts and their inter-relations, from tracking the navigation behavior of users19, or a combination thereof, and to construct therefrom one or more vectors for each concept107. Syntactically, a “concept” is a single word or a word sequence (e.g., “gravity”, “supreme court”, “Newton's second law”, “Albert Einstein”) which becomes a semantic “concept” once it has been designated by a community to have a special role, namely—as representing more than just a sequence of words. In addition, a concept has many attributes: field of endeavor, origin, history, an associated body of work and/or knowledge, cultural and/or historical connotation and more. So, although superficially, words, phrases and concepts seem similar, a word sequence becomes a concept when it embeds a wider cultural context and a designation by a community, encompassing a significant meaning and presence in an area, in a historical context, in its relationships to other concepts and in ways it influences events and perceptions. It is worth emphasizing the point that not every well-known sequence of words is a concept, and the declaration of a sequence of words to be a concept is a community decision which has implications regarding naturally-arising sequences of concepts. With this understanding, the concept vector engine11may include a concept sequence identifier12, such as an annotator, which accesses sources17-19for sequences of concepts embedded in texts of various kinds and/or which arise by tracking concept exploration behavior from examining non-text sources, such as click streams. As different concept sequences are identified, the adjacency of the concepts is tied to the closeness of the concepts themselves. Once concept sequences are available, a concept vector extractor13acts as a learning device to extract vector representations for the identified concepts. The resulting concept vectors107may be stored in the knowledge database108or directly accessed by one or more vector processing applications14which may be executed, for example, to construct concept vectors corresponding to the skills and interests of different individuals, and then to process theses concept vectors with reference to one or more reference concept vectors to identify a team of individuals based on their distances to the reference concept vector(s) rather than to each other.

To identify or otherwise obtain a sequence of concepts, a concept sequence identifier12may be provided to (i) access one or more wiki pages17or other text source which contains these concepts by filtering out words that are not concepts, (ii) algorithmically derive concept sequences from a graph18(e.g., a Concept Graph (CG)), (iii) track one or more actual users' navigation behavior19over concepts, or some modification or combination of one of the foregoing. For example, the concept sequence identifier12may be configured to extract concepts from a text source, but also some text words extracted per concept in the context surrounding the concept's textual description, in which case the concepts are “converted” to new unique words.

To provide a first illustrative example, the concept sequence identifier12may be configured to derive concept sequences12A from one or more Wikipedia pages17by eliminating all words from a page that are not concepts (i.e., Wikipedia entries). For example, consider the following snippet from the Wikipedia page for Photonics at http://en.wikipedia.org/wiki/Photonics in which the concepts are underlined:Photonics as a field began with the invention of the laser in 1960. Other developments followed: the laser diode in the 1970s, optical fibers for transmitting information, and the erbium-doped fiber amplifier. These inventions formed the basis for the telecommunications revolution of the late 20th century and provided the infrastructure for the Internet.Though coined earlier, the term photonics came into common use in the 1980s as fiber-optic data transmission was adopted by telecommunications network operators. At that time, the term was used widely at Bell Laboratories. Its use was confirmed when the IEEE Lasers and Electro-Optics Society established an archival journal named Photonics Technology Letters at the end of the 1980s.During the period leading up to the dot-com crash circa 2001, photonics as a field focused largely on optical telecommunications.

In this example, the concept sequence12A derived by the concept sequence identifier12is: laser, laser diode, optical fibers, erbium-doped fiber amplifier, Internet, Bell Laboratories, IEEE Lasers and Electro-Optics Society, Photonics Technology Letters, dot-com crash. However, it will be appreciated that the concept sequence identifier12may examine a “dump” of Wikipedia pages17to obtain long concept sequences reflecting the whole collection of Wikipedia concepts.

In another illustrative example, the concept sequence identifier12may be configured to derive concept sequences12A from one or more specific domains. For example, a pharmaceutical company's collection of concerned diseases, treatments, drugs, laboratory tests, clinical trials, relevant chemical structures and processes, or even biological pathways may be accessed by the concept sequence identifier12to extract domain-specific concept sequences. In this example, concept sequences may be extracted from company manuals, emails, publications, reports, and other company-related text sources.

In another illustrative example, the concept sequence identifier12may be configured to derive concept sequences12A which also include non-concept text. For example, an identified concept sequence may include inserted “ordinary” or non-concept words which are used for learning. One option would be to use all the words from the original source text by converting “concept” words into “new” words by appending a predetermined suffix (e.g., “_01”) to each concept. In the example “Photonics” page listed above, this approach would lead to the following first paragraph: “Photonics as a field began with the invention of the laser 01 in 1960. Other developments followed: the laser diode 01 in the 1970s, optical fibers 01 for transmitting information, and the erbium-doped fiber amplifier 01. These inventions formed the basis for the telecommunications revolution of the late 20th century and provided the infrastructure for the Internet 01.”

Another option for deriving concept sequences with text would be to process the original source text by a filtering process that retains only the parts of the text relevant to a specific theme. For example, if the original source text consists of a collection of medical documents, a search procedure can be applied to identify and retrieve only the documents containing the word “cancer.” The retrieved documents are taken as the theme-restricted collection for deriving the concept sequences.

Another option for deriving concept sequences with text would be to process the original source text to keep only words that are somewhat infrequent as indicated by an occurrence threshold, and that are in close proximity to a concept. In the example “Photonics” page listed above, this approach would lead to the following first paragraph: “invention laser 01 1960. developments laser diode 01 1970s, optical fibers 01 transmitting information erbium-doped fiber amplifier 01 telecommunications revolution infrastructure Internet 01.”

Another option for deriving concept sequences is to construct sequences of concepts and words in units and (potentially rearranged) orderings, as determined by a natural language parser.

Another option for deriving concept sequences with text would be to explicitly specify a collection of words or types of words to be retained in the concept sequence. For example, one may have a specified collection of words connected to medicine (e.g., nurse, doctor, ward and operation), and the derived concept sequence would limit retained non-concept words or text to this specified collection.

To provide a second illustrative example of the concept sequence identifier process, the concept sequence identifier12may be configured to derive concept sequences (e.g.,12A) from one or more concept graphs18having nodes which represent concepts (e.g., Wikipedia concepts). As will be appreciated, a graph18may be constructed by any desired method (e.g., Google, etc.) to define “concept” nodes which may be tagged with weights indicating their relative importance. In addition, an edge of the graph is labeled with the strength of the connection between the concept nodes it connects. When edge weights are given, they indicate the strength or closeness of these concepts, or observed and recorded visits by users in temporal proximity. An example way of relating the edge weights to user visits is to define the edge weight connecting concept “A” to concept “B” to be the number of times users examined concept “A” and, within a short time window, examined concept “B”.

Using the Wikipedia example, if a Wikipedia page “A” has a link to another Wikipedia page “B,” then the graph18would include an edge connecting the “A” concept to the “B” concept. The weight of a node (importance) or the weight (strength) of an edge of an edge may be derived using any desired technique, such as a personalized Pagerank of the graph or other techniques. In addition, each concept i in the graph18may be associated with a (high dimensional) P-vector such that the jthentry of the P-vector corresponding to concept i is the strength of the connection between concept i and concept j. The entries of the P-vector may be used to assign weights to graph edges. To derive concept sequences from the concept graph(s)18, the concept sequence identifier12may be configured to perform random walks on the concept graph(s)18and view these walks as concept sequences. For example, starting with a randomly chosen starting node v, the concept sequence identifier12examines the G-neighbors of v and the weights on the edges connecting v and its neighboring nodes. Based on the available weights (if none are available, the weights are considered to be equal), the next node is randomly chosen to identify the next node (concept) in the sequence where the probability to proceed to a node depends on the edge weight and the neighboring node's weight relative to other edges and neighboring nodes. This random walk process may be continued until a concept sequence of length H is obtained, where H may be a specified parametric value (e.g., 10,000). Then, the random walk process may be repeated with a new randomly selected starting point. If desired, the probability of selecting a node as a starting node may be proportional to its weight (when available). The result of a plurality of random walks on the graph18is a collection of length H sequences of concepts12A.

Extracting sequences from the concept graph(s)18may also be done by using a random walk process in which each step has a specified probability that the sequence jumps back to the starting concept node (a.k.a., “teleportation”), thereby mimicking typical navigation behavior. Alternatively, a random walk process may be used in which each step has a specified probability that the sequence jumps back to the previous concept node, thereby mimicking other typical navigation behavior. If desired, a combination of the foregoing step sequences may be used to derive a concept sequence. Alternatively, a concept sequence may be derived by using a specified user behavior model M that determines the next concept to explore. Such a model M may employ a more elaborate scheme in order to determine to which concept a user will examine next, based on when previous concepts were examined and for what duration.

The resulting concept sequences12A may be stored in the knowledge database108or directly accessed by the concept vector extractor13. In addition, whenever changes are made to a concept graph18, the foregoing process may be repeated to dynamically maintain concept sequences by adding new concept sequences12A and/or removing obsolete ones. By revisiting the changed concept graph18, previously identified concept sequences can be replaced with new concept sequences that would have been used, thereby providing a controlled time travel effect.

In addition to extracting concepts from annotated text17and/or graph representations18, concept sequences12A may be derived using graph-based vector techniques whereby an identified concept sequence12A also includes a vector representation of the concept in the context of graph G (e.g., Pagerank-derived vectors). This added information about the concepts in the sequence12A can be used to expedite and qualitatively improve the learning of parameters process, and learning quality, by providing grouping, i.e., additional information about concepts and their vicinity as embedded in these G-associated vectors.

To provide a third illustrative example of the concept sequence identifier process, the concept sequence identifier12may be configured to derive concept sequences (e.g.,12A) from the user navigation behavior19where selected pages visited by a user (or group of users) represent concepts. For example, the sequences of concepts may be the Wikipedia set of entries explored in succession by (a) a particular user, or (b) a collection of users. The definition of succession may allow non-Wikipedia intervening web exploration either limited by duration T (before resuming Wikipedia), number of intervening non-Wikipedia explorations, or a combination of theses or related criteria. As will be appreciated, user navigation behavior19may be captured and recorded using any desired method for tracking a sequence of web pages a user visits to capture or retain the “concepts” corresponding to each visited page and to ignore or disregard the pages that do not correspond to concepts. Each concept sequence12A derived from the captured navigation behavior19may correspond to a particular user, and may be concatenated or combined with other user's concept sequences to obtain a long concept sequence for use with concept vector training. In other embodiments, the navigation behavior of a collection of users may be tracked to temporally record a concept sequence from all users. While such collective tracking blurs the distinction between individual users, this provides a mechanism for exposing a group effort. For example, if the group is a limited-size departmental unit (say, up to 20), the resulting group sequence12A can reveal interesting relationships between the concepts captured from the user navigation behavior19. The underlying assumption is that the group of users is working on an interrelated set of topics.

To provide another illustrative example of the concept sequence identifier process, the concept sequence identifier12may be configured to generate concept sequences using concept annotations created by two or more different annotators, where each annotator uses its chosen set of names to refer to the collection of concepts included in a text source. For example, one annotator applied to a text source may mark up all occurrences of the concept of “The United States of America” as “U.S.A.”, whereas another may mark it up as “The United States”. In operation, a first concept sequence may be generated by extracting a first plurality of concepts from a first set of concept annotations for the one or more content sources, and a second concept sequence may be generated by extracting a second plurality of concepts from a second set of concept annotations for the one or more content sources. In this way, the concept sequence identifier12may be used to bring together different annotated versions of a corpus. In another example, a first set of concept annotations may be a large collection of medical papers that are marked up with concepts that are represented in the Unified Medical Language System (UMLS) Metathesaurus. The second set of concept annotations may the same collection of medical papers that are marked up with concepts that are defined in the English Wikipedia. Since these two dictionaries have good overlap but they are not identical, they may refer to the same thing (e.g., leukemia) differently in the different sets of concept annotations.

In addition to identifying concept sequences12A from one or more external sources17-19, general concept sequences may be constructed out of extracted concept sequences. For example, previously captured concept sequences106may include a plurality of concept sequences S1, S2, . . . , Sm which originate from various sources. Using these concept sequences, the concept sequence identifier12may be configured to form a long sequence S by concatenating the sequences S=S1S2. . . Sm.

Once concept sequences12A are available (or stored106), a concept vector extractor13may be configured to extract concept vectors13A based on the collected concept sequences. For example, the concept vector extractor13may employ a vector embedding system (e.g., Neural-Network-based, matrix-based, log-linear classifier-based or the like) to compute a distributed representation (vectors) of concepts13A from the statistics of associations embedded within the concept sequences12A. More generally, the concept vector extractor13embodies a machine learning component which may use Natural Language Processing or other techniques to receive concept sequences as input. These sequences may be scanned repeatedly to generate a vector representation for each concept in the sequence by using a method, such as word2vec. Alternatively, a matrix may be derived from these sequences and a function is optimized over this matrix and word vectors, and possibly context vectors, resulting in a vector representation for each concept in the sequence. Other vector generating methods, such as using Neural Networks presented by a sequence of examples derived from the sequences, are possible. The resulting concept vector may be a low dimension (about 100-300) representation for the concept which can be used to compute the semantic and/or grammatical closeness of concepts, to test for analogies (e.g., “a king to a man is like a queen to what?”) and to serve as features in classifiers or other predictive models. The resulting concept vectors13A may be stored in the knowledge database107or directly accessed by one or more vector processing applications14.

To generate concept vectors13A, the concept vector extractor13may process semantic information or statistical properties deduced from word vectors extracted from the one or more external sources17-19. To this end, the captured concept sequences12A may be directed to the concept vector extraction function or module13which may use Natural Language Processing (NLP) or machine learning processes to analyze the concept sequences12A to construct one or more concept vectors13A, where “NLP” refers to the field of computer science, artificial intelligence, and linguistics concerned with the interactions between computers and human (natural) languages. In this context, NLP is related to the area of human-to-computer interaction and natural language understanding by computer systems that enable computer systems to derive meaning from human or natural language input. To process the concept sequences12A, the concept vector extractor13may include a learning or optimization component which receives concept sequence examples12A as Neural Network examples, via scanning text, and the like. In the learning component, parameters (Neural Network weights, matrix entries, coefficients in support vector machines (SVMs), etc.) are adjusted to optimize a desired goal, usually reducing an error or other specified quantity. For example, the learning task in the concept vector extractor13may be configured to implement a scanning method where learning takes place by presenting examples from a very large corpus of Natural Language (NL) sentences. The examples may be presented as Neural Network examples, in which the text is transformed into a sequence of examples where each example is encoded in a way convenient for the Neural Network intake, or via scanning text where a window of text is handled as a word sequence with no further encoding. In scanning methods, the learning task is usually to predict the next concept in a sequence, the middle concept in a sequence, concepts in the context looked at as a “bag of words,” or other similar tasks. The learning task in the concept vector extractor13may be also configured to implement a matrix method wherein text characteristics are extracted into a matrix form and an optimization method is utilized to minimize a function expressing desired word vector representation. The learning results in a matrix (weights, parameters) from which one can extract concept vectors, or directly in concept vectors (one, or two per concept), where each vector Vi is associated with a corresponding concept Ci. Once the learning task is complete, the produced concept vectors may have other usages such as measuring “closeness” of concepts (usually in terms of cosine distance) or solving analogy problems of the form “a to b is like c to what?”

To provide a first illustrative example for computing concept vectors from concept sequences, the concept vector extractor13may be configured to employ vector embedding techniques (e.g., word2vec or other matrix factorization and dimensionality reduction techniques, such as NN, matrix-based, log-linear classifier or the like) whereby “windows” of k (e.g., 5-10) consecutive concepts are presented and one is “taken out” as the concept to be predicted. The result is a vector representation for each concept. Alternatively, the concept vector extractor13may be configured to use a concept to predict its neighboring concepts, and the training result produces the vectors. As will be appreciated, other vector producing methods may be used. Another interesting learning task by which vectors may be created is that of predicting the next few concepts or the previous few concepts (one sided windows).

To provide another illustrative example for computing concept vectors13A from concept sequences12A, the concept vector extractor13may be configured to employ NLP processing techniques to extract a distributed representation of NLP words and obtain vectors for the concept identifiers. As will be appreciated, the size of the window may be larger than those used in the NLP applications so as to allow for concepts to appear together in the window. In addition, a filter F which can be applied to retain non-concept words effectively restricts the words to only the ones that have a strong affinity to their nearby concepts as measured (for example, by their cosine distance to the concept viewed as a phrase in an NLP word vector production, e.g., by using word2vec).

To provide another illustrative example for computing concept vectors13A from concept sequences12A, the concept vector extractor13may be configured to employ NLP processing techniques to generate different concept vectors from different concept sequences by supplying a first plurality of concepts (extracted from a first set of concept annotations) as input to the vector learning component to generate the first concept vector and by supplying a second plurality of concepts (extracted from a second set of concept annotations) as input to the vector learning component to generate a second concept vector. If both versions of concept sequence annotations are brought together to obtain first and second concept vectors, the resulting vectors generated from the different concept sequence annotations can be compared to one another by computing similarities therebetween. As will be appreciated, different annotators do not always mark up the same text spans in exactly the same way, and when different annotation algorithms choose to mark up different occurrences of the term, a direct comparison of the resulting concept vectors just by text alignment techniques is not trivial. However, if both versions of annotated text sources are included in the embedding process, by way of association with other concepts and non-concept words, the respective concept vectors can be brought to close proximity in the embedding space. Computing similarities between the vectors could reveal the linkage between such alternative annotations.

Once concept vectors13A are available (or stored107), they can be manipulated in order to answer questions such as “a king is to man is like a queen is to what?”, cluster similar words based on a similarity measure (e.g., cosine distance), or use these vectors in other analytical models such as a classification/regression model for making various predictions. For example, one or more vector processing applications14may be applied to carry out useful tasks in the domain of concepts and user-concept interaction, allowing better presentation and visualization of concepts and their inter-relations (e.g., hierarchical presentation, grouping, and for a richer and more efficient user navigation over the concept graph). For example, an application14may access n vectors V1, . . . , Vn of dimension d which represent n corresponding concepts C1, . . . , Cn, where a vector Vi is a tuple (vi1, . . . , vid) of entries where each entry is a real number. Concept vector processing may include using a similarity calculation engine15to calculate a similarity metric value between (1) one or more concepts (or nodes) in an extracted concept sequence (e.g.,106) and/or (2) one or more extracted concept vectors (e.g.,107). Such concept/vector processing at the similarity calculation engine15may include the computation of the dot product of two vectors Vh and Vi, denoted dot(Vh,Vi) is Σj=1, . . . , d Vhj*Vij. In concept vector processing, the length of vector Vi is defined as the square root of dot(Vi,Vi), i.e., SQRT(dot(Vi,Vi)). In addition, concept vector processing at the similarity calculation engine15may include computation of the cosine distance between Vh and Vi, denoted cos(Vh,Vi), is dot(Vh,Vi)/(length(Vh)*length(Vi)). The cosine distance is a measure of similarity, where a value of “1” indicates very high similarity and a value of “−1” indicates very weak similarity. As will be appreciated, there are other measures of similarity that may be used to process concept vectors, such as soft cosine similarity. In addition, it will be appreciated that the concept vector processing may employ the similarity calculation engine15as part of the process for extracting concept sequences12, as part of the process of concept vector extraction13, or as concept vector processing step for constructing team recommendations.

To provide a first illustrative example application for processing concept vectors13A, a vector processing application14may be configured to form teams of researchers/employees by obtaining the Wikipedia concepts in which each is interested and forming a group in which the total mutual interests as derived from the closeness of the involved concepts is high. For example, information is collected to identify the interests of a group of users, such as by using each user's homepage, navigation behavior, emails, etc. For each person p, the concept vector engine11and/or vector processing application14may be configured to form a vector Vp representing that person's interests in concepts. For example, Vp may be formed by adding the concept vectors corresponding to p's interests which may be weighted based on priority information (or weighted equally if priority information is unavailable) such that the weights sum up to 1. The collection of person vectors Vp may then be clustered, such as by using a K-means algorithm or another clustering method, into U classes, where U is a programmable parameter. Subsequently, the vector processing application14may be configured to assign each person p to the group represented by the cluster to which p's vector Vp belongs. The team formation application may also be used to form teams dealing with different specified concepts. For example, to choose N individuals for a group G dealing primarily with ConceptA (weight 0.5) but also with ConceptB (weight 0.25) and ConceptC (weight 0.25), the vector processing application14may be configured to construct a vector VG=0.5 VconceptA+0.25 VconceptB+0.25 VconceptC, and then choose the N persons each of whose vector's cosine distance to VG is maximal.

To provide another illustrative example application for processing concept vectors13A, a vector processing application14may be configured to form teams from a set of individuals or persons who are each associated with a document and a vector having an associated concept. In these embodiments, a list of candidates for a focused team of individuals with similar skills is obtained by using the concept vector engine11and/or vector processing application14to form a vector Vp representing each individual's interests in concepts. In constructing the concept vector for each individual, the concept vector engine11may be configured to associate one dimension to each concept and assign the correspondent vector in the canonical basis, i.e., a vector where all entries are 0 except the one corresponding to the dimension associated with the concept, which is set to 1. Upon selecting or identifying a key concept that should be familiar or known to all individuals in the focused team, the vector processing application14may be configured to search for vectors that are most similar to the key concept vector among all vectors associated with the individuals. For example, the vectors associated with the individuals may be processed using a recommendation system, such as collaborative filtering techniques (e.g., the Slope One method), such as by using the number of mentions of a concept in the individual's document as a proxy for a user rating. In this case, the vector associated with an individual is the vector of predicted ratings for all the concepts. While a similarity measure, such as cosine similarity, can be used to evaluate the individual vectors against the key concept vector, but it will be appreciated that other methods for the construction of the concept and individuals vectors can be used, as well as different similarity measures.

To provide another illustrative example application for processing concept vectors13A, a vector processing application14may be configured to form teams from a set of individuals or persons by taking into account specific characteristics or parameters values of each individual. In these embodiments, a list of candidates for a team of individuals may be obtained by using the concept vector engine11and/or vector processing application14to form a vector Vp representing each individual's specific characteristics or parameters, such as for example, language spoken, geographical location, psychological compatibility indicator factors, personality traits, and/or availability. In these embodiments, a project description would be specified as a project concept vector for the project the team is to work on, where the project concept vector is derived from project description topics or concepts to be covered by the team and additional information such as a project schedule. With a specified project concept vector, the vector processing application14and/or similarity calculation engine15may be configured to select a list of candidate team members based on the associated individual vectors Vp that have the highest similarities to the concepts in the project concept vector, and then perform a re-ranking of the candidates based on the additional parameters. Re-ranking can be performed through on-off methods, such as requirements on certain maximum geographical distance from a geographical location. This would lead to an effective pruning of the list of candidates. Re-ranking can also be performed using soft metrics, such as overlap of available time on the candidate's calendar and the candidates. In this case, the various metrics may be combined through a combination function, such as a linear convex combination, to form a new ranking of the candidates.

To provide another illustrative example application for processing concept vectors13A, a vector processing application14may be configured to form a team for a defined project by selecting a set of individuals or persons whose skills or experience provide full coverage to the set of concepts specified by the project description. In these embodiments, a list of candidate team members is obtained by using the concept vector engine11and/or vector processing application14to form a vector Vp representing each individual's skills or experience in concepts. In addition, a project description would be specified as a project concept vector for the project the team is to work on, where the project concept vector is derived from project description topics or concepts to be covered by the team and additional information such as a project schedule. With a specified project concept vector, the vector processing application14and/or similarity calculation engine15may be configured to select a team of individuals having the highest similarities to the concepts in the project concept vector, where the similarity function is applied to candidate teams as opposed to candidate individuals. For example, the vector processing application14may be configured to compute a team similarity metric by computing, for each team member, a similarity metric between the member and each of the project concepts; assigning the top scoring team member to the concept where the team member scores highest; removing the team member from the team; and the continuing with the process until all concepts from the project concept vector have been assigned a team member. The overall team similarity metric is the minimum of such scores. Demanding that a candidate team has the overall team similarity metric passing a pre-selected threshold would provide full coverage of each topic in the project, guaranteeing at least one person per concept.

To provide another illustrative example application for processing concept vectors13A, a vector processing application14may be configured to form multiple teams to work on multiple projects, where each project-role and each individual can be represented by a list of key concepts. In these embodiments, the individual and project concept vectors may be derived as described hereinabove. In addition, the vector processing application14and/or similarity calculation engine15may be configured to compute a matching score between each person/project-role pair using the vectors representing the relevant concepts, and to apply a rule that takes into account the importance of the concepts to the individual/project-role. To illustrate the application of a rule, consider an example where an individual has strong expertise in areas represented by concepts A and B, and where a project-role has needs in areas represented by concepts P, Q, and R. In this example, the vector processing application14could be configured to score the match by applying the rule [max_(X in {P, Q, R}) cos(A,X)+max_(Y in {P, Q, R}−{X}) cos(B,Y)]/2. Other rules of a similar nature could be designed to account for other factors (e.g., relative levels of expertise, relative levels of demand, and for score normalization). According to matching scores between the individuals and the project-roles, each individual may have a preference order on which project-role the individual most wishes to take. Likewise, each project-role may have a preference order for which individual(s) to employ. An optimal assignment of individuals to project-roles can then be computed using any desired matching algorithm (e.g., the Gale-Shapley algorithm for the stable matching/marriage problem). Alternative optimization criteria and other constraints can be imposed on the assignment that may turn the problem into one of its several variants (e.g., the standard assignment problem, the hospitals/residents problem, or the hospitals/residents problem with couples). These can be solved using established optimal or heuristic algorithms.

Types of information handling systems that can use the QA system100range from small handheld devices, such as handheld computer/mobile telephone110to large mainframe systems, such as mainframe computer170. Examples of handheld computer110include personal digital assistants (PDAs), personal entertainment devices, such as MP3 players, portable televisions, and compact disc players. Other examples of information handling systems include a pen or tablet computer120, laptop or notebook computer130, personal computer system150, and server160. As shown, the various information handling systems can be networked together using computer network102. Types of computer network102that can be used to interconnect the various information handling systems include Local Area Networks (LANs), Wireless Local Area Networks (WLANs), the Internet, the Public Switched Telephone Network (PSTN), other wireless networks, and any other network topology that can be used to interconnect the information handling systems. Many of the information handling systems include nonvolatile data stores, such as hard drives and/or nonvolatile memory. Some of the information handling systems may use separate nonvolatile data stores (e.g., server160utilizes nonvolatile data store165, and mainframe computer170utilizes nonvolatile data store175). The nonvolatile data store can be a component that is external to the various information handling systems or can be internal to one of the information handling systems.

FIG. 2illustrates an illustrative example of an information handling system200, more particularly, a processor and common components, which is a simplified example of a computer system capable of performing the computing operations described herein. Information handling system200includes one or more processors210coupled to processor interface bus212. Processor interface bus212connects processors210to Northbridge215, which is also known as the Memory Controller Hub (MCH). Northbridge215connects to system memory220and provides a means for processor(s)210to access the system memory. In the system memory220, a variety of programs may be stored in one or more memory device, including a team formation engine module221which may be invoked to extract concept vectors from candidate individuals and one or more project concepts and to construct therefrom concept vectors which may compared and processed to identify team groups of individuals based on the generation and manipulation of similarity metrics. Graphics controller225also connects to Northbridge215. In one embodiment, PCI Express bus218connects Northbridge215to graphics controller225. Graphics controller225connects to display device230, such as a computer monitor.

Northbridge215and Southbridge235connect to each other using bus219. In one embodiment, the bus is a Direct Media Interface (DMI) bus that transfers data at high speeds in each direction between Northbridge215and Southbridge235. In another embodiment, a Peripheral Component Interconnect (PCI) bus connects the Northbridge and the Southbridge. Southbridge235, also known as the I/O Controller Hub (ICH) is a chip that generally implements capabilities that operate at slower speeds than the capabilities provided by the Northbridge. Southbridge235typically provides various busses used to connect various components. These busses include, for example, PCI and PCI Express busses, an ISA bus, a System Management Bus (SMBus or SMB), and/or a Low Pin Count (LPC) bus. The LPC bus often connects low-bandwidth devices, such as boot ROM296and “legacy” I/O devices (using a “super I/O” chip). The “legacy” I/O devices (298) can include, for example, serial and parallel ports, keyboard, mouse, and/or a floppy disk controller. Other components often included in Southbridge235include a Direct Memory Access (DMA) controller, a Programmable Interrupt Controller (PIC), and a storage device controller, which connects Southbridge235to nonvolatile storage device285, such as a hard disk drive, using bus284.

ExpressCard255is a slot that connects hot-pluggable devices to the information handling system. ExpressCard255supports both PCI Express and USB connectivity as it connects to Southbridge235using both the Universal Serial Bus (USB) and the PCI Express bus. Southbridge235includes USB Controller240that provides USB connectivity to devices that connect to the USB. These devices include webcam (camera)250, infrared (IR) receiver248, keyboard and trackpad244, and Bluetooth device246, which provides for wireless personal area networks (PANs). USB Controller240also provides USB connectivity to other miscellaneous USB connected devices242, such as a mouse, removable nonvolatile storage device245, modems, network cards, ISDN connectors, fax, printers, USB hubs, and many other types of USB connected devices. While removable nonvolatile storage device245is shown as a USB-connected device, removable nonvolatile storage device245could be connected using a different interface, such as a Firewire interface, etc.

Wireless Local Area Network (LAN) device275connects to Southbridge235via the PCI or PCI Express bus272. LAN device275typically implements one of the IEEE 802.11 standards for over-the-air modulation techniques to wireless communicate between information handling system200and another computer system or device. Extensible Firmware Interface (EFI) manager280connects to Southbridge235via Serial Peripheral Interface (SPI) bus278and is used to interface between an operating system and platform firmware. Optical storage device290connects to Southbridge235using Serial ATA (SATA) bus288. Serial ATA adapters and devices communicate over a high-speed serial link. The Serial ATA bus also connects Southbridge235to other forms of storage devices, such as hard disk drives. Audio circuitry260, such as a sound card, connects to Southbridge235via bus258. Audio circuitry260also provides functionality such as audio line-in and optical digital audio in port262, optical digital output and headphone jack264, internal speakers266, and internal microphone268. Ethernet controller270connects to Southbridge235using a bus, such as the PCI or PCI Express bus. Ethernet controller270connects information handling system200to a computer network, such as a Local Area Network (LAN), the Internet, and other public and private computer networks.

WhileFIG. 2shows one example configuration for an information handling system200, an information handling system may take many forms, some of which are shown inFIG. 1. For example, an information handling system may take the form of a desktop, server, portable, laptop, notebook, or other form factor computer or data processing system. In addition, an information handling system may take other form factors such as a personal digital assistant (PDA), a gaming device, ATM machine, a portable telephone device, a communication device or other devices that include a processor and memory. In addition, an information handling system need not necessarily embody the north bridge/south bridge controller architecture, as it will be appreciated that other architectures may also be employed. In additional embodiments, the system can be implemented in a computing cloud environment, where the various elementary operations of the invention are performed by servers interconnected through a computer network. Each operation implemented by a specific type of server and the number of such kind of servers running in the cloud possibly controlled to be proportional to the load of the system.

To provide additional details for an improved understanding of selected embodiments of the present disclosure, reference is now made toFIG. 3which depicts a simplified flow chart300showing the logic for obtaining and using a distributed representation of concepts as vectors. The processing shown inFIG. 3may be performed in whole or in part by a cognitive system, such as the QA information handing system15, QA system100, or other natural language question answering system which identifies sequences of concepts to extract concept vectors (e.g., distributed representations of the concept) which may be processed to carry out useful tasks in the domain of concepts and user-concept interaction.

FIG. 3processing commences at301whereupon, at step302, a question or inquiry from one or more end users is processed to generate an answer with associated evidence and confidence measures for the end user(s), and the resulting question and answer interactions are stored in an interaction history database. The processing at step302may be performed at the QA system100or other NLP question answering system, though any desired information processing system for processing questions and answers may be used. As described herein, a Natural Language Processing (NLP) routine may be used to process the received questions and/or generate a computed answer with associated evidence and confidence measures. In this context, NLP is related to the area of human-computer interaction and natural language understanding by computer systems that enable computer systems to derive meaning from human or natural language input.

In the course of processing questions to generate answers, a collection or sequence of concepts may be processed at step310. The concept sequence processing at step310may be performed at the QA system100or concept vector engine13by employing NLP processing and/or extraction algorithms, machine learning techniques, and/or manual processing to collect concepts from one or more external sources (such as the Wikipedia or some other restricted domain, one or more concept graph sources, and/or captured user navigation behavior) to generate training input comprising concept sequences. As will be appreciated, one or more processing steps may be employed to obtain the concept sequences.

For example, the concept sequence processing at step310may employ one or more concept graphs to generate concept sequences at step303. To this end, the concept graph derivation step303may construct a graph G using any desired technique (e.g., a graph consisting of Wikipedia articles as nodes and the links between them as edges) to define concepts at each graph node which may be tagged with weights indicating its relative importance. In addition, the graph edges may be weighted to indicate concept proximity. By traversing the graph G using the indicated weights to affect the probability of navigating via an edge, a sequence of concepts may be constructed at step303. In contrast to existing approaches for performing short random walks on graph nodes which view these as sentences and extract a vector representation for each node, the graph derivation step303may employ a random walk that is directed by the edge weights such that there is a higher probability to traverse heavier weight edges, thereby indicating closeness of concepts. In addition, the concept graphs employed by the graph derivation step303encodes many distinct domains may be represented as graphs that are derived non-trivially from the conventional web graph. In addition, the graph derivation step303may allow a graph traversal with a “one step back” that is not conventionally available. As a result, the resulting concept vectors are quite different.

In addition or in the alternative, the concept sequence processing at step310may employ one or more text sources to extract concept sequences at step304. In selected embodiments, the text source is the Wikipedia set of entries or some other restricted domain. By analyzing a large corpus of documents mentioning Wikipedia entries (e.g., Wikipedia itself and other documents mentioning its entries), the text source extraction step304may extract the sequence of concepts, including the title, but ignoring all other text. In addition, the text source extraction step304may extract the sequence of appearing concepts along with additional words that are extracted with the concept in the context of surrounding its textual description while using a filter to remove other words not related to the extracted concepts. Alternatively, the text source extraction step304may extract a mixture of concepts and text by parsing a text source to identify concepts contained therein, replacing all concept occurrences with unique concept identifiers (e.g., by appending a suffix to each concept or associating critical words with concepts).

In addition or in the alternative, the concept sequence processing at step310may employ behavior tracking to derive concept sequences at step305. In selected embodiments, the actual user's navigation behavior is tracked to use the actual sequence of explored concepts by a single user or a collection of users to derive the concept sequence at step305. In selected embodiments, the tracking of user navigation behavior may allow non-Wikipedia intervening web exploration that is limited by duration T before resuming Wikipedia, by the number of intervening non-Wikipedia explorations, by elapsed time or a combination of these or related criteria.

After the concept sequence processing step310, the collected concept sequences may be processed to compute concept vectors using known vector embedding methods at step311. As disclosed herein, the concept vector computation processing at step311may be performed at the QA system100or concept vector extractor12by employing machine learning techniques and/or NLP techniques to compute a distributed representation (vectors) of concepts from the statistics of associations. As will be appreciated, one or more processing steps may be employed to compute the concept vectors. For example, the concept vector computation processing at step311may employ NL processing technique such as word2vec or to implement a neural network (NN) method at step306to perform “brute force” learning from training examples derived from concept sequences provided by step310. In addition or in the alternative, the concept vector computation processing at step311may employ various matrix formulations at method step307and/or extended with SVM-based methods at step308. In each case, the vector computation process may use a learning component in which selected parameters (e.g., NN weights, matrix entries, vector entries, etc.) are repeatedly adjusted until a desired level of learning is achieved.

After the concept vector computation processing step311, the computed concept vectors may be used in various applications at step312which may be performed at the QA system100or the concept vector application module14by employing NLP processing, artificial intelligence, extraction algorithms, machine learning model processing, and/or manual processing to process the distributed representation (concept vectors) to carry out useful tasks in the domain of concepts and user-concept interaction. For example, a team building application309performed at step312may generate recommendations for groupings of persons based on their skills/knowledge being related to a set of concepts. To generate team recommendations, the QA system100may process information for each individual (e.g., the individual's homepage, navigation behavior, emails, skills, experience, or other identifying parameters, such as language spoken, geographical location, psychological compatibility indicator factors such as personality traits, and availability) into concept sequences which are processed into concept vectors using vector embedding methods. However generated, the concept vectors may be processed to identify teams or groups of individuals by comparing the vectors representing different individuals to one another or to project description vectors on the basis of similarity metric distances therebetween to identify a “team” of individuals having skills or knowledge that is related the team's project. As will be appreciated, each of the concept vector applications309executed at step312can be tailored or constrained to a specified domain by restricting the corpus input to only documents relevant to the domain and/or restricting concept sequences to the domain and/or restricting remaining words to those of significance to the domain.

To provide additional details for an improved understanding of selected embodiments of the present disclosure, reference is now made toFIG. 4which depicts a simplified flow chart400showing the logic and method steps401-406for generating team recommendations based on the skills and knowledge of individuals being related to a set of concepts. The processing shown inFIG. 4may be performed in whole or in part by a cognitive system, such as the QA information handing system16, QA system100, or other natural language question answering system which uses concept vectors to generate concept hierarchies.

FIG. 4processing commences at step401by collecting identifying information from each individual that may be assigned to a team. The collected information may identify the interests of an individual, such as the individual's homepage, navigation behavior, emails, etc. In addition, the collected information may specify as individual parameters, such as experience, expertise, qualifications, computer skills, language spoken, geographical location, psychological compatibility, personality traits, or availability. In selected embodiments, the collected information may be assembled from a document associated with the individual.

At step402, the process continues by capturing, retrieving, or otherwise obtaining, for each individual, at least one input set of concepts, such as a concept sequence S1over a set of concepts {C1, . . . , Cn}. In selected embodiments, the input concept sequence S1may be retrieved from storage in a database, or may be generated by a concept sequence identifier (e.g., 12) that extracts a sequence of concepts from the identifying information for each individual. In selected embodiments, the collected concept sequence can be weighted based on priority information, or may be restricted to a set of concepts {C1, . . . , Ck} by deleting selected concepts (e.g., Ck+1, . . . , Cn). Alternatively, the concept sequence S1can be restricted to selected concepts (e.g., C1, . . . , Ck) and concepts that are highly related to them, i.e., those whose cosine distance to some concept C in {C1, . . . , Ck} is among the U (a parameter, e.g. 3) highest cosine distances to these concepts.

At step403, one or more concept vectors VC1, . . . , VCn, may be generated for each individual to serve as representations for the individual's associated concepts C1, . . . , Cn, such as by using concept sequences obtained at step402to compute or train concept vectors VC1, . . . , VCn using any desired vector embedding techniques. As disclosed herein, the concept vector computation processing at step403may be performed at the QA system100or concept vector extractor13by employing machine learning techniques and/or NLP techniques to compute a distributed representation (vectors) of concepts VC1, . . . , VCn which are trained on the concepts from the input sequence S1. For example, the concept vector computation processing at step403may employ NL processing technique such as word2vec or to implement a neural network (NN) method to perform “brute force” learning from training examples derived from concept sequences that contain those concepts in S1. In addition or in the alternative, the concept vector computation processing at step403may employ various matrix formulations and/or extended with SVM-based methods. In each case, the vector computation process may use a learning component in which selected parameters (e.g., NN weights, matrix entries, vector entries, etc.) are repeatedly adjusted until a desired level of learning is achieved. Though illustrated as occurring after step402, the vector extraction step403may be skipped in situations where the concept vectors were previously extracted or computed. In selected embodiments, a set of individual vector representations based on a selected concept subset C1, . . . , Ck can be learned by first restricting the sequence of concepts to C1, . . . , Ck (by deleting the others) and then learning the vector representation VC1, . . . VCk and then combining VC1, . . . , VCk with a set of weights specific to each individual to form the individual's vector representation.

At step404, one or more team alignment or project concept vectors Vt may be generated to serve as representations for the project at hand or to otherwise align the formation of the team around one or more specified concepts. For example, a team alignment vector may be formed from concepts relating to a specified team project by assembling a set of concepts relating to the specified team project which are used to compute or train team alignment vectors Vt using any desired vector embedding techniques. Alternatively, a project concept vector may be formed from by selecting a key concept all individuals in the team need to be familiar with and then using the key concept to compute or train project concept vectors Vt using any desired vector embedding techniques. As disclosed herein, the vector computation processing at step404may be performed at the QA system100or concept vector extractor13by employing machine learning techniques and/or NLP techniques to compute a distributed representation (vectors) of the team alignment/project concept vector(s)Vt.

At step405, one or more teams may be formed by grouping individuals with corresponding project(s) based on computed vector similarity metrics between team alignment/project concept vectors and individual vectors. As disclosed herein, the team formation processing at step405may be performed at the QA system100or vector processing application14by using the similarity calculation engine15to process the distributed representations (vectors) of individual concepts to group the individuals into teams based on their skills/knowledge being related to the set of concepts in the team alignment/project concept vector(s). In an example embodiment, the team formation processing may be implemented by configuring the QA system100or vector processing applications14to obtain concepts (e.g., Wikipedia concepts) in which each individual is interested as identified by each individual's homepage, navigation behavior, emails, etc., and to then form a group in which the total mutual interests of individuals as derived from the closeness of the involved concepts is high, where the group formation process constructs vectors Vp representing each individual's interests in concepts and then clusters the individual vectors into teams. In other embodiments, the team formation processing may be implemented by configuring the QA system100or vector processing applications14to obtain a list of individuals having similar skills for a focused team by selecting a key concept that all individuals in the team should be familiar with, and then searching for the individual vectors that are most similar to the key concept vector. In other embodiments, the team formation processing may be implemented by configuring the QA system100or vector processing applications14to select a team of individuals by using specified characteristics or parameters for the individuals (e.g., language skills, work experience, geographical location, psychological compatibility, personality traits, and availability) to match individuals with concepts or topics for a project (e.g., project location and schedule). In other embodiments, the team formation processing may be implemented by configuring the QA system100or vector processing applications14to select a team of individuals such that their corresponding skills and experience (captured in the concepts of each individual vector) fully meet the set of concepts specified for a particular project. In these embodiments, each concept Ci in the individual's concept sequence S1may be sequentially processed to find a nearest neighbor to a project concept Cj from the team alignment/project concept vectors by iteratively computing a similarity metric sim(VCi, VCj) for i,j=1, . . . , N, j≠i, and then determining which individual vector or vectors is closest to the team alignment/project concept vector(s). In an example embodiment, the vector similarity metric values may be computed by configuring the QA system100or vector processing applications14to compute, for each concept Ci, the cosine similarity metric value cos(VCi,VCj) for i,j=1, . . . , N, j≠i. However, it will be appreciated that the QA system100or vector processing applications14may use any desired similarity metric computation to compute a vector distance measure, such as the L_infinity norm (max norm), Euclidean distance, etc. At step406, the process ends.

By now, it will be appreciated that there is disclosed herein a system, method, apparatus, and computer program product for grouping individuals into at least a first team with an information handling system having a processor and a memory. As disclosed, the system, method, apparatus, and computer program product generate at least an individual concept set for each of a plurality of individuals, where each individual concept set includes one or more individual concepts associated with a corresponding individual, such as by capturing concepts relating to the individual's interests, skills, availability, or qualifications. In addition, the disclosed system, method, apparatus, and computer program product generate a project concept set for a first project, where the project concept set includes one or more project concepts corresponding to a specified project. In selected embodiments, the project concept set is generated by capturing concepts relating to specified project parameters. A vector representation of each individual concept set and each project concept set is generated, retrieved, constructed, or otherwise obtained. The vectors are processed by performing a natural language processing (NLP) comparison analysis of the vector representation of each individual concept set to a vector representation of each project concept set to determine a similarity measure between each individual concept set and each project concept set. In selected embodiments, the NLP analysis includes analyzing a vector similarity function sim(Vi,Vj) between (1) a vector representation Vi of each individual concept set and (2) a vector representation Vj for the project concept set to identify a group of individual concept sets that are most strongly connected to the project concept set. In other embodiments, the NLP analysis includes selecting a key project concept from the project concept set that all individuals in the first team should be familiar with, and then analyzing, by the system, a vector similarity function sim(Vi,Vj) between (1) a vector representation Vi of each individual concept set and (2) a vector representation Vj for the key project concept set to identify a group of individual concept sets that are most strongly connected to the key project concept set. In addition, the system may select one or more of the plurality of individuals for grouping into a first team by using the similarity measure to determine which individual concept associated with a corresponding individual are similar to the project concept set. In addition, team identification information may be displayed by the system to visually identify which individuals are included in the first team. In selected embodiments, the system may select one or more of the plurality of individuals for grouping into the first team by selecting individuals having individual concepts associated with corresponding skills and experience which fully meet specified project requirements specified in the project concepts of the project concept set. In addition, the system may generate a second project concept set for a second project, where the second project concept set includes one or more second project concepts corresponding to the second project. In such embodiments, the selection of individuals for grouping may be implemented by selecting one or more of the plurality of individuals for grouping into first and second teams by using the similarity measure to determine which individual concept set associated with a corresponding individual are similar to the project concept set