Intelligent recommendation of guidance instructions

Embodiments for providing intelligent recommendation of guidance instructions to a user by a processor. A user may be cognitively recognized performing a selected task on an item using one or more Internet of Things (“IoT”) computing devices. One or more guidance instructions may be provided/suggested for assisting with performing the selected task according to identified contextual factors. The user may be cognitively guided to perform the selected task using the one or more guidance instructions.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates in general to computing systems, and more particularly to, various embodiments for intelligent recommendation of guidance instructions to a user by a processor.

Description of the Related Art

In today's society, consumers, business persons, educators, and others use various computing network systems with increasing frequency in a variety of settings. The advent of computers and networking technologies have made possible the increase in the quality of life while enhancing day-to-day activities. Computing systems can include an Internet of Things (IoT), which is the interconnection of computing devices scattered across the globe using the existing Internet infrastructure. IoT devices may be embedded in a variety of physical devices or products.

As great strides and advances in technologies come to fruition, these technological advances can be then brought to bear in everyday life. For example, the vast amount of available data made possible by computing and networking technologies may then assist in improvements to quality of life and appropriate living conditions.

SUMMARY OF THE INVENTION

Various embodiments for providing intelligent recommendation of guidance instructions to a user using one or more processors, are provided. In one embodiment, by way of example only, a method for cognitively recommending a set of guidance instructions to accomplish a task, again by a processor, is provided. A user may be cognitively recognized performing a selected task on an item (e.g., fixing/repairing or assembling the item) using one or more Internet of Things (“IoT”) computing devices. One or more guidance instructions may be provided/suggested for assisting with performing the selected task according to identified contextual factors. The user may be cognitively guided to perform the selected task using the one or more guidance instructions.

DETAILED DESCRIPTION OF THE DRAWINGS

Computing systems may include large scale computing called “cloud computing,” in which resources may interact and/or be accessed via a communication system, such as a computer network. Resources may be software-rendered simulations and/or emulations of computing devices, storage devices, applications, and/or other computer-related devices and/or services run on one or more computing devices, such as a server. For example, a plurality of servers may communicate and/or share information that may expand and/or contract across servers depending on an amount of processing power, storage space, and/or other computing resources needed to accomplish requested tasks. The word “cloud” alludes to the cloud-shaped appearance of a diagram of interconnectivity between computing devices, computer networks, and/or other computer related devices that interact in such an arrangement.

Additionally, the Internet of Things (IoT) is an emerging concept of computing devices that may be embedded in objects, especially appliances, and connected through a network. An IoT network may include one or more IoT devices or “smart devices”, which are physical objects such as appliances with computing devices embedded therein. Many of these objects are devices that are independently operable, but they may also be paired with a control system or alternatively a distributed control system such as one running over a cloud computing environment.

The prolific increase in use of IoT appliances in computing systems, particularly within the cloud computing environment, in a variety of settings provide various beneficial uses to a user such as, for example, a medical patient. For example, often, when a particular item needs to be assembled and/or repaired (e.g., an electronic device, wooden assembly, a plumbing task, etc.) and there is limited knowledge pertaining to assembly and/or repair, a subject matter expert is often summonsed to perform the task. Alternatively, attempts are made by a user themselves to perform the tasks. However, considering the user has limited knowledge about assembling/repairing the item, the user often times becomes frustrated, angry, and/or confused in performing the task. This is often an iterative process and user ends up spending unnecessary time in a reckless state without correctly assembling/repairing the item.

Accordingly, the present invention provides smart guidance system which can detect the real-time activity being performed by a user, fetch/retrieve one or more useful guidance instructions from one or more trusted sources, and then cognitively guide the user via step-by-step guidance instructions to complete the task.

In an additional aspect, the present invention provides for cognitively recommending a useful set of guidance instructions to accomplish a task such as, for example, assembling and/or repairing an item/object. A selected task being performed by a user on an item/object may be cognitively recognized (e.g., fixing/repairing or assembling the item) using one or more IoT computing devices.

The term “usefulness” or “useful” as used herein may refer to one or more actions having a positive impact upon assembling, repairing, and/or performing one or more actions, and/or efforts in relation to a selected task. Useful may be generally defined as the ability to be used for a practical purpose, capable of being put to use, serviceable for an end or purpose, and/or having value or productivity. It should be noted that one or more calculations may be performed using various mathematical operations or functions that may involve one or more mathematical operations (e.g., solving differential equations or partial differential equations analytically or computationally, using addition, subtraction, division, multiplication, standard deviations, means, averages, percentages, statistical modeling using statistical distributions, by finding minimums, maximums or similar thresholds for combined variables, etc.).

Other examples of various aspects of the illustrated embodiments, and corresponding benefits, will be described further herein.

It is understood in advance that although this disclosure includes a detailed description on cloud computing, implementation of the teachings recited herein are not limited to a cloud computing environment and/or computing systems associated with one or more medium/means. Rather, embodiments of the present invention are capable of being implemented in conjunction with any other type of computing environment now known or later developed.

Characteristics are as follows:

Service Models are as follows:

Deployment Models are as follows:

System memory28can include computer system readable media in the form of volatile memory, such as random access memory (RAM)30and/or cache memory32.

As stated above, the present invention provides a novel solution for intelligent recommendation of guidance instructions. A cognitive system fetches instructional steps or videos from the cloud database or corpus. Once the retrieval operation is performed, the cognitive system may monitor in real-time, current activities performed by a user in order to cross verify the instructional steps based on the contextual situation and the actions associated with the item/object. Based on the monitoring, the cognitive system automatically pauses, stops, and/or provides additional audio data, text data, and/or video data (e.g., visual aids) when the user is unable to perform one or more steps of the instructional steps (e.g., stuck at a particular step). The cognitive system detects that the user is stuck (unable to continue performing a step or next step) and attempts to locate additional information from one or more ranked sources (e.g., a first data source such as an online search engine, a second data source such as a website or audio/video sharing website, a text based website such as a web-based encyclopedia/dictionary) that may guide the user through the process in a further granular form. The cognitive system may collect and gather user's inputs in relation to each step/task (either manual driven or automatically detected). Upon the user successfully completing a specific step/stage of the instructions, a next step/stage of the instructions may be provided.

For example, an intelligent recommendation of guidance instructions service may analyze and detect an activity being performed in real-time by the user on an object (e.g., a computer). The user can point to a specific object for a camera device to detect and/or bring the specific item to be assembled or repaired in front of the camera lens of an IoT computing device. A neural network operation may be performed on the captured video/image of the scene in order to recognize the object. The neural network may divide the image into regions and predicts bounding boxes (e.g., “predicting boundaries” for each region) and probabilities for each region. For example, a virtual (dotted) box may be created around the image while processing by camera scanning system to understand what is being observed (e.g., object recognition by machine learning and using video analytics). These bounding boxes may be weighted by one or more predicted probabilities (e.g., probability of correctly identifying a device which has been enclosed in a certain boundary while recording the activities of the user engaged with a certain device such as, for example, cycle repair, laptop repair, microwave device repair, cooking items etc.).

The intelligent recommendation of guidance instructions service may perform real-time detection on an IoT device (e.g., personal digital assistant (PDA) or cellular telephone54A, desktop computer54B, laptop computer54C, and/or automobile computer system54N ofFIG. 2). The real-time detection may be performed by running machine learning operations with a parallel computing platform and application programming interface “API” (e.g., CUDA) and open source computer vision (“OpenCV”). The visual analysis generates label files that a machine learning operation uses and models that may be trained accordingly in order to identify the objects or items in space. Once the item has been detected and recognized, the intelligent recommendation of guidance instructions service may record/store in a database the activity of the user. Feature matching of the object/item in space with the set of instructions associated with assembling or repairing an item (e.g., either pre-loaded at the time of creation and increasing the database as the intelligent recommendation of guidance instructions service evolves or loaded at run-time in real time based on the contextual situation) are fetched from the cloud database in a sequential form. That is, a machine learning operation may provide two components: 1) feature extraction that identifies features (via pixel by pixel understanding) of a particular object in a space (e.g., laptop) and 2) feature matching for matching or correlating the object with a known library of objects in the database. For example, the machine learning operation may learn and understand, via IoT sensor feeds/camera, the objects/items being observed (e.g., DRAMS, laptop, bicycle, screw driver etc.) and then the respective features (vectored matrix) may be extracted based on color, type, size etc. and then matched or correlated with known objects to classify the object properly.

An IoT device may detect or monitors the user's activity in order to inspect the progress of the specific user. A Visual Surveillance and Monitoring (“VSAM”) operation can be used in order to detect the progress of the user. VSAM with automated video understanding operations may allow a network of active video sensors to automatically monitor objects and events. A machine learning component may also perform a syntactic and semantic analysis in order to re-organize the instructions. A device inculcated with a back-end corpus of the items detected in space and instructional aid will act as an instructional kit to guide the user through the process and also tell the user where the user made mistakes. The information parsed via the machine learning component to the user and provided in a sequential form can be based on information obtained one or more ranked sources based on micro-tracking the user's activity. One or more mistakes committed by the users can be stored in the feature matching operation in order to trigger or activate the mistaken steps before the actual steps so that the user keeps track of the correct instructions or steps to be executed. That is, if the user commits certain mistakes (e.g., skipping some steps or proceeding incorrectly with repairing something), then feature matching system, which is a part of monitoring user's activities correlating with ideal activity to be performed while fixing or repairing something (e.g., continuous software polling being performed by monitoring the user's activity and the preferred way of doing the activity), may calculate a threshold deviation from ideal way of performing the activity or completing a certain task and then corrective actions can be provided to the user by interjecting at a certain point and guiding the user with a modified set of instructions.

Turning now toFIG. 4A-4B, are block diagrams depicting exemplary functional components400and475according to various mechanisms of the illustrated embodiments is shown. In one aspect, each of the devices, components, modules, and/or functions described inFIGS. 1-3may also apply to the devices, components, modules, and functions ofFIG. 4A-4B. Also, one or more of the operations and steps ofFIGS. 1-3may also be included in one or more operations or actions ofFIG. 4A-4B. Computer system/server12is again shown, which may incorporate an intelligent service402or “intelligent recommendation of guidance instructions service402” (e.g., an Artificial Intelligence Simulated Humanoid Assistant “A.I.S.H.A.”).

As illustrated inFIG. 4A, in one aspect, the computer system/server12may provide virtualized computing services (i.e., virtualized computing, virtualized storage, virtualized networking, etc.) to one or more computing devices, as described herein. More specifically, the computer system/server12may provide virtualized computing, virtualized storage, virtualized networking and other virtualized services that are executing on a hardware substrate.

The intelligent service402(e.g., intelligent recommendation of guidance instructions service402) depicted inFIG. 4may be in communication with and/or association with one or more computing devices such as, for example, a voice activated hub420(e.g., an personal assistant IoT computing device), and/or IoT computing device430(e.g., a camera, television, smart phone, desktop computer, laptop computer, tablet, smart watch and/or another electronic/wireless communication device that may have one or more processors, memory, and/or wireless communication technology for displaying/streaming audio/video data).

The intelligent recommendation of guidance instructions service402, the voice activated hub420, and the IoT computing device430may each be associated with and/or in communication with each other, by one or more communication methods, such as a computing network, wireless communication network, or other network means enabling communication (each collectively referred to inFIG. 4as “network18”). In one aspect, the intelligent recommendation of guidance instructions service402may be installed locally on the voice activated hub420and/or the IoT computing device430. Alternatively, the intelligent recommendation of guidance instructions service402may be located external to (e.g., via a cloud computing server) the voice activated hub420and/or the IoT computing device430.

The intelligent recommendation of guidance instructions service402may incorporate processing unit16to perform various computational, data processing and other functionality in accordance with various aspects of the present invention. A domain knowledge412(e.g., a database that may include an ontology) is shown, along with a guidance instructions component404, an analysis component406, a monitoring component408, a machine learning component410, and/or a recognition component414.

The domain knowledge412may include and/or be associated with an ontology of concepts, keywords, expressions representing a domain of knowledge. A thesaurus or ontology may be used as the database and may also be used to identify semantic relationships between observed and/or unobserved variables by the machine learning component410(e.g., a cognitive component). In one aspect, the term “domain” is a term intended to have its ordinary meaning. In addition, the term “domain” may include an area of expertise for a system or a collection of material, information, content and/or other resources related to a particular subject or subjects. A domain can refer to information related to any particular subject matter or a combination of selected subjects.

The term ontology is also a term intended to have its ordinary meaning. In one aspect, the term ontology in its broadest sense may include anything that can be modeled as an ontology, including but not limited to, taxonomies, thesauri, vocabularies, and the like. For example, an ontology may include information or content relevant to a domain of interest or content of a particular class or concept. The ontology can be continuously updated with the information synchronized with the sources, adding information from the sources to the ontology as models, attributes of models, or associations between models within the ontology.

Additionally, the domain knowledge412may include one or more external resources such as, for example, links to one or more Internet domains, webpages, and the like. For example, text data may be hyperlinked to a webpage that may describe, explain, or provide additional information relating to the text data. Thus, a summary may be enhanced via links to external resources that further explain, instruct, illustrate, provide context, and/or additional information to support a decision, alternative suggestion, alternative choice, and/or criteria.

In an additional aspect, the analysis component406of the computer system/server12may work in concert with processing unit16to accomplish various aspects of the present invention. For example, analysis component406may undergo various data analytics functions to analyze data communicated from one or more devices such as, for example, the voice activated hub420and/or the IoT computing device430.

The analysis component406may receive and analyze each physical property associated with media data (e.g., audio data and/or video data). The analysis component406may cognitively receive and/or detect audio data and/or video data for the guidance instructions component404.

The analysis component406, the monitoring component408, and/or the machine learning component410may access and monitor one or more audio data sources and/or video data sources (e.g., website, audio storage systems, video storage systems, cloud computing systems, etc.) to provide the audio data, video data, and or text data for providing guidance instructions for performing a task. The analysis component406may cognitively analyze the data retrieved from the domain knowledge412, one or more online sources, a cloud computing system, a text corpus, or a combination thereof. The analysis component406and/or the machine learning component410may extract one or more keywords, phrases, instructions, and/or transcripts (e.g., transcribe audio data into text data) using natural language processing (“NLP”).

The monitoring component408may monitor performance of the one or more guidance instructions while performing the selected task. The recognition component414may recognize a user performing a selected task on an item using the voice activated hub420and/or the IoT computing device430. For example, the voice activated hub420and/or the IoT computing device430and/or other IoT devices such as, for example, computing devices used by cloud consumers, such as, for example, personal digital assistant (PDA) or cellular telephone54A, desktop computer54B, laptop computer54C, and/or automobile computer system54N ofFIG. 2, may identify one or more activities, bodily movements and/or features (e.g., facial recognition, facial expressions, hand/feet gestures, etc.), behaviors, audio data (e.g., voice detection and/or recognition), environmental surroundings, or other defined parameters/features that may identify, locate, and/or recognize a user and/or a task being performed by the user.

As part of detecting the data, the analysis component406, the monitoring component408, and/or the machine learning component410may identify, from one or more sources audio data, video data, text data, and/or contextual factors associated with the audio data, video data, and/or text data, or a combination thereof. Also, the machine learning component410may initiate a machine learning operation to learn the contextual factors associated with the audio data, video data, and/or text data associated with guidance instructions for performing a task such as, for example, assembling and/or fixing/repairing an item (e.g., assembling a new bicycle or fixing a computer).

The guidance instructions component404may provide one or more guidance instructions450for assisting with performing the selected task according to identified contextual factors. The guidance instructions450may be text data, audio data, and/or video data. For example, the voice activated hub420may audibly communicate the guidance instructions450. The IoT computing device430may visually provide the guidance instructions450as illustrated as in image/video data485of a graphical user interface (“GUI”) of the IoT computing device430.

The guidance instructions component404may cognitively guide the user to perform the selected task using the one or more guidance instructions450. The guidance instructions component404may provide a sequence of guidance instructions retrieved from a domain knowledge, one or more online sources, a cloud computing system, a text corpus, or a combination thereof. The guidance instructions component404may provide media data from one or more online sources, a cloud computing system, or a combination thereof.

The guidance instructions component404may verify each step of the one or more guidance instructions for assisting with performing the selected task. The guidance instructions component404may also identify a level of difficulty (e.g., a level of stress, frustration, anxiety, excitability, or other emotional responses) by the user in performing a set of tasks associated with the one or more guidance instructions delivered via streamed media, pause/stop/terminate delivery of the streamed media for a selected period of time, and/or provide the user with a modified set of guidance instructions to guide the user through an enhanced level of instructions.

The guidance instructions component404may provide additional guidance information relating to the guidance instructions collected from a domain knowledge, one or more online sources, a cloud computing system, a text corpus, or a combination thereof for performing the selected task. For example, if a first set of instructions are insufficient for the user, an additional set that may further explain one or more of the original instructions may be provided.

The intelligent recommendation of guidance instructions service402may adjust a tone, volume, pace of the speech and/or frequency of the guidance instructions audio/media data based on the speed/pace of a user following the guidance instructions. Also, words, phrases, and/or compete sentences (e.g., all or portions of a conversation) by other parties relating to the audio data may be transcribed in a text form based on an NLP extraction operation (e.g., NLP based keyword extraction). The text data may be relayed, sent, stored, or further processed so that the same audio/video data (e.g., all or portions of the conversations) may be heard or listened to while simultaneously providing the text version of the guidance instruction.

As previously indicated, the intelligent recommendation of guidance instructions service402may also communicate with other linked devices such as, for example, the voice activated hub420and/or the IoT computing device430. Moreover, the analysis component406and/or the machine learning component410may even access one or more online data sources such as, for example, a social media network, website, or data site for providing one or more guidance instructions for assisting with performing the selected task according to identified contextual factors. That is, the analysis component406, the recognition component414, and/or the machine learning component410may learn and observe, for a user, a degree or level of attention, a level difficulty by a user in performing a task, a type of response, and/or feedback as to various topics and/or guidance instructions. The learned and observed behavior of the user may be linked to various data sources that provide personal information, social media data, or user profile information to learn, establish and/or determine a confidence factor relating to the performance of the guidance instructions.

In one aspect, the machine learning component410, as described herein, may be performed by a wide variety of methods or combinations of methods, such as supervised learning, unsupervised learning, temporal difference learning, reinforcement learning and so forth. Some non-limiting examples of supervised learning which may be used with the present technology include AODE (averaged one-dependence estimators), artificial neural network, backpropagation, Bayesian statistics, naive bays classifier, Bayesian network, Bayesian knowledge base, case-based reasoning, decision trees, inductive logic programming, Gaussian process regression, gene expression programming, group method of data handling (GMDH), learning automata, learning vector quantization, minimum message length (decision trees, decision graphs, etc.), lazy learning, instance-based learning, nearest neighbor algorithm, analogical modeling, probably approximately correct (PAC) learning, ripple down rules, a knowledge acquisition methodology, symbolic machine learning algorithms, sub symbolic machine learning algorithms, support vector machines, random forests, ensembles of classifiers, bootstrap aggregating (bagging), boosting (meta-algorithm), ordinal classification, regression analysis, information fuzzy networks (IFN), statistical classification, linear classifiers, fisher's linear discriminant, logistic regression, perceptron, support vector machines, quadratic classifiers, k-nearest neighbor, hidden Markov models and boosting. Some non-limiting examples of unsupervised learning which may be used with the present technology include artificial neural network, data clustering, expectation-maximization, self-organizing map, radial basis function network, vector quantization, generative topographic map, information bottleneck method, IBSEAD (distributed autonomous entity systems based interaction), association rule learning, apriori algorithm, eclat algorithm, FP-growth algorithm, hierarchical clustering, single-linkage clustering, conceptual clustering, partitional clustering, k-means algorithm, fuzzy clustering, and reinforcement learning. Some non-limiting example of temporal difference learning may include Q-learning and learning automata. Specific details regarding any of the examples of supervised, unsupervised, temporal difference or other machine learning described in this paragraph are known and are within the scope of this disclosure. Also, when deploying one or more machine learning models, a computing device may be first tested in a controlled environment before being deployed in a public setting. Also even when deployed in a public environment (e.g., external to the controlled, testing environment), the computing devices may be monitored for compliance.

In one aspect, the intelligent recommendation of guidance instructions service402may perform one or more calculations according to mathematical operations or functions that may involve one or more mathematical operations (e.g., solving differential equations or partial differential equations analytically or computationally, using addition, subtraction, division, multiplication, standard deviations, means, averages, percentages, statistical modeling using statistical distributions, by finding minimums, maximums or similar thresholds for combined variables, etc.) Thus, as used herein, a calculation operation may include all or part of the one or more mathematical operations.

In one aspect, if the activity that the user wants to perform is initially unable to be detect, a user may provide (e.g., verbally via the voice activated hub420and/or via an interactive GUI interface of the IoT computing device430) activity data into the intelligent recommendation of guidance instructions service402as an input so that the intelligent recommendation of guidance instructions service402may start with object scanning, instruction scanning (after downloading into the corpus if it hasn't done already) and guiding the step-by-step instructions to the user based on monitoring the activity of the user.

Turning now toFIG. 4B, consider the following exemplary use case. As previously stated, guidance instructions component404may provide one or more guidance instructions450for assisting with performing a selected task such as, for example, changing random access memory (“RAM”) in laptop. The guidance instructions450may be text data, audio data, and/or video data. For example, the IoT computing device430may visually provide the guidance instructions450as illustrated as in the image/video data485with visual aids such as, for example, “laptop detected,” “dynamic “DRAM” detected,” and instructions450pointing to the image/video data485.

For changing the RAM in a laptop, step 1) step-by-step guidance instructions450for taking out the RAM and changing the RAM of the computer may be provided. In step 2) relevant steps may be automatically accessed/identified on a web-based encyclopedia/dictionary or similar ranked platform and the information may be stored in the domain knowledge412(or corpus or). In step 3) the voice activated hub420and/or the IoT computing device430may provide micro-instructional assistance. In step 4) Audio or visual aids on image/video data485may be provided to the user at every step and pauses while the user operates through that particular step. Thus, as illustrated inFIG. 4B, the voice activated hub420and/or the IoT computing device430may provide an interactive dialog with a user (e.g., “John Doe”) for providing the guidance instructions450.

Turning now toFIG. 5, a block diagram of exemplary functionality500relating to an intelligent recommendation of guidance instructions service is depicted. As shown, the various blocks of functionality are depicted with arrows designating the blocks'500relationships with each other and to show process flow. Additionally, descriptive information is also seen relating each of the functional blocks500. As will be seen, many of the functional blocks may also be considered “modules” of functionality, in the same descriptive sense as has been previously described inFIG. 4A-4B. With the foregoing in mind, the module blocks500may also be incorporated into various hardware and software components of a system for image enhancement in accordance with the present invention. Many of the functional blocks500may execute as background processes on various components, either in distributed computing components, or on the user device, or elsewhere, and generally unaware to the user performing generalized tasks.

As a preliminary matter,FIG. 5illustrates the intelligent recommendation of guidance instructions for assembling a bicycle. Starting with block502, one or more IoT computing devices may recognize a user performing a selected task on an item (e.g., “detect John Doe attempting to assemble a bike”) and display the recognized user and task via an image. A message may be communicated, via one or more IoT devices such as, “I'll launch bike assembly instructions or you can scan a copy of instructions in the system (e.g., intelligent recommendation of guidance instructions system) and I'll guide you though it by detecting your activity,” as in block506. In block506, a list of items needed for performing the task may be provided according to a search operation on one or more resources such as, for example, “A.I.S.H.A: So, this is what you need based on a corpus or scanned documents! Let's proceed.”

In block508, a video image of instructions (e.g., step 1) may be provided along with the instructions as text data displayed on an IoT device and/or the instructions audibly communicated via an IoT device such as, for example, “Step 1” cut the cable tires for the wheels first. Please take the wheels out of the crankset arm with care.” Also, intelligent recommendation of guidance instructions service may wait until the user completes the instructions of block508. Also, if the user is having difficulty performing the task (e.g., step 1 instructions), an additional video/instructions may be launched with one or more furnished sets of visual instructions from an alternative source as additional aids for accomplishing the particular step/instruction the user is having difficulty in performing.

Upon completion of block510, a second video image of an instructions (e.g., step 2) may be provided along with the instructions as text data displayed (and/or audibly communicated) on an IoT device and/or the instructions audibly communicated via an IoT device such as, for example, “Step 2: Insert the seat post into the seat tube now. Look at me if you need the image to be zoomed in. Use a 4 mm Allen key to tighten the seat claim” As indicated, the instructions may also provide hints or additional actions/bodily gestures (e.g., a request for the user to look at the IoT device to indicate a particular action needed by the user such as, for example, the IoT device zooming in on the video image). Thus, the present invention may cognitively guide the user to perform the selected task using the one or more guidance instructions at a granular level.

Turning now toFIG. 6, a method600for intelligent recommendation of guidance instructions to a user by a processor is depicted, in which various aspects of the illustrated embodiments may be implemented. The functionality600may be implemented as a method executed as instructions on a machine, where the instructions are included on at least one computer readable medium or one non-transitory machine-readable storage medium. The functionality600may start in block602.

A user may be cognitively recognized performing a selected task on an item using one or more IoT computing devices, as in block604. One or more guidance instructions may be provided/suggested for assisting with performing the selected task according to identified contextual factors, as in block606. The user may be cognitively guided to perform the selected task using the one or more guidance instructions, as in block608. The functionality600may end, as in block610.

Turning now toFIG. 7, a method700for intelligent recommendation of guidance instructions to a user by a processor is depicted, in which various aspects of the illustrated embodiments may be implemented. The functionality700may be implemented as a method executed as instructions on a machine, where the instructions are included on at least one computer readable medium or one non-transitory machine-readable storage medium. The functionality700may start in block702.

A user may be cognitively recognized performing a selected task on an item using one or more IoT computing devices, as in block704. One or more audio/video guidance instructions (e.g., audio data and/or video data) may be identified for assisting with performing the selected task according to identified contextual factors, as in block706. The audio/video guidance instructions may be delivered (e.g., real-time streaming, on-demand streaming, broadcasting, and/or communicating, etc.) to one or more IoT device to cognitively guide the user to perform the selected task using the one or more guidance instructions, as in block708. A level of difficulty by the user in performing a set of tasks associated with the one or more guidance instructions, as in block710. The delivered audio/video guidance instructions may be paused (e.g., stopped) for a selected time period, as in block712. Additional audio/video guidance instructions may be provided for assisting with performing the selected task according to identified contextual factors, as in block714. The functionality700may end, as in block716.

In one aspect, in conjunction with and/or as part of at least one block ofFIGS. 6-7, the operations of methods600and700may include each of the following. The operations of methods600and700may provide a sequence of guidance instructions retrieved from a domain knowledge, one or more online sources, a cloud computing system, a text corpus, or a combination thereof, and/or provides media data from one or more online sources, a cloud computing system, or a combination thereof. The performance of the one or more guidance instructions may be monitored while performing the selected task. Each step of the one or more guidance instructions may be verified (e.g., actions performed by the user may be compared to the instructions for determining accuracy of performance) for assisting with performing the selected task.

The operations of methods600and700may identify a level of difficulty by the user in performing a set of tasks associated with the one or more guidance instructions delivered via streamed media, and/or pause delivery of the streamed media for a selected period of times, and/or provide the user with a modified set of guidance instructions to guide the user through an enhanced level of instructions. The operations of methods600and700may provide additional guidance information relating to the guidance instructions collected from a domain knowledge, one or more online sources, a cloud computing system, a text corpus, or a combination thereof for performing the selected task.