Deep convolutional neural network with self-transfer learning

Systems and techniques for facilitating a deep convolutional neural network with self-transfer learning are presented. In one example, a system includes a machine learning component, a medical imaging diagnosis component and a visualization component. The machine learning component generates learned medical imaging output regarding an anatomical region based on a convolutional neural network that receives medical imaging data. The machine learning component also performs a plurality of sequential downsampling and upsampling of the medical imaging data associated with convolutional layers of the convolutional neural network. The medical imaging diagnosis component determines a classification and an associated localization for a portion of the anatomical region based on the learned medical imaging output associated with the convolutional neural network. The visualization component generates a multi-dimensional visualization associated with the classification and the localization for the portion of the anatomical region.

TECHNICAL FIELD

This disclosure relates generally to artificial intelligence.

BACKGROUND

Artificial Intelligence (AI) can be employed for classification and/or analysis of digital images. For instance, AI can be employed for image recognition. In certain technical applications, AI can be employed to enhance medical imaging diagnosis. Diseases of a patient can be classified, for example, by analyzing medical images of the patient using a deep neural network. In an example, region-of-interest based deep neural networks can be employed to localize a disease in an anatomical region of a patient. However, accuracy and/or efficiency of a classification and/or an analysis of digital images using conventional artificial techniques is generally difficult to achieve. Furthermore, conventional artificial techniques for classification and/or analysis of digital images generally requires labor-intensive processes such as, for example, pixel annotations, voxel level annotations, etc. As such, conventional artificial techniques for classification and/or analysis of digital images can be improved.

SUMMARY

According to an embodiment, a system includes a machine learning component, a medical imaging diagnosis component and a visualization component. The machine learning component generates learned medical imaging output regarding an anatomical region based on a convolutional neural network that receives medical imaging data. The machine learning component also performs a plurality of sequential downsampling and upsampling of the medical imaging data associated with convolutional layers of the convolutional neural network. The medical imaging diagnosis component determines a classification and an associated localization for a portion of the anatomical region based on the learned medical imaging output associated with the convolutional neural network. The visualization component generates a multi-dimensional visualization associated with the classification and the localization for the portion of the anatomical region.

According to another embodiment, a method is provided. The method provides for receiving, by a system comprising a processor, medical imaging data for a patient body. The method also provides for performing, by the system, iterative sequential downsampling and upsampling of the medical imaging data associated with convolutional layers of a convolutional neural network to generate learned medical imaging output regarding the patient body. Furthermore, the method provides for classifying, by the system, a disease for a portion of the patient body based on the learned medical imaging output associated with the convolutional neural network. The method also provides for generating, by the system, a multi-dimensional visualization associated with the classifying of the disease for the portion of the patient body.

According to yet another embodiment, a method is provided. The method provides for receiving, by a system comprising a processor, medical imaging data that comprises a set of medical images. The method also provides for training, by the system, a convolutional neural network by performing iterative sequential downsampling and upsampling of the medical imaging data associated with convolutional layers of the convolutional neural network. Furthermore, the method provides for generating, by the system, a set of filter values for the convolutional neural network based on the iterative sequential downsampling and upsampling of the medical imaging data.

DETAILED DESCRIPTION

Various aspects of this disclosure are now described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more aspects. It should be understood, however, that certain aspects of this disclosure may be practiced without these specific details, or with other methods, components, materials, etc. In other instances, well-known structures and devices are shown in block diagram form to facilitate describing one or more aspects.

Systems and techniques for performing self-transfer learning associated with a deep convolutional network are presented. For example, as compared to conventional artificial intelligence (AI) techniques, the subject innovations provide for a novel weakly supervised AI framework with self-transfer learning. The novel weakly supervised AI framework can perform machine learning (e.g., deep learning) associated with a deep convolutional network by employing image-level labels to classify and/or localize a disease over a particular anatomical region associated with medical imaging data. In an aspect, the novel weakly supervised AI framework can comprise shared convolutional layers containing sequential down sampling and/or sequential up sampling. Additionally or alternatively, the novel weakly supervised AI framework can comprise fully connected layers and/or class activation maps. As such, by employing the novel weakly supervised AI framework to analyze medical imaging data, detection and/or localization of diseases for a patient associated with the medical imaging data can be improved. Furthermore, accuracy and/or efficiency for classification and/or analysis of digital images (e.g., medical imaging data) can be improved. Moreover, effectiveness of a machine learning model for classification and/or analysis of digital images (e.g., medical imaging data) can be improved, performance of one or more processors that execute a machine learning model for classification and/or analysis of digital images (e.g., medical imaging data) can be improved, and/or efficiency of one or more processors that execute a machine learning model for classification and/or analysis of digital images (e.g., medical imaging data) can be improved.

Referring initially toFIG. 1, there is illustrated an example system100that provides a deep convolutional neural network with self-transfer learning, according to an aspect of the subject disclosure. The system100can be employed by various systems, such as, but not limited to medical device systems, medical imaging systems, medical diagnostic systems, medical systems, medical modeling systems, enterprise imaging solution systems, advanced diagnostic tool systems, simulation systems, image management platform systems, care delivery management systems, artificial intelligence systems, machine learning systems, neural network systems, modeling systems, aviation systems, power systems, distributed power systems, energy management systems, thermal management systems, transportation systems, oil and gas systems, mechanical systems, machine systems, device systems, cloud-based systems, heating systems, HVAC systems, medical systems, automobile systems, aircraft systems, water craft systems, water filtration systems, cooling systems, pump systems, engine systems, prognostics systems, machine design systems, and the like. In one example, the system100can be associated with a viewer system to facilitate visualization and/or interpretation of medical imaging data. Moreover, the system100and/or the components of the system100can be employed to use hardware and/or software to solve problems that are highly technical in nature (e.g., related to processing digital data, related to processing medical imaging data, related to medical modeling, related to medical imaging, related to artificial intelligence, etc.), that are not abstract and that cannot be performed as a set of mental acts by a human.

The system100can include a deep learning component102that can include a machine learning component104, a medical imaging diagnosis component106and a visualization component108. Aspects of the systems, apparatuses or processes explained in this disclosure can constitute machine-executable component(s) embodied within machine(s), e.g., embodied in one or more computer readable mediums (or media) associated with one or more machines. Such component(s), when executed by the one or more machines, e.g., computer(s), computing device(s), virtual machine(s), etc. can cause the machine(s) to perform the operations described. The system100(e.g., the deep learning component102) can include memory112for storing computer executable components and instructions. The system100(e.g., the deep learning component102) can further include a processor110to facilitate operation of the instructions (e.g., computer executable components and instructions) by the system100(e.g., the deep learning component102).

The deep learning component102(e.g., the machine learning component104) can receive medical imaging data (e.g., MEDICAL IMAGING DATA shown inFIG. 1). The medical imaging data can be two-dimensional medical imaging data and/or three-dimensional medical imaging data generated by one or more medical imaging devices. For instance, the medical imaging data can be electromagnetic radiation imagery captured via a set of sensors (e.g., a set of sensors associated with a medical imaging device). In certain embodiments, the medical imaging data can be a series of electromagnetic radiation imagery captured via a set of sensors (e.g., a set of sensors associated with a medical imaging device) during an interval of time. The medical imaging data can be received directly from one or more medical imaging devices. Alternatively, the medical imaging data can be stored in one or more databases that receives and/or stores the medical imaging data associated with the one or more medical imaging devices. A medical imaging device can be, for example, an x-ray device, a computed tomography (CT) device, another type of medical imaging device, etc. The machine learning component104can perform a machine learning process (e.g., an artificial intelligence process for machine learning) based on the medical imaging data. In an aspect, the machine learning component104can perform deep learning to facilitate classification and/or localization of one or more diseases associated with the medical imaging data. In another aspect, the machine learning component104can perform deep learning based on a convolutional neural network that receives the medical imaging data.

In an embodiment, the machine learning component104can perform a training phase for the machine learning process. For example, the medical imaging data can be a set of medical images (e.g., a set of x-ray images, etc.) stored in a data store. Furthermore, the machine learning component104can perform the training phase for the machine learning process based on the set of medical images stored in a data store to train a neural network model (e.g., a neural network model for the convolutional neural network). In certain embodiments, the machine learning component104can employ a first portion of the medical imaging data for training associated with the convolutional neural network, a second portion of the medical imaging data for validation associated with the convolutional neural network, and a third portion of the medical imaging data for testing associated with the convolutional neural network. Additionally or alternatively, the machine learning component104can randomly select a set of medical images from a training set associated with the medical imaging data for data augmentation associated with the medical imaging data. In an aspect, the machine learning component104can modify an orientation of the set of medical images for the data augmentation associated with the medical imaging data. In one example, the machine learning component104can modify the set of medical images through at least one affine transformation for the data augmentation associated with the medical imaging data. In another embodiment, the machine learning component104can perform an inference phase. For example, the medical imaging data can be a medical image for an anatomical region of a patient associated with the medical image. Furthermore, the machine learning component104can perform the training phase for the machine learning process based on the medical image. For an inference phase associated with the machine learning component104, the machine learning component104can generate learned medical imaging output regarding an anatomical region based on the convolutional neural network that receives medical imaging data.

In an aspect, the machine learning component104can employ a spring network of convolutional layers. The machine learning component104can employ the spring network of convolutional layers to generate the learned medical imaging output based on the medical imaging data. In an aspect, the machine learning component104can generate the learned medical imaging output based on a first convolutional layer process associated with sequential downsampling of the medical imaging data and a second convolutional layer process associated with sequential upsampling of the medical imaging data. The spring network of convolutional layers can include the first convolutional layer process associated with the sequential downsampling and the second convolutional layer process associated with sequential upsampling. In one example, the machine learning component104can perform a plurality of sequential downsampling and upsampling of the medical imaging data associated with convolutional layers of the convolutional neural network. The spring network of convolutional layers employed by the machine learning component104can alter convolutional layer filters similar to functionality of a spring. For instance, the machine learning component104can analyze the medical imaging data based on a first convolutional layer filter that comprises a first size, a second convolutional layer filter that comprises a second size that is different than the first size, and a third convolutional layer filter that comprises the first size associated with the first convolutional layer filter.

In certain embodiments, the machine learning component104can extract information that is indicative of correlations, inferences and/or expressions from the medical imaging data based on the spring network of convolutional layers. The machine learning component104can generate the learned medical imaging output based on the execution of at least one machine learning model associated with the spring network of convolutional layers. The learned medical imaging output generated by the machine learning component104can include, for example, learning, correlations, inferences and/or expressions associated with the medical imaging data. In an aspect, the machine learning component104can perform learning with respect to the medical imaging data explicitly or implicitly using the spring network of convolutional layers. The machine learning component104can also employ an automatic classification system and/or an automatic classification process to facilitate analysis of the medical imaging data. For example, the machine learning component104can employ a probabilistic and/or statistical-based analysis (e.g., factoring into the analysis utilities and costs) to learn and/or generate inferences with respect to the medical imaging data. The machine learning component104can employ, for example, a support vector machine (SVM) classifier to learn and/or generate inferences for medical imaging data. Additionally or alternatively, the machine learning component104can employ other classification techniques associated with Bayesian networks, decision trees and/or probabilistic classification models. Classifiers employed by the machine learning component104can be explicitly trained (e.g., via a generic training data) as well as implicitly trained (e.g., via receiving extrinsic information). For example, with respect to SVM's, SVM's can be configured via a learning or training phase within a classifier constructor and feature selection module. A classifier can be a function that maps an input attribute vector, x=(x1, x2, x3, x4, xn), to a confidence that the input belongs to a class—that is, f(x)=confidence(class).

To facilitate localization of one or more diseases associated with the medical imaging data, the machine learning component104can perform a local pooling process for an activation map associated with a convolutional layer of the convolutional neural network prior to performing a global pooling process associated with the convolutional neural network. Additionally or alternatively, the machine learning component104can generate the learned medical imaging output based on a class activation mapping process that applies a set of weights to a set of heat maps associated with the medical imaging data. Additionally or alternatively, the machine learning component104can process the medical imaging data based on one or more regularization techniques to classify one or more portions of the medical imaging data. In an aspect, the machine learning component104can also merge a set of classifier layers associated with the convolutional neural network and a set of activation maps associated with the convolutional neural network to generate the learned medical imaging output.

The medical imaging diagnosis component106can employ information provided by the machine learning component104(e.g., the learned medical imaging output) to classify and/or localize a disease associated with the medical imaging data. In an embodiment, the medical imaging diagnosis component106can determine a classification and an associated localization for a portion of the anatomical region based on the learned medical imaging output associated with the convolutional neural network. In certain embodiments, the medical imaging diagnosis component106can determine one or more confidence scores for the classification and/or the localization. For example, a first portion of the anatomical region with a greatest likelihood of a disease can be assigned a first confidence score, a second portion of the anatomical region with a lesser degree of likelihood of a disease can be assigned a second confidence score, etc. A disease classified and/or localized by the medical imaging diagnosis component106can include, for example, a lung disease, a heart disease, a tissue disease, a bone disease, a tumor, a cancer, tuberculosis, cardiomegaly, hypoinflation of a lung, opacity of a lung, hyperdistension, a spine degenerative disease, calcinosis, or another type of disease associated with an anatomical region of a patient body. In an aspect, the medical imaging diagnosis component106can determine a prediction for a disease associated with the medical imaging data. For example, the medical imaging diagnosis component106can determine a probability score for a disease associated with the medical imaging data (e.g., a first percentage value representing likelihood of a negative prognosis for the disease and a second value representing a likelihood of a positive prognosis for the disease).

The visualization component108can generate deep learning data (e.g., DEEP LEARNING DATA shown inFIG. 1) based on the classification and/or the localization for the portion of the anatomical region. In an embodiment, the deep learning data can include a classification and/or a location for one or more diseases located in the medical imaging data. In certain embodiments, the deep learning data can include probability data indicative of a probability for one or more diseases being located in the medical imaging data. The probability data can be, for example, a probability array of data values for one or more diseases being located in the medical imaging data. In another embodiment, the visualization component108can generate a multi-dimensional visualization associated with the classification and/or the localization for the portion of the anatomical region. The multi-dimensional visualization can be a graphical representation of the medical imaging data that shows a classification and/or a location of one or more diseases with respect to a patient body. The visualization component108can also generate a display of the multi-dimensional visualization of the diagnosis provided by the medical imaging diagnosis component106. For example, the visualization component108can render a 2D visualization of the portion of the anatomical region on a user interface associated with a display of a user device such as, but not limited to, a computing device, a computer, a desktop computer, a laptop computer, a monitor device, a smart device, a smart phone, a mobile device, a handheld device, a tablet, a portable computing device or another type of user device associated with a display. In an aspect, the multi-dimensional visualization can include the deep learning data. The deep learning data associated with the multi-dimensional visualization can be indicative of a visual representation of the classification and/or the localization for the portion of the anatomical region. The deep learning data can also be rendered on the 3D model as one or more dynamic visual elements. In an aspect, the visualization component108can alter visual characteristics (e.g., color, size, hues, shading, etc.) of at least a portion of the deep learning data associated with the multi-dimensional visualization based on the classification and/or the localization for the portion of the anatomical region. For example, the classification and/or the localization for the portion of the anatomical region can be presented as different visual characteristics (e.g., colors, sizes, hues or shades, etc.), based on a result of deep learning and/or medical imaging diagnosis by the machine learning component104and/or the medical imaging diagnosis component106. In another aspect, the visualization component108can allow a user to zoom into or out with respect to the deep learning data associated with the multi-dimensional visualization. For example, the visualization component108can allow a user to zoom into or out with respect to a classification and/or a location of one or more diseases identified in the anatomical region of the patient body. As such, a user can view, analyze and/or interact with the deep learning data associated with the multi-dimensional visualization.

It is to be appreciated that technical features of the deep learning component102are highly technical in nature and not abstract ideas. Processing threads of the deep learning component102that process and/or analyze the medical imaging data, determine deep learning data, etc. cannot be performed by a human (e.g., are greater than the capability of a single human mind). For example, the amount of the medical imaging data processed, the speed of processing of the medical imaging data and/or the data types of the medical imaging data processed by the deep learning component102over a certain period of time can be respectively greater, faster and different than the amount, speed and data type that can be processed by a single human mind over the same period of time. Furthermore, the medical imaging data processed by the deep learning component102can be one or more medical images generated by sensors of a medical imaging device. Moreover, the deep learning component102can be fully operational towards performing one or more other functions (e.g., fully powered on, fully executed, etc.) while also processing the medical imaging data.

Referring now toFIG. 2, there is illustrated a non-limiting implementation of a system200in accordance with various aspects and implementations of this disclosure. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity. The system200can include the deep learning component102, and the deep learning can include the machine learning component104, the medical imaging diagnosis component106, the visualization component108, the processor110and/or the memory112. The machine learning component104can include a spring self-transfer learning (STL) network component202.

The spring STL network component202can provide a weakly supervised framework with self-transfer learning. For instance, a convolutional neural network model employed by the spring STL network component202can be pre-trained using a set of medical images formatted as image-level labels (e.g., weak-labeled images) without location information with respect to localization of one or more features of the set of medical images. The convolutional neural network model employed by the spring STL network component202can employ a plurality of sequential downsampling and upsampling of the medical imaging data received by the spring STL network component202. For example, the plurality of sequential downsampling and upsampling can be performed by shared spring convolutional layers that behave in a spring-like manner. The shared spring convolutional layers can include convolutional layer filters with various sizes. Furthermore, one or more convolutional layer filters from the shared spring convolutional layers can be repeated. In an aspect, the spring STL network component202can generate learned medical imaging output associated with the medical imaging data based on the shared spring convolutional layers. The spring STL network component202can additionally employ classification layers and/or localization layers to generate the learned medical imaging output. For instance, the learned medical imaging output generated by the spring STL network component202can include one or more classifications for the medical imaging data that is determined based on the shared spring convolutional layers and the classification layers. Additionally or alternatively, the learned medical imaging output generated by the spring STL network component202can include one or more localizations for the medical imaging data that is determined based on the shared spring convolutional layers and the localization layers.

Referring now toFIG. 3, there is illustrated a non-limiting implementation of a system300in accordance with various aspects and implementations of this disclosure. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity. The system300can include the deep learning component102, and the deep learning can include the machine learning component104, the medical imaging diagnosis component106, the visualization component108, the processor110and/or the memory112. The machine learning component104can include the spring STL network component202. The spring STL network component202can include a spring convolutional layers component302.

The spring convolutional layers component302can execute the shared spring convolutional layers. The shared spring convolutional layers can be associated with a machine learning convolutional layer process. The shared spring convolutional layers executed by the spring convolutional layers component302can behave in a spring-like manner. For example, the shared spring convolutional layers executed by the spring convolutional layers component302can include convolutional layer filters with various sizes. Furthermore, one or more convolutional layer filters from the shared spring convolutional layers executed by the spring convolutional layers component302can be repeated. For instance, shared spring convolutional layers executed by the spring convolutional layers component302can include a first convolutional layer filter that comprises a first size, a second convolutional layer filter that comprises a second size that is different than the first size, a third convolutional layer filter that comprises the first size associated with the first convolutional layer filter, etc. In an aspect, spring convolutional layers component302can extract feature information from the medical imaging data using the shared spring convolutional layers. The feature information can include, for example, a set of data matrices (e.g., a set of feature maps) extracted from the medical imaging data. A size of the set of data matrices can be smaller than a size of a data matrix associated with the medical imaging data.

Referring now toFIG. 4, there is illustrated a non-limiting implementation of a system400in accordance with various aspects and implementations of this disclosure. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity. The system400can include the deep learning component102, and the deep learning can include the machine learning component104, the medical imaging diagnosis component106, the visualization component108, the processor110and/or the memory112. The machine learning component104can include the spring STL network component202. The spring STL network component202can include the spring convolutional layers component302and/or a classification layers component402.

The classification layers component402can be a classifier that performs a classification process to classify the medical imaging data. The classification layers component402can employ a set of classification layers (e.g., a set of fully connected layers) to perform the classification process. For instance, the classification layers component402can classify the medical imaging data into a class based on the set of classification layers (e.g., the set of fully connected layers). The classification layers component402can also employ a training dataset to facilitate the classification of the medical imaging data. For instance, the classification layers component402can also classify the medical imaging data into the class based on the training dataset. The training dataset can be generated during a training phase that trains the convolutional neural network model employed by the spring convolutional layers component302. In an embodiment, the classification layers component402can employ an automatic classification system and/or an automatic classification process to facilitate analysis of the medical imaging data. For example, the classification layers component402can employ a probabilistic and/or statistical-based analysis (e.g., factoring into the analysis utilities and costs) to learn and/or generate inferences with respect to the medical imaging data. The classification layers component402can employ, for example, a support vector machine (SVM) classifier to learn and/or generate inferences for medical imaging data. Additionally or alternatively, the classification layers component402can employ other classification techniques associated with Bayesian networks, decision trees and/or probabilistic classification models. Classifiers employed by the classification layers component402can be explicitly trained (e.g., via a generic training data) as well as implicitly trained (e.g., via receiving extrinsic information). For example, with respect to SVM's, SVM's can be configured via a learning or training phase within a classifier constructor and feature selection module. A classifier can be a function that maps an input attribute vector, x=(x1, x2, x3, x4, xn), to a confidence that the input belongs to a class—that is, f(x)=confidence(class).

Referring now toFIG. 5, there is illustrated a non-limiting implementation of a system500in accordance with various aspects and implementations of this disclosure. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity. The system500can include the deep learning component102, and the deep learning can include the machine learning component104, the medical imaging diagnosis component106, the visualization component108, the processor110and/or the memory112. The machine learning component104can include the spring STL network component202. The spring STL network component202can include the spring convolutional layers component302, the classification layers component402and/or a localization layers component502. The localization layers component502can include an activation map component504, a regularizer component506and/or a global pooling component508.

The localization layers component502can be a localizer that performs a localization process to localize one or more classifications of the medical imaging data. The localization layers component502can employ a set of localization layers to perform the localization process. For instance, the localization layers component502can localize a classification of the medical imaging data based on the set of localization layers. In an embodiment, the activation map component504can generate a set of activation maps. The set of activation maps can be score maps for each class associated with the classification layers component402. For instance, a number of activation maps included in the set of activation maps can correspond to a number of classes determined by the classification layers component402. In another embodiment, the regularizer component506can be employed to reduce overfitting associated with the set of activation maps. The regularizer component506can perform a local pooling process that reduces dimensionality of the set of activation maps. For example, the regularizer component506can include a local pooling layer to reduce singularity issues and/or to improve localization for the localization layers component502. In yet another embodiment, the global pooling component508can perform a global pooling process that further reduces dimensionality of the set of activation maps. In an aspect, a size of a filter associated with the global pooling process can be larger than a size of a filter associated with the local pooling layer. In one example, a size of a filter associated with the global pooling process can correspond to a size of the medical imaging data.

Referring now toFIG. 6, there is illustrated a non-limiting implementation of a system600in accordance with various aspects and implementations of this disclosure. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity. The system600can include the deep learning component102, and the deep learning can include the machine learning component104, the medical imaging diagnosis component106, the visualization component108, the processor110and/or the memory112. The machine learning component104can include the spring STL network component202. The spring STL network component202can include the spring convolutional layers component302, the classification layers component402, the localization layers component502and/or a training component602. The localization layers component502can include the activation map component504, the regularizer component506and/or the global pooling component508.

The training component602can perform a training phase for a neural network model employed by the spring convolutional layers component302. For example, the medical imaging data can be a set of medical images (e.g., a set of x-ray images, etc.) stored in a data store. Furthermore, the training component602can perform the training phase for a neural network model (e.g., a convolutional neural network model) based on the set of medical images stored in a data store to train the neural network model. In an embodiment, training component602can train a convolutional neural network (e.g., the neural network model) by performing iterative sequential downsampling and upsampling of the medical imaging data associated with convolutional layers of the convolutional neural network. In an aspect, the training component602can generate a set of filter values for the convolutional neural network (e.g., the neural network model) based on the iterative sequential downsampling and upsampling of the medical imaging data. For example, the training component602can generate a set of as set of weights for a set of filters associated with the convolutional neural network (e.g., the neural network model) based on the iterative sequential downsampling and upsampling of the medical imaging data. In certain embodiments, the training component602can analyze the medical imaging data based on a first convolutional layer filter that comprises a first size, analyze the medical imaging database on a second convolutional layer filter that comprises a second size that is different than the first size, analyze the medical imaging database on a third convolutional layer filter that comprises the first size associated with the first convolutional layer filter, etc.

Referring now toFIG. 7, there is illustrated a non-limiting implementation of a system700in accordance with various aspects and implementations of this disclosure. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity. The system700can include spring convolutional layers702, classification layers704, localization layers706, a weighted add phase708and/or a training phase710. The localization layers706can include activation maps712, a regularizer714and/or global pooling716.

The system700can be, for example, a spring self-transfer learning network. The spring convolutional layers702can receive the medical imaging data. In an embodiment, the spring convolutional layers702can be executed by the spring convolutional layers component302. The spring convolutional layers702can behave in a spring-like manner. For example, spring convolutional layers702can include convolutional layer filters with various sizes. Furthermore, one or more convolutional layer filters from the spring convolutional layers702can be repeated. For instance, the spring convolutional layers702can include a first convolutional layer filter that comprises a first size, a second convolutional layer filter that comprises a second size that is different than the first size, a third convolutional layer filter that comprises the first size associated with the first convolutional layer filter, etc. In an aspect, the spring convolutional layers702can extract feature information from the medical imaging data. The feature information can include, for example, a set of data matrices (e.g., a set of feature maps) extracted from the medical imaging data. A size of the set of data matrices can be smaller than a size of a data matrix associated with the medical imaging data.

The classification layers704can further process the feature information extracted from the spring convolutional layers702. In an embodiment, the classification layers704can be executed by the classification layers component402. The classification layers704can determine one or more classifications for the medical imaging data. In an aspect, each CNN neuron in a first layer (e.g., a previous layer) from the classification layers704can be connected to each neuron in a second layer (e.g., a next layer) from the classification layers704. In another aspect, the one or more classifications for the medical imaging data can be received by the weighted add phase708.

The localization layers706can also further process the feature information extracted from the spring convolutional layers702. In an embodiment, the localization layers706can be executed by the localization layers component502. The localization layers706can determine one or more localizations for the medical imaging data. In an aspect, the localization layers706can employ the activation maps712, the regularizer714and/or the global pooling716to determine the one or more localizations for the medical imaging data. In an embodiment, the activation maps712can be executed by the activation map component504, the regularizer714can correspond to the regularizer component506, and/or the global pooling716can be performed by the global pooling component508. The activation maps712can be score maps (e.g., class activation maps) for each class associated with the spring convolutional layers702. For instance, a number of the activation maps712can correspond to a number of classes associated with the spring convolutional layers702.

The regularizer714can reduce overfitting associated with the activation maps712. The regularizer714can be, for example, a local pooling process that reduces dimensionality of the activation maps712. For example, the regularizer714can include a local pooling layer that reduces dimensionality of the activation maps712. The global pooling716can be a global pooling process that further reduces dimensionality of the activation maps712. In an aspect, a size of a filter associated with the global pooling716can be larger than a size of a filter associated with the regularizer714. In one example, a size of a filter associated with the global pooling716can correspond to a size of the medical imaging data received by the spring convolutional layers702. As such, the regularizer714(e.g., a local pooling layer associated with the regularizer714) can be performed prior to the global pooling716with respect to the activation maps712to, for example, overcome one or more singularity issues and/or to improve localization by the localization layers706. In an aspect, the one or more localizations for the medical imaging data can be received by the weighted add phase708. The weighted add phase708can combine the one or more classifications and the one or more localizations to generate the learned medical imaging output. For example, the learned medical imaging output can provide a classification and/or a location for one or more features associated with the medical imaging data.

In certain embodiments, the system700can employ the training phase710. The training phase710can perform a training phase for a neural network model employed by the spring convolutional layers702. For example, the medical imaging data can be a set of medical images (e.g., a set of x-ray images, etc.) stored in a data store. Furthermore, the training phase710can perform the training phase for a neural network model (e.g., a convolutional neural network model) based on the set of medical images stored in a data store to train the neural network model. In an embodiment, training phase710can train a convolutional neural network (e.g., the neural network model) by performing iterative sequential downsampling and upsampling of the medical imaging data associated with convolutional layers of the convolutional neural network. In an aspect, the training phase710can generate a set of filter values for the convolutional neural network (e.g., the neural network model) based on the iterative sequential downsampling and upsampling of the medical imaging data. For example, the training phase710can generate a set of as set of weights for a set of filters associated with the convolutional neural network (e.g., the neural network model) based on the iterative sequential downsampling and upsampling of the medical imaging data. In an embodiment, the training phase710can analyze the medical imaging data based on a first convolutional layer filter that comprises a first size, analyze the medical imaging database on a second convolutional layer filter that comprises a second size that is different than the first size, analyze the medical imaging database on a third convolutional layer filter that comprises the first size associated with the first convolutional layer filter, etc.

Referring now toFIG. 8, there is illustrated a non-limiting implementation of a system800in accordance with various aspects and implementations of this disclosure. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity. The system800can be associated with an inference phase for a spring self-transfer learning network. The system800can include medical imaging data802that is received by a pooling layer process804. In an embodiment, the medical imaging data802can correspond to the medical imaging data received by the deep learning component102. The medical imaging data802can be, for example, two-dimensional medical imaging data and/or three-dimensional medical imaging data generated by a medical imaging device. For instance, the medical imaging data802can be electromagnetic radiation imagery captured via a set of sensors (e.g., a set of sensors associated with a medical imaging device). In one example, the medical imaging data802can be an x-ray image. The pooling layer process804can format the medical imaging data802for processing by the spring convolutional layer process806. For example, the pooling layer process804can convert the medical imaging data802into a data matrix with a particular size. In certain embodiments, the pooling layer process804can reduce dimensionality of the medical imaging data802. For example, the pooling layer process804can reduce the particular size of the data matrix. The pooling layer process804can be followed by a spring convolutional layer process806. The spring convolutional layer process806can be a machine learning convolutional layer process. Furthermore, the spring convolutional layer process806can be, for example, a sequential convolutional layer process that behaves in a spring-like manner. The spring convolutional layer process806can include a plurality of sequential downsampling and upsampling of the medical imaging data802. For example, the plurality of sequential downsampling and upsampling of the spring convolutional layer process806can be performed by shared spring convolutional layers that behave in a spring-like manner. The shared spring convolutional layers of the spring convolutional layer process806can include convolutional layer filters with various sizes. Furthermore, one or more convolutional layer filters from the shared spring convolutional layers of the spring convolutional layer process806can be repeated.

The spring convolutional layer process806can be followed by a fully connected layer process808implemented in parallel to an activation map process810. The fully connected layer process808can be a machine learning classification process that classifies the medical imaging data802. In an aspect, the fully connected layer process808can determine one or more classes for the medical imaging data802. The activation map process810can generate a set of activation maps for the medical imaging data802. For example, the set of activation maps generated by the activation map process810can be a set of score maps associated with the one or more classes determined by the fully connected layer process808. In an aspect, a number of activation maps included in the set of activation maps can correspond to a number of classes determined by the fully connected layer process808. The activation map process810can be followed by a pooling layer process812. The pooling layer process812can reduce dimensionality of the set of activation maps generated by the activation map process810.

A heat map814can be generated following the pooling layer process812. The heat map814can include one or more localizations to localize one or more classifications of the medical imaging data802. For example, the heat map814can be a graphical representation of data generated by the spring convolutional layer process806and/or the activation map process810. The data generated by the spring convolutional layer process806and/or the activation map process810can be represented as different colors based on a value of the data. For example, one or more data values that satisfy a first defined criterion (e.g., one or more data values that represents a high degree of localization) can be represented as a red color, one or more data values that satisfy a second defined criterion (e.g., one or more data values that represents a medium degree of localization) can be represented as a green color, one or more data values that satisfy a third defined criterion (e.g., one or more data values that represents a low degree of localization) can be represented as a green color, etc. The heat map814can be combined with the medical imaging data802to generate a multi-dimensional visualization816. The pooling layer process812can also be followed by a global pooling layer process818. The global pooling layer process818can further alter dimensionality of the of the set of activation maps generated by the activation map process810. For example, the global pooling layer process818alter dimensionality of the of the set of activation maps generated by the activation map process810to correspond to dimensionality of the medical imaging data802. The global pooling layer process818can be followed by a weighted add process820. The fully connected layer process808can also be followed by the weighted add process820. The weighted add process820can employ information from the spring convolutional layer process806(e.g., one or more localizations) and the fully connected layer process808(e.g., one or more classifications) to generate output probability data.

Referring now toFIG. 9, there is illustrated a non-limiting implementation of a system900in accordance with various aspects and implementations of this disclosure. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity. The system900can be associated with sequential upsampling and downsampling for a spring self-transfer learning network. In an embodiment, the system900can be associated with the spring convolutional layers component302.

The system900can include a convolutional layer902. The convolutional layer902can be a first convolutional layer of a convolutional neural network that processes medical imaging data. Furthermore, the convolutional layer902can be associated with a first filter size. The convolutional layer902can be followed by a pooling layer (down)904. The pooling layer (down)904can be associated with downsampling. For instance, the pooling layer (down)904can reduce dimensionality of data generated by the convolutional layer902. In one example, the pooling layer (down)904can reduce dimensionality of a feature map for medical imaging data processed by the convolutional layer902. The pooling layer (down)904can be followed by a convolutional layer906. The convolutional layer906can be a second convolutional layer of the convolutional neural network that processes the medical imaging data. Furthermore, the convolutional layer906can be associated with a second filter size that is different than the first filter size associated with the convolutional layer902. For example, the second filter size associated with the convolutional layer906can be smaller than the first filter size associated with the convolutional layer902. The convolutional layer906can be followed by a pooling layer (down)908. The pooling layer (down)908can be associated with downsampling. For instance, the pooling layer (down)908can reduce dimensionality of data generated by the convolutional layer906. In one example, the pooling layer (down)908can reduce dimensionality of a feature map for medical imaging data processed by the convolutional layer906. The pooling layer (down)908can be followed by a convolutional layer (not shown), which, in turn, can be followed by a pooling layer (up)910. However, in certain embodiments, the pooling layer (down)910can be followed by one or more other convolutional layers and/or one or more other pooling layers (down) prior to the pooling layer (up)910to further process medical imaging data with different filter sizes and/or further reduction to dimensionality of data. The pooling layer (up)910can be associated with upsampling. For instance, the pooling layer (up)910can increase dimensionality of data generated by one or more convolutional layers. In one example, the pooling layer (up)910can increase dimensionality of a feature map for medical imaging data processed by one or more convolutional layers. The pooling layer (up)910can be followed by a convolutional layer912. The convolutional layer912can be, for example, a third convolutional layer of the convolutional neural network that processes the medical imaging data. Furthermore, the convolutional layer912can be associated with the second filter size associated with the convolutional layer906.

The convolutional layer912can be followed by a pooling layer (up)914. The pooling layer (up)914can be associated with upsampling. For instance, the pooling layer (up)914can increase dimensionality of data generated by the convolutional layer912. In one example, the pooling layer (up)914can increase dimensionality of a feature map for medical imaging data processed by the convolutional layer912. The pooling layer (up)914can be followed by a convolutional layer916. The convolutional layer916can be, for example, a fourth convolutional layer of the convolutional neural network that processes the medical imaging data. Furthermore, the convolutional layer916can be associated with the first filter size associated with the convolutional layer912. As such, the system900can behave similar to functionality of a spring where a filter size for one or more convolutional layers are repeated while processing medical imaging data via a convolutional neural network.

FIG. 10illustrates an example multi-dimensional visualization1000, in accordance with various aspects and implementations described herein. The multi-dimensional visualization1000can, for example, display a medical imaging diagnosis for a patient. For example, the multi-dimensional visualization1000can display one or more classifications and/or one or more localizations for one or more diseases identified in medical imaging data. In an aspect, the multi-dimensional visualization1000can include localization data1002for a medical imaging diagnosis. The localization data1002can be a predicted location for a disease associated with medical imaging data processed by the machine learning component104and/or the medical imaging diagnosis component106. Visual characteristics (e.g., a color, a size, hues, shading, etc.) of the localization data1002can be dynamic based on information provided by the machine learning component104and/or the medical imaging diagnosis component106. For instance, a first portion of the localization data1002can comprise a first visual characteristic, a second portion of the localization data1002can comprise a second visual characteristic, a third portion of the localization data1002can comprise a third visual characteristic, etc. In an embodiment, a display environment associated with the multi-dimensional visualization1000can include a heat bar1004. The heat bar1004can include a set of colors that correspond to different values for the localization data1002. For example, a first color (e.g., a color red) in the heat bar1004can correspond to a first value for the localization data1002, a second color (e.g., a color green) in the heat bar1004can correspond to a second value for the localization data1002, a third color (e.g., a color blue) in the heat bar1004can correspond to a third value for the localization data1002, etc.

FIG. 11illustrates an example user interface1100, in accordance with various aspects and implementations described herein. The user interface1100can be a display environment for medical imaging data and/or deep learning data associated with medical imaging data. The user interface1100can include medical imaging data1102. In one embodiment, the medical imaging data1102can be displayed as a multi-dimensional visualization that presents a medical imaging diagnosis for a patient. For example, in certain embodiments, the medical imaging data1102can be displayed as a multi-dimensional visualization that presents one or more classifications and/or one or more localizations for one or more diseases identified in medical imaging data1102. In certain embodiments, the medical imaging data1102can be displayed as a multi-dimensional visualization that presents localization data for a medical imaging diagnosis. In another embodiment, the user interface1100can include a heat bar1104. The heat bar1104can include a set of colors that correspond to different values for the localization data. The user interface1100can also include a prediction section1106to present one or more predictions associated with the medical imaging data1102. The prediction section1106can include a patient name1108for a patient (e.g., a patient body) associated with the medical imaging data1102. The prediction section1106can also include a condition portion1110and a prediction portion1112. The condition portion1110can include one or more conditions such as, for example, a tuberculosis condition1110a, a lateral view condition1110b, a cardiomegaly condition1110c, an opacity/lung condition1110d, a lung/hypoinflation condition1110e, a hyperdistention condition1110f, a spine degenerative condition1110g, a calcinosis condition1110hand/or another type of condition. The prediction portion1112can include corresponding predictions1112a-hfor the conditions included in the condition portion1110. For example, the prediction1112acan include a prediction for the medical imaging data1102being associated with tuberculosis (e.g., a 38.42% chance of a negative prognosis for tuberculosis and a 61.58% chance of a positive prognosis for tuberculosis). In another example, the prediction1112hcan include a prediction for the medical imaging data1102being associated with calcinosis (e.g., a 40.99% chance of a negative prognosis for calcinosis and a 59.01% chance of a positive prognosis for calcinosis). In certain embodiments, the prediction section1106can also include a patient age1114, a patient gender1116and/or other information regarding a patient associated with the patient name1108. As such, in certain embodiments, the medical imaging data1102can be associated with multiple diseases. Furthermore, multiple inferencing models can be employed and aggregated as deep learning data shown in the user interface1100.

Referring toFIG. 12, there is illustrated a non-limiting implementation of a methodology1200for facilitating a deep convolutional neural network with self-transfer learning, according to an aspect of the subject innovation. At1202, medical imaging data for a patient body is received by a system comprising a processor (e.g., by machine learning component104). The medical imaging data can be, for example, a medical image such as electromagnetic radiation imagery, an x-ray image, a CT scan image, another type of medical image, etc. In an embodiment, the medical imaging data can be electromagnetic radiation imagery captured via a set of sensors (e.g., a set of sensors associated with a medical imaging device).

At1204, it is determined whether new medical imaging data is available. If yes, methodology1200returns to1202. If no, methodology1200proceeds to1206.

At1206, iterative sequential downsampling and upsampling of the medical imaging data associated with convolutional layers of a convolutional neural network is performed, by the system (e.g., by machine learning component104), to generate learned medical imaging output regarding the patient body. The iterative sequential downsampling and upsampling of the medical imaging data can behave in a spring-like manner. For example, the iterative sequential downsampling and upsampling of the medical imaging data can be associated with convolutional layer filters with various sizes. One or more convolutional layer filters can be repeated. For instance, the iterative sequential downsampling and upsampling of the medical imaging data can include a first convolutional layer filter that comprises a first size, a second convolutional layer filter that comprises a second size that is different than the first size, a third convolutional layer filter that comprises the first size associated with the first convolutional layer filter, etc. In certain embodiments, the performing the iterative sequential downsampling and upsampling of the medical imaging data can include analyzing the medical imaging data based on a first filter that comprises a first size, analyzing the medical imaging data based on a second filter that comprises a second size that is different than the first size, analyzing the medical imaging data based on a third filter that comprises the first size associated with the first filter, etc. In another embodiment, the performing the iterative sequential downsampling and upsampling of the medical imaging data can include generating the learned medical imaging output based on a first convolutional layer process associated with downsampling of the medical imaging data and a second convolutional layer process associated with upsampling of the medical imaging data. In an aspect, the performing the iterative sequential downsampling and upsampling of the medical imaging data can be associated with automated feature detection for the medical imaging data.

At1208, a disease for a portion of the patient body classifying, by the system (e.g., by medical imaging diagnosis component106), based on the learned medical imaging output associated with the convolutional neural network. A disease can include, for example, a lung disease, a heart disease, a tissue disease, a bone disease, a tumor, a cancer, tuberculosis, cardiomegaly, hypoinflation of a lung, opacity of a lung, hyperdistension, a spine degenerative disease, calcinosis, or another type of disease associated with an anatomical region of a patient body. In an embodiment, a prediction for the disease can be determined. For example, a probability score for the disease can be determined (e.g., a first percentage value representing likelihood of a negative prognosis for the disease and a second value representing a likelihood of a positive prognosis for the disease can be determined).

At1210, a multi-dimensional visualization associated with the classifying of the disease for the portion of the patient body is generated by the system (e.g., by visualization component108. The multi-dimensional visualization can be a graphical representation of the medical imaging data that shows a classification and/or a location of one or more diseases with respect to a patient body. In an aspect, visual characteristics (e.g., color, size, hues, shading, etc.) of at least a portion of the multi-dimensional visualization can be altered based on the classification and/or a location of one or more diseases with respect to a patient body.

In certain embodiments, the methodology1200can additionally include performing, by the system, a local pooling process for an activation map associated with a convolutional layer of the convolutional neural network prior to performing a global pooling process associated with the convolutional neural network. In certain embodiments, the methodology1200can additionally or alternatively include generating, by the system, the learned medical imaging output based on a class activation mapping process that applies a set of weights to a set of heat maps associated with the medical imaging data. Furthermore, in certain embodiments, the methodology1200can additionally or alternatively include merging, by the system, a set of classifier layers associated with the convolutional neural network and a set of activation maps associated with the convolutional neural network to generate the learned medical imaging output.

Referring toFIG. 13, there is illustrated a non-limiting implementation of a methodology1300for facilitating training of a deep convolutional neural network with self-transfer learning, according to an aspect of the subject innovation. At1302, medical imaging data that comprises a set of medical images is received by a system comprising a processor (e.g., by machine learning component104). The medical imaging data can be, for example, a set of medical images stored in a data store. In one example, the set of medical images can be a set of x-ray images and/or a set of CT scan images.

At1304, it is determined whether new medical imaging data is available. If yes, methodology1300returns to1302. If no, methodology1300proceeds to1306.

At1306, a convolutional neural network is training, by the system (e.g., by training component602), by performing iterative sequential downsampling and upsampling of the medical imaging data associated with convolutional layers of the convolutional neural network. The iterative sequential downsampling and upsampling of the medical imaging data can behave in a spring-like manner. For example, the iterative sequential downsampling and upsampling of the medical imaging data can be associated with convolutional layer filters with various sizes. One or more convolutional layer filters can be repeated. For instance, the iterative sequential downsampling and upsampling of the medical imaging data can include a first convolutional layer filter that comprises a first size, a second convolutional layer filter that comprises a second size that is different than the first size, a third convolutional layer filter that comprises the first size associated with the first convolutional layer filter, etc.

At1308, a set of filter values for the convolutional neural network is generated, by the system (e.g., by training component602), based on the iterative sequential downsampling and upsampling of the medical imaging data. For example, a set of weights for a set of filters associated with the convolutional neural network can be generated based on the iterative sequential downsampling and upsampling of the medical imaging data.

In order to provide a context for the various aspects of the disclosed subject matter,FIGS. 14 and 15as well as the following discussion are intended to provide a brief, general description of a suitable environment in which the various aspects of the disclosed subject matter may be implemented.

With reference toFIG. 14, a suitable environment1400for implementing various aspects of this disclosure includes a computer1412. The computer1412includes a processing unit1414, a system memory1416, and a system bus1418. The system bus1418couples system components including, but not limited to, the system memory1416to the processing unit1414. The processing unit1414can be any of various available processors. Dual microprocessors and other multiprocessor architectures also can be employed as the processing unit1414.

The system memory1416includes volatile memory1420and nonvolatile memory1422. The basic input/output system (BIOS), containing the basic routines to transfer information between elements within the computer1412, such as during start-up, is stored in nonvolatile memory1422. By way of illustration, and not limitation, nonvolatile memory1422can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, or nonvolatile random access memory (RAM) (e.g., ferroelectric RAM (FeRAM). Volatile memory1420includes random access memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM.

FIG. 14also depicts software that acts as an intermediary between users and the basic computer resources described in the suitable operating environment1400. Such software includes, for example, an operating system1428. Operating system1428, which can be stored on disk storage1424, acts to control and allocate resources of the computer system1412. System applications1430take advantage of the management of resources by operating system1428through program modules1432and program data1434, e.g., stored either in system memory1416or on disk storage1424. It is to be appreciated that this disclosure can be implemented with various operating systems or combinations of operating systems.

A user enters commands or information into the computer1412through input device(s)1436. Input devices1436include, but are not limited to, a pointing device such as a mouse, trackball, stylus, touch pad, keyboard, microphone, joystick, game pad, satellite dish, scanner, TV tuner card, digital camera, digital video camera, web camera, and the like. These and other input devices connect to the processing unit1414through the system bus1418via interface port(s)1438. Interface port(s)1438include, for example, a serial port, a parallel port, a game port, and a universal serial bus (USB). Output device(s)1440use some of the same type of ports as input device(s)1436. Thus, for example, a USB port may be used to provide input to computer1412, and to output information from computer1412to an output device1440. Output adapter1442is provided to illustrate that there are some output devices1440like monitors, speakers, and printers, among other output devices1440, which require special adapters. The output adapters1442include, by way of illustration and not limitation, video and sound cards that provide a means of connection between the output device1440and the system bus1418. It should be noted that other devices and/or systems of devices provide both input and output capabilities such as remote computer(s)1444.

Computer1412can operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s)1444. The remote computer(s)1444can be a personal computer, a server, a router, a network PC, a workstation, a microprocessor based appliance, a peer device or other common network node and the like, and typically includes many or all of the elements described relative to computer1412. For purposes of brevity, only a memory storage device1446is illustrated with remote computer(s)1444. Remote computer(s)1444is logically connected to computer1412through a network interface1448and then physically connected via communication connection1450. Network interface1448encompasses wire and/or wireless communication networks such as local-area networks (LAN), wide-area networks (WAN), cellular networks, etc. LAN technologies include Fiber Distributed Data Interface (FDDI), Copper Distributed Data Interface (CDDI), Ethernet, Token Ring and the like. WAN technologies include, but are not limited to, point-to-point links, circuit switching networks like Integrated Services Digital Networks (ISDN) and variations thereon, packet switching networks, and Digital Subscriber Lines (DSL).

Communication connection(s)1450refers to the hardware/software employed to connect the network interface1448to the bus1418. While communication connection1450is shown for illustrative clarity inside computer1412, it can also be external to computer1412. The hardware/software necessary for connection to the network interface1448includes, for exemplary purposes only, internal and external technologies such as, modems including regular telephone grade modems, cable modems and DSL modems, ISDN adapters, and Ethernet cards.

FIG. 15is a schematic block diagram of a sample-computing environment1500with which the subject matter of this disclosure can interact. The system1500includes one or more client(s)1510. The client(s)1510can be hardware and/or software (e.g., threads, processes, computing devices). The system1500also includes one or more server(s)1530. Thus, system1500can correspond to a two-tier client server model or a multi-tier model (e.g., client, middle tier server, data server), amongst other models. The server(s)1530can also be hardware and/or software (e.g., threads, processes, computing devices). The servers1530can house threads to perform transformations by employing this disclosure, for example. One possible communication between a client1510and a server1530may be in the form of a data packet transmitted between two or more computer processes.

The system1500includes a communication framework1550that can be employed to facilitate communications between the client(s)1510and the server(s)1530. The client(s)1510are operatively connected to one or more client data store(s)1520that can be employed to store information local to the client(s)1510. Similarly, the server(s)1530are operatively connected to one or more server data store(s)1540that can be employed to store information local to the servers1530.

It is to be noted that aspects or features of this disclosure can be exploited in substantially any wireless telecommunication or radio technology, e.g., Wi-Fi; Bluetooth; Worldwide Interoperability for Microwave Access (WiMAX); Enhanced General Packet Radio Service (Enhanced GPRS); Third Generation Partnership Project (3GPP) Long Term Evolution (LTE); Third Generation Partnership Project 2 (3GPP2) Ultra Mobile Broadband (UMB); 3GPP Universal Mobile Telecommunication System (UMTS); High Speed Packet Access (HSPA); High Speed Downlink Packet Access (HSDPA); High Speed Uplink Packet Access (HSUPA); GSM (Global System for Mobile Communications) EDGE (Enhanced Data Rates for GSM Evolution) Radio Access Network (GERAN); UMTS Terrestrial Radio Access Network (UTRAN); LTE Advanced (LTE-A); etc. Additionally, some or all of the aspects described herein can be exploited in legacy telecommunication technologies, e.g., GSM. In addition, mobile as well non-mobile networks (e.g., the Internet, data service network such as internet protocol television (IPTV), etc.) can exploit aspects or features described herein.

While the subject matter has been described above in the general context of computer-executable instructions of a computer program that runs on a computer and/or computers, those skilled in the art will recognize that this disclosure also can or may be implemented in combination with other program modules. Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks and/or implement particular abstract data types. Moreover, those skilled in the art will appreciate that the inventive methods may be practiced with other computer system configurations, including single-processor or multiprocessor computer systems, mini-computing devices, mainframe computers, as well as personal computers, hand-held computing devices (e.g., PDA, phone), microprocessor-based or programmable consumer or industrial electronics, and the like. The illustrated aspects may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. However, some, if not all aspects of this disclosure can be practiced on stand-alone computers. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

As used in this application, the terms “component,” “system,” “platform,” “interface,” and the like, can refer to and/or can include a computer-related entity or an entity related to an operational machine with one or more specific functionalities. The entities disclosed herein can be either hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a server and the server can be a component. One or more components may reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers.

In another example, respective components can execute from various computer readable media having various data structures stored thereon. The components may communicate via local and/or remote processes such as in accordance with a signal having one or more data packets (e.g., data from one component interacting with another component in a local system, distributed system, and/or across a network such as the Internet with other systems via the signal). As another example, a component can be an apparatus with specific functionality provided by mechanical parts operated by electric or electronic circuitry, which is operated by a software or firmware application executed by a processor. In such a case, the processor can be internal or external to the apparatus and can execute at least a part of the software or firmware application. As yet another example, a component can be an apparatus that provides specific functionality through electronic components without mechanical parts, wherein the electronic components can include a processor or other means to execute software or firmware that confers at least in part the functionality of the electronic components. In an aspect, a component can emulate an electronic component via a virtual machine, e.g., within a cloud computing system.

As used herein, the terms “example” and/or “exemplary” are utilized to mean serving as an example, instance, or illustration. For the avoidance of doubt, the subject matter disclosed herein is not limited by such examples. In addition, any aspect or design described herein as an “example” and/or “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs, nor is it meant to preclude equivalent exemplary structures and techniques known to those of ordinary skill in the art.

Various aspects or features described herein can be implemented as a method, apparatus, system, or article of manufacture using standard programming or engineering techniques. In addition, various aspects or features disclosed in this disclosure can be realized through program modules that implement at least one or more of the methods disclosed herein, the program modules being stored in a memory and executed by at least a processor. Other combinations of hardware and software or hardware and firmware can enable or implement aspects described herein, including a disclosed method(s). The term “article of manufacture” as used herein can encompass a computer program accessible from any computer-readable device, carrier, or storage media. For example, computer readable storage media can include but are not limited to magnetic storage devices (e.g., hard disk, floppy disk, magnetic strips . . . ), optical discs (e.g., compact disc (CD), digital versatile disc (DVD), blu-ray disc (BD) . . . ), smart cards, and flash memory devices (e.g., card, stick, key drive . . . ), or the like.

In this disclosure, terms such as “store,” “storage,” “data store,” data storage,” “database,” and substantially any other information storage component relevant to operation and functionality of a component are utilized to refer to “memory components,” entities embodied in a “memory,” or components comprising a memory. It is to be appreciated that memory and/or memory components described herein can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory.

By way of illustration, and not limitation, nonvolatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), flash memory, or nonvolatile random access memory (RAM) (e.g., ferroelectric RAM (FeRAM). Volatile memory can include RAM, which can act as external cache memory, for example. By way of illustration and not limitation, RAM is available in many forms such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM). Additionally, the disclosed memory components of systems or methods herein are intended to include, without being limited to including, these and any other suitable types of memory.

It is to be appreciated and understood that components, as described with regard to a particular system or method, can include the same or similar functionality as respective components (e.g., respectively named components or similarly named components) as described with regard to other systems or methods disclosed herein.