Methods, systems, articles of manufacture and apparatus for code review assistance for dynamically typed languages

Methods, apparatus, systems and articles of manufacture are disclosed for code review assistance for dynamically typed languages. An example apparatus to analyze a segment of code includes a function identifier to identify a first input of a first function call included in the segment of the code, a parameter type vector (PTV) estimator model to estimate a first data structure based on the first input, the PTV estimator model generated via a set of reviewed code, a PTV determiner to generate a second data structure based on a data parameter type of the first input, an error comparator to determine a first reconstruction error based on the first data structure, and the second data structure and a recommendation generator to, if the first reconstruction error does not satisfy a recommendation threshold, generate a first recommendation to review the first function call.

FIELD OF THE DISCLOSURE

This disclosure relates generally to code review technology, and, more particularly, to methods, systems, articles of manufacture, and apparatus for code review assistance for dynamically typed languages.

BACKGROUND

Statically typed languages (e.g., C, C+, Java, Fortran, etc.) check the type of variables (e.g., float, integer, string, etc.) while the code is compiling. Dynamically typed languages (e.g., Python, JavaScript, Ruby, Perl, etc.) do not require compiling and the types of variables used in these languages are checked at code run-time. Accordingly, errors in scripts written with dynamically typed languages are often discovered during run-time.

Code review ensures software quality by allowing multiple programmers to check software by viewing and editing source code. Source code is often uploaded to an online repository, such as Github. After source code has been uploaded, reviewers are able to look for and correct errors in the source code. A thorough and iterative code review process can help ensure quality software.

The figures are not to scale. Instead, the thickness of the layers or regions may be enlarged in the drawings. In general, the same reference numbers will be used throughout the drawing(s) and accompanying written description to refer to the same or like parts.

DETAILED DESCRIPTION

In the following detailed description, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration specific examples that may be practiced. These examples are described in sufficient detail to enable one skilled in the art to practice the subject matter, and it is to be understood that other examples may be utilized and that logical, mechanical, electrical and other changes may be made without departing from the scope of the subject matter of this disclosure. The following detailed description is, therefore, provided to describe an exemplary implementation and not to be taken as limiting on the scope of the subject matter described in this disclosure. Certain features from different aspects of the following description may be combined to form yet new aspects of the subject matter discussed below.

Artificial intelligence (AI), including machine learning (ML), deep learning (DL), and/or other artificial machine-driven logic, enables machines (e.g., computers, logic circuits, etc.) to use a model to process input data to generate an output based on patterns and/or associations previously learned by the model via a training process. For instance, the model may be trained with data to recognize patterns and/or associations and follow such patterns and/or associations when processing input data such that other input(s) result in output(s) consistent with the recognized patterns and/or associations.

Throughout the specification and claims, the following terms take the meanings explicitly associated herein unless the context clearly dictates otherwise. The term “neural network” refers to a computing system or other processor system that learns to perform a task by analyzing pre-classified training examples. Neural networks include a plurality of densely connected processing nodes inspired by the human brain. In certain examples, the nodes of neural networks can be organized into layers in which data moves in the forward direction (e.g., data in the first layer moves into the second layer, data in the second layer moves into the third layer, etc.), for example, to drive one or more outputs based on one or more inputs via correlations (e.g., connections) represented by the nodes and their interconnections. Deep learning and/or machine learning can be implemented via a neural network to process incoming data to generate an output and benefit from feedback to improve its processing. A “recurrent neural network” or “RNN” is a type of neural network in which nodes or cells include loops to allow information to persist over time. Thus, the RNN can leverage reasoning about previous events to inform subsequent processing. In an RNN, a memory or other internal state is used to process input sequence(s) in an element-by-element process wherein an output for each element is dependent on the output of previous and/or other elements (e.g., a directed graph driving a sequence).

Many different types of machine learning models and/or machine learning architectures exist. In examples disclosed herein, a “long short-term memory” network is used. However, other types of machine learning models could additionally or alternatively be used such as, for example, a support vector machine (SVM), a different type of RNN, a convolutional neural network (CNN), etc. “Long short-term memory” networks or “LSTM” networks are RNNs designed to handle long-term dependencies. Generally, LSTM networks are organized into cells and gates which interact to optimize the output of the network. Information from outside the processing of the current element (e.g., information from previous elements) is stored in gated cells. These gates release information based on the weight of the gates, which are adjusted and optimized during the training phase of the AI. In an LSTM network (or its pared-down variant gated recurrent unit network), the nodes or cells in the network have storage and an associated stored state under control of the neural network to aid in establishing correlations and processing input data. An “encoder-decoder LSTM” is a type of LSTM that is comprised of two RNN models, an encoder model and a decoder model. Generally, the encoder model reads an input and encodes it into a fixed length vector. The decoder model maps the fixed length vector into a variable length output. A common application of encoder-decoder LSTM networks is text translation.

Different types of training may be performed based on the type of machine learning model and/or the expected output. For example, supervised training uses inputs and corresponding expected (e.g., labeled) outputs to select parameters (e.g., by iterating over combinations of select parameters) for the machine learning model that reduce model error. Unsupervised learning refers to an expected output of the machine learning model (e.g., a classification, an expected output value, etc.) being assumed from unlabeled input data by the neural network. Additionally or alternatively, unsupervised training (e.g., used in deep learning, a subset of machine learning, etc.) involves inferring patterns from inputs to select parameters for the machine learning model (e.g., without the benefit of expected (e.g., labeled) outputs). Some examples disclosed herein include models that are trained via supervised training. However, any other type of training/learning can be used.

Dynamically typed languages are rapidly gaining popularity. Traditional code review methods of dynamically typed languages often require a significant time investment. For example, because code errors are discovered while the code is running, errors in the code are often discovered serially. That is, each error in the code is encountered one at a time. In such examples, if the code contains two or more errors (e.g., a first error, a second error, etc.), the first error would cause the code to crash and would require a programmer to fix it. After fixing the first error, the second error would cause the code to crash and require a similar fix from the programmer. As such, each error in a dynamically typed language must be discovered and fixed individually before the next error can be found. Accordingly, the review of code corresponding to dynamically typed languages can be time-consuming and inefficient.

Examples disclosed herein overcome the above obstacles and improve code review techniques by analyzing source code and providing review suggestions using a neural network tuned to process the function calls of code and generate review suggestions based on this analysis. Examples disclosed herein reduce the amount of iterative and serial review required to develop software code with dynamically typed languages by generating review recommendations for new code simultaneously. In some examples disclosed herein, the inputs of the function calls of a segment of code are determined and analyzed by a neural network (e.g., an encoder-decoder LSTM, etc.). In some examples disclosed herein, the neural network is generated and trained using previously checked and error-free code retrieved from a repository. In some examples disclosed herein, a neural network can generate a predicted data structure corresponding to the inputs of a function call. In such examples disclosed herein, the predicted data structure can be compared to the actual data structure associated with the function call. In such examples, the difference between the predicted data structure and the actual data structure can be compared to determine if there is an error associated with the function call.

FIG. 1is an illustration of a code review system100constructed in accordance with the teachings of this disclosure. The example code review system100includes an example new code102, an example repository104, an example code review platform106, an example machine programming engine108, an example model storage110, an example data collector112, an example common vulnerable and exposures (CVE) feed114, an example training database116and an example model generator118.

The example new code interface101acquires, retrieves, and/or otherwise receives example new code102drafted by one or more programmers associated with the code review system100. For example, one or more programmers associated with the code review systems100can manually input the new code102into the new code interface101. In the illustrated example ofFIG. 1, the new code102is written in a dynamically typed language (e.g., Python, JavaScript, Ruby, Perl, Matlab, etc.). In some examples, the new code102includes one or more function calls, each of which includes one or more inputs. In some examples, the function inputs are of a particular data type (e.g., integer, floating point, complex number, string, Boolean, mapping, set, etc.). In some examples, errors in the function input can be fatal. As used herein, the term “fatal” refers to a condition that causes the new code102obtained by the new code interface101to stop executing (e.g., exit, etc.) and, in some examples output (e.g., print, etc.) an error message. In some examples, the new code102obtained by the new code interface101can include one or more function calls including inputs with incorrectly assigned data types (e.g., a type error). As used herein, the term “type error” refers to an error that occurs when a function is applied to an input of an inappropriate type. In some examples, a type error can be fatal. Additionally or alternatively, a type error can cause the new code102obtained by the new code interface101to be unable to execute its intended function.

The example repository104is a file archive and hosting service where the obtained by the new code interface101transmits the new code102to be stored. In some examples, the repository104is an online hosted service (e.g., Github®, Assembla®, SourceForge®, etc.). In other examples, the repository104is a physical location connected to the code review platform106that can be accessed via a physical connection, an intranet, etc. In some examples, the repository104allows revision control, error tracking, and other code review related functionality.

The example code review platform106is a code review tool (e.g., Gerrit®, Crucible®, etc.) that allows multiple developers to simultaneously work on the new code102. In some examples, the code review platform106assists in the review of new code102by checking for coding style, naming conventions, reviewing for errors, etc. In some examples, the code review platform106is a tool incorporated in the repository104and/or the new code interface101. In other examples, the code review platform106, the new code interface101, and the repository104can be independent software.

The example machine programming engine108analyzes the new code102to determine if there are any potential errors (e.g., type errors, etc.) that need to be corrected. In some examples, the machine programming engine108analyzes each function call of the new code102. For example, the machine programming engine108can determine a parameter type vector (PTV) for each function call based on the inputs of the function call. As used herein, the term “parameter type vector” and the acronym “PTV” refers to a data structure (e.g., a vector, etc.) that indicates the parameters types (e.g., integer, floating point number, etc.) of the inputs of a function. In such examples, the machine programming engine108can compare the determined PTV to a predicted PTV generated by a machine learning model trained on a set of reviewed code. In some examples, if the difference (e.g., a Euclidean distance, etc.) between the determined PTV and the predicted PTV satisfies an error threshold, the machine programming engine108can generate an error warning associated with the analyzed function call of the new code102. The example machine programming engine108can also identify structural issues (e.g., code/logic separation of functional blocks, etc.), suggest design patterns (e.g., enforcing a singleton, etc.), detect system-wide errors (e.g., race conditions, deadlocks, etc.), detect security issues (e.g., via a model trained using the CVE feed114, etc.) and/or recommend language features. In some examples, the machine programming engine108is incorporated into the code review platform106and/or the repository104. In other examples, the machine programming engine108hosted on the web and/or a local computer.

The example model storage110is the location where the current model(s) utilized by the machine programming engine108are stored. In some examples, the model storage110is an online location (e.g., a drop box, etc.). In other examples, the model storage110can be implemented by physical storage (e.g., a hard drive, etc.). In the illustrated example ofFIG. 1, the model(s) stored in the model storage110were transmitted to the model storage110by the model generator118. In some examples, when an updated model is received or otherwise retrieved from the model storage110, the previous iteration of the model is deleted. In other examples, the previous models are stored in model storage110after an updated model has been received.

The example data collector112retrieves model generator data from the repository104and/or CVE feed114for use by the model generator118. For example, the data collector112can query the repository104for fully reviewed code (e.g., codes with no known errors, etc.) using a delta feed. In other examples, the data collector112can pull (e.g., on a periodic, on an aperiodic, or scheduled basis, etc.) fully reviewed code from the repository104. In some examples, the data collector112can also query the CVE feed114for periodic updates on common vulnerabilities and exposures of the new code102. In some examples, the data collector112can tag blocks of code, functions and/or classes for use by the model generator118. For example, the data collector112can tag collected code based on the function of the collected code (e.g., using metadata from the repository104, using analytics, etc.). In some examples, the data collector112can determine the time complexity and/or space complexity of collected code. In some examples, the data collector112can use batch analytics.

The CVE feed114is a data feed which includes a list of common vulnerabilities and exposures in code. In some examples, the CVE feed114is generated by a publicly available CVE system operated by a concerned entity (e.g., The MITRE Corporation®, a government agency, etc.). In such examples, the CVE feed114is composed of unique identifiers for publicly known vulnerabilities in software. In some examples, the CVE feed114is continuously updated as new vulnerabilities are discovered. In some examples, the data collector112stores the collected data in the example training database116. In some examples, the data collector112can modify and normalize the data to be stored in the example training database116.

The example model generator118trains a machine learning model using the data collected by the data collector112. For example, the example model generator118trains a machine learning model (e.g., a PTV estimator model210ofFIG. 2as described in further detail below, etc.) based on the data contained within the training database116. In such examples, the model generator118can divide the data of the training database116into training data (e.g., data used to shape the machine learning model, etc.) and verification data (e.g., data used to test the machine learning model). In some examples, the model generator118generates the model using hyperparameters (e.g., a learning rate, a number of layers to be used in the machine learning model, etc.). In some examples, the model generator118can generate a new machine learning model (e.g., retrain the existing model, etc.). The model generator118can retrain the model in response to additional training data becoming available in the training database116, in response to a threshold amount of time elapsing since generation of a prior model, etc.

In operation, a developer of the new code102pushes (e.g., commits, etc.) the new code102from the new code interface101to the repository104via an example first message122. In other examples, the code review platform106and/or the machine programming engine108can detect when the new code102is ready for review and query the new code interface101for the new code102. When the example first message122is transmitted by the new code interface101, an example code review request120is transmitted by the new code interface101to the code review platform106. In some examples, the code review request120is generated by the new code interface101in response to the first message122. In other examples, the developer manually creates the code review request120. Upon receiving the new code102, the code review platform106generates a review request124and transmits the review request124to the machine programming engine108. The machine programming engine108will then transmit an example request126for the committed code and associated metadata (e.g., information identifying the new code102, etc.) from the repository104. The machine programming engine108then performs an example analysis130using an example model132transmitted from the model storage110. In some examples, the analysis130determines what function calls of the new code102should be flagged for reviewed. An example implementation of the example analysis130is described in greater detail below in conjunction withFIGS. 4-6.

To train the model132used by the machine programming engine108, the data collector112collects example reviewed code134from the repository104. In other examples, the example reviewed code134can come from any other suitable source (e.g., a different repository, locally from the programmers, etc.). In the illustrated example ofFIG. 1, the data collector112can also gather data from the CVE feed114. In some examples, the data collector112can preprocess (e.g., normalize, clean, transform, enrich, etc.) the reviewed code134to facilitate the training of the model. In other examples, the model generator118can preprocess the reviewed code134. In the illustrated example ofFIG. 1, the data collector112can transmit the preprocessed code136to the training database116. After some period of time (e.g., periodically, aperiodically, scheduled, after a threshold period of time, after an error rate of the current model is exceeded, etc.), the example model generator118transmits a query138to the training database116for model generation data140. In some examples, the model generation data140is divided into training data and verification data for use during example model generation142. Once the example model generation142has concluded, the example model132is transmitted to the model storage110until the model132is requested by the machine programming engine108. In some examples, the model132can be pushed to the machine programming engine108as soon as it is generated by the model generator118.

FIG. 2is a block diagram of the machine programming engine108ofFIG. 1. The example machine programming engine108ofFIG. 2includes an example code review interface202, an example repository interface204, an example function identifier206, an example PTV determiner208, an example PTV estimator model210, an example error comparator212, an example recommendation generator214, and an example recommendation aggregator216. In the illustrated example ofFIG. 2, the example code review interface202is a first transmitting means. In the illustrated example ofFIG. 2, the example repository interface204is a second transmitting means. In the illustrated example ofFIG. 2, the example function identifier206is an identifying means. In the illustrated example ofFIG. 2, the example PTV determiner208is a PTV determining means. In the illustrated example ofFIG. 2, the example PTV estimator model210is a PTV estimating means. In the illustrated example ofFIG. 2, the example error comparator212is an error comparing means. In the illustrated example ofFIG. 2, an example recommendation generator214is a recommendation generating means. In the illustrated example ofFIG. 2, an example recommendation aggregator216is a recommendation aggregating means. As used herein, the example first transmitting means, the example second transmitting means, the example identifying means, the example PTV determining means, the example PTV estimating means, error comparing means, recommendation generating means and the recommendation aggregating means are hardware.

The example code review interface202is an interface that facilitates communication between the code review platform106and the machine programming engine108. For example, the code review interface202packages and transmits code review recommendations (e.g., the recommendations128, recommendations generated by recommendation generator214and recommendation aggregator216, etc.). In some examples, the code review interface202can receive review requests (e.g., the review request124, etc.). In some examples, the code review interface202converts received requests into a format readable by the machine programming engine108. In some examples, the code review interface202can be implemented by hardware (e.g., a modem, a direct connection, etc.) and/or software (e.g., an application programming interface, etc.).

The example repository interface204is an interface that facilitates communication between the machine programming engine108and the repository104. For example, the repository interface204packages and transmits requests for the committed new code102and associated metadata. In some examples, the repository interface204receives the committed new code102and associated metadata. In some examples, the repository interface204converts received requests into a format readable by the machine programming engine108. In some examples, the repository interface204can be implemented by hardware (e.g., a modem, a direct connection, etc.) and/or software (e.g., an application programming interface, etc.).

The example function identifier206analyzes the new code102and identifies the function calls contained therein. For example, the function identifier206identifies the function type, input parameters, outputs, arguments, etc. associated with each function of the new code102. In some examples, the function identifier206creates a data structure (e.g., a vector, a matrix, etc.) containing data associated with each identified function and associated parameters. In some examples, the function identifier206preprocesses each function identified in the new code102. For example, the function identifier206can normalize, vectorize, etc. each function identified in a way suitable to ensure the PTV estimator model210can process the identified function calls.

The example PTV determiner208analyzes an identified function to determine the parameter type vector (PTV) associated with each function identified by the function identifier206. For example, the PTV determiner208generates a vector based on the input parameter type(s) of each function of the new code102. In some examples, the PTV determined by the PTV determiner208employs conventional mathematics, operations, and/or lookup tables (e.g., not determined via machine learning, etc.). In some examples, the PTV determined by the PTV determiner208is a one-hot encoded vector. An example implementation of the PTV determiner208is described in further detail in conjunction withFIG. 2.

The example PTV estimator model210is a machine learning algorithm and/or neural network that analyzes each function identified by function identifier206. As noted above, artificial intelligence (AI), including machine learning (ML), deep learning (DL), and/or other artificial machine-driven logic, enables the example PTV estimator model210to analyze each function identified by function identifier206to estimate a PTV. For example, the PTV estimator model210can be an encoder-decoder LSTM model that is trained and verified using reviewed code from the repository104. In other examples, the PTV estimator model210can be any other suitable type of machine learning network (e.g., an RNN, a gated recurrent unit (GRU), a deep convolutional network (DCN), an echo state network (ESN), a deep residual network (DRN), etc.) In some examples, the PTV estimator model210is periodically updated with a delta feed from the repository104and/or any other suitable source of data (e.g., the CVE feed114, etc.). In some examples, the PTV estimator model210can be multiple models that are used based on the software type of the new code102. For example, the PTV estimator model210can include a model associated with a first type of code (e.g., operating system software, etc.) and a model associated with a second type of code (e.g., gaming software, etc.). In such examples, the PTV estimator model210can be based on the metadata received from the repository104via the repository interface204.

The example error comparator212compares the PTV determined by the PTV determiner208and the PTV estimated by the PTV estimator model210to determine the difference between them. For example, the error comparator212determines a reconstruction error based on the comparison. For example, the error comparator212can determine the Euclidean distance between PTV determined by the PTV determiner208and the PTV estimated by the PTV estimator model210. In other examples, the error comparator212can use any other suitable technique of comparing the determined PTV by the PTV determiner208and the PTV estimated by the PTV estimator model210(e.g., cosine distance, etc). Additionally or alternatively, the error comparator212can compare any other suitable characteristic of the determined PTV and the estimated PTV (e.g., the value of a specific dimension, etc.).

The example recommendation generator214generates a review recommendation based on the reconstruction error determined by error comparator212. For example, the recommendation generator214can determine if a function needs to have a review recommendation generated based on if the reconstruction error does not satisfy a reconstruction threshold. In some examples, the reconstruction threshold can be determined based on the reconstruction errors determined during the training of the PTV estimator model210. In other examples, an operator of the machining learning engine108has a tuning parameter to control the frequency of the code review recommendations occurring. In some examples, the recommendation generator214does not generate a recommendation if the construction error does satisfy the threshold.

The example recommendation aggregator216receives the generated recommendation(s) from the recommendation generator214. In some examples, the example recommendation aggregator216waits for each function of the new code102to be analyzed by the PTV determiner208, the PTV estimator model210and/or the error comparator212. In such examples, the recommendation aggregator216transmits a list of functions that require review to the code review platform106via the code review interface202. In other examples, the recommendation aggregator216transmits the recommendation(s) to the code review interface as the recommendations are generated.

While an example manner of implementing the machine programming engine108ofFIG. 1is illustrated inFIG. 2, one or more of the elements, processes and/or devices illustrated inFIG. 4may be combined, divided, re-arranged, omitted, eliminated and/or implemented in any other way. Further, the example code review interface202, the example repository interface204, the example function identifier206, the example PTV determiner208, the example PTV estimator model210, the example error comparator212, the example recommendation generator214, the example recommendation aggregator216, and/or, more generally, the example machine programming engine108ofFIGS. 1 and/or 2may be implemented by hardware, software, firmware and/or any combination of hardware, software and/or firmware. Thus, for example, any of the example code review interface202, the example repository interface204, the example function identifier206, the example PTV determiner208, the example PTV estimator model210, the example error comparator212, the example recommendation generator214, the example recommendation aggregator216, and/or, more generally, the example machine programming engine108ofFIGS. 1 and/or 2and/or, more generally, the example machine programming engine108could be implemented by one or more analog or digital circuit(s), logic circuits, programmable processor(s), programmable controller(s), graphics processing unit(s) (GPU(s)), digital signal processor(s) (DSP(s)), application specific integrated circuit(s) (ASIC(s)), programmable logic device(s) (PLD(s)) and/or field programmable logic device(s) (FPLD(s)). When reading any of the apparatus or system claims of this patent to cover a purely software and/or firmware implementation, at least one of the example code review interface202, the example repository interface204, the example function identifier206, the example PTV determiner208, the example PTV estimator model210, the example error comparator212, the example recommendation generator214, the example recommendation aggregator216is/are hereby expressly defined to include a non-transitory computer readable storage device or storage disk such as a memory, a digital versatile disk (DVD), a compact disk (CD), a Blu-ray disk, etc. including the software and/or firmware. Further still, the example machine programming engine108ofFIG. 1may include one or more elements, processes and/or devices in addition to, or instead of, those illustrated inFIGS. 1 and 2, and/or may include more than one of any or all of the illustrated elements, processes and devices. As used herein, the phrase “in communication,” including variations thereof, encompasses direct communication and/or indirect communication through one or more intermediary components, and does not require direct physical (e.g., wired) communication and/or constant communication, but rather additionally includes selective communication at periodic intervals, scheduled intervals, aperiodic intervals, and/or one-time events.

FIG. 3is a block diagram of the example PTV determiner208ofFIG. 2. The example PTV determiner208includes an example literal parameters type estimator302, an example variable type static analyzer304, an example one-hot encoder306, and an example concatenator308. In the illustrated example ofFIG. 3, the example literal parameters type estimator302is a literal parameter type estimating means. In the illustrated example of FIG.3, the example variable type static analyzer304is a static variable analyzing means. In the illustrated example ofFIG. 3, the example one-hot encoder306is an encoding means. In the illustrated example ofFIG. 3, the example concatenator308is a concatenating means. As used herein, the example literal parameter type estimating means, the example static variable analyzing means, the example encoding means, the example concatenating means are hardware.

The example literal parameter type estimator302ofFIG. 2performs literal type detection and embedding for each input of the function of the new code102. For example, the literal parameter type estimator302can analyze each input of each function of the new code102to detect if the new input of the function is a literal type object and identify the literal type of the input. As used herein, “literal type” objects are commonly used objects with a fixed value in the programming language of the new code102. For example, the literal object types can include string, integer, floating point number, Boolean, Unicode, etc. In some examples, the literal parameter type estimator302outputs a data structure (e.g., a scalar, a vector, etc.) representative of the identified literal type. In some examples, the literal parameter type estimator302outputs a data structure if an input is not a literal type object (e.g., an empty marker, etc.).

The example variable type static analyzer304analyzes the function call to determine the primitive data types of inputs. For example, the variable type static analyzer304can analyze the log files (e.g., the results of previous function calls, etc.) to determine if the input objects are associated with a primitive data type. As used herein, “primitive type” objects are basic building block objects that include characters, integers, floating point number, Boolean, reference, etc. For example, if a log file of the new code102indicates that an input object of a function call is assigned “integer” four times and “character” one time, the example variable type static analyzer304determines that the first object is an integer. In other examples, any other method of determining the input variable type can be used by the variable type static analyzer304. In some examples, the variable type static analyzer304outputs a data structure (e.g., a scalar, a vector, etc.) representative of the identified primitive type.

The example one-hot encoder306receives the outputs of the literal parameter type estimator302and the variable type static analyzer304and converts the outputs into a vector. In some examples, the vectors generated by the one-hot encoder306are a binary sparse vector in which one dimension (e.g. one index, etc.) has a value of “1” and each of the other dimensions have a value of “0.” In some examples, each dimension of the vector represents a different possible data type. For example, if the example variable type static analyzer304can identify 10 different data types, each vector generated by the one-hot encoder306will be ten dimensions long with each dimension corresponding to a specific data type.

The example concatenator308concatenates (e.g., “stacks,” appends, etc.) the outputs of the variable type static analyzer304and literal parameter type estimator302into a PTV. In some examples, to save memory, the concatenator308scalarizes each output of the one-hot encoder306into a scalar value representing the would-be index of the “1” in the associated sparse vector. In such examples, the PTV can be a scalar value. Additionally or alternatively, the concatenator308can create any suitable type of data structure

While an example manner of implementing the example PTV determiner208ofFIG. 2is illustrated inFIG. 3, one or more of the elements, processes and/or devices illustrated inFIG. 3may be combined, divided, re-arranged, omitted, eliminated and/or implemented in any other way. Further, the example literal parameters type estimator302, the example variable type static analyzer304, the example one-hot encoder306, the example concatenator308and/or, more generally, the example PTV determiner208ofFIGS. 2and/or3may be implemented by hardware, software, firmware and/or any combination of hardware, software and/or firmware. Thus, for example, any of the example literal parameters type estimator302, the example variable type static analyzer304, the example one-hot encoder306, the example concatenator308and/or, more generally, the example PTV determiner208could be implemented by one or more analog or digital circuit(s), logic circuits, programmable processor(s), programmable controller(s), graphics processing unit(s) (GPU(s)), digital signal processor(s) (DSP(s)), application specific integrated circuit(s) (ASIC(s)), programmable logic device(s) (PLD(s)) and/or field programmable logic device(s) (FPLD(s)). When reading any of the apparatus or system claims of this patent to cover a purely software and/or firmware implementation, at least one of the example literal parameters type estimator302, the example variable type static analyzer304, the example one-hot encoder306, the example concatenator308is/are hereby expressly defined to include a non-transitory computer readable storage device or storage disk such as a memory, a digital versatile disk (DVD), a compact disk (CD), a Blu-ray disk, etc. including the software and/or firmware. Further still, the example PTV determiner208ofFIGS. 2 and 3may include one or more elements, processes and/or devices in addition to, or instead of, those illustrated inFIG. 3, and/or may include more than one of any or all of the illustrated elements, processes and devices. As used herein, the phrase “in communication,” including variations thereof, encompasses direct communication and/or indirect communication through one or more intermediary components, and does not require direct physical (e.g., wired) communication and/or constant communication, but rather additionally includes selective communication at periodic intervals, scheduled intervals, aperiodic intervals, and/or one-time events.

Flowchart representative of example hardware logic, machine readable instructions, hardware implemented state machines, and/or any combination thereof for implementing the machine programming engine108ofFIGS. 1 and 3is shown inFIGS. 4-6. The machine readable instructions may be one or more executable programs or portion(s) of an executable program for execution by a computer processor such as the processor712shown in the example processor platform700discussed below in connection withFIG. 7. The program may be embodied in software stored on a non-transitory computer readable storage medium such as a CD-ROM, a floppy disk, a hard drive, a DVD, a Blu-ray disk, or a memory associated with the processor712, but the entire program(s) and/or parts thereof could alternatively be executed by a device other than the processor712and/or embodied in firmware or dedicated hardware. Further, although the example program(s) is/are described with reference to the flowcharts illustrated inFIGS. 4-6, many other methods of implementing the example machine programming engine108may alternatively be used. For example, the order of execution of the blocks may be changed, and/or some of the blocks described may be changed, eliminated, or combined. Additionally or alternatively, any or all of the blocks may be implemented by one or more hardware circuits (e.g., discrete and/or integrated analog and/or digital circuitry, an FPGA, an ASIC, a comparator, an operational-amplifier (op-amp), a logic circuit, etc.) structured to perform the corresponding operation without executing software or firmware.

The machine readable instructions described herein may be stored in one or more of a compressed format, an encrypted format, a fragmented format, a packaged format, etc. Machine readable instructions as described herein may be stored as data (e.g., portions of instructions, code, representations of code, etc.) that may be utilized to create, manufacture, and/or produce machine executable instructions. For example, the machine readable instructions may be fragmented and stored on one or more storage devices and/or computing devices (e.g., servers). The machine readable instructions may require one or more of installation, modification, adaptation, updating, combining, supplementing, configuring, decryption, decompression, unpacking, distribution, reassignment, etc. in order to make them directly readable and/or executable by a computing device and/or other machine. For example, the machine readable instructions may be stored in multiple parts, which are individually compressed, encrypted, and stored on separate computing devices, wherein the parts when decrypted, decompressed, and combined form a set of executable instructions that implement a program such as that described herein. In another example, the machine readable instructions may be stored in a state in which they may be read by a computer, but require addition of a library (e.g., a dynamic link library (DLL)), a software development kit (SDK), an application programming interface (API), etc. in order to execute the instructions on a particular computing device or other device. In another example, the machine readable instructions may need to be configured (e.g., settings stored, data input, network addresses recorded, etc.) before the machine readable instructions and/or the corresponding program(s) can be executed in whole or in part. Thus, the disclosed machine readable instructions and/or corresponding program(s) are intended to encompass such machine readable instructions and/or program(s) regardless of the particular format or state of the machine readable instructions and/or program(s) when stored or otherwise at rest or in transit.

The process400ofFIG. 4includes block402. At block402, the example code review interface202receives a request for code analysis from the code review platform106. For example, the code review platform106can automatically generate a code review request (e.g., the review request124, etc.) when the new code102is committed to the repository104. In some examples, the request received from the code review interface202can include the information required to receive (e.g., the location) of the new code102in the repository104. In some examples, the received request includes information regarding the analysis of the new code102(e.g., the tolerance of the construction threshold, the model to be used, etc.).

At block404, the repository interface204requests committed code and metadata from repository104. For example, the repository interface204can transmit a request (e.g., the request126, etc.) to repository104for the code based on the review request124. In some examples, the repository interface204can further request the metadata associated with the new code102. In other examples, the repository interface204can acquire the new code102via any other means (e.g., directly input by a developer, from the code review platform106, etc.).

At block406, the machine programming engine108performs code review analysis. For example, the machine programming engine108can determine the what functions of the new code102are to be tagged for review. The execution of block406is described in greater detail below in conjunction withFIG. 5.

At block408, the recommendation aggregator216determines if the code analysis generate at least one recommendation. For example, the recommendation aggregator216can determine if the recommendation generator214generated a recommendation during the execution of block406. If the recommendation aggregator216determines that a recommendation was generated, the process400advances to block410. If the recommendation generator214determines that no recommendations were generated, the process400advances to block412.

At block410, the code review interface202transmits the generated recommendations to the code review platform106. For example, the code review interface202transmits an annotated version of the code with the functions to be reviewed to the code review platform106. In some examples, the code review platform106transmits a list of the functions to be reviewed to the code review platform106. In other examples, the code review interface202recommendations can be transmitted to code review platform106be any other suitable means. The process400ends. At block412, the code review interface202transmits an indication to the code review platform106that no recommendations were generated during the code analysis.

The process500ofFIG. 5includes block502. At block502, the function identifier206identifies the function calls in the new code102. For example, the function identifier can parse the new code102to determine the function calls contained therein. In some examples, the function identifier206can identify other parameters (e.g., the input objects, etc.) associated with the functions of the new code102.

At block506, the PTV determiner208determines the PTV for the selected function. For example, the PTV determiner208can analyze the inputs of the selected function to determine the PTV of the function. The execution of block506is described in greater detail in conjunction withFIG. 6.

At block508, the PTV estimator model210analyzes the function to predict a PTV of the function. For example, the PTV estimator model210analyzes the function using a neural network (e.g., a general RNN, an LSTM, etc.) to generate a PTV. In some examples, the PTV estimator model210is trained via a bank of reviewed source code (e.g. from the repository104. In such examples, the functions of the reviewed source code acts as a training input from the PTV estimator model210and the corresponding PTVs are used as the training output. In some examples, the corresponding PTVs of the reviewed source code can be determined via the PTV determiner208. In some examples, the PTVs of the functions of the new code102can be predicted via the topology of the PTV estimator model210.

At block510, the error comparator212determines the reconstruction error of the selected function based on the predicted PTV and the determined PTV. For example, the error comparator212can calculate the Euclidean distance between the predicted PTV (e.g., as determined by the PTV estimator model210at block508, etc.) and the PTV determiner208(e.g., as determined by the PTV determiner208at block506, etc.). In other examples, the error comparator212can use any other suitable means to determine the reconstruction error.

At block512, the recommendation generator214determines if the reconstruction error satisfies the recommendation threshold. In some examples, the recommendation threshold is determined empirically during the training of the PTV estimator model210. In some examples, the recommendation threshold can be set when the request is sent to the machine programming engine108. In other examples, the recommendation threshold can be determined by any other suitable means. If the recommendation generator214determines the reconstruction error satisfies the recommendation threshold, the process500advances to the block512. If the recommendation generator214determines the reconstruction error does not satisfy the recommendation threshold, the process500advances to the block514.

At block514, the recommendation generator214generates a recommendation to review the selected function call. For example, the recommendation generator214could tag the function call for review. In some examples, the recommendation generator214can generate a specific suggestion for review. In some examples, the recommendation generator214can add the function call to a list of function calls for review.

At block516, the function identifier206determines if another function call is to be analyzed. For example, the function identifier206can determine if there are function calls in the new code102that have yet to be analyzed. In other examples, the function identifier206can use any other suitable means of determining if another function call is to be analyzed. If another function call is to be analyzed, the process500returns to block504. If another function call is not be analyzed, the process500ends.

The process600ofFIG. 6includes block602. At block602, the literal parameter type estimator302identifies literal data types of inputs of the function call. For example, the literal parameter type estimator302can identify the literal type(s) (e.g., string, floating point number, integer, etc.) associated with each function call. In some examples, the literal parameter type estimator302can determine the literal type parameters of the input of the function call by analyzing the new code102. In other examples, the literal parameter type estimator302can determine the literal data type via any other suitable means (e.g., prompting the developer for manual input, metadata, etc.).

At block604, the one-hot encoder306one hot encodes the literal data types into a first vector. For example, the one-hot encoder306can generate a data structure (e.g., a sparse binary vector, etc.) in which one dimension (e.g., one index, etc.) has a value of “l” and each of the other dimensions have a value of “0.” In such examples, each dimension of the data structure represents a different possible literal data type of the function call. For example, if there are ten possible literal data types of the function input, the one-hot encoder306creates a data structure with ten dimensions with each dimension corresponding to one of the specific literal data types.

At block606, the variable type static analyzer304identifies primitive data types of the function calls using static analysis based on log files. For example, the variable type static analyzer304can analyze the log files of the function to determine what data type(s) was assigned to the function inputs. For example, if a function input was assigned as a string four times and an integer one time, the variable type static analyzer304identifies the

At block608, the one-hot encoder306one hot encodes the primitive data types into a second vector. For example, the one-hot encoder306can generate a data structure (e.g., a sparse binary vector, etc.) in which one dimension (e.g., one index, etc.) has a value of “1” and each of the other dimensions are “0.” In such examples, each dimensions of the data structure represents a different possible primitive data type(s) of the function call. For example, if there are ten possible primitive data type(s) of the function input, the one-hot encoder306creates a data structure with ten dimensions with each dimension corresponding to one of the specific primitive data type(s).

At block610, the concatenator308generates a PTV based on the first vector and the second vector. For example, the concatenator308can combine the data structure generated by the one-hot encoder306during the execution of the block604and the data structure generated by the one-hot encoder306during the execution of the block608. In some examples, the concatenator308concatenates the data structure(s). The process600ends.

FIG. 7is a block diagram of an example processor platform700structured to execute the instructions ofFIGS. 4-6to implement the machine programming engine108ofFIGS. 1-3. The processor platform700can be, for example, a server, a personal computer, a workstation, a self-learning machine (e.g., a neural network), a mobile device (e.g., a cell phone, a smart phone, a tablet such as an iPad™), a personal digital assistant (PDA), an Internet appliance, a headset or other wearable device, or any other type of computing device.

The processor platform700of the illustrated example includes a processor712. The processor1012of the illustrated example is hardware. For example, the processor712can be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. The hardware processor may be a semiconductor based (e.g., silicon based) device. In this example, the processor implements the example function identifier206, the example PTV determiner208, the example PTV estimator model210, the example error comparator212, the example recommendation generatorz, and the example recommendation aggregator216.

The processor712of the illustrated example includes a local memory713(e.g., a cache). The processor712of the illustrated example is in communication with a main memory including a volatile memory714and a non-volatile memory716via a bus718. The volatile memory714may be implemented by Synchronous Dynamic Random Access Memory (SDRAM). Dynamic Random Access Memory (DRAM), RAMBUS® Dynamic Random Access Memory (RDRAM®) and/or any other type of random access memory device. The non-volatile memory716may be implemented by flash memory and/or any other desired type of memory device. Access to the main memory714,716is controlled by a memory controller.

The processor platform700of the illustrated example also includes an interface circuit720. The interface circuit720may be implemented by any type of interface standard, such as an Ethernet interface, a universal serial bus (USB), a Bluetooth® interface, a near field communication (NFC) interface, and/or a PCI express interface.

In the illustrated example, one or more input devices722are connected to the interface circuit720. The input device(s)722permit(s) a user to enter data and/or commands into the processor712. The input device(s) can be implemented by, for example, an audio sensor, a microphone, a camera (still or video), a keyboard, a button, a mouse, a touchscreen, a track-pad, a trackball, isopoint and/or a voice recognition system.

The processor platform700of the illustrated example also includes one or more mass storage devices728for storing software and/or data. Examples of such mass storage devices728include floppy disk drives, hard drive disks, compact disk drives, Blu-ray disk drives, redundant array of independent disks (RAID) systems, and digital versatile disk (DVD) drives.

The machine executable instructions732ofFIGS. 4-6may be stored in the mass storage device728, in the volatile memory714, in the non-volatile memory716, and/or on a removable non-transitory computer readable storage medium such as a CD or DVD.

From the foregoing, it will be appreciated that example methods, apparatus and articles of manufacture have been disclosed that provide code review assistance for dynamically typed languages. The disclosed methods, apparatus and articles of manufacture improve the efficiency of using a computing device by reducing the amount of time associated with reviewing and correcting dynamically typed languages. The disclosed methods, apparatus and articles of manufacture are accordingly directed to one or more improvement(s) in the functioning of a computer.

Example methods, apparatus, systems, and articles of manufacture for code review assistance for dynamically typed languages are disclosed herein. Further examples and combinations thereof include the following: Example 1 includes an apparatus to analyze a segment of code, the apparatus comprising a function identifier to identify a first input of a first function call included in the segment of the code, a parameter type vector (PTV) estimator model to estimate a first data structure based on the first input, the PTV estimator model generated via a set of reviewed code, a PTV determiner to generate a second data structure based on a data parameter type of the first input, an error comparator to determine a first reconstruction error based on the first data structure and the second data structure, and a recommendation generator to, if the first reconstruction error does not satisfy a recommendation threshold, generate a first recommendation to review the first function call.

Example 2 includes the apparatus of example 1, wherein the function identifier is to identify a second input of a second function call included in the segment of the code, the second function call after the first function call in the segment of the code, the PTV estimator model is to estimate a third data structure based on the first input and the second input, the PTV determiner is to calculate a fourth data structure based on the data parameter type of the second input, the error comparator is to determine a second reconstruction error based on the third data structure and the fourth data structure, and the recommendation generator is to, if the second reconstruction error does not satisfy the second recommendation threshold, generate a second recommendation to review the second function call.

Example 3 includes the apparatus of example 2, wherein the PTV estimator model is further to estimate the first data structure based on the second input.

Example 4 includes the apparatus of example 1, wherein the PTV estimator model is an encoder-decoder LSTM.

Example 5 includes the apparatus example 1, wherein the PTV determiner includes a literal parameter type estimator to determine a literal type associated with the first input, a variable type static analyzer to determine, by analyzing a log file associated with the segment of the code, a primitive type associated with the first input, and a concatenator to generate the first data structure based on at least one of the determined literal type or the primitive type.

Example 6 includes the apparatus of example 1, wherein the error comparator is to determine the first reconstruction error based on an Euclidean distance between the first data structure and the second data structure.

Example 7 includes the apparatus of example 1, wherein the set of reviewed code is queried and normalized from an online repository.

Example 8 includes a method to analyze a segment of code, the method comprising identifying, by executing an instruction with at least one processor, a first input of a first function call included in the segment of the code, estimating, by executing an instruction with the at least one processor via a neural network, a first data structure based on the first input, the neural network generated via a set of reviewed code, generating, by executing an instruction with the at least one processor, a second data structure based on a data parameter type of the first input, determining, by executing an instruction with the at least one processor, a first reconstruction error based on the first data structure and the second data structure, and if the first reconstruction error does not satisfy a recommendation threshold, generating, by executing an instruction with the at least one processor, a first recommendation to review the first function call.

Example 9 includes the method of example 8, further including identifying a second input of a second function call included in the segment of the code, the second function call after the first function call in the segment of the code, estimating, via the neural network, a third data structure based on the first input and the second input calculating a fourth data structure based on the data parameter type of the second input, determining a second reconstruction error based on the third data structure and the fourth structure, and if the second reconstruction error does not satisfy a recommendation threshold, generating a second recommendation to review the second function call.

Example 10 includes the method of example 9, wherein the estimating, via the neural network, the first data structure is further based on the second input.

Example 11 includes the method of example 8, wherein the neural network is an encoder-decoder LSTM.

Example 12 includes the method of example 8, wherein the calculating a second data structure based on the data parameter type of the first input includes determining a literal type associated with the first input, determining, by analyzing a log file associated with the segment of the code, a primitive type associated with the first input, and generating the first data structure based on at least one of the determined literal type or the primitive type.

Example 13 includes the method of example 8, wherein the determining a first reconstruction error based on the first data structure and the second structure includes determining an Euclidean distance between the first data structure and the second data structure.

Example 14 includes the method of example 8, wherein the set of reviewed code is queried and normalized from an online repository.

Example 15 includes a non-transitory computer readable storage medium which, when executed, cause a machine to at least identify a first input of a first function call included in the segment of the code, estimate, via a neural network, a first data structure based on the first input, the neural network generated via a set of reviewed code, generate a second data structure based on a data parameter type of the first input, determine a first reconstruction error based on the first data structure and the second data structure, and if the first reconstruction error does not satisfy a recommendation threshold, generate a first recommendation to review the first function call.

Example 16 includes the non-transitory computer readable storage medium of example 15, wherein the instructions cause the machine further to identify a second input of a second function call included in the segment of the code, the second function call after the first function call in the segment of the code, estimate, via the neural network, a third data structure based on the first input and the second input calculate a fourth data structure based on the data parameter type of the second input, determine a second reconstruction error based on the third data structure and the fourth structure, and if the second reconstruction error does not satisfy a recommendation threshold, generate a second recommendation to review the second function call.

Example 17 includes the non-transitory computer readable storage medium of example 16, the first data structure is further based on the second input.

Example 18 includes the non-transitory computer readable storage medium of example 15, wherein the neural network is an encoder-decoder LSTM.

Example 19 includes the non-transitory computer readable storage medium of example 15, wherein the instructions cause the machine to calculate a second data structure based on the data parameter type of the first input by determining a literal type associated with the first input, determining, by analyzing a log file associated with the segment of the code, a primitive type associated with the first input, and generating the first data structure based on at least one of the determined literal type or the primitive type.

Example 20 includes the non-transitory computer readable storage medium of example 15, wherein the first reconstruction error is on an Euclidean distance between the first data structure and the second data structure. Although certain example methods, apparatus and articles of manufacture have been disclosed herein, the scope of coverage of this patent is not limited thereto. On the contrary, this patent covers all methods, apparatus and articles of manufacture fairly falling within the scope of the claims of this patent.