Patent Description:
In recent years, machine learning and/or artificial intelligence have increased in popularity. For example, machine learning and/or artificial intelligence may be implemented using neural networks. Neural networks are computing systems inspired by the neural networks of human brains. A neural network can receive an input and generate an output. The neural network can be trained (e.g., can learn) based on feedback so that the output corresponds a desired result. Once trained, the neural network can make decisions to generate an output based on any input. Neural networks are used for the emerging fields of artificial intelligence and/or machine learning. A Bayesian neural network is a particular type of neural network that includes neurons that generate a variable weight as opposed to a fixed weight. The variable weight falls within a probability distribution defined by a mean value and a variance determined during training of the Bayesian neural network.

<NPL>) proposes an efficient performance trade-off modeling method for analog circuit based on Bayesian Neural Network (BNN).

<NPL>) targets a low-power ADC to relax the requirements on a neurochip's surface temperature.

Pattavina Jeffrey S. , "Charge-Pump Phase-Locked Loop - A Tutorial-Part I", (<NUM>) describes the fundamental properties of a charge-pump PLL. The focus of discussion is the methods suitable for ASIC design.

The figures are not to scale. In general, the same reference numbers will be used throughout the drawing(s) and accompanying written description to refer to the same or like parts. Connection references (e.g., attached, coupled, connected, and joined) are to be construed broadly and may include intermediate members between a collection of elements and relative movement between elements unless otherwise indicated. As such, connection references do not necessarily infer that two elements are directly connected and in fixed relation to each other. Although the figures show layers and regions with clean lines and boundaries, some or all of these lines and/or boundaries may be idealized. In reality, the boundaries and/or lines may be unobservable, blended, and/or irregular.

Machine learning models, such as neural networks, are used to be able to perform a task (e.g., classify data). Machine learning can include a training stage to train the model using ground truth data (e.g., data correctly labelled with a particular classification). Training a traditional neural network adjusts the weights of neurons of the neural network. After trained, data is input into the trained neural network and the weights of the neurons are applied to input data to be able to process the input data to perform a function (e.g., classify data).

Overfitting and sensitivity to malicious attacks negatively affect the performance and/or reliability of traditional neural networks. Overfitting occurs when a model is trained to have too small of an error. If the training results in too small of an error, the model has a difficult time generalizing for new situations. Malicious attacks can exploit a combination of overfitting and/or knowledge of the underlying neural network model. Sensitivity to malicious attacks is the result of a trained model being overconfident in its outputs. If a model is overconfident, small perturbations to the inputs can result in undesired and/or unpredictable changes in the output. Both the above problems are caused by the failure of traditional neural networks to include uncertainty information in a finite set of training data.

Bayesian neural networks (BNNs) introduce uncertainty information to overcome the problems of overfitting and sensitivity to malicious attacks. Instead of using fixed weights, BNNs introduce weights associated with conditioned probability distribution (e.g., the output weight may be a value within a probability distribution defined by a mean (herein also referred to as mu or u) and standard deviation and/or variance). Because BNNs introduce some amount of randomness, BNNs can be trained with smaller training data without sacrificing accuracy. However, traditional BNNs with neurons that generate weights corresponding to a probability distribution require a lot of power and/or hardware to implement. Therefore, such traditional BNNs are expensive, complex, and energy inefficient. Examples disclosed herein correspond to a BNN that is implemented with less hardware (and thus less expensive) and is more energy efficient than traditional BNNs.

Examples disclosed herein leverage the Gaussian distribution corresponding to randomly dithering charge in the analog domain. Examples disclosed herein generates a BNN using (<NUM>) a C-2C ladder that converts the mean weight into an electric charge level, (<NUM>) a jittery oscillator sampling-based entropy source that provides programmable randomness, and (<NUM>) a charge pump controlled by the entropy source to dither (e.g., adjust) the charge generated by the C-2C ladder to give the final, programmable, and Gaussian distributed output weight. Examples disclosed herein results in an analog-based, compute-in-memory (CiM) BNN implementation of a neural network.

In general, implementing a ML/AI system involves two phases, a learning/training phase and an inference phase. In the learning/training phase, a training algorithm is used to train a model to operate in accordance with patterns and/or associations based on, for example, training data. In general, the model includes internal parameters that guide how input data is transformed into output data, such as through a series of nodes and connections within the model to transform input data into output data. Additionally, hyperparameters may be used as part of the training process to control how the learning is performed (e.g., a learning rate, a number of layers to be used in the machine learning model, etc.). Hyperparameters are defined to be training parameters that are determined prior to initiating the training process.

Different types of training may be performed based on the type of ML/AI model and/or the expected output. As used herein, labelling refers to an expected output of the machine learning model (e.g., a classification, an expected output value, etc.). Alternatively, unsupervised training (e.g., used in deep learning, a subset of machine learning, etc.) involves inferring patterns from inputs to select parameters for the ML/AI model (e.g., without the benefit of expected (e.g., labeled) outputs).

In examples disclosed herein, training is performed until a threshold number of actions have been predicted. In examples disclosed herein, training is performed either locally (e.g. in the device) or remotely (e.g., in the cloud and/or at a server). Training may be performed using hyperparameters that control how the learning is performed (e.g., a learning rate, a number of layers to be used in the machine learning model, etc.). In some examples re-training may be performed. Such re-training may be performed in response to a new program being implemented or a new user using the device. Training is performed using training data. When supervised training may be used, the training data is labeled. In some examples, the training data is pre-processed.

Once training is complete, the model is deployed for use as an executable construct that processes an input and provides an output based on the network of nodes and connections defined in the model. The model is stored locally in memory (e.g., cache and moved into memory after trained) or may be stored in the cloud. The model may then be executed by the computer cores.

Once trained, the deployed model may be operated in an inference phase to process data. In the inference phase, data to be analyzed (e.g., live data) is input to the model, and the model executes to create an output. This inference phase can be thought of as the AI "thinking" to generate the output based on what it learned from the training (e.g., by executing the model to apply the learned patterns and/or associations to the live data). In some examples, input data undergoes pre-processing before being used as an input to the machine learning model. Moreover, in some examples, the output data may undergo post-processing after it is generated by the AI model to transform the output into a useful result (e.g., a display of data, an instruction to be executed by a machine, etc.).

In some examples, output of the deployed model may be captured and provided as feedback. By analyzing the feedback, an accuracy of the deployed model can be determined. If the feedback indicates that the accuracy of the deployed model is less than a threshold or other criterion, training of an updated model can be triggered using the feedback and an updated training data set, hyperparameters, etc., to generate an updated, deployed model.

<FIG> is a schematic illustration of an example neural network (NN) trainer <NUM> to train example BNN <NUM>. The example BNN <NUM> includes an example reference clock <NUM>, an example memory <NUM>, and example neurons <NUM>. Although the illustrated neurons <NUM> of <FIG> include six neurons, there may be any number of neurons in any type of configuration. Although the example of <FIG> is described in conjunction with the BNN <NUM>, examples disclosed herein may be utilized in any AI-based system or model that includes weights.

The example NN trainer <NUM> of <FIG> trains the BNN <NUM> by selecting a mean weight and an amount of deviation for the mean weight for each of the neurons <NUM>. Initially, the BNN <NUM> is untrained (e.g., the neurons are not yet weighted with a mean and deviation). To train the BNN <NUM>, the example NN trainer <NUM> of <FIG> uses training data (e.g., input data labelled with known classifications and/or outputs) to configure the BNN <NUM> to be able to predict output classifications for input data with unknown classification. The NN trainer <NUM> may train a model with a first set of training data and test the model with a second set of the training data. If, based on the results of the testing, the accuracy of the model is below a threshold, the NN trainer <NUM> can tune (e.g., adjust, further train, etc.) the parameters of the model using additional sets of the training data and continue testing until the accuracy is above the threshold. After the NN trainer <NUM> has trained the BNN <NUM>, the example NN trainer <NUM> stores the corresponding means and deviations for the respective neurons <NUM> in the example memory <NUM> of the example BNN <NUM>. The example NN trainer <NUM> may be implemented in the same device as the BNN <NUM> and/or in a separate device in communication with the example BNN <NUM>. For example, the NN trainer <NUM> may be located remotely, develop the weight data locally, and deploy the weight data (e.g., means and deviation for the respective neurons <NUM>) to the BNN <NUM> for implementation.

The example BNN <NUM> of <FIG> includes the example reference clock <NUM>. The reference clock <NUM> generates a periodic signal (e.g., a clock signal, a pulse signal, etc.). The periodic signal can be used by one or more of the example neurons <NUM> to sample a voltage of a jittery oscillator to infuse randomness into a weight generated by the respective neuron <NUM>, as further described below.

The example BNN <NUM> of <FIG> further includes the example memory <NUM>. The example memory <NUM> stores the weight data from the example NN trainer <NUM> in conjunction with a particular neuron. For example, a first section of the memory <NUM> is dedicated for a first mean value and a first variance value for a first neuron, a second section of the memory <NUM> is dedicated to a second mean value and a second variance value for a second neuron, etc. The mean value may be stored in the dedicated section as a bit value representative of the mean value. As further described below, the bit value corresponds to control of particular switches in a C-2C ladder of one the neurons <NUM> (e.g., each bit value corresponding to one or more switches in a single bit cell). In this manner, the bit value controls the bit cells of the C-2C ladder to ensure that the output voltage is equivalent to the mean value. The variance value is used to control a variable resistor to adjust the slope of the clock signal from the reference clock <NUM>. Because the reference clock <NUM> is sampled based on a rising edge of a jitter clock signal from a jittery oscillator, adjusting the slope of the clock signal adjusts the variance, as further described below.

The example neurons <NUM> of <FIG> receive input data, generate a weight that corresponds to a probability distribution and apply the weight to the input data to generate an output. For example, if the probability distribution of a neuron follows a standard normal distribution, the mean weight of the neuron is <NUM>, and the variance of the neuron is <NUM> (e.g., the standard deviation is <NUM>), then the will be a <NUM>% chance that the neuron will output a weight between <NUM> and <NUM> (e.g., one standard deviation away from the mean), a <NUM>% chance that the neuron will output a weight between <NUM> and <NUM> (e.g., two standard deviations away from the mean), etc. Accordingly, the weight generated by the output may be different every time a weight is generated but will follow the probability distribution. Thus, the example neurons <NUM> provide randomness that can counteract overfitting and sensitivity to malicious attacks. The structure of one of the example neurons <NUM> is further described below in conjunction with <FIG> and <FIG>.

<FIG> is a block diagram of one of the example neurons <NUM> of <FIG>. The example neuron <NUM> includes an example memory interface <NUM>, an example C-2C ladder <NUM>, an example variance to resistance converter <NUM>, an example clock interface <NUM>, an example entropy source <NUM>, an example voltage sampler <NUM>, an example charge pump <NUM>, an example weight applicator <NUM>, and an example sleep controller <NUM>.

The example memory interface <NUM> of <FIG> interfaces with the example memory <NUM> of <FIG> to access the bit value of the mean and the variance corresponding to the neuron <NUM>. The memory interface <NUM> transmits the bit value representative of the mean for the output weight to the example C-2C ladder <NUM> (e.g., with a line for each bit). For example, if the mean is represented by an <NUM>-bit value, the memory interface <NUM> transmits eight signals so the C-2C ladder (e.g., in series or in parallel), one signal for each bit of the <NUM>-bit mean value. The memory interface <NUM> transmits the variance to the example variance-to-resistance converter <NUM>.

The example C-2C ladder <NUM> of <FIG> obtains the bit values corresponding to the mean weight value (also referred to as mean value or mean weight) from the example memory interface <NUM> and generates a voltage corresponding to the mean weight. For example, the C-2C ladder may include X bit cells corresponding to the X bit values of the mean. The C-2C bit cells include capacitors to store charge and switches to discharge and/or charge the capacitor (e.g., by grounding and/or apply a voltage supply to the capacitors). By controlling the switches, one or more of the bit cells can charge and/or discharge to generate an output voltage that corresponds to the mean value. For example, if the switches of all the bit cells are enabled to allow the supply voltage to charge the respective capacitors, the output voltage will be equivalent to <NUM> (e.g., <NUM>% of the supply voltage), if the switches of half the bit cells are enabled and the switches of half the bit cells are disabled, the output voltage will be equivalent to <NUM> (e.g., <NUM>% of the supply voltage), etc. A hardware implementation of the C-2C ladder <NUM> is further described below in conjunction with <FIG>. The example C-2C ladder <NUM> outputs the voltage corresponding to the mean value to the example charge pump <NUM>.

The example variance-to-resistance converter <NUM> of <FIG> obtains the variance value stored in the memory <NUM> via the example memory interface <NUM> and converts the variance into a resistance value. The example variance-to-resistance converter <NUM> converts the resistance into a signal (e.g., a voltage) that is used to adjust the resistance of a resistive element (e.g., a variable resistor) of the entropy source <NUM>. The example variance-to-resistance converter <NUM> may be implemented by one or more multiplexers and/or look up tables that convert a variance value to a signal representative of a resistance to tune a variable resistor to the corresponding resistance. In this manner, the variable resistor of the entropy source <NUM> can adjust the resistance to adjust the slope a jitter free reference clock signal corresponding to an amount of variance when being sampled based on a raising edge of a jittery clock signal from a jittery oscillator, as further described below.

The example clock interface <NUM> of <FIG> interfaces with the example reference clock <NUM> of <FIG> to obtain the periodic signal (e.g., a jitter free referenced clock signal) from the reference clock <NUM>. The example clock interface <NUM> passes the clock signal from the reference clock <NUM> to a buffer of the example entropy source <NUM>.

The example entropy source <NUM> of <FIG> obtains the clock signal from the clock interface <NUM> and the signal corresponding to a resistance from the variance-to-resistance converter <NUM>. The example entropy source <NUM> includes a buffer to delay the rising edge of the clock signal. The buffer includes a variable resistor that is controlled (e.g., to adjust the resistance) based on the signal from the variance-to-resistance converter <NUM>. In this manner, the example entropy source <NUM> adjusts the slope of the rising edge of the clock signal based on the signal from the variance-to-resistance converter <NUM>. The example entropy source <NUM> outputs the buffered clock signal to the example voltage sampler <NUM>. Additionally, the entropy source <NUM> includes a jittery oscillator that generates a clock signal that includes some amount of jitter. The jitter of the oscillator is leveraged to create an amount of randomness that is added to the mean value to generate a weight that corresponds to a probability distribution. For example, the rising edge of the clock signal generated by the jittery oscillator may be slightly different (e.g., with respect to when the rising edge occurs) based on the intrinsic characteristics of the analog components of the oscillator. As further described below, the voltage sampler <NUM> samples the voltage output by the jitter free output of the buffered reference clock signal based on the raising edge of the jittery output voltage of the jittery oscillator. An example hardware implementation of the entropy source <NUM> is further described below in conjunction with <FIG>.

The example voltage sampler <NUM> of <FIG> samples the voltage output by the buffer of the example entropy source <NUM> when the jittery clock signal reaches a threshold voltage. The example voltage sampler <NUM> outputs the sampled voltage until a subsequent sample is obtained. The example voltage sampler <NUM> may be implemented by one or more comparators and/or one or more latches. Because of the jitter of the jittery oscillator, the example voltage sampler <NUM> may sample different voltages for each sample that corresponds to a probability distribution. The slope of the buffered clock signal (e.g., based on the resistance of the variable resistor) results in the amount of variance to add to the output voltage of the example C-2C ladder <NUM>. The example voltage sampler <NUM> outputs the sampled voltage to the example charge pump <NUM> of <FIG>.

The example charge pump <NUM> of <FIG> adjusts the weight output (e.g., a voltage that corresponds to the determined mean) by the C-2C ladder by adjusting the weight output voltage during two adjacent clock cycles (e.g., two rising edges of a clock signal) to discharge and charge the weight output voltage at the output node of the charge pump <NUM>, thereby generating an adjusted weight that can be applied to input data. For example, the charge pump <NUM> may discharge the output of the C-2C ladder <NUM> using a first sampled voltage from the voltage sampler <NUM> during a first cycle (e.g., corresponding to the clock signal from the reference clock <NUM>) and charge the output of the C-2C ladder <NUM> using a second sampled voltage from the voltage sampler <NUM> during a second subsequent cycle. The resulting voltage output by the charge pump <NUM> will be the mean value plus the difference between the sampled voltage at the first cycle and the sampled voltage at the second cycle. Thus, the resulting voltage output after the first and second cycles (e.g., corresponding to one discharge and one charge) corresponds to some voltage along the probability distribution about the mean based on the variance caused by the jitter oscillator of the entropy source <NUM>. In some examples, the charge pump <NUM> uses the first cycle to charge the voltage at the output node and uses the second cycle to discharge the voltage at the output node to generate the adjusted weight. The charge pump <NUM> outputs the resulting adjusted weight (e.g., the mean value plus or minus the variance (e.g., the difference between the first sampled voltage and the second subsequent sampled voltage)) to the example weight applicator <NUM>.

The example weight applicator <NUM> of <FIG> receives input data (e.g., image data, video data, audio data, files, word docs, applications, code, etc.) and applies the generated weight (e.g., output by the charge pump <NUM>) to the input data to generate output data. For example, the weight applicator <NUM> may multiply the input data by the generated weight. In some examples, the charge pump <NUM> may generate multiple weights to be applied to the input data.

The example sleep controller <NUM> monitors when data is input into the example weight applicator <NUM> to implement a sleep/wake protocol. For example, after the weight applicator <NUM> outputs the output data, the example sleep controller <NUM> may control one or more components of the neuron <NUM> to cause the one or more components to power down and/or enter into a sleep or low power mode (e.g., to conserve power). When new input data is received at the weight applicator <NUM>, the example sleep controller <NUM> sends a signal to the one or more components of the neuron <NUM> to wake up and/or exit from the sleep and/or low power mode. In some examples, the sleep controller <NUM> can control components of the neuron <NUM> to operate in different modes. For example, the sleep controller <NUM> may power down or disconnect the entropy source <NUM> and/or the charge pump <NUM>. In such an example, the neuron <NUM> can operate like a traditional neural network where the C-2C ladder <NUM> generates a constant output weight (e.g., that is not adjusted).

<FIG> illustrates a hardware implementation of the example C-2C ladder <NUM>, the example entropy source <NUM> and the example charge pump <NUM> of <FIG>. The example C-2C ladder <NUM> includes example bit cells 300an, example switches <NUM>, and example capacitors <NUM>, the example entropy source <NUM> includes an example ring oscillator <NUM>, an example reference clock buffer <NUM>, and an example variable resistor <NUM>. The example charge pump <NUM> includes an example charge-side variable current source <NUM>, example switches <NUM>, <NUM>, and an example discharge-side variable current source <NUM>.

The example C-2C ladder <NUM> of <FIG> includes X bit cells 300a-n for the X bits of the mu value (e.g., the mean value of the weight). Each bit cell 300a-n includes the switch(es) <NUM> that can be controlled to charge the respective capacitor <NUM> and/or discharge the respective capacitor <NUM>. As described above, the bit values of mu are used to control the switches <NUM>. For example, if a bit value of mu corresponds to <NUM>, <NUM>, <NUM>, <NUM>, then the first switch and the last switch will be controlled to operate a first state (e.g., to ground) and the middle switches will be controlled to operate a second state (e.g., the reference voltage Vref). The sum of the charges stored at the example bit cells 300a-n corresponds to the output voltage that corresponds to mu (e.g., <NUM> in the illustrated example of <FIG>). In some examples, each switch <NUM> may be implemented by four transistors, as further shown below in conjunction with <FIG>. Accordingly, the example C-2C ladder <NUM> can be implemented with <NUM>*X capacitors and <NUM>*X capacitors, where X is the number of bits to represent mu.

The example entropy source <NUM> of <FIG> includes the example ring oscillator <NUM>. The example ring oscillator <NUM> includes three inventors (e.g., analog components). However, the ring oscillator <NUM> can be implemented with any odd number of stages. The ring oscillator <NUM> is a jittery oscillator. Thus, the rising edge of the ring oscillator <NUM> may not occur at the same time within a cycle as other cycles. The distribution of the jitter corresponds a probability distribution. Accordingly, the jitter of the ring oscillator <NUM> can be leveraged to add randomness to the output voltage of the C-2C ladder <NUM> via the charge pump <NUM>. The example entropy source <NUM> further includes the example reference clock buffer <NUM>. The reference clock buffer <NUM> obtains the input reference clock from the clock interface <NUM> of <FIG>. The reference block buffer delays the clock signal by some time t and adjusts the slope of the rising edge of the clock using the example variable resistor <NUM>. Although the example of <FIG> includes the variable resistor <NUM>, the variable resistor <NUM> can be replaced with any type of resistive element (e.g., a transistor with a tuned gate voltage). The higher the resistance of the example variable resistor <NUM>, the slower the slope of the clock signal output by the reference clock buffer <NUM> and the lower the resistance of the example variable resistor <NUM>, the faster the slope of the clock signal output by the reference clock buffer <NUM>. As described above, the example variance-to-resistance converter <NUM> outputs a signal to the example variable resistor <NUM> to adjust the resistance so that the probability distribution for a sampled voltage corresponds to the target variance stored in the example memory <NUM> of <FIG>. Although the example entropy source <NUM> includes a buffer to adjust a clock signal and a jittery oscillator, the example entropy source <NUM> may include two jittery oscillators (e.g., a first oscillator to compare to a threshold and a second oscillator to sample when the voltage of the first oscillator satisfies the threshold). However, in such examples, to program the output of such an entropy source <NUM> to correspond to a desired probability distribution, additional hardware and/or operation overhead may be needed to achieve the desired probability distribution.

As described above, the example voltage sampler <NUM> of <FIG> obtains the jittery output clock signal of the example ring oscillator <NUM> and the jitter free output buffered clock signal from the example reference clock buffer <NUM>. The example voltage sampler <NUM> monitors the voltage of the jittery output of the example ring oscillator <NUM> until the voltage rises above a threshold voltage (e.g., during the rising edge). When the voltage sampler <NUM> determines that the jittery output voltage of the ring oscillator <NUM> is above a threshold, the voltage sampler <NUM> samples the voltage of jitter free buffered output voltage of the reference clock buffer <NUM> and outputs the sampled voltage. Because of the jitter of the ring oscillator <NUM>, the point of time when the output voltage of the ring oscillator <NUM> will be above the threshold may vary with respect to difference cycles. Accordingly, the sampled voltage with likewise vary, thereby corresponding to the randomness to be added to the weight generated by the example C-2C ladder <NUM>. In some examples, the voltage sampler <NUM> uses a falling edge instead of a rising edge to trigger sampling (e.g., when the voltage is below a threshold voltage, trigger sampling). In some examples, the voltage sampler <NUM> may monitor the jitter free output voltage of the reference clock buffer <NUM>, sample the jittery output voltage of the ring oscillator <NUM> when the jitter free voltage reaches a threshold voltage, and output the sampled voltage to the charge pump <NUM>.

The example charge pump <NUM> of <FIG> is a phase-locked loop (PLL) charge pump. However, the charge pump <NUM> can be any type of charge pump. The charge pump <NUM> increases or decreases the output voltage of C-2C ladder <NUM> based on a difference between two samples (e.g., of two neighboring cycles) from the example voltage sampler <NUM>. The charge pump <NUM> includes a first example switch(es) <NUM> that is enabled for a first duration of time (e.g. Clk[n], based on the clock signal from the reference clock <NUM> of <FIG>) after the voltage sampler <NUM> outputs a first sampled voltage to the variable current source <NUM>. The variable current source <NUM> pumps current toward ground to discharge the charge at the output of the C-2C ladder <NUM>, thereby decreasing the voltage at the output of the C-2C ladder <NUM>. The amount of current pumped out (e.g., corresponding to the amount of voltage decrease at the output of the C-2C ladder <NUM>) depends on the first sampled voltage, which controls the variable current source <NUM>.

During a subsequent cycle, the voltage second example switch(es) <NUM> is enabled for a second duration of time (e.g., Clk [n+<NUM>]) after the voltage sampler outputs a second sampled voltage to the variable current source <NUM>. The variable current source <NUM> pumps current into the output node of the C-2C ladder <NUM> to charge the output of the C-2C ladder <NUM>, thereby increasing the voltage at the output of the C-2C ladder <NUM>. The amount of current pumped in depends on the depends on the second sampled voltage, which controls the variable current source <NUM>. Accordingly, after the two cycles, the charge pump decreases and then increases the voltage output by the example C-2C ladder <NUM>. Thus, the output of the charge pump <NUM> corresponds to the mean weight value plus the difference corresponding to the first voltage sample and the second voltage sample. The combination of the entropy source <NUM> and the charge pump <NUM> can be implemented with <NUM> transistors and <NUM> resistors. An example timing diagram that further describes the functionality of the entropy source <NUM> and the charge pump <NUM> is further described below in conjunction with <FIG>.

<FIG> illustrates an example hardware implementation of a section of the example memory <NUM> that stores a bit-wise value of mu and the corresponding bit cells 300a-d of the example C-2C ladder <NUM>. As described above, operation of the bit cells 300a-d generate an output voltage corresponding to the stored value of mu. The example section of the memory <NUM> includes example four <NUM> transistors (6T) storage cells 400a-d corresponding to the four bit cells 300a-d. Although the illustrated example of <FIG> includes four bit cells 300a-d and four 6T storage cells 400a-d (e.g., corresponding to a <NUM> bit mu value), <FIG> may include any number of bit cells and/or 6T storage cells corresponding to any number of bits to represent mu. As shown in <FIG>, to implement and X bit value of mu, <NUM>*Xtransistors (<NUM> transistors to implement the 6T SRAM cell plus <NUM> transistors to implement the switches <NUM> in a bit cell 300a-d) where X is the number of bits to represent mu, and <NUM>*X capacitance (e.g., a first capacitor with a first capacitance and a second capacitor with twice the first capacitance, per cell) can be used.

While an example manner of implementing the BNN <NUM> of <FIG> is illustrated in <FIG>, one or more of the elements, processes and/or devices illustrated in <FIG> may be combined, divided, rearranged, omitted, eliminated and/or implemented in any other way. Further, the example reference clock <NUM>, the example memory <NUM>, the example memory interface <NUM>, the example variance-to-resistance converter <NUM>, the example clock interface <NUM>, the example entropy source <NUM>, the example voltage sampler <NUM>, the example charge pump <NUM>, the example weight applicator <NUM>, the example sleep controller <NUM>, and/or, more generally, the example BNN <NUM> of <FIG> may be implemented by hardware, software, firmware and/or any combination of hardware, software and/or firmware. Thus, for example, any of the example reference clock <NUM>, the example memory <NUM>, the example memory interface <NUM>, the example variance-to-resistance converter <NUM>, the example clock interface <NUM>, the example entropy source <NUM>, the example voltage sampler <NUM>, the example charge pump <NUM>, the example weight applicator <NUM>, the example sleep controller <NUM>, and/or, more generally, the example BNN <NUM> of <FIG> could be implemented by one or more analog or digital circuit(s), logic circuits, programmable processor(s), programmable controller(s), graphics processing unit(s) (GPU(s)), digital signal processor(s) (DSP(s)), application specific integrated circuit(s) (ASIC(s)), programmable logic device(s) (PLD(s)) and/or field programmable logic device(s) (FPLD(s)). When reading any of the apparatus or system claims of this patent to cover a purely software and/or firmware implementation, at least one of the example reference clock <NUM>, the example memory <NUM>, the example memory interface <NUM>, the example variance-to-resistance converter <NUM>, the example clock interface <NUM>, the example entropy source <NUM>, the example voltage sampler <NUM>, the example charge pump <NUM>, the example weight applicator <NUM>, the example sleep controller <NUM>, and/or, more generally, the example BNN <NUM> of <FIG> is/are hereby expressly defined to include a non-transitory computer readable storage device or storage disk such as a memory, a digital versatile disk (DVD), a compact disk (CD), a Blu-ray disk, etc. including the software and/or firmware. Further still, the example BNN <NUM> of <FIG> may include one or more elements, processes and/or devices in addition to, or instead of, those illustrated in <FIG>, and/or may include more than one of any or all of the illustrated elements, processes, and devices. As used herein, the phrase "in communication," including variations thereof, encompasses direct communication and/or indirect communication through one or more intermediary components, and does not require direct physical (e.g., wired) communication and/or constant communication, but rather additionally includes selective communication at periodic intervals, scheduled intervals, aperiodic intervals, and/or one-time events.

Flowcharts representative of example hardware logic, machine readable instructions, hardware implemented state machines, and/or any combination thereof for implementing the example BNN <NUM> of <FIG> are shown in <FIG>. The machine readable instructions may be one or more executable programs or portion(s) of an executable program for execution by a computer processor such as the processor <NUM> shown in the example processor platform <NUM> discussed below in connection with <FIG>. The program may be embodied in software stored on a non-transitory computer readable storage medium such as a CD-ROM, a floppy disk, a hard drive, a DVD, a Blu-ray disk, or a memory associated with the processor <NUM>, but the entire program and/or parts thereof could alternatively be executed by a device other than the processor <NUM> and/or embodied in firmware or dedicated hardware. Further, although the example program is described with reference to the flowcharts illustrated in <FIG>, many other methods of implementing the example BNN <NUM> may alternatively be used. For example, the order of execution of the blocks may be changed, and/or some of the blocks described may be changed, eliminated, or combined. Additionally or alternatively, any or all of the blocks may be implemented by one or more hardware circuits (e.g., discrete and/or integrated analog and/or digital circuitry, an FPGA, an ASIC, a comparator, an operational-amplifier (op-amp), a logic circuit, etc.) structured to perform the corresponding operation without executing software or firmware.

The machine readable instructions described herein may be stored in one or more of a compressed format, an encrypted format, a fragmented format, a compiled format, an executable format, a packaged format, etc. Machine readable instructions as described herein may be stored as data (e.g., portions of instructions, code, representations of code, etc.) that may be utilized to create, manufacture, and/or produce machine executable instructions. For example, the machine readable instructions may be fragmented and stored on one or more storage devices and/or computing devices (e.g., servers). The machine readable instructions may require one or more of installation, modification, adaptation, updating, combining, supplementing, configuring, decryption, decompression, unpacking, distribution, reassignment, compilation, etc. in order to make them directly readable, interpretable, and/or executable by a computing device and/or other machine. For example, the machine readable instructions may be stored in multiple parts, which are individually compressed, encrypted, and stored on separate computing devices, wherein the parts when decrypted, decompressed, and combined form a set of executable instructions that implement a program such as that described herein.

In another example, the machine readable instructions may be stored in a state in which they may be read by a computer, but require addition of a library (e.g., a dynamic link library (DLL)), a software development kit (SDK), an application programming interface (API), etc. in order to execute the instructions on a particular computing device or other device. In another example, the machine readable instructions may need to be configured (e.g., settings stored, data input, network addresses recorded, etc.) before the machine readable instructions and/or the corresponding program(s) can be executed in whole or in part. Thus, the disclosed machine readable instructions and/or corresponding program(s) are intended to encompass such machine readable instructions and/or program(s) regardless of the particular format or state of the machine readable instructions and/or program(s) when stored or otherwise at rest or in transit.

As mentioned above, the example processes of <FIG> may be implemented using executable instructions (e.g., computer and/or machine readable instructions) stored on a non-transitory computer and/or machine readable medium such as a hard disk drive, a flash memory, a read-only memory, a compact disk, a digital versatile disk, a cache, a random-access memory and/or any other storage device or storage disk in which information is stored for any duration (e.g., for extended time periods, permanently, for brief instances, for temporarily buffering, and/or for caching of the information). As used herein, the term non-transitory computer readable medium is expressly defined to include any type of computer readable storage device and/or storage disk and to exclude propagating signals and to exclude transmission media.

<FIG> and <FIG> is a flowchart representative of example machine readable instructions <NUM> which may be executed to implement the neuron <NUM> of <FIG> of a BNN generate a weight corresponding to a probability distribution to apply to input data. Although the instructions <NUM> are described in conjunction with the example neurons <NUM> of <FIG>, the instructions <NUM> may be described in conjunction with any neuron in any type of neural network that includes a weight generator, an entropy source, and a charge pump.

At block <NUM>, the example sleep controller <NUM> determines if input data was received at the weight applicator <NUM>. As described above, the sleep controller <NUM> monitors the weight applicator <NUM> to determine when input data is received to wake up component(s), power up component(s), and/or exit a low power mode of component(s) of the example neuron <NUM>. If the example sleep controller <NUM> determines that input data is not received (block <NUM>: NO), instructions return to block <NUM> until input data is received (e.g., the component(s) of the neuron <NUM> remain asleep, off, and/or in low power mode). If the example sleep controller <NUM> determines that input data is received (block <NUM>: YES), the example sleep controller <NUM> powers up (e.g., wakens and/or instructs to exit low power mode) one or more components of the neuron <NUM> that are asleep, off, or in low power mode (block <NUM>).

At block <NUM>, the example memory interface <NUM> accesses the bit values from the example memory <NUM> to control the C-2C switches <NUM> based on the values of stored in the memory <NUM> to output the mean voltage according to the stored mu value. At block <NUM>, the example variance-to-resistance converter <NUM> converts the variance value stored in the example memory <NUM> into a signal corresponding to a resistance value. As described above, the resistance signal is used to adjust the resistance of the variable resistor <NUM> or other resistive element of the example reference clock buffer <NUM>. As described above, adjusting the resistance adjusts the edge of the rising slope of the clock signal, there by adjusting the probability distribution corresponding to the ring oscillator <NUM> to correspond to the stored variance value.

At block <NUM>, the example variance-to-resistance converter <NUM> outputs a signal that is applied to the variable resistor <NUM> to adjust the variable resistance of the variable resistor <NUM> to a particular resistance (e.g., defined by the output of the variance-to-resistance converter <NUM> and corresponding to the stored variance). At block <NUM>, the example reference clock buffer <NUM> of the example entropy source <NUM> obtains a clock signal from the reference clock <NUM> (e.g., via the clock interface <NUM>). At block <NUM>, the example reference clock buffer <NUM> of the entropy source <NUM> buffers the jitter free reference clock signal (e.g., using an even number of inverters). Additionally, the example reference clock buffer <NUM> adjusts the slope of the rising edge of the jitter free reference clock signal (e.g., using the example variable resistor <NUM>).

At block <NUM>, the example voltage sampler <NUM> determines if the jittery output voltage from the ring oscillator <NUM> of the entropy source <NUM> satisfies a voltage (e.g., is above a voltage when monitoring a rising edge or is below a voltage when monitoring a falling edge). The voltage sampler <NUM> samples the jitter free output of the reference clock buffer <NUM> when the voltage of jittery output clock signal from the ring oscillator <NUM> satisfies the threshold. Alternatively, the voltage sampler <NUM> samples the jittery output of the ring oscillator <NUM> when the voltage of jitter free reference clock signal from the reference clock buffer <NUM> satisfies the threshold. If the example voltage sampler <NUM> determines that the jittery output voltage of the ring oscillator <NUM> does not satisfy the threshold voltage (block <NUM>: NO), control returns to block <NUM> until the jittery output voltage satisfies the threshold voltage.

If the example voltage sampler <NUM> determines that the jittery output voltage of the ring oscillator <NUM> satisfies the threshold voltage (block <NUM>: YES), the example voltage sampler <NUM> samples the output voltage of the reference clock buffer <NUM> (e.g., the buffered clock signal) (block <NUM>). At block <NUM>, the example voltage sampler <NUM> outputs a signal based on the sample to the variable current source <NUM> to pump current toward ground. At block <NUM>, the example switch <NUM> of the example charge pump <NUM> determines (e.g., based on the jitter free reference clock signal which controls the switch <NUM>) if the reference clock corresponds to a discharge duration (e.g., Clk[n] of <FIG>). The discharge duration can correspond to the high duration of the clock signal or the low duration of the clock signal for the first cycle.

If the example switch <NUM> determines that the reference clock signal does not correspond to the discharge duration (block <NUM>: NO), control returns to block <NUM> until the reference clock signal corresponds to the discharge duration. If the example switch <NUM> determines that the reference clock signal corresponds to the discharge duration (block <NUM>: YES), the example switch <NUM> (e.g., the switch at the discharging portion of the charge pump <NUM>) closes (e.g., enables) (block <NUM>). Enabling the switch <NUM> while the variable current source pumps current toward ground causes the charge at the output of the C-2C ladder <NUM> to discharge, thereby decreasing the output voltage at the output of the C-2C ladder <NUM>.

At block <NUM> the example switch <NUM> of the example charge pump <NUM> determines (e.g., based on the reference clock signal which controls the switch <NUM>) if the reference clock still corresponds to a discharge duration (e.g., Clk[n] of <FIG>). If the example switch <NUM> determines that the reference clock still corresponds to the discharge duration (block <NUM>: YES), control returns to block <NUM> until the reference clock no longer corresponds to the discharge duration. If the example switch <NUM> determines that the reference clock no longer corresponds to the discharge duration (block <NUM>: NO), the example switch <NUM> (e.g., at the discharging portion of the charge pump <NUM>) opens (e.g., disables) (block <NUM>), thereby stopping the discharging at the output node of the example C-2C ladder <NUM>.

At block <NUM>, the example reference clock buffer <NUM> of the example entropy source <NUM> obtains a clock signal from the reference clock <NUM> (e.g., via the clock interface <NUM>). At block <NUM>, the example reference clock buffer <NUM> of the entropy source <NUM> buffers the reference clock signal (e.g., using an even number of inverters). Additionally, the example reference clock buffer <NUM> adjusts the slope of the rising edge of the reference clock signal (e.g., using the example variable resistor <NUM>).

At block <NUM>, the example voltage sampler <NUM> determines if the jittery output voltage from the ring oscillator <NUM> of the entropy source <NUM> satisfies a voltage (e.g., is above a voltage when monitoring a rising edge or is below a voltage when monitoring a falling edge). The voltage sampler <NUM> samples the jitter free output of the reference clock buffer <NUM> when the voltage of jittery clock signal from the ring oscillator <NUM> satisfies the threshold. Alternatively, the voltage sampler <NUM> may sample the jittery block signal of the ring oscillator <NUM> when the voltage of jitter free clock signal from the reference clock buffer <NUM> satisfies the threshold. If the example voltage sampler <NUM> determines that the jittery output voltage of the ring oscillator <NUM> does not satisfy the threshold voltage (block <NUM>: NO), control returns to block <NUM> until the jittery output voltage satisfies the threshold voltage.

If the example voltage sampler <NUM> determines that the jittery output voltage of the ring oscillator <NUM> satisfies the threshold voltage (block <NUM>: YES), the example voltage sampler <NUM> samples the output voltage of the reference clock buffer <NUM> (e.g., the buffered clock signal) (block <NUM>). At block <NUM>, the example voltage sampler <NUM> outputs a signal based on the sample to the variable current source <NUM> to pump current toward the output of the example C-2C ladder <NUM>. At block <NUM>, the example switch <NUM> of the example charge pump <NUM> determines (e.g., based on the reference clock signal which controls the switch <NUM>) if the reference clock corresponds to a charge duration (e.g., Clk[n+<NUM>] of <FIG>). The charging duration can correspond to the high duration of the clock signal or the low duration of the clock signal for the second cycle.

If the example switch <NUM> determines that the reference clock signal does not correspond to the charge duration (block <NUM>: NO), control returns to block <NUM> until the reference clock signal corresponds to the charge duration. If the example switch <NUM> determines that the reference clock signal corresponds to the charge duration (block <NUM>: YES), the example switch <NUM> (e.g., the switch at the charging portion of the charge pump <NUM>) closes (e.g., enables) (block <NUM>). Enabling the switch <NUM> while the variable current source pumps current toward the output node causes the charge at the output of the C-2C ladder <NUM> to increase, thereby increasing the output voltage at the output of the C-2C ladder <NUM>.

At block <NUM>, the example switch <NUM> of the example charge pump <NUM> determines (e.g., based on the reference clock signal which controls the switch <NUM>) if the reference clock still corresponds to a charge duration (e.g., Clk[n+<NUM>] of <FIG>). If the example switch <NUM> determines that the reference clock still corresponds to the charge duration (block <NUM>: YES), control returns to block <NUM> until the reference clock no longer corresponds to the charge duration. If the example switch <NUM> determines that the reference clock no longer corresponds to the charge duration (block <NUM>: NO), the example switch <NUM> opens (e.g., disables) (block <NUM>), thereby stopping the charging at the output of the example C-2C ladder <NUM>. As described above, the charge pump <NUM> discharges and then charges the charge at the output of the C-2C ladder <NUM> so that the output voltage corresponds to the mean value plus some voltage that corresponds to the difference between the first sampled voltage and the second sampled voltage, thereby adding some randomness based on the probability distribution of the jitter of the ring oscillator <NUM>. At block <NUM>, the example weight applicator <NUM> applies (e.g., multiple) the adjusted weight output at the output node to the input data to generate the output data.

At block <NUM>, the example sleep controller <NUM> determines if an additional iteration is to be performed (e.g., for an additional weight generation corresponding to the same and/or additional input data). If the example sleep controller <NUM> determines that an additional iteration is to be performed (block <NUM>: YES), control returns to block <NUM>. If the example sleep controller <NUM> determines that an additional iteration is not to be performed (block <NUM>: NO), the example sleep controller <NUM> powers down one or more of the components of the neuron <NUM> (block <NUM>).

<FIG> illustrates an example timing diagram <NUM> of an example input clock reference clock signal <NUM> (e.g., output by the example reference clock <NUM> of <FIG>), an example modulated clock signal <NUM> (e.g., output by the example reference clock buffer <NUM> of <FIG>), and an example jittery signal <NUM> (e.g., output by the example oscillator <NUM> of <FIG>). <FIG> further includes the example charge pump <NUM> of <FIG> for illustrative purposes.

As shown in the example modulated signal <NUM> of <FIG> (e.g., modulated with respect to time and the slope of the raising/falling edge), the signal is delayed (e.g., due to the inventors of the reference clock buffer <NUM>) and has a slower slope due to the variable resistor <NUM> of <FIG>. Additionally, the example jittery signal <NUM> illustrates the jitter produced by the example ring oscillator <NUM>. Accordingly, when the jittery signal <NUM> reaches a threshold voltage during the first cycle, the example voltage sampler <NUM> samples the voltage at the modulated/buffered signal <NUM> (e.g., Vn) and outputs the first sampled voltage (Vn) to the example current source Vn <NUM> to pump current to ground when the switch <NUM> is enabled (e.g., during the duration corresponding to Clk[n] of the input reference clock signal <NUM>). When the jittery signal <NUM> reaches a threshold voltage during the second cycle, the example voltage sampler <NUM> samples the voltage at the modulated/buffered signal <NUM> (e.g., Vn+<NUM>) and outputs the first sampled voltage (Vn+<NUM>) to the example current source Vn <NUM> to pump current to the output of the C-2C ladder <NUM> when the switch <NUM> is enabled (e.g., during the duration corresponding to Clk[n+<NUM>] of the input reference clock signal <NUM>).

<FIG> is a block diagram of an example processor platform <NUM> structured to execute the instructions of <FIG> to implement the example BNN <NUM> of <FIG> The processor platform <NUM> can be, for example, a server, a personal computer, a workstation, a self-learning machine (e.g., a neural network), a mobile device (e.g., a cell phone, a smart phone, a tablet such as an iPad™), a personal digital assistant (PDA), an Internet appliance, or any other type of computing device.

For example, the processor <NUM> can be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. The hardware processor may be a semiconductor based (e.g., silicon based) device. In this example, the processor <NUM> implements at least one of the example reference clock <NUM>, the example memory interface <NUM>, the example variance-to-resistance converter <NUM>, the example clock interface <NUM>, the example entropy source <NUM>, the example voltage sampler <NUM>, the example charge pump <NUM>, the example weight applicator <NUM>, and the example sleep controller <NUM>.

The example local memory <NUM>, the example volatile memory <NUM>, and/or the example non-volatile memory <NUM> can implement the memory <NUM> of <FIG>.

The input device(s) can be implemented by, for example, an audio sensor, a microphone, a camera (still or video), a keyboard, a button, a mouse, a touchscreen, a track-pad, a trackball, and/or a voice recognition system.

The output devices <NUM> can be implemented, for example, by display devices (e.g., a light emitting diode (LED), an organic light emitting diode (OLED), a liquid crystal display (LCD), a cathode ray tube display (CRT), an in-place switching (IPS) display, a touchscreen, etc.), a tactile output device, and/or speaker.

The communication can be via, for example, an Ethernet connection, a digital subscriber line (DSL) connection, a telephone line connection, a coaxial cable system, a satellite system, a line-of-site wireless system, a cellular system, etc..

The machine executable instructions <NUM> of <FIG> and/or 5B may be stored in the mass storage device <NUM>, in the volatile memory <NUM>, in the non-volatile memory <NUM>, and/or on a removable non-transitory computer readable storage medium such as a CD or DVD.

Example methods, apparatus, systems, and articles of manufacture to provide and operate an improved Bayesian neural network disclosed herein. Further examples and combinations thereof include the following: Example <NUM> includes an apparatus comprising an oscillator to generate a first clock signal, a resistive element to adjust a slope of a rising edge of a second clock signal, a voltage sampler to generate a sample based on at least one of (a) a first voltage of the first clock signal when a second voltage of the second clock signal satisfies a threshold or (b) a third voltage of the second clock signal when a fourth voltage of the first clock signal satisfies the threshold, and a charge pump to adjust a weight based on the sample, the weight to adjust data in a model.

Example <NUM> includes the apparatus of example <NUM>, wherein the oscillator is a jittery ring oscillator.

Example <NUM> includes the apparatus of example <NUM>, wherein the resistive element is a variable resistor, further including a converter to adjust a resistance of the variable resistor based on a value determined during training.

Example <NUM> includes the apparatus of example <NUM>, wherein the value corresponds to an amount of variance for the adjusted weight.

Example <NUM> includes the apparatus of example <NUM>, wherein the second clock signal is a jitter free reference signal corresponding to a reference clock.

Example <NUM> includes the apparatus of example <NUM>, further including a C-2C ladder to generate the weight.

Example <NUM> includes the apparatus of example <NUM>, wherein the C-2C ladder includes a number of bit cells corresponds to a bit-wise value of a mean value determined during training, the bit cells coupled to the respective storage cells corresponding to the mean value.

Example <NUM> includes the apparatus of example <NUM>, wherein the sample is a first sample the voltage sampler to generate the first sample during a first clock cycle, and generate a second sample during a second clock cycle, and the charge pump to adjust the weight based on a difference between the first sample and a second sample.

Example <NUM> includes the apparatus of example <NUM>, wherein the charge pump is to adjust the weight by pumping a first amount of current toward ground during a first duration of time, the first amount of current corresponding to the first sample, and pumping a second amount of current toward an output node during a second duration of time, the second amount of corresponding to the second sample.

Example <NUM> includes the apparatus of example <NUM>, wherein the adjusted weight follows a probability distribution corresponding to a mean value and a variance value determined during training.

Example <NUM> includes a neuron in a neural network, the neuron including a C-2C ladder to generate an output voltage corresponding to a mean weight value determined during training a voltage sampler to sample a first voltage of a first clock signal when a second voltage of a second clock signal satisfies a threshold, and a charge pump to adjust the output voltage based on the sample.

Example <NUM> includes the neuron of example <NUM>, further including a jitter oscillator to generate the second clock signal.

Example <NUM> includes the neuron of example <NUM>, further including a buffer to output the first clock signal, the first clock signal corresponding to a reference clock signal after being modulated.

Example <NUM> includes the neuron of example <NUM>, wherein the C-2C ladder includes a number of bit cells corresponds to a bit-wise value of the mean value, the bit cells coupled to the respective storage cells corresponding to the mean value.

Example <NUM> includes the neuron of example <NUM>, wherein the sample is a first sample the voltage sampler to generate the first sample during a first clock cycle, and generate a second sample during a second clock cycle, and the charge pump to adjust the weight based on a difference between the first sample and a second sample.

Example <NUM> includes the neuron of example <NUM>, wherein the charge pump is to adjust the weigh by pumping a first amount of current toward ground during a first duration of time, the amount of current corresponding to the first sample, and pumping a second amount of current toward an output node during a second duration of time, the second amount of corresponding to the second sample.

Example <NUM> includes the neuron of example <NUM>, wherein the adjusted weight follows a probability distribution corresponding to a mean value and a variance value determined during training.

Example <NUM> includes a neural network comprising memory to store a mean value and a variance value, a reference clock to output a reference clock signal, and a neuron coupled to the memory and the reference clock, the neuron to generate an output voltage corresponding to the mean value using a C-2C ladder, and adjust the output voltage corresponding to the variance value using, an oscillator, a charge pump, and the reference clock signal.

Example <NUM> includes the neural network of example <NUM>, further including a voltage sampler to sample a voltage corresponding to the reference clock signal based on an output signal of the oscillator.

Example <NUM> includes the neural network of example <NUM>, wherein the charge pump is to adjust the output voltage based on the sample.

From the foregoing, it will be appreciated that example methods, apparatus and articles of manufacture have been disclosed that provide an improved Bayesian neural network and methods and apparatus to operate the same. Examples disclosed herein leverage the Gaussian distribution corresponding to randomly dithering charge in the analog domain. Examples disclosed herein generates a BNN using (<NUM>) a C-2C ladder to converts the mean weight into an electric charge level, (<NUM>) a jittery oscillator sampling-based entropy source that provides programmable randomness, and (<NUM>) a charge pump controlled by the entropy source to dither the charge generated by the C-2C ladder to give the final, programmable, and Gaussian distributed output.

BNNs introduce uncertainty information to overcome the problems of overfitting and sensitivity to malicious attacks. Instead of using fixed weights, BNNs introduce weights associated with conditioned probability distribution (e.g., the output weight may be a value within a probability distribution defined by a mean and standard deviation). Because BNNs introduce some amount of randomness, BNNs can be trained with smaller training data without sacrificing accuracy. However, traditional BNNs with neurons that generate values corresponding to a probability distribution require a lot of power and/or hardware to implement. Therefore, such traditional BNNs are expensive, complex, and energy inefficient. Examples disclosed herein correspond to a BNN that is implemented with less hardware (and thus less expensive) and is more energy efficient than traditional BNNs. Accordingly, the disclosed methods, apparatus and articles of manufacture are accordingly directed to one or more improvement(s) in the functioning of a neural network.

Although certain example methods, apparatus and articles of manufacture have been disclosed herein, the scope of coverage of this patent is not limited thereto. On the contrary, this patent covers all methods, apparatus and articles of manufacture fairly falling within the scope of the claims of this patent.

Claim 1:
An apparatus comprising:
an oscillator (<NUM>) configured to generate a first clock signal, wherein the oscillator (<NUM>) is a jittery ring oscillator;
a resistive element configured to adjust a slope of a rising edge of a second clock signal;
a C-2C ladder configured to generate an output voltage corresponding to the mean value of a weight in a neural network;
a voltage sampler (<NUM>) configured to sample a voltage based on at least one of (a) a first voltage of the first clock signal when a second voltage of the second clock signal satisfies a threshold or (b) a third voltage of the second clock signal when a fourth voltage of the first clock signal satisfies the threshold; and
a charge pump (<NUM>) configured to adjust the output voltage based on the sampled voltage.