> are neural nets deterministic at all? Given all their randomness, I'd assume not.

That's actually an interesting question; I'm probably not the person to answer it, since my experience with NNs is admittedly on the "still learning" side of things, and that learning has not been via any credentialed sources...but I'm willing to take a stab at it.

NN models - that is, the thing that results from training a neural network, are typically "fixed", in that once trained, the model doesn't change - at least classically. I imagine that somewhere out there, there may be NN models which can change as data they are processing is run through them, learning on the fly so to speak. I'd have to research it; I'm sure it's something that's done or been done?

But normally, once a NN has been trained, it's model is "fixed" and doesn't change when inputs are presented to it. The training phase is - or appears to be - fairly stochastic. I'm not sure I'd want to call it random, though, because I don't recall any kind of random numbers being used during back-propagation.

But the data that is presented to the neural network is usually randomized in presentation - and sometimes content. That is, say you're training the NN on recognizing horses. You might have several thousand images of horses, but you don't want to show just those. You might want to generate many more - rotate each one just a bit, skew it just a smidge, maybe change the color and/or contrast/brightness, etc - to in effect generate a bunch more positive (and just as many negative - so you don't get bias or overfitting happening) examples to train the NN on what is a "horse" vs what is a "not horse".

So that data is somewhat "random" - but from what I recall, the actual mechanics of training - the algorithms of forward passes and backwards passes (backpropagation) don't have any randomness to them; just to be sure, I checked this - which is a great explanation (not the simplest, but not impossible to follow):

https://mattmazur.com/2015/03/17/a-step-by-step-backpropagat...

I don't know what your math skill level is here, so don't let any of the calculus and "chain rule" stuff get to you if you aren't familiar with it (tbh - I suck at it), just look at the equations and explanations. It's plain that there is no random number generator to be found in the process.

So - in theory - if the model, after it has been trained and "baked in place" so to speak - is presented with the exact same inputs, it should generate the exact same output.

But the input has to be exactly the same; in the case of an image, the neural network's input layer is usually an "unrolled" 1-dimensional array representing the pixel values of the image (left-to-right, top-to-bottom - as a 1D array - usually). Those values are usually grayscale or color values, presented either as integer data or floating-point values.

As long as that image data is presented exactly the same to the NN model, the output should be the same; for instance, if shown a set of pixels values that represent something, the output of the model will always be the same if shown those exact same pixel values.

But usually, these systems aren't built to take in "exact data" but rather data from sensors of the real world. So - the data that would probably be fed into the NN model likely comes from, say, a camera - and that sensor will not always present the exact same data to the NN model, even if purposefully set up to do so - because all such sensors have noise and aren't perfect (different pixels from the camera's sensor can and will return different values, even if shown a calibrated blank image in a fixed mode, with consistent fixed lighting, with the best camera sensor available, etc - it's just a fact of the real world).

So, because of this - the output of the NN model will in effect be "random" - but only because the input is effectively "random". That's actually ok, because what the model outputs (even if set up as a classifier) are values of probability - percentage values, where (ideally) the "spike" in the overall set of values represents the actual identification for the network, and that any inherent randomness in the system (sensors and whatnot) is filtered out and (hopefully) doesn't effect the outcome.

Though as we know, this too can be exploited; because the model is "baked in", you can show a series of images to the network, and get an identification on the other end, and probably with some statistical analysis you can work out what the layers in between might actually represent (probably not exactly though) - and identify flaws that could cause misidentifications on the output. More or less "hacking the NN model middle layers" and using that information to craft an "exploit" for the input to cause a particular mis-identification on the output side.

As we know - this is possible; at least, we know it's possible to get a network to output a wrong result by simply changing the pixel values of the input image slightly (even subtly that can't be seen by a person looking at the image - basically a steganographic style attack?). Such a "hack" could be used for any number of purposes, but mostly they show how fragile such NN models can be.

Does this mean they are random, or deterministic? I'd argue for the latter at this point, but again, I'm not really the person to ask. Hopefully someone else can answer (or has, by the time I post this).