I hope I'm not too late to your thread to ask questions. I find your field absolutely fascinating, though I've never taken my interest further than basic undergrad image processing or simple machine learning. I wish I had more time to follow all of my interests.

Anyway, this one particular gray area has been bugging me, and I've been hoping to run into a researcher or someone that has the appropriate context to clarify it for me. It's not technical question, per se.--it's a set of questions that build up to an uncertainty related to model training, reuse, and sharing. (I hope I'm asking the relevant questions...) I don't expect answers to all of this as it would take way too much of anyone's time, but maybe you can pick a few key points and discuss. I'm also really dying to get feedback on the last part as to its feasibility.

1. Do computer vision / modeling researchers typically share common training and evaluation data sets, or is it mostly kept proprietary? If good data sets do exist out in the open, do they undergo continual improvement, or are they frozen? Are these training sets typically amenable to only one type of training--ie., black box, offline vs online, etc.? Does it take a lot of practice/skill to know how much to hold back for post-training evaluation? Do you have an estimate for how much time and effort is involved in manually annotating and curating these data sets?

2. Once an algorithm is developed, does a model get trained under different parameters? Does tweaking those parameters lead to vastly different results? Are there typically distinct optima for a given classification task, or can it vary? Does the training set have a big impact on the performance of the algorithm? Which is more important, the training data or the algorithm?

3. Once models undergo offline training, are they fast to run? Is there a typical runtime complexity, or do different types of models operate significantly differently under the hood (by principle or by computational complexity)? Can you run an extremely complicated and robust model with a ton of classification outputs in sub-millisecond time on commodity hardware?

4. Can trained models be packaged and redistributed as a kind of "open source"? Are there any obvious barriers preventing this, such as the existence (or proliferation) of patents in your field? Do computer vision researchers like to share their code / results? If it's not a common practice to share code, would a large number of researchers be downright opposed to having their (patentable) algorithms and (copyrightable?) models made available to others?

5. Are trained models too complicated for the layman to use and produce good, consistent results? (For our purposes, I would consider a layman to be someone with at least some understanding of basic computer science, including exposure to data structures, algorithms, and a little bit of mathematical ability. Perhaps none of this too deep.)

6. Highly related to the last question, would there be a lot of specialized knowledge required to tweak model input parameters? Would these parameters correlate to mathematical operations? Would there be any arcane and seemingly arbitrary weights to adjust that are deeply encoded into the model itself? (Not sure if I'm far left field here or not.)

I think what I'm ultimately hitting at is that it would be freaking awesome if there were a good set of robust pre-trained classifier models available for the layman programmer. Models that continually undergo development as improvements are made to training sets, algorithms, the literature--what have you. Please tell me if this kind of thing already exists.

Anyway, I feel like I'm about to go on a long-winded talk about a technology I've been imagining. It's something along the lines of a specification that allows for the broad spectrum sharing of reusable, generic, containerized ML and classifier training results with others. In the world I envision, you would share trained models and import them just as you would code libraries.

Let me reiterate: if this sort of thing happens to exist in the wild already, please point me to it! I can think of tons of uses. :)

Note: replied to my own comment because original text was too long.