I'd say that we'd need some more law to be certain, either some precedent ruling or explicit new legislation, but as of now some of the relevant and irrelevant arguments would be:

1) Purely mechanistic transformations aren't copyrightable, so if a model is determined to be that (it may be debated if it is or isn't), then it has no independent protection from copyright law other than any potential restrictions imposed by fragments of other copyrighted works included in that particular model.

2) On the other hand, even the tiniest bit of creativity is sufficient to make something copyrightable. Some work on ML definitely has some creativity, however is the "creative part" fully within the code used for training and, or is part of that creativity embedded also in the weights output by that code?

3) Facts about a copyrighted work; e.g. the word frequency statistics of a copyrighted book do not violate the work's copyright and also are not copyrightable themselves, so at least some language models - the very simplest word n-gram models, which were used both not that long ago for statistical machine translation and also long, long ago in pre-computer dictionary work - aren't covered by copyright, just as any other collection of facts. Databases may have specific ('sui generis') extra protection granted by law, but then that's a separate debate than general copyright.

4) In general, there are different things which often are conflated but in this complex situation the difference may be meaningful and even critical - e.g. (a) being a derived work of the training data; (b) violating the (limited, explicitly enumerated) exclusive rights of the author of that training data; and (c) being copyrightable itself - those are three separate things that might have different answers for the same type of model.

5) "sweat of the brow" doctrine asserts that (at least in USA) it doesn't matter how much effort and money it took to make something, so the effort/expense of training a model is not relevant to it 'deserving' protection and being copyrightable.

6) Not all creative creation is protected by copyright - the law lists very many protected types, but the rest are not protected unless courts get convinced that they fall under the umbrella of something explicitly listed in copyright law; for example, copyright applies to software because (and only because) program code is considered to be a type of literary work, which is one of the things listed in https://www.copyright.gov/title17/92chap1.html#102 or equivalent legislation elsewhere. Are model weights a type of literary work? That's a tricky question, and my opinion on it doesn't really matter, we'd need to hear a precendent-setting judge assert if they are or not.

... and there's probably many more arguments that should be considered before even trying to answer the original questions. And I think that it's illustrative to see what all the megacorps with megapaid megalawyers are doing; it seems that they're writing all their legal documents from a perspective where models might not be copyrightable and trying to assert whatever conditions they want under some type of contract, since conditions properly agreed under contract law will apply no matter how the copyright law will be interpreted.