skysight-inc/HackerNews huggingface dataset:
Unfortunately this is true of any encoding scheme, short of scrambling the order of the paragraphs. And although scrambling the order might seem tempting, it destroys the ability to train large context windows — 32k context tokens is enough to fit most of a book into a prompt, and this window will only grow bigger.

If your (quite reasonable) argument holds water, then it sinks our ability to ever share copyrighted training data with large context windows. That’s why it seemed important to justify a plain text training format, since anything on top of it would be equivalent.

Indeed, one alternate training format would be to ship the raw html from every epub file, then process it into text at runtime. But this makes it trivial to reconstruct the original epub file and use it in an actual book reader.

It’s frustrating that we can’t share the epub files, because there are so many advantages: you can scrape the metadata, you can tweak the rendering to plain text, you can get semantic info from the images (and even OCR them — it turns out that lots of coding epubs show code examples as screenshots, because epub html rendering is so primitive, so this would be the only way to let your model learn from those).

All of that is why I’m leaning towards "make a script to spider all of libgen and cache the epub files locally". But I haven’t finished calculating how much disk space this would require.

I’m sad that researchers will have to wait days for their training data instead of downloading it in a few hours from a high speed cache, but it seems like any such cache would be swiftly DMCA’ed, so there’s no alternative.
Rigor is not the only variable, agreed. The issue (again) is that the other variables are many and they are non-linear in the main. (For example, the variable of ‘competence of development organization’ is not smoothly distributed. Our field is not a “smooth” domain it is ‘chunky’.)

So where does that leave us? Opinions are one option - comparative views to other ‘industrial’ age type of activities may be informative.

I propose to you that “we moderns” live in a typed world. It is not strongly typed but it is typed. One could argue that that is a side-effect of physical world artifacts produced at scale. I would be interested in hearing the argument as to why that near universal phenomena* does not apply to software, in your opinion.

(* Industrial production at scale and emergence of standards)
Hypothesis:

People gaming clicks using popularity of subjects from past years would want to drift (heh!) the time forward slightly, these topics probably normally arise around the time clocks change for Winter (29 October this year is the end of British Summer Time). So, I speculate that this is a drifted "clocks go back, but will your computer adjust itself?" topic area.
To anyone knowledgeable: where does geometric deep learning fit in here? Is this paper just another geometric view, or is it an attempt at a formalization for transformer mathematics? I don't see prominent GDL authors in this paper's references (Bronstein, Cohen, Bruna, Veličković, ...).
And Judaism and Islam.
If they found a 50k year old giant wooden ship, what would that prove about god or genesis? Or rather, to better answer your questions: what would be verified by the finding and how?
I know nothing about this topic, but the server also has the version id of the last state (that you fetch) before disconnect. Your update can then be placed on top of that version and not some version far in the future when you reconnect.
Not sure what you mean, I read it without an account, but it has to be hosted (originate) somewhere.
(not the parent poster):

Treat data like oil, a public resource, and redistribute the profits to everyone equally.

I don't love the idea of taxing profits and letting the government send checks out, so being creative, how about we force dilution of the stockholders of whoever wins the AI wars and make every citizen a beneficiary of dividends.
> Indeed, but learning is effectively a form of lossy compression

Yeah, that copies in the brain of a human who has experienced the work and not reduced to any othe media are neither “copies” nor “phonorecords” as covered by US copyright law is pretty clear, and that this does not apply to data stored by clever algorithms in computer storage is also pretty clear, so I’m not sure what your point is.

If you want to argue LLMs are people to escape this, then, sure, copyright stops being a problem for their training data, but it still is for their output plus you end up a whole set of new legal problems with using them as people do now, starting with 13th Amendment problems.
This seems super relevant to tech, and not just something you'd find on reddit /r/atheism
> If the Ark was found (and verified somehow), would it change your mind?

Maybe, but it'd definitely have to be verified by Top Men.
And I can't use HTTP/3 over Tor cause Tor only forwards TCP lol

We'll get there
They forgot to add, however, "97.5% of our sales were in that 5th category last quarter, suck it liberals!". And we are only starting in the last 3 days of calendar year 2025.
Frankly, the point of copyright - an artificial monopoly on the reproduction of a work - is to encourage people to create by allowing them to financially benefit from their works.

If letting AI train on whatever their masters want and it results in a reduction of creative works being created ... I think governments might just amend the law to fix that - rendering all the "lawyering" by OpenAI over current copyright laws moot.
Food corresponds to 10~30% of individual carbon footprint. Dairy + meat amount to 3/4 of total emissions from food production. So going vegan should theoretically reduce from 7.5% to 22.5% total individual emissions[1]. It may not be the largest factor in every situation, but it is still a very important one.

[1]https://css.umich.edu/publications/factsheets/sustainability...
You can get centrists over to your side, or people with loosely held beliefs. You aren't getting 99% of the extremists.

Proselytization likely works most effectively on the above groups, and while it might be a big win to convert 1 X believer to a Y believer, you could probably just convert 10 non-believers into Y much more effectively.

Deprogramming sounds exactly the opposite of 'empathy and patience'.
Oh, you're right! And by that logic, Hacker News is a malware vector, too, with all these random links hanging around. Maybe we should petition the moderators to integrate Google's tracking links here as well!
We just create labels, which are rooted in Earths' rotation around the Sun at regular intervals measured by radiation and call it time.
I'm all about getting rid of copyright.

This is...bullshit.

> patterns inherent in human-generated media

That's what you hope your model is, not what your model literally is!

> The aim of this process

That's a lovely thought. Now let's get back to talking about the process itself.

I'm not interested in treating LLM tech like a black box. That's intentional stupidity.

OpenAI's entire argument hangs on not understanding how its tech works! I can't see any difference between that and fraud.
I agree.

The thing I hate about this whole situation is that it's going to be decided solely on which side has more lobbying power[1], since just as you said, it's something in between.

[1]: One could say all the policies are decided this way...
Is it just me, or is the basic knowledge of database normalization and indexing kind of a lost art these days?

How many teams out there just "add cache and more hardware" when no one ran an EXPLAIN on that one core query in a giant table?

Perhaps we should do that before talking about anything "at scale".
Ruby on Rails 7.1.0 Released
There are people claiming everything coming out of these tools is transformed and wholly unencumbered.
Dude. Everyone is asleep.

The largest, most frequent, and easiest to exploit but also easiest to fix issue continues to DOMINATE hacks:

Not sanitizing user entered inputs.

Not saying this is an excuse to rid of TLS, but TLS is not really where the main focus of people hosting home blogs needs to be.

Ah. I see the “it’s easier to reason about” crowd of hype driven developers are here, as indicated by how the voting is going.
You're not reading that correctly. Google tried to get away with one overly broad privacy policy, allowing them to combine personal data from various sources/services, and that is not allowed as per the decision.
but it also fails on the very philosophical framework used to come up with those laws

they're using philosophical frameworks older than digital computers; which is ok, and is as should be. up to the point where those ways to understand fail to capture certain qualities of computers and digital technology which break the whole notion of copyright and have been doing it for a few decades now.

those lawmakers are using obsolete philosophy!

but we gotta wait until they all die off, is not like people, specially older people, are willing or possibly able to change how they think

I blame digital technology, it just doesn't work like the rest of reality does...
DON'T UPGRADE TO ANDROID 14, I am 4a 5G user, work as charm on last version of Android 13 September version, I feel like a lot of dropping frame while using Android 14. I feel upset and dissapointed towards Google now.
> I absolutely do not trust Microsoft

Sentence should have stopped there.
No, it would not likely change my mind. I'd see it as the origin of the man-with-a-boat-in-a-flood stories, of which the biblical story is just one.

An old wooden boatlike structure would be an interesting artifact, but in itself not evidence of anything supernatural.
and after that court case, I’ll go after other accepted fair use works under a 14th amendment challenge saying it needs to apply equally to other use cases simply because ads are being used, or a university charges for a book, or a publication charges for subscription access

which I’m fine with, dont think I’m trying to deter you with a slippery slope fallacy, we’re already at the bottom of it
Interestingly, the Go memory model[1] does in fact guarantee atomicity of word-sized reads/writes:

> Otherwise, each read of a single-word-sized or sub-word-sized memory location must observe a value actually written to that location (perhaps by a concurrent executing goroutine) and not yet overwritten.

However, it allows the implementation to immediately exit and report an error as well.

[1]: https://go.dev/ref/mem
Political Disinformation and AI
The Go runtime has quite a few places where it assumes atomic load/store behavior of int sized words as well as the eventual propagation of the stores.

I always get a little anxious when looking at such code. But it seems to work well in practice?
Or fire, as stated in Genesis 8:20
Imagine you entering a non-crowded venue and prices on wireless e-Ink tags are changing according to your profile got by face-recognition camera at the front door.
Google didn't force anyone to put ads on their blogspam sites.
I wonder when disabling autoplay for media will be enabled as a setting to be toggled.
Sure if you utterly ignore all of the fair use parts of US copyright law.
Can you provide a reference for this? My web search didn't find a result.
Makes sense. Thanks for sharing all that you have. It's an incredible resource.
It was all that. I don't have eidetic memory but I the visuals presented in this book I recall vividly.
Or perhaps just some group of hotel owners that's tired of not being paid?
>what happens to the cost of medical plastics,

Plastic production is an insanely small part of oil and gas usage, 4%. Anyone bringing up plastic as a reason to not stop using oil isn't being intellectually honest. We could literally decimate the oil industry and still produce way more oil than the plastic industry needs. It's just not a concern.
You can own a car, you can own a flamethrower, you can't fire your flamethrower out of your car while driving on public roads.
I would assume that if an old boat were found that it had nothing to do with the Ark of Christian mythos. There would be no way to "verify" it because the entire story rests on a foundation of faith, not fact. Where faith and fact do coincide faith likes to use that overlap as evidence of truth whereas fact cannot do that because faith does not require evidence (and there are so many competing faiths but facts are a bit more consistent).

Kind of like how if all religions and all scientific and mathematical knowledge were somehow irrevocably lost, all of the science and math would eventually come back just as it was but none of the religions would (in spite of the protestations of fervent believers).

The flood story is present in so very many mythologies that there likely was some historical event in that region but people back then could be forgiven from thinking that such a flood covered the entire world as it covered what little they new of it.
Making thermostats that are aware of dynamic energy pricing or, better yet, are in part controlled by the energy company (I want my house to be 21-24 C, I don't care when the cooling happens) would give us massive flexibility.

All power hungry devices should at least have that capability (maybe other than the kettle, lol). This is literally a cost of $20 hardware in many cases.
The Power of Synthetic Data: Techniques, Benefits, and Challenges
Anything that satisfies intellectual curiosity belongs on HN.
But the human that consumed the content paid for it didn't they?
The ICE1 being the longer version with locomotives ("Triebköpfe", literally "driving heads") at both ends, while the ICE2 is only half as long, with only one locomotive, and two of them can be coupled - the restriction you mentioned only applies to these.

...but actually the ones to first use this configuration were the French with their TGV, which was then copied by the Germans and Italians for their respective high-speed trains (https://de.wikipedia.org/wiki/TGV#Allgemein). And then the Austrians implemented a "budget version" of a high-speed train with their Railjet (https://en.wikipedia.org/wiki/Railjet) which uses an unmodified Taurus locomotive and a control car which has the same design as the locomotive.
This sounds like "I don't know what a wheel is, but if I chisel this square to be more efficient it might work". Sometimes, it's better to not reinvent the wheel, but just use the wheel.

Pretty much everyone serving video uses DASH or HLS so that there are many versions of the encoding at different bit rates, frame sizes, and audio settings. The player determines if it can play the streams and keeps stepping down until it finds one it can use.

Edit:
>Putting a glorified powerpoint in an mp4 to distribute doesn't feel to me like it is the kind of task where the prerequisite knowledge includes what the difference between yuv420 and yuv422 is or what Annex B or AVC are.

This is the beauty of using mature software. You don't need to know this any more. Encoders can now set the profile/level and bit depth to what is appropriate. I don't have the charts memorized for when to use what profile at what level. In the early days, the decoders were so immature that you absolutely needed to know the decoder's abilities to ensure a compatible encode was made. Now, the decoder is so mature and is even native to the CPU, that the only limitation is bandwidth.

Of course, all of this is strictly talking about the video/audio. Most people are totally unawares that you can put programming inside of an MP4 container that allows for interaction similar to DVD menus to jump to different videos, select different audio tracks, etc.
I see no problem with this as long as the dividends are equal to the potential of anyone displaced or otherwise having their marketshare impacted.

EDIT: otherwise, if I lose a 120k+ (or whatever) job a year programming because one engineer and a cadre of llms is 'good enough' for management, and all I get is $15 a month for my portion of the collective dole, it's not really a solution.
I don't know why this is getting downvoted? Tesla is a perfect example of the "shine wearing off".
Hey chx, got a response to the details listed here?
> would it change your mind?

about what?

I ask that because I believe it's possible to interpret the new testament, the old testament, Egyptian writings about Hyksos, various other historical sources from Greece and Rome and other older cultures all in differing ways, yet find overlapping "lossy" recountings of what was even then ancient history.
>OpenAI's entire argument hangs on not understanding how its tech works! I can't see any difference between that and fraud.

This is how I see it as well.

If you can't explain, in detail, how your mechanism is not violating copyright then you have no defense.
Incidentally this will create a reaction that companies will make their content to be unavailable on public sources to prevent they working for free to feed AI freeloaders.

The opposite view is also valid. SEO-types will figure out on how to deploy their BS into models so that they will recommend their stuff.

As AI models tend to replace the “search” market, they will become as useless as today search tech.
Well of course it won't look like a traffic ticket, when they just pay with a credit card online and call it done. There is paperwork and "promises" and agreements. They have to have a compliance "officer" now and they have to report that propellant tracking has been implemented. That's still a drop in the bucket to have to shorten the life of their satellites. The fuel alone probably costs more than $150k.
If companies would have to pay for externalities as well as the exploitation of the less developed world: capitalism wouldn't be “competitive”.
In other words: the seatbelt is not a deciding factor.

Ultimately deaths cannot be prevented by seatbelts. People will die regardless whether they wear a seatbelt or not.

We can argue its because of a lack of seatbelts, roll cages, airbags and so on. I'd argue nothing can force force a person to be perfectly safe while driving.

> The examples are bona fide safety issues

I'd disagree because those are driving errors. You can present those as cars missing safety features, but you could also not use those safety systems and just drive carefully. Whether we use safety systems or not is a design choice.

Those can be considered "safety issues" (or missing safety? a type of safety obsession?) in a safety design perspective. But it's still an issue with the driver's behavior.

bit of hyperbole for flavour:

If we could just ditch the heavy roll cage, we could go much faster!
It's not superficial in the slightest. There is a cult of creativity that leads people to say things like "a neural network can never do anything creative" and instead that it just parrots back exactly what it's been told but no, it has an ability to generalize, which is why diffusion models trained to draw pornography draw girls with 7 belly buttons and you don't have to be worried that ChatGPT will tell you how to build a bomb because it will tell you how to build a hydrogen bomb using shaving cream and ski wax.

When I read a book it changes connections weights between the synapses in my brain. I don't retain an exact copy but I recall the gist of the book and some facts (imperfectly) I have the ability to generalize this information (I can read The Tale of Genji and retell a story based on it that is set on a Mars Colony in 4712AD) and it can improve my linguistic abilities in general.
The use is using it for training data. I'm pretty sure there isn't much of an effect on the market for training data.
A multi-billion dollar corporation is withholding payments from smaller businesses. Is that not enough reason to publish such an article?
I would imagine looking at this fragile marble from space and then coming back to see things like decades-long killings over "disputed territories", or people getting grey hairs over completely insignificant things, all of that would seem incredibly petty and unnecessary.
IMO, the most likely outcome will be a move towards stronger checks to validate personhood.

The sociotechnical changes necessary to do this effectively are already in motion, facilitated through the big three consumer identity providers - Apple, Google and Microsoft. State-run services could also play a part.

There are privacy preserving means of doing this safely, but they would need to be extremely robust in order to gain and maintain the public’s trust.
I wrote a system that used the database as essentially a rules engine for "what work to execute next?" There were constraints about max throughput per key, waiting periods, etc.

That system broke me down and built me back up over time. Really deep explaining, understanding the internals of how the data is actually being processed, then finding ways to improve it. I had 1-2 other people who were really interested in postgres that would help work on debugging.

We would continuously get 100x query improvements every time we dove in. Including after we thought we maxed it out.

I hope everyone gets the chance to work on something like that. It was really fun.
I think people are just disappointed that their beloved bubble is now controlled by the opposition. Musk went from god to devil in these circles in a very short time. Meanwhile I've never cared about either and am just trying to understand the mass hysteria. Twitter was a cesspool a decade ago, just that they didn't notice because they agreed with the bullshit.
Its worth noting that Fair Use, while now also incorporated into statute which Congress can change by refular law, was first articulated by courts as a First Amendment limit on the copyright power, and later incorporated as statute law (which is why its kind of a fuzzy statute: it just literally reproduced the judicially-articulated multifactor test.)

So, fair use arguments probably cannot be overcome by simple statutory change.
What is historically wrong?

And if you do not need Marxism, where are all other success cases in the hundreds of countries in the world outside these cases? You know that we have almost 200 countries, only a small minority manage to develop themselves. What explain the incredible coincidence that we have a so small number of developed countries, all them only divided in that 4 cases? What explain that the only countries that manage to became competitor to major capitalist powers are countries like USSR and now China?
Why is an AI model allowed to read a book without paying for access to it, but I as a human have to pay for it to read it?
Agreed in full.
Don't say that to anyone from Northern England, they had to suffer these things for a generation: https://en.wikipedia.org/wiki/Pacer_(British_Rail)
You keep the gold in the form of relatively small coins, not large bars.

> It seems pretty useless for spending on supplies and food if you ask me.

Seem pretty useful to me. At least historically, you can generally exchange gold for goods even in a total disaster scenario. You won't be weighing anything out or getting change, of course. You'll be seriously overpaying by the standards of a functioning economy. But nonetheless, you'll be able to engage in enough commerce to help you survive.

That said, I don't keep gold around. I do keep cash -- but if things have devolved to the point where cash isn't useful, then I'll have much larger problems than figuring out how to buy stuff.
Today algos can identify people by their walking habits. (cue Ministry of Silly Walks)
Agree, but topics sounded like an improvement to me.
Give me a case where the fact an original "was created for human consumption" was used as a successful fair use defense.
From the article:

Some of the attendees had lied about their gender identity on their conference registrations, said Cullen White, the chief impact officer with AnitaB.org, the nonprofit that organizes the conference.