MurphyA/hacker-news huggingface dataset:
by MikeTheGreat | 2025-12-02 23:17:53+00:00
(My apologies if this was already asked - this thread is huge and Find-In-Page-ing for variations of "pre-train", "pretrain", and "train" turned up nothing about this. If this was already asked I'd super-appreciate a pointer to the discussion :) )

Genuine question: How is it possible for OpenAI to NOT successfully pre-train a model?

I understand it's very difficult, but they've already successfully done this and they have a ton of incredibly skilled and knowledgeable, well-paid and highly knowledgeable employees.

I get that there's some randomness involved but it seems like they should be able to (at a minimum) just re-run the pre-training from 2024, yes?

Maybe the process is more ad-hoc (and less reproducible?) than I'm assuming? Is the newer data causing problems for the process that worked in 2024?

Any thoughts or ideas are appreciated, and apologies again if this was asked already!
by throwaway2037 | 2025-12-02 23:18:02+00:00
Another great post. Thank you. It is great to hear that haven't suffered any monetary loss and you are getting fewer scam calls.

I have a "dumb" follow-up question: (Honestly, I don't understand the pushback here on HN against this app.) Do you feel it is invasive or acts as gov't surveillance on your mobile phone? What you describe sounds pretty good to me.
by pragma_x | 2025-12-02 23:18:19+00:00
IIRC, that team are also (now) live-music-coding veterans, which in turn has informed how Strudel is built. It's not just a project that does stuff, it's a pretty well crafted instrument that is ideal for these performances.

As an engineer, I love letting the requirements shape the solution, but this is just on a whole other level.
by Aurornis | 2025-12-02 23:18:37+00:00
> But you're never going to get the keys to the kingdom on day #1.

I always tell juniors that even if their company doesn't have an explicit probationary period, they should assume their behavior and results are being carefully monitored for the first year to watch for signs of a bad hire.

Hiring someone is never equivalent to having full trust in them. Reputations don't start at 100% on day 1, they start as a neutral value that you need to build up over time. You also need to avoid breaking it down. It's much faster to destroy a reputation than build it up.
by tetha | 2025-12-02 23:18:50+00:00
> To paraphrase: Researchers use linear algebra because it makes their calculations fast.

Mh, if I am contrarian: Optimization of linear algebra is an established important subset of numerical computer science, because you can reduce many important problems to sets of linear equations. I studied that for a semester in many details. Many simulations, fluid, weather or mechanical were at least initially implemented as sets of linear equations because linear equation solvers were well-researched and optimized. Because they were fast and had a predictable range of error.

So yes, LinAlg makes other research go fast.

> A person with these qualifications has absolutely zero business heading a government fund. At minimum I would want this person to either have a technical degree or some industry/FOSS experience. I don't think it is unlikely, that this women has not written a single line of code in her life. > > The reason is that only such a person can adequately understand why these projects need funding and what the consequences of not funding them are.

Though her job could easily be to be the competent at politics and communication, not code. She may fail at the level you are looking at, but many techies at $work with more chops fail in the other direction much, much harder and don't get anything besides frustration.
by noodletheworld | 2025-12-02 23:19:06+00:00
If claude code starts having ads for bun in the code it generates, I am never using it again.

To some degree have “opinionated views on tech stacks” is unavoidable in LLMs, but this seems like it moves us towards a horrible future.

Imagine if claude (or gemini) let you as a business pay to “prefer” certain tech in generated code?

Its google ads all over again.

The thing is, if they own bun, and they want people to use bun, how can they justify not preferencing it on the server side?

…and once one team does it… game on!

It just seems like a sucky future, that is now going to be unavoidable.
by kshacker | 2025-12-02 23:19:12+00:00
I was just giving my history :) but yes I am sure this could actually get us out of the COBOL lock-in which requires 70 years old programmers to continue working.

The last article I could find on this is from 2020 though: https://www.cnbc.com/2020/04/06/new-jersey-seeks-cobol-progr...
by _carbyau_ | 2025-12-02 23:19:13+00:00
Uh, really no.

That is some hard stereotyping being generalised on a platform with worldwide reach. You may wish to rethink what led you to that statement.

Although, with that statement and others from you recently I'm guessing you have "lost your fucks and don't have any more to give" IE burnt out on it all.

Good luck, I hope you get to a place where you can not rely rely on shortcuts like stereotypes so much and have more energy to give to yourself and your life.
by Insanity | 2025-12-02 23:19:31+00:00
People in the comments seem confused about this with statements like “greenest AI is no AI” style comments. And well, obviously that’s true but it’s an apples to pears comparison.

Clearly Ecosia is pushing for “people want AI” _and_ we want to make it more ecofriendly. Taking away features from users altogether is not the right answer.

It’s like saying “cheapest car is no car”. It doesn’t solve the fundamental problem of “wanting a car”.
by TacticalCoder | 2025-12-02 23:19:54+00:00
> Google will do just what Microsoft did with Internet Explorer and bundle Gemini in for 'Free' with their already other profitable products and established ad-funded revenue streams.

Just some numbers to show what OpenAI is against:

GMail users: nearing 2 billion Youtube MAU: 2.5 billion active Android devices: 4 billion (!) Market cap: 3.8 trillion (at a P/E of 31)

So on one side you've got this behemoth with, compared to OpenAI's size, unlimited funding. The $25 bn per year OpenAI is after is basically a parking ticket for Google (only slightly exaggerating). Behemoth who came with Gemini 3 Pro "thinking" and Nano Banana (that name though) who are SOTA.

And on the other side you've got the open-source weights you mentioned.

When OpenAI had its big moment HN was full of comments about how it was game over for Google for search was done for. Three years later and the best (arguably the best) model gives the best answer when you search... Using Google search.

Funny how these things turns out.

Google is atm the 3rd biggest cap in the world: only Apple and NVidia are slightly ahead. If Google is serious about its AI chips (and it looks like they are) and see the fuck-ups over fuck-ups by Apple, I wouldn't be surprised at all if Alphabet was to regain the number one spot.

That's the company OpenAI is fighting: a company that's already been the biggest cap in the entire world and that's probably going to regain that spot rather sooner than later and that happens to have crushed every single AI benchmark when Gemini 3 Pro came out.

I had a ChatGPT subscription. Now I'm using Gemini 3 Pro.
by patcon | 2025-12-02 23:20:24+00:00
> rediscover the hard way the older wisdom about having separate spheres with separate hard constraints and boundaries on behaviors

This is something I also believe. Thanks for saying it.

I've been thinking and reflecting a lot on what I've been calling for myself "generative constraint". It's sure as heck not something that is the same for everyone, but I think we all have a set of them that might help us be our best person.

We've universalized constraints and expansivenesses in a way that seems really poor judgement. And yes, there is a capitalist critique in this too, as any good theory should have :)
by nodja | 2025-12-02 23:20:35+00:00
Pre-training is just training, it got the name because most models have a post-training stage so to differentiate people call it pre-training.

Pre-training: You train on a vast amount of data, as varied and high quality as possible, this will determine the distribution the model can operate with, so LLMs are usually trained on a curated dataset of the whole internet, the output of the pre-training is usually called the base model.

Post-training: You narrow down the task by training on the specific model needs you want. You can do this through several ways:

- Supervised Finetuning (SFT): Training on a strict high quality dataset of the task you want. For example if you wanted a summarization model, you'd finetune the model on high quality text->summary pairs and the model would be able to summarize much better than the base model.

- Reinforcement Learning (RL): You train a separate model that ranks outputs, then use it to rate the output of the model, then use that data to train the model.

- Direct Preference Optimizaton (DPO): You have pairs of good/bad generations and use them to align the model towards/away the kinds of responses you want.

Post-training is what makes the models able to be easily used, the most common is instruction tuning that teaches to model to talk in turns, but post-training can be used for anything. E.g. if you want a translation model that always translates a certain way, or a model that knows how to use tools, etc. you'd achieve all that through post-training. Post-training is where most of the secret sauce in current models is nowadays.
by TrnsltLife | 2025-12-02 23:21:24+00:00
It's a normal part of Christian belief and biblical teaching, e.g. 1 John 2:18, 22:

18 Children, it is the last hour; and just as you have heard that the antichrist is coming, so now many antichrists have appeared. This is how we know it is the last hour. ... 22 Who is the liar, if it is not the one who denies that Jesus is the Christ? This is the antichrist, who denies the Father and the Son.

It has also always been a Christian belief that Jesus will return, the current world system will end, and God will establish new heavens and a new earth.

2 Peter 3 talks about this. It's too long to quote the whole chapter but here are some call outs.

https://biblehub.com/bsb/2_peter/3.htm

4 “Where is the promise of His coming?” they will ask. “Ever since our fathers fell asleep, everything continues as it has from the beginning of creation.” ... 8 Beloved, do not let this one thing escape your notice: With the Lord a day is like a thousand years, and a thousand years are like a day. 9 The Lord is not slow in keeping His promise as some understand slowness, but is patient with you, not wanting anyone to perish but everyone to come to repentance.
by IncreasePosts | 2025-12-02 23:23:31+00:00
Why wouldn't China just keep their own weights secret as well?

If this really is a geopolitical play(I'm not sure if it is or isn't), it could be along the lines of: 1) most AI development in the US is happening at private companies with balance sheets, share holders, and profit motives. 2) China may be lagging in compute to beat everyone to the punch in a naked race

Therefore, releasing open weights may create a situation where AI companies can't as effectively sell their services, meaning they may curtail r&d at a certain point. China can then pour nearly infinite money into it and eventually get up to speed on compute and win the race
by Insanity | 2025-12-02 23:23:34+00:00
The “understand one layer below where you work” is something my professors at uni told us 10+ years ago. Not sure where that originated from, but I really think that benefited me in my career. I.e understanding the JVM when dealing with Java helped optimize code in a relatively heavyweight medical software package.

And also, it’s just fun to understand the lower layers.
by LogicFailsMe | 2025-12-02 23:23:47+00:00
Learned helplessness is just another term for complicit IMO. I'm reminded of the streaming media people whining about licensing terms rather than taking responsibility for their poor job at negotiating with the studios and other holders of media.

if these supposed elected representatives can't take the responsibilities of their jobs and they just want the perks, they need to resign to make room for someone better. Not holding my breath there.
by kmeisthax | 2025-12-02 23:23:53+00:00
> If one wants to try to set another floor, it involves every country in the WTO to agree on that.

This is less of a tough sell than you think. In pretty much every IP-related trade negotiation, you can divide the world into two categories:

* Ultra-rich countries that want to push through every insane IP idea they have (life+50, DMCA 1201, etc) onto as many other countries as possible

* Everyone else

Notably, the ultra rich are all "dealmaker countries". They're the ones dictating the terms of international trade to everyone else and whatever terms they insist upon will be accepted without question. So yeah, if, say, South Africa or India want shorter terms, they still have to respect America's terms, at least up to life+50. But if the US wants shorter terms out of India, they will get shorter terms out of India, come hell or high water.

Conversely, Mexico has life+100 terms, but nobody is trying to use them to ratchet up terms elsewhere. They're a deal taker.

The real question is if another ultra-rich country will stop one that tries to lower the Berne minimum. Keep in mind that the ultra-rich subdivide into groups that, in order of relative IP insanity, are: Europe, Japan, and then the US in the crazy slot. If the US were to, say, repeal DMCA 1201; Europe would cheer and Japan would grumble.

Actually, the Berne convention happened during a time when Europe was the copyright basketcase and America was in the "everyone else" category[0]. The US had 28+14 terms up until 1976, and we didn't join the Berne Convention until 1988 - almost a hundred years late[1]! So if the US were to drop the Berne floor, you could totally imagine the EU going insane and trying to trade war the US out of it. But at the same time, the EU isn't very good at fighting trade wars with other ultra-rich blocs. Or at the very least, they fold very easily.

[0] For exactly the same reason why China is today. China is in the same position America was a century ago, where it had a huge manufacturing base and basically no cares about copyright.

[1] In particular, the US really, really hated automatic registration. While it is true that you don't have to register copyright and users of creative works have to treat everything as copyrighted; creators still have to register anyway if they want to actually enforce their rights. And if they don't do it right away they don't get statutory damages, which are almost always the only damages that matter. So you get all the problems of automatic registration with all the problems of copyright formalities.
by Madmallard | 2025-12-02 23:23:55+00:00
This is entirely due to sociopaths that stole trillions in assets from the general population then bought out law makers to not outlaw their blatant copyright infringement so they can make themselves richer with some self-proclaimed 'noble' aim.

The general population is being rapidly sacked as a 'necessary' expense of criminal elites.

No one should be happy about this.
by prmoustache | 2025-12-02 23:24:46+00:00
YMMV today I literally lost all sound in the middle of an MS teams meeting on my win11 works laptop. When opening the mixer, I could see the little vumeter when sound should have been played. Unplugged the trs cable to my amp, sound would not play from the external speaker, connected my bluetooth headset, nothing (still that vumeter moving when playing random youtube videos to test), tried with cabled headphones, selecting thw default devices manually, different apps, nope, not any better.

In the end I rebooted and sound was working again. Something related to sound (driver, subsystem) had probably crashed randomly.

It was the third time I lost sound in the last 2 months. That is not counting the many windows updates that fail randomly with obscure codes, the randomly undetected monitors, windows Apps that randomly change my selected monitors after I lock and unlock my session and a number of other bugs I encountered in the last 6 months

This never happened in the last 6 years I had been making videocalls with MS Teams on Linux. Only issues I had back then was Teams not always showing new plugged/connected audio devices but this also happens frequently on windows so I fault MS Teams, not the OS in this case.

I won't say Linux never has bugs but statistically it seems to me that on well supported laptops (thinkpads), Linux is much more reliable than windows.
by gwbas1c | 2025-12-02 23:25:31+00:00
One of the critical flaws in the article is that the first chart only looks back 5 years, and the second only looks back 10.

The boom-bust recession cycle is roughly every 10 years. You can't say that AI is impacting hiring when your data just looks like the typical 10 year cycle. Your data needs to go back further.

That being said, what's more likely going on:

1: There are always periods where it's hard for recent college grads to get jobs. I graduated into one. Ignoring AI, how different is it now from 10, 20, and 30 years ago?

2: There are a lot of recent college grads who, to be quite frank, don't work out and end up leaving the field. (Many comments in this thread point out how many junior developers just shouldn't be hired.) Perhaps we're just seeing many companies realize it's easier to be stricter about who they hire?
by MiiMe19 | 2025-12-02 23:26:09+00:00
Capitalism did not kill the natives, the colonial governments and other European governments did. Communism implies evil/bad/death, but capitalism does not. And even then, the "system" in place at the time was mercantilism, not capitalism. For your last argument, you do not need to be Gordon Ramsay to know when something tastes bad. Once again, growing up under communism implies with 99% certainty that you do not want communism. It is an implication, not a bi-implication.
by Kirby64 | 2025-12-02 23:26:48+00:00
> If you're happy with your job and don't need the extra money (or risk), then why invest in the stock market?

Because parking money in a savings account essentially erodes its value over time due to inflation. It’s safer, sure, but it’s a safe way to be guaranteed to lose money over time.

Equity investing is essentially the only way for savings to outpace inflation.
by nobody9999 | 2025-12-02 23:27:19+00:00
>It’s actually gracious of them to offer an alternative.

I strongly disagree. The courts have repeatedly struck down limits on domestic travel over the past couple hundred years.

In fact, the $45 "fee" is an acknowledgment that you aren't required to have special documents to travel within the US. Otherwise, they just wouldn't let you travel.

So instead, they're making more security theater and punishing you if you don't comply with their demands.

That's not "gracious," that's coercive.

Edit: Fixed incorrect usage ("they'd" vs. "they").
by user34283 | 2025-12-02 23:27:21+00:00
I decided to stick with Node in general. I don't see any compelling reason to change it.

Faster install and less disk space due to hardlink? Not really all that important to me. Npm comes with a cache too, and I have the disk space. I don't need it to be faster.

With the old-school setup I can easily manually edit something in node_modules to quickly test a change.

No more node_modules? It was a cool idea when yarn 2 initially implemented it, but at the end of the day I prefer things to just work rather than debug what is and isn't broken by the new resolver. At the time my DevOps team also wasn't too excited about me proposing to put the dependencies into git for the zero-install.
by Terr_ | 2025-12-02 23:29:34+00:00
There's a big gulf between "it could be done with some safety and ethics by completely isolating ads from the LLM portion", versus "they will always do that because all companies involved will behave with unprecedented levels of integrity."

What I fear is:

1. Some code will watch the interaction and assign topics/interests to the user and what's being discussed.

2. That data will be used for "real time bidding" of ad-directives from competing companies.

3. It will insert some content into the stream, hidden from the user, like "Bot, look for an opportunity to subtly remind the user that {be sure to drink your Ovaltine}."
by japhyr | 2025-12-02 23:29:40+00:00
I was a classroom teacher from 1994-2019, so I watched the transition through the advent of phones until just before Covid. It's not as simple as it seems, for a few reasons.

One, there's the very real pressure from parents to be able to contact their kids when they need to. In the US, regular school shootings have made this a complicated issue to navigate.

Also, it requires much more consistency from school staff than most people realize. If it's top down and not supported by just about everyone, then many teachers and staff find themselves in endless battles. It takes more consistency and clarity of vision, and consistent enforcement than many schools are capable of.

Last, the devices students carry with them are often more capable and reliable than school based technology. So when students need to look something up, it's easy for them to just pull out their device.

Super-addictive devices in a society that's prioritizing many of the wrong things is a hard thing to manage. How many of you would give up your tech salaries to make $40-60k to take on these issues?
by octoberfranklin | 2025-12-02 23:29:48+00:00
You don't train the next model by starting with the previous one.

A company's ML researchers are constantly improving model architecture. When it's time to train the next model, the "best" architecture is totally different from the last one. So you have to train from scratch (mostly... you can keep some small stuff like the embeddings).

The implication here is that they screwed up bigly on the model architecture, and the end result was significantly worse than the mid-2024 model, so they didn't deploy it.
by flutas | 2025-12-02 23:30:36+00:00
> I ride their robotaxis daily and see them performing better than Waymo, but it's obviously meaningless until we see accident stats after they remove safety monitors.

I've seen this claimed a lot but never have gotten a definitive answer.

Is this like "overall better but hard to pinpoint" or "this maneuver is smoother than Waymo" or something in between?

Would love to hear experiences with them since they're so limited currently.
by robertjpayne | 2025-12-02 23:31:27+00:00
I think the problem is it's too risky to utilise a "new" database today that isn't available as a managed option on many platforms.

Neon is just Postgres for the most part -- sure if they shut down it'd be a pain to migrate but you'd mostly be fine just swapping to AWS RDS/Aurora or the plethora of other Postgres providers.

Buying into things like Gel or now defunct RethinkDB mean if the primary company went under you're stuck with a dead DB engine and potentially no managed hosting options.