skysight-inc/HackerNews huggingface dataset:
This idea was championed by the Stochastic Parrots paper. They assumed LLMs are just pattern learning, which doesn't make sense.

- the simple-to-hard RL method recently discovered is one argument against it, the models can reason their way out on harder and harder problems

- zero shot translation shows the models really develop an interlingua, a semantic representation, otherwise they wouldn't be able to translate between unseen pairs of languages

- the human in the room is also important. LLMs are like pianos, we play prompts on the keyboard to them, and they play back language to us. The quality and originality of the output aren't solely inherent to the model, but are co-created in the dialogue with the prompter. It's not just about the instrument, but also the 'musician' playing it.
A lab leaking wouldn’t do any harm to the CCP. The harm they caused themselves was a mixture of lying and trying to cover up the whole thing. Downplaying it. Etc. The WHO says any unknown phenomena must be treated as airborn until proven otherwise. But they followed the CCP in telling us it was fine, it wasn’t transmissible.
You can still regurgitate a chain of thought response…

You people are so close to getting it. So close to understanding that you're the ones doing the regurgitating.
My test of an 040 (no A1 change, D1 changed) was on a chip with the following markings:

XC68LC040RC25B

02E23G
QEDP9348D
MALAYSIA
Automate MongoDB Atlas Backup
I have written a blog that explains in detail how to automate mongodb backup. You can do it for free and nothing special is required.
more random support for your critique of a post-hoc, and historically ignorant, classification of "small cities". Offhand the CA capital contest was ongoing one and tracked from south/central CA towards the north in the mid 1800s. It's entirely unsurprising that they landed in the Sacramento courthouse; it was ~1850 and Sacramento was 1) last inland port for shipping 2) major terminus for the gold fields 3) an obvious rail hub for the connectivity in the 1850-60s, esp bay area to Sacramento and the east. Even a trivial look at the (flawed) 1850 census shows that basically all of the (mostly new) immigrant population was in Sacramento and the goldfields. eg Los Angeles was 3,500 people compared to 9,000 in Sacramento.

Similar for the Washington Territory which was barely a collection of homesteads and tiny villages. IIRC Olympia was the first significant immigrant settlement in the northern Oregon Territory, the major (only?) early trade connection to shipping on the pacific, and connected to the few other settlers moving north in to the territory. Its not very surprising that it was the territorial capitol and remained so for the state. Yes, post statehood a lot of rail & shipping traffic moved north. But at least through the turn of the 20th century both Tacoma and Seattle were roughly equivalent in population and industry. Arguably the easier ship access for the Klondike is what cemented the prosperity of Seattle pre-war, and Boeing & related defence activity post war. But _at the time of territory & statehood_ Olympia was the major town in the region.
A year ago I was a believer, and I am more of a believer now. This isnt stopping, I think we have a generalized method to learn from data. It's just a matter of getting the data into it.
https://archive.ph/UJJYp
There was a very small presence of Americans in South Vietnam at the time - around 5,000. Mostly US government workers (e.g. USAID), CIA and some military.

When the US disengaged in 1973, it left most of the military equipment and gave it to South Vietnam.

So most of the helicopters and such that fled to the US fleet were South Vietnamese aircraft, not US aircraft.

So I assume the US has pretty much written them off before that.
They in fact were working on coronaviruses in a BSL-2 facility, which is another thing that helps make the lab leak hypothesis more plausible. E.g. from Vanity Fair:

https://www.vanityfair.com/news/story/ralph-baric-wuhan-lab-...

Baric testified that he had specifically warned Shi Zhengli that the WIV’s critical coronavirus research was being conducted in labs with insufficient biosafety protections. When he urged her to move the work to a more secure biosafety level 3 (BSL-3) lab, he testified that she did not heed his recommendation. Because the WIV continued to perform coronavirus research at what he considers an inappropriately low biosafety level, Baric said of a laboratory accident, “You can’t rule that out…. You just can’t.”
I suppose the missing part of the story is why they held back on pursuing this market.

At the time, the console market was wide open, with little innovation in terms of hardware, until Nintendo released the Switch.

Even now, I'd be quite happy to own a Valve branded, small form PC that plugs into a TV.

The Steam Link was a kop out to me.
Wow what a work, this is truly joy to go back in years. Great job.
The lab leak theory was started by Chinese netizens. It was mentioned on Chinese media. Hell the name Wu Flu came from Chinese media.
The question depends what you mean by first principles. Usage of the phrase "first principles" has sprawled into many different things since (I think) Musk first mentioned it as a way to learn. The original, philosophical meaning of first principles meant a fundamental truth which could be used to derive others. Much of the philosophising of thinkers like Aristotle or Descartes was to uncover these truths (eg I think, therefore I am). In physics and other sciences, it means calculations using established laws, rather than approximations or assumptions. Then it got borrowed into certain circles of the tech crowd with the vague meaning of thinking about what's important or true and ignoring the rest. Then it trickled down into the learning/self help world as a hack of some sort to learn. If we take the original meaning of first principles, there aren't a great deal of absolute truths in machine learning. It is a very empirical, approximated and engineering oriented endeavor. Most of the research involves thinking of a new approach, building it and trying it on new datasets.

The other big question is why you want to learn it. If you want to learn ML in itself, than anything including the search algorithms (which used to be considered core to ML a long time ago) you mentioned is part of that. But if you want to learn ML to contribute to modern developments like LLMs, then search algorithms are virtually useless. If you aren't going to be engineering any ML or ML products, what you want is to gain some insight into it's future and the business of it. So learning things like transformer architecture is going to be far more unhelpful than say, reading about the economics of compute clusters.

Given the empirical/engineering quality of current ML, I'd say building it from scratch is really good for getting the handful of possible first principles (the fundamental functions involved, data cleaning, training, etc)
That sword cuts both ways.
Why not just buy a minisforum and install popos and steam?
Each language has unique tools. If you have a word for something or even better a whole set of words the conversation works a lot better than in a language that has nothing of the kind. English elaborately talks about all kinds of Communities. Dutch does have a word like it but it is almost never used. Or, how should we talk about the kind of snow if we have only one word? https://watchingtheswedes.com/2018/02/28/50-words-for-snow/
Investigation of multiple hypothesis simultaneously is the norm, not an exception.
I've been looking for a new space to lurk on for a bit now (hard to replace HN). Hadn't heard of Lobsters might check it out.

Appreciate the post.
Money can be destroyed with inflation.
How do guessers arrive at a consensus about what card to touch, if they are forbidden from communicating at all?
Can't trust a word out of the New York Times.
As far as I know you're not missing anything and this is why gain-of-function research was banned in the US for a while. EcoHealth Alliance outsourced it to China in the mid-2010s because of the ban, so technically none of it was happening in the US.
Yep, another continent from the home is the best place for such kind of research.
They can do split-screen 3d, albeit at a lower refresh rate. When you put them into that mode they appear to the computer as one screen of double width.
Know your history.

The Vietnam War had significant opposition from the start. It only grew from '65 to '73.

Plenty of Americans back then wanted out, immediately.
If you followed events at the time and the suppressed rumours from doctors in China end of 2019, the new illness began exactly around that area actually (https://en.wikipedia.org/wiki/Li_Wenliang etc).

There were no similar reports in another place on this planet. (Since 99% of other places do not have full control of media and many have better healthcare so if it happened it would be less likely to go unnoticed)
Very well said, sometimes I feel the modern stack could re-learn and improve.
I posted about genocide and nobody gave a fuck. Your website was never there, sorry. It's just old school reddit
I never looked into prql; does the ordering matter? As if not , that would be great; aka, is this the same:

from users
take 50
filter id == 42 # conditionally added only if needed
filter username == param # again, only if the param is present
derive [full_name = name || ' ' || surname]

? As that's more how I tend to think and write code, but in sql, I always jump around in the query as I don't work in the order sql works.

I usually use knex or EF or such where ordering doesn't matter; it's a joy however, I prefer writing queries directly as it's easier.
Honestly, I'm welcoming this move to Notion. It's much less cluttered than Medium.
World's Dumbest Game
Mosaic GPU and Pallas: a JAX kernel language [video]
Not as much as the Chinese, apparently.
It's true that building an MVP is a lot easier than creating a polished product that would scale to tons of users. I would even say that the skill set needed for the two are significantly different. Somebody trained at quick prototyping of full stack app may not have the skills to scale it for thousands of concurrent users and vice versa. But isn't that the reason behind raising funds? If the product demand is proven, funds can be raised to polish and scale the product out.
Where do you see 5%?

At the top I see: 271,489 MAU and 532,524 total users (so 51%)

At the bottom (graphs) I see: 27,412 active and 316,151 user count (so 8.6%)

I don't know how to understand those numbers, wondering if you have a better source.
There’s a lot of GoF research on a lot of different diseases with a wide range of goals.

One goal for disease likely to cause pandemics is ultimately to create better treatments for those already infected. There’s a long lag between a vaccine being designed and scaling production and distribution to actually protect people. That means there’s going to be a lot of people infected in an outbreak, including many vaccinated people.
why do you lie, it is blatantly obvious chatgpt censors a ton of things and has a bit of left-tilt too while trying hard to stay neutral.

If you think these tech companies are censoring all of this “just because” and instead of being completely torched by the media, and government who’ll use it as an excuse to take control of AI, then you’re sadly lying to yourself.

Think about it for a moment, why did Trump (and im not a trump supporter) re-appeal Biden’s AI Executive Order 2023 ? , what was in it ? , it is literally a propaganda enforcement article, written in sweet sounding, well meaning words.

It’s ok, no country is angel, even the american founding fathers would except americans to be critical of its government during moments, there’s no need for thinking that America = Good and China = Bad. We do have a ton of censorship in the “free world” too and it is government enforced, or else you wouldnt have seen so many platforms turn the tables on moderation, the moment trump got elected, the blessing for censorship directly comes from government.
Very closely follows the style of Total Annihilation, a 4X game I spent a fantastic amount of time in.

Perhaps still the only game whose loading screen dedicates time to loading explosions!
Well, assuming you’re part of the “we” that resides in the US, I think we’ve made it pretty clear we’re aren’t taking any of it seriously. Pulling out of the WHO is akin to burying our heads in the sand.

Sure, technically it “isn’t our problem” when some new disease breaks out in another country. But when (not if) it is eventually our problem, it’ll be a very big problem.
Why do you believe that is how humans reason?
You mean deep seek r1 generated this?

With what prompt?
The correct result of 119 million * 10MB is 1.19 PB.

Also the images on the first page of the author's own account average only 157kB, bringing your estimate to 18.6 TB...

pxscdn.com resolves to Digital Ocean, if he's using their S3-compatible storage, that's $372/mo + $0.01 per GB downloaded
What's with AI boosters and not viewing other people as human?
Right. "Low confidence". Read [1] People who deal with intel data need to work with possibly wrong info. Most spy novels don't get this. Real military planning works more like: "Intel says the enemy is at A or B, most likely at A. How much resources should we devote to A vs B, or hold in reserve until we find out by encountering the enemy? What's our plan if we guess wrong?" Planning has to assume that intel might be wrong.

Go watch "A Bridge Too Far" (WWII). Read the story of the Son Tay raid (Vietnam). The many overestimates of Soviet capability during the Cold War. The underestimates of North Korean missile capability. Sometimes uncertain intel really works, as with the attack on the bin Laden compound in Pakistan.

Retrospective intel questions may never be answered. It's known that the US atomic bomb program had another Russian spy who is mentioned by code name in VENONA transcripts, but that spy was never identified. There are still arguments over whether the explosion of the Maine in Havana harbor in 1889 was an accident or a hostile act. It's still not clear why Turret 2 of the USS Iowa blew up in 1989. Huge amounts of effort were expended on all three of those questions, all of which were important at the time, and yet they remain unsettled.

[1] https://www.cia.gov/resources/csi/static/Article-Principles-...
Why would you go through the effort of creating a sockpuppet just to hurt someone being uncommonly genuine and sharing insights from their lived experience? It’s pathetic. (Not to mention incredibly insecure. Phew!)

If you don’t change, don’t be surprised to find yourself utterly alone in a few decades’ time.
There's a couple of probability distributions we don't know. And whether this leaked in Wuhan or not doesn't affect them.

1. What's the probability distribution and damage distribution of GoF research lab leaks? It's not zero-- it likely has enormous long tail risk. But:

2. What's the probability distribution and damage distribution of not knowing as much about how gain of function happens in the wild? Because nature is doing some of these GOF experiments on its own, without much effort at containment.
An interesting offshoot of this type of question is: how long does it take to hit every stop on a given transit system? BART has quite a few people making attempts; the official BART website even maintains a little article [1]! A couple have made videos of attempts and theory [2, 3], but the king of these things is tomo tawa linja, who I believe is the current record holder at 5:30:26 [4].

[1] https://www.bart.gov/news/fun/speedrun

[2] https://www.youtube.com/watch?v=6o9JGsamQF0

[3] https://www.youtube.com/watch?v=XFWp_LH3X5k

[4] https://www.youtube.com/watch?v=rmQTnSlL4C4
However, look at the figure for R1-zero. The x-axis is effectively the number of RL steps, measured in the thousands. Each of them involves a whole group of inferences, but compare that to the gradient updates required for consuming 15 trillion tokens during pretraining, and it is still a bargain. Direct RL on the smaller models was not effective as quickly as with DeepSeek v3, so although in principle it might work at some level of compute, it was much cheaper to do SFT of these small models using reasoning traces of the big model. The distillation SFT on 800k example traces probably took much less than 0.1% of the pretraining compute of these smaller models, so this is the compute budget they compare RL against in the snippet that you quote.
Why not try to actually sell the property? It sounds like it's property that's much less valuable than it looks on paper, and these spammers are only looking at the paper value.

If they buy it and use it, that's better than the land sitting unused in the middle of the city as it is now.
Was the idea sold the same way? You obviously need to get buy in across the org, which in itself can be a daunting task and requires you to create various pitch decks, project plans etc.
The challenge I’ve seen, is women’s groups tend to not have/allow boundaries. At least in a ‘you can’t say no’ type of way. Lying/hiding stuff is of course pervasive, as a defense. It’s a really common pattern. ‘Mean girls’, ‘gossip group’, etc.

In many, it’s typical to discuss everything from the sex habits of them and their partners (in excruciating detail), their own and others affairs, to every embarrassing detail of their kids lives. It often seems to be a competition to see who can get the most exciting ‘tea’ out of each other.

In my experience, having indirectly seen/overheard many of these discussions, most men would be horrified if they knew what was really going on.

It happens in some men’s groups, but is much, much rarer.

As for my story - I’ve seen quite a few.

Office politics where a senior woman leader was essentially running a ‘sex for leverage’ campaign against all the men (and a couple women) in the group, using the women in the group as ‘bait’.

A church where the pastor got convicted of child molestation, but where the community insisted he be forgiven (after getting out), and he was indeed reinstated - while another part of the congregation had their entire family driven from the group (and harassed socially in the community for years) because the father divorced his spouse because of infidelity and physically abusive behaviors.

Oh, and the classic ‘ex wives club’ stalking and harassing an ex, and any new wife - and manipulating her into ruining herself and joining the club.

I’ve seen all these play out first hand, and they are just a drop in the bucket. I’ve ceased to be amazed at the cruelty often demonstrated.

Abusive men tend to work a bit differently, so their setups often look more directly hierarchical and have less information sharing going on. They tend to operate more off secrecy and/or threats of explicit violence, than manipulation.

But I’ve seen a few (rare) instances of similar setups. People can be awesome. People can be terrible.
If you want to research a cure from engineered virus from one hostile country, paying another hostile country to do virus engineering research for you is surely great logic.
For those who haven't realized it yet, Deepseek-R1 is better than claude 3.5 and better than OpenAI o1-pro, better than Gemini.

It is simply smarter -- a lot less stupid, more careful, more astute, more aware, more meta-aware, etc.

We know that Anthropic and OpenAI and Meta are panicking. They should be. The bar is a lot higher now.

The justification for keeping the sauce secret just seems a lot more absurd. None of the top secret sauce that those companies have been hyping up is worth anything now that there is a superior open source model. Let that sink in.

This is real competition. If we can't have it in EVs at least we can have it in AI models!
I think you need to discount possible farther future benefits, because so much change can intervene and make the analysis invalid.

That is, when people want to do something-- risks tend to be understated and possible future benefits tend to be overstated.

I don't back the precautionary principle, but I do think risk in cost-benefit analysis has to be viewed from a pretty cautious place, in general (not just science).
Just pointing out a factual error: "He who rides tigers cannot dismount" is not said by Mencius, but comes from Fang Xuanling of the Tang Dynasty.

However, it is still highly literate (both in English and Chinese), which I believe is one of its advantages
Reasoning is a human, social and embodied activity. TFA is about machines that output text reminiscent of the results of reasoning, but it's obviously fake since the machine is neither human, social or embodied.

It's an attempt at fixing perceived problems with the query planner in an irreversibly compressed database.
The way you taught chain-of-thought before was with supervised fine tuning (SFT). During training, you have to rate every sentence of reasoning the model writes, many times, to nudge it to reason correctly.

But this approach to teach chain-of-thought doesn’t do that. In this post, they take a small model (7B) that already knows math. Then they give it a relatively small number of problems to solve (8k). They use a simple reinforcement learning loop where the only goal is to get the problem right. They don’t care how the model got the right answer, just that it’s correct.

This is part of the “recipe” that DeepSeek used to create R1.

After many iterations, just like DeepSeek, they found that the model has an “aha” moment. It starts emitting chains-of-thought where it wasn’t before. And then it starts getting the math answers right.

This is the gist of it. I don’t fully understand the recipe involved.

Can you teach small models to “think” just using RL? How small can they be? What tasks does this work for? Is just RL best for this? RL+SFT? Everyone’s trying to figure it out.
Put the no procrastination setting on to short value with a long interval and take a break for a few months. I do the same from time to time and it stops the muscle memory of hitting compile in my IDE and then opening a tab with hn.
https://www.science.org/do/10.1126/science.z7pm10i/full/acti...

This is the order. Good luck finding "literally stop everything" in it.
Endless but Pointless Litigation Guaranteed
M4 or M4 Pro?
Yeah you’re actually right. I should have said “Americans”, meaning the individuals. You know history is truly shaped by the victors as no one should forget the unthinkable horrors the country inflicted on civilians yet here we are.
>Politicians hunting for excuses to do what they already want to do though, is definitely a thing.

Just to be clear: The people in charge are the CIA, not GWB. GWB was simply the right useful idiot at the right time in the right place.
Only played with DeepSeek-R1-Distill-Qwen-14B, but the knowledge is definitely still there.

https://pastebin.com/H2UTdi78

Seems more than happy to talk about Tienanmen, Xi, etc. starting at line 170 with the very primitive method of wrapping the query in its own "<think>...</think>" syntax even though it's the user role. Uyghurs are more strictly forbidden as a topic, as are its actual system prompts. None of this is serious jailbreaking, it was just interesting to see where and when it drew lines and that it switched to simplified Chinese at the end of the last scenario.
This is just an unfair clause set up to solve the employment problem of people within the system, to play a supervisory role and prevent companies from doing evil. In reality, it has little effect, and they still have to abide by the law.
That’s not what reason is. There’s nothing inherently human about reason.
I said your statement was weak I didn't say YOU were weak. Don't take it personally. It wasn't meant to be that way. If you take offense then I apologize.

That being said, my argument was a statement about a general fact that is very true. The sentiment not too long ago was these things are just regurgitators with no similarity to human intelligence.

I think it's clear now that all the previous claims were just completely baseless.