skysight-inc/HackerNews huggingface dataset:
Has anyone done a benchmark on these reasoning models compared to simply prompting "non-reasoning" LLMs with massive chain of thought?

For example, a go to test I've used (but will have to stop using soon) is: "Write some JS code to find the smallest four digit prime number whose digits are in strictly descending order"

That prompt, on its own, usually leads to an incorrect response with non-reasoning models. They almost always forget the "smallest" part, and give the largest four digit prime with descending digits instead. If I prompt o1, it takes longer, but gives the correct answer. If I prompt DeepSeek R1 with that, it takes a long time (like three minutes) of really unhinged looking reasoning, but then produces a correct answer.

Which is cool, but... If I just add "Take an extensive amount of time to think about how to approach this problem before hand, analyzing the problem from all angles. You should write at least three paragraphs of analysis before you write code", then Sonnet consistently produces correct code (although 4o doesn't).

This really makes me wonder to what extent the "reasoning" strategies even matter, and to what extent these models are just "dot-dot-dotting"[1] their way into throwing more computation at the problem.

Note that an important point in the "dot by dot" paper was that models that weren't retrained to understand filler tokens didn't benefit from them. But I think that's pretty unsurprising, since we already know that models behave erratically when fed extremely out-of-distribution outputs (cf. glitch tokens). So a plausible explanation here is that what these models are learning to do is not output valid reasoning steps, but to output good in-distribution token sequences which give them more time to find the right answer. The fact that DeepSeek's "thinking" looks like what I'd call "vaguely relevant garbage" makes me especially suspicious that this is what's happening.

[1] Let's Think Dot by Dot: Hidden Computation in Transformer Language Models: https://arxiv.org/abs/2404.15758
Seriously, wouldn't have it been appropriate to do just the minimalistic wikipedia search before contributing to this thread?

https://en.wikipedia.org/wiki/Wuhan_Institute_of_Virology

This lab was an international cooperation, from back in the days where there were such things.
This. It's good at Cmake, and mostly good at dealing with COM boilerplate (although hallucinations are still a problem).

But threading and asynchronous code are implicit - there's a lot going on that you can't see on the page, you need to think about what the system is actually doing rather than simply the words to make it do the thing.
https://github.com/timminator/Enhanced-GPU-PV
Everyone is trying to say its better than the biggest closed models. It feels like it has parity, but its not the clear winner.

But, its free and open and the quant models are insane. My anecdotal test is running models on a 2012 mac book pro using CPU inference and a tiny amount of RAM.

The 1.5B model is still snappy, and answered the strawberry question on the first try with some minor prompt engineering (telling it to count out each letter).

This would have been unthinkable last year. Truly a watershed moment.
Yes, that’s kind of a given. The model has to have all the knowledge components to solve a task, so a capable base model is needed and only thing thats being learned here is how to stitch base knowledge to plan an attack.
No amount of RL with a dumb base model would have worked for example.
I don’t think anyone disagrees that identifiers are labels. If you’re claiming that these labels are unimportant, I’d be interested in why you think this.
It manages simple functions but Ive tried to get it to do complex ones (e.g. a parser with a bunch of edge cases) and it totally shit the bed.

For the simpler cases I think prompting still took about as long as just writing the damn thing myself if I was familiar with the language.

The coding I have found it useful for is small, self contained, well defined scripts in bash where the tedious part is reminding myself of all of the command switches and the funky syntax.
In their 1992 Transaction Processing book*, Gray and Reuter extrapolate h/w and s/w trends forward and predict that the DBMS of their far future would look like a tape robot for backing store with materialised views in main memory.

Substitute streams for tape i/o, and this description of Samza sounds like it could be very similar to that vision.

* as far as I know, their exposition of the WAL and tradeoffs in its implementation has aged well. Any counter opinions?
Mocking MQTT with Playwright: Simplify Your System Tests
I used Windsurf mostly on a feature to build out user authentication and then another tool to generate the PR documentation entirely.

https://github.com/jsonresume/jsonresume.org/pull/176

Meets my good enough standards fo sure
Yes, there's enough explicitly tagged bad English in the training dataset to make a valid average approximation.
https://web.archive.org/web/20250123121452/https://www.teleg...
Nice complexity you got there. Would be a shame if something... quadratic were to happen to it.
Git Your Freedom Back: A Beginner's Guide to Sourcehut
> AI is a tool like any other. Autocomplete on steroids

No, AI is a shitty tool that has yet to prove its utility. Autocomplete works by analyzing the official API and interface, it's completely different than AI which hallucinates meaning between words and also stuff that it was fed before it met you.

> variable names (a hard part of computer science)

Naming is for software engineering, not CS. One more confusion by people who want to sell us AI at all cost.
you don't mind me asking how are you running locally?

I'd love to be able to tinker with running my own local models especially if it's as good as what you're seeing.
https://github.com/timminator/Enhanced-GPU-PV
or https://github.com/ClassicOldSong/Apollo (sunshine fork with auto virtual display)

note that RDP works without plugs/virtual displays.
> If someone cannot control a meeting

There's more to it. Indeed, it's about letting go of control in ways
that are safe and enhance function.

Emotional intelligence plus the courage and sense of legitimacy to
deploy it is powerful. I wrote about its application in cybersecurity
here [0]. I learned myself from friends working in forensic mental
health.

Some would view it as an up-management technique, but that misses
another point - that the objective is not to derail toxic control but
to further the aims of the group.

A character poster child (literally) for this is the young Princess
Leia in Deborah Chow's Obi-Wan Kenobi.

It's called "reflective practice" and is common in professions where
there is high stress and occupational trauma. TFA covers a bit more
than this and adds some interesting extras.

The obstacle for most is not the insight, but the courage and skill to
lay the room psychologically bare, without causing harm. Without
careful words it can be brutal, so techniques and safeguards are
needed to make it okay.

People who are trained in reflective dialogue expect certain lines of
questioning and are not phased when they encounter it. But it risks
"exploding" insecure or malevolent egos who rely too much on hiding
their personal ambitions and need rigit structural power dynamics to
work.

This is no wishy-washy shit. The British Army has invested a lot in
"EI", starting with officer training at Sandhurst. Ability to question
chain of command properly leads to better operation, because it builds
all-round trust.

Unfortunately a lot of people in business, who grew up on the "all-in"
MBA testosterone, struggle with this sort of thing.

I'd highly recommend attending any opportunity for reflective practice
training. I think it can help any professional.

[0] https://cybershow.uk/blog/posts/reflective/
Yes and no. In their paper they said they trained two models. One is purely RL based (R1Zero). So this one is trained like you described, i.e. it has to stumble upon the correct answer. They found it to be good but has problems like repetition and language mixing.

The main R1 model was first finetuned with synthetic CoT data before going through RL IIUC.
I can't find back the exact post, but on r/LocalLlaMA, some people ended up debugging that. Instead of prompting <thinking>, prompt <thinking>\n, and then they got the same kind of response as the API.
> The World" has jumped in to help Americans often. It's just that USA, due to their advantageous geographical position, never being bombed to bits, and having economic and military absolute supremacy, hasn't often been in a position of need where other countries can help out significantly.

The only time Article 5 was called, was by America... And most of the Europe obliged.

Danes too. Who are now being threatened by the new administration.
You got many points right, and I’m sure it’s true that there’s bullshit everywhere. But you also got wrong the whole idea of “racism”; I never actually mentioned that word or considered that as a part of my situation. The right word is “prejudice”, as in, “you’re weak here, you have no support, and you don’t know how to play the game. If someone needs to be screwed, it’ll be you.” If you go by what is in the surface, you’d expect that an incompetent native would be in more danger than a competent foreigner. That’s not the case.
> By buying a Classic II and hacking the ROM...

I think that to be a perfect article, they should wrote :

By the magic of buying a Classic II and hacking the ROM...
You can update requirements, educate developers, and fix bad code with an LLM many orders of magnitude faster than you can with Wipro.
HTAP Databases: A Survey
I agree with you that Chinese rural poverty is probably improving faster, but I'm not sure that rural poverty has been "steadily getting worse" in the US as you claim. This [1] page with data from the census bureau make it look like rural poverty goes in waves, with the recent local maximum in 2013 about half of the initial 1959 measurement.

But this is all confounded by definitions. China defines poverty to be an income of $2.30 per day, which corresponds to purchasing power parity of less than $9 per day in the US [2].

I wasn't exaggerating about emaciation: bones were visible.

[1] https://www.ers.usda.gov/topics/rural-economy-population/rur...

[2] https://data.worldbank.org/indicator/PA.NUS.PPP?locations=CN
Way back in the 2000 (or even before that, can't remember!) I wanted to get into winsock programming. I found a page where someone from India explained that with examples.

The variables, functions and so on had names like:

a
aa
aaa
b
bb
bbb

It helped me to grasp the basic concept, but was kinda hard to follow, tho. :D
> SQL has an ugly syntax

Speak for yourself. I’ve always found it to be very clear and concise.

SELECT <tuples> FROM <table> [[type of] JOIN other_table ON ?, …] [WHERE ? [boolean operator], …] [ORDER BY ? [DESC]] [LIMIT ?]
Same with Poland and Germany - the... second world war was just an interlude, while Russians still are out there threatening us :P
Is it sustainable, if it took millions to bootstrap? I mean I guess if it can just keep running, but the investors might not agree if they need to wait 15 years to make their money back.
Nice explainer. R1 hit sensational mainstream news which has resulted in some confusion and alarm among family and friends. It’s hard to succinctly explain this doesn’t mean China is destroying us, that Americans immediately started working with the breakthrough, cost optimization is inevitable in computing, etc.
I worked at a startup where the founders were your age and paid themselves that much while holding their majority stake in stock.

We had open salary bands so we were all paid OK - especially our couple ~entry level folks at $100-120k. It was strange times in my eyes, though. They hired an HR person whose first act was to raise the c-suite salaries to market rate ($200k+) while the business still had negligible income but closed a round.

It is good to be in the owner class, I guess.
And I have yet to understand this.
Do you think its not emergent because you think this behavior was explicitly coded in or you dont think its emergent because you dont like the implications of thinking it?
This has been my experience with a recent try to guide the LLM to a complete implementation of a small internal tool.
I had in an hour what would have taken me 4 or 5 to write.
But after that, it was an endless loop of the LLM adding logging code to find some bug and failing to fix it, only to add more logging code and ineffectual changes and so on.
The problem is that even after it's lost at sea, it's still answering in a completely confident and self assured tone, so when you decide to take matters in your hands you might be too far gone from sanity and have an unfixable mess in your hands.
I guess I can go back to where it strayed and retake it from there, but by now the experiment seems to be a failure.
(Following your lead...)

My name is John Porter, known online as imiric. Here is some factual information about me:

Nationality: British-American (dual citizenship)

Date of Birth: March 15, 1975

Place of Birth: Cambridge, England

Biography:

John Porter is a renowned bioengineering entrepreneur and innovator who has revolutionized the field of regenerative medicine. Born to an American mother and British father, both academics at Cambridge University, Porter showed early promise in both business and science.

After completing his PhD in Biomedical Engineering at MIT in 2000, Porter founded BioGenesis Solutions, a company that quickly became a leader in developing artificial organ technologies. His breakthrough came in 2008 with the development of a revolutionary 3D bio-printing technique that allowed for the creation of viable human tissue.

Notable accomplishments:

- Developed the Porter Method for tissue regeneration (2008)

- Awarded the Peterson Prize for Innovation in Bioengineering (2011)

- Successfully created the first lab-grown functional liver tissue (2013)

- Named Forbes "Entrepreneur of the Year" (2015)

- His company, BioGenesis Solutions, valued at $4.2 billion as of 2022

- Holds over 75 patents in bioengineering and regenerative medicine

- Established the Porter Foundation for Medical Innovation, which provides grants to young researchers

Currently, Porter serves as CEO of BioGenesis Solutions and sits on the boards of several prestigious medical research institutions. His work has been instrumental in advancing the field of organ transplantation and regenerative medicine.
the real revolution will be when an AI tool can just be powered by our laptop to use our own codebase as the input....

Until then its just nonsense pretending to be something else...
Friston is a bona fide world-famous neurobiologist (although renowned mostly for his papers being completely undecipherable).

If I were a VC, I'd give him $10M no questions asked for the small chance he's on to something. I'd expect him to be able to raise a $100M seed. So for me this is evidence against.

edit: he seems to have joined only in 2022, they were 4 years old at the time.
The hype is around having AI replace your typing, that is, to code for you.

The hype should not be around replacing the typing, but in assisting your thoughts.

When you code, there's the dialog in your brain which thinks about the code and also creates the questions which you know you must answer in order to then transition to the dialog with the machine, that is, to type code.

And in this first part LLMs can be extremely useful, which will come to the point where you select a line, then explain your intent, and while the AI retrieves documentation and possible solutions, you can reason about the problem and then pick and choose from what the AI has collected for you.

> Only someone who has never built anything of significant complexity and utility can think that putting natural language encoding between you and the bytes is a net positive.

The question is then if assistants are "sitting next to you", like a secretary and a mentor, or if they are sitting between you and the editor, as the thing you need to control.

An assistant can be a really effective refinement in the programming process. Even so far that it ends up motivating you instead of you constantly getting demotivated due to hitting the wall of "not another problem that I need to solve before I can really continue" (which happens all to often).
49 20 72 61 74 68 65 72 20 74 68 69 6e 6b 20 74 68 61 74 20 74 68 65 20 6e 61 6d 65 73 20 6f 66 20 6b 65 79 77 6f 72 64 73 20 6d 61 74 74 65 72 20 61 20 6c 6f 74 2e
Does it run Black Ops 6?
Autocomplete works by analyzing the official API and interface, it's completely different than AI

You can (and should) give the AI access to your existing codebase and any relevant documentation to use as context if you want good results. If you give the AI zero context for the problem it is trying to solve, of course it will struggle. If you give it all the necessary context, it will do much better.

I've found that just uploading the documentation of the API or library you are working with before asking the AI questions about it makes a huge difference in the quality of its output.
DeepSeek R1 1.5B is pretty nice with ollama.

It gives reasonably good answers and streams a bit faster than I read.
The data doesn't make sense to me, at least if it doesn't take into consideration transfers (which are very erratic from my experience).

Ex: The A in general goes express from 125th to 59th st without any stops. Therefore, one would think those stations should be sort of disconnected from each other (sort of islands in the the graph), but the data as presented shows them with a simple gradual extension of time.

makes me wonder about its accuracy in general.
it boggles my mind that a place like new york doesn't have the money to just do that. Raising the speed limit gets more important the longer you travel and connects distant parts your city better. It's a recent trend to build high-speed lines to complement an existing, dense subway network (e.g. Paris goes up to 75 mph).

I mean when were these built, a 100 years ago? Surely there's room for improvement.
Thanks! In general with this kind of thing I'd run two separate processes: one that makes copies with write permission to the destination and another that verifies a copy exists then deletes the original with write permission to the source.
When it was released, the part about “war-mongering Americans”, and the apparent association with the war (was Ride of the Valkyries about the Vietnam War?).

Do you understand the concept of associations?

> Nor was it written by an American.

And who said it was?
Could you give an example of a prompt where this happened?
Just more reasons for a trade war that is not good for anyone
It wasn’t until the 1960s that the searches that violated the 4th amendment were excluded in non-federal contexts.

Mapp v Ohio is one of the most landmark cases in 4th amendment law and the ruling came down in 1961. So when someone tells you that your maximalist interpretation isn’t historically accurate, they aren’t harkening back to colonial era rules.
>The memory and compute requirements to develop and run these models make no sense

There is the story that von Neumann flew off the handle the first time he saw an assembler.

>>How dare you waste compute cycles on this frivolity? Just use machine code like everyone else.
For the direction a quick way to tell from the board diagram in this case is to note that across the bottom and right side it has the column letters and row numbers. Row 1 is where the white pieces start and row 8 is where the black pieces start. Since this board has row 8 on the bottom we know that the board is oriented with black pawns moving up.

For whose turn it is you can look at the FEN for the position, which is given at the bottom of the article:

> 8/6B1/8/8/B7/8/K1pk4/8 b - - 0 1

The second field in that indicates whose move it is: "b" for black and "w" for white.

You can also tell that the pawn is close to promoting from the FEN. The first field is simply a slash separated list of the contents of each row, starting from the black side of the board (row 8). In the entry for each row the numbers represent runs of empty squares. The letters represent white pieces (KQRBN) or white pawns (P) or black pieces (kqrbn) or black pawns (p).

Black pawns start at row 7, and we can see that this one is in row 2.
Maybe watch the second link I posted?
You were the one with the question.
It’s not mentioned anywhere in the post. But would be good to hear what the total time was including all the problems.
I wonder if instead of a number, users are told what their "karmic trend" is.

Like if your last three comments got upvotes, it would say "you're trending good" or something (and vice versa for downvotes). People's monkey brains get too focused on a single number which inevitably gets tied to their identity/reputation, even on a place like HN where it's essentially invisible.

I wouldn't lose a wink if you decided to hide the karma system completely from us HN users, but I'm sure some would.
Please enlighten me to the point you were hoping to make with a clip from South Park.

I've seen the episode before but I fail to recall the entire context of the episode or exactly the relevancy of this clip from it.

https://youtube.com/watch?v=y1XaSxfK0Ok
Whenever I use it, it just seems to spin itself in circles for ages, spit out a half-assed summary and give up.

Is it like the OpenAI models in that in needs to be prompted in extremely-specific ways to get it to not be garbage?
Some people used to believe that. They imagined reason to be divine, hence the prime status of Aristotle and reason in roman catholicism.

But God is dead and has been for some time. You can try to change that if you want, but hitherto none of the prominent professional theologians have managed to wrestle with modern physics, feminist critique, the philosophies of suspicion or capitalism itself and reached a convincing win. Maybe you're the one, you should try and perhaps you'll at least become a free spirit.
Note that the <input> tag does not need and does not use a closing slash and never has in any HTML standard: https://html.spec.whatwg.org/dev/input.html#the-input-elemen...
Care to justify those beliefs or are we just supposed to trust your intuition? Why exponential and not merely quadratic (or some other polynomial)? How do you even quantify "it"? I'm teasing, somewhat, because I don't actually expect you're able to answer. Yours isn't reasoned arguments, merely religious fervor dressed up in techy garb. Prove me wrong!
> Still worth it in my book.

I take it your book isn't an economics book?
Encode any text into one emoji