skysight-inc/HackerNews huggingface dataset:
> And as anyone who’s spent time using Claude 3.5 Sonnet / GPT-4o can attest, these things really are useful and smart!

I have spent significant time with GPT-4o, and I disagree. LLMs are as useful as a random forum dweller who recognises your question as something they read somewhere at some point but are too lazy to check so they just say the first thing which comes to mind.

Here’s a recent example I shared before: I asked GPT-4o which Monty Python members have been knighted (not a trick question, I wanted to know). It answered Michael Palin and Terry Gilliam, and that they had been knighted for X, Y, and Z (I don’t recall the exact reasons). Then I verified the answer on the BBC, Wikipedia, and a few others, and determined only Michael Palin has been knighted, and those weren’t even the reasons.

Just for kicks, I then said I didn’t think Michael Palin had been knighted. It promptly apologised, told me I was right, and that only Terry Gilliam had been knighted. Worse than useless.

Coding-wise, it’s been hit or miss with way more misses. It can be half-right if you ask it uninteresting boilerplate crap everyone has done hundreds of times, but for anything even remotely interesting it falls flatter than a pancake under a steam roller.
Attention is about similarity/statistical correlation which is fundamentally stochastic , while reasoning needs to be truthful and exact to be successful.
It is open in the sense that everyone can use it.
Actually the article tells be an air conditioner/heater. Because being a thermostat means just leave the awkward meeting.
> my gut reaction is this negatively impacts model usability, but i'm having a hard time putting my finger on why.

This will make it harder for things like DSPy to work, which rely using "good" CoT examples as few-shot examples.
Avoid This Bug with Numeric Environment Variables in Ruby
Had this wild old power transformer, forget what it came out of. 25lb thing, big steel can full of oil and copper, spark-plug looking post terminals all over the top. As I recall it has a 1960's date on it somewhere.

The thing had, for some reason, 240/120 input taps, and output taps for every 6 volt interval up to 1:1.

I soldered springs onto the terminals, like the Radio shack "200 in one" learning kits used to have. That transformer was a part of innumerable bodge machines on my desk for 20 years or so.

I think I finally melted it using it backwards as a step up in a spark gap kinda thing. The bolts melted and the gap was no longer a gap, leading to rich chunky amps being dispensed to the limits of that tool's ability.
So, again, this describes a scenario that can be addressed with tooling and management/communication.

The question I asked was why reviewers should flag code that correctly iterates over a list but does so in a different way than other code in the codebase. Are we straying into “foolish consistency” if we flag this, or is there a tangible benefit?
> Average:18/2=9

> 9 corresponds to 'i'(9='i')

> But 'i' is 9, so that seems off by 1.

Still seems bad at counting, as ever.
It's the same, for sure the proximity of that little scandal to this announcement is no coincidence.
Same, the first thing I looked for is what tip it uses.

I want to like the miniware, pine, etc irons, but I'd really like being able to buy T15 tips from my local electronics supplier, who carries Hakko.

If the product isn't sucessful and/or ifixit stops producing tips for whatever reason, a perfectly good iron is effectively bricked.
No matter the prompt, there's a significant difference between how it handles common problems in popular languages (python, JS) versus esoteric algorithms in niche languages or tools.

I had a funny one a while back (granted this was probably ChatGPT 3.5) where I was trying to figure out what payload would get AWS CloudFormation to fix an authentication problem between 2 services and ChatGPT confidently proposed adding some OAuth querystring parameters to the AWS API endpoint.
This power law behavior of test-time improvement seems to be pretty ubiquitous now. In more agents is all you need [1], they start to see this as a function of ensemble size. It also shows up in: Large Language Monkeys: Scaling Inference Compute with Repeated Sampling [2]

I sorta wish everyone would plot their y-axis with logit y-axis, rather than 0->100 accuracy (including the openai post), to help show the power-law behavior. This is especially important when talking about incremental gains in the ~90->95, 95->99%. When the values (like the open ai post) are between 20->80, logit and linear look pretty similar, so you can "see" the inference power-law

[1] https://arxiv.org/abs/2402.05120
[2] https://arxiv.org/abs/2407.21787
I have an idea! We should train an LLM with reasoning capabilities to sift through all the attempts! /s
The option isn't there for us (though the blogpost says otherwise), even after CTRL-SHIFT-R, hence the parent comment.
This is only true for some situations. In some test conditions it has not been passed. I can't remember the exact name, but there used to be a competition where PhD level participants blindly chat for several minutes with each other and are incentivized to discover who is a bot and who is a human. I can't remember if they still run it, but that bar has never been passed from what I recall.
Maybe I missed it, but do the tokens used for internal chain of thought count against the output tokens of the response (priced at spicy level of $60.00 / 1M output tokens)?
lol at the graphs at the top. Logarithmic scaling for test/compute time should make everyone who thinks AGI is possible with this architecture take pause.
Sure, web apps 10 years ago looked and worked largely the same as current ones, but they were usually monolithic and custom mountains of JS which were difficult to write, deploy and maintain. Few organizations could deploy something like Gmail.

So a good chunk of this emerging complexity tries to improve web developer productivity (at the cost of stack complexity and fragility). It's the infamous indirection treadmill, laser focused on a single thing: overall productivity and fast delivery of the final product by low skilled employees.
Hmm, the power supply has a knob, that’s ok. Why does it mention a web console? Video didn’t mention.
Much lighter impact on system performance that world-GC. Simpler algorithms as well, so lower risk that you’ll ever get performance regressions - or worse.
Sure, but Intel never got to the point of packaging it as memory (DIMMs) since the performance wasn't there.
I’m not surprised there’s no comparison to GPT-4. Was 4o a rewrite on lower specced hardware and a more quantized model, where the goal was to reduce costs while trying to maintain functionality? Do we know if that is so? That’s my guess. If so is O1 an upgrade in reasoning complexity that also runs on cheaper hardware?
lol come on it’s not the exact same thing. At best this is like gagging yourself while you talk about it then engaging yourself when you say the answer. And that presupposing LLMs are thinking in, your words, exactly the same way as humans.

At best it maybe vaguely resembles thinking
> Am I wrong?

That's the thing, did the operator select the correct result or did the model check it's own attempts? No info given whatsoever in the article.
My phone only has a 16 Wh battery; I'm not sure I'd want it to dump 100 W out its USB-C port even if it could!
At least they're open about not being open. Very meta OpenAI.
I'm a UNIX zealot and UNIX-like would be the most compatible with existing software for backwards compatibility. I would view the UNIX-like aspect as more of a subsystem or personality as the architecture itself is outdated, IMHO.
I think the problem is that basically none of these technologies are optimized for a solo developer. I remember deploying cool websites on simple LAMP servers in college a few months after learning to code. I had implemented so many things with just jQuery, sqlite, and php (sign on, media player, messaging) Then I was told I needed to learn a javascript framework and react, I started doing tutorials but never deployed anything. What a waste! I was having fun and deploying real apps.
Even compatibility with TS100/Pinecil V2 tips would be better. TS100 is meant to be open source, and the Pinecil V2 tips are just shorter with a different resistance.
An alternative for python is to flip what you're iterating over at the outermost level. It's certainly not as clean as F# but neither is it as bad as the original example if there's a lot of functions:

iter = [1,2,3,4]
f1 = lambda x: x*2
f2 = lambda x: x+4
f3 = lambda x: x*1.25

result = iter
for f in [f1, f2, f3]:
result = [f(v) for v in result]

Then the list comprehension can be moved up to mimic more closely what you're doing with F#, allowing for operations other than "map":

result = iter
for f in [
lambda a: [f1(v) for v in a],
lambda a: [f2(v) for v in a],
lambda a: [f3(v) for v in a],
]:
result = f(result)

And a step further if you don't like the "result" reassignment:

from functools import reduce
result = reduce(lambda step, f: f(step), [
lambda a: [f1(v) for v in a],
lambda a: [f2(v) for v in a],
lambda a: [f3(v) for v in a],
], iter)
The point above is the said amazing multimodal version of ChatGPT was announced in May and are still not the actual offered way to interact with the service in September (despite the model choice being called 4 omni it's still not actually using multimodal IO). It could be a giant leap in practical improvements but it doesn't matter if you can't actually use what is announced.

This one, oddly, seems to actually be launching before that one despite just being announced though.
In the performance tests they said they used "consensus among 64 samples" and "re-ranking 1000 samples with a learned scoring function" for the best results.

If they did something similar for these human evaluations, rather than just use the single sample, you could see how that would be horrible for personal writing.
One of them would have been named gpt-5, but people forget what an absolute panic there was about gpt-5 for quite a few people. That caused Altman to reassure people they would not release 'gpt-5' any time soon.

The funny thing is, after a certain amount of time, the gpt-5 panic eventually morphed into people basically begging for gpt-5. But he already said he wouldn't release something called 'gpt-5'.

Another funny thing is, just because he didn't name any of them 'gpt-5', everyone assumes that there is something called 'gpt-5' that has been in the works and still is not released.
People have been celebrating the fact that tokens got 100x cheaper and now here's a new system that will use 100x more tokens.
https://chatgpt.com/?model=o1-preview --> defaults back to 4o
Seeing the "hmmm", "perfect!" etc. one can easily imagine the kind of training data that humans created for this. Being told to literally speak their mind as they work out complex problems.
metformin can also cause gastroparesis, which is a nasty condition I wouldn't wish on anyone.
Microsoft, Google, Facebook have all said in recent weeks that they fully expect their AI datacenter spend to accelerate. They are effectively all-in on AI. Demand for nvidia chips is effectively infinite.
Indeed.

I guess we'll have to wait for an iFixit teardown to see how this new widget actually differs in internal construction.
This model is currently available for those accounts in Tier 5 and above, which requires "$1,000 paid [to date] and 30+ days since first successful payment"

More info here: https://platform.openai.com/docs/guides/rate-limits/usage-ti...
Yes! I really love how ChromeOS is architected as well, even though it's not for me (power user/programmer) in its current form.
You're kidding right? The tests they gave it are probably better tests than IQ tests at determining actually useful problem solving skills...
https://openai.com/index/learning-to-reason-with-llms/
The progress in AI is incredibly depressing, at this point I don't think there's much to look forward to in life.

It's sad that due to unearned hubris and a complete lack of second-order thinking we are automating ourselves out of existence.

EDIT: I understand you guys might not agree with my comments. But don't you thinking that flagging them is going a bit too far?
For those "downvoting" this comment, please: I wrote it right after my initial post, before any answer, to make my initial post clearer. I certainly should have added this to the main post.

Now that I have all these answers, I understand better. But cant delete or modify this comment. So sadly it's here for eternity :-(

Thanks a lot for your insightfull comments to the original post. Actually, I now think that I will use these method to help getting more feedback from users
When one axis is on log scale and the other is linear with the plot points appearing linear-ish, doesn't it mean there's a roughly exponential relationship between the two axis?
When are they going to change the name to reflect their complete change of direction?

Also, what is going to be their excuse to defend themselves against copyright lawsuits if they are going to "understandably" keep their models closed?
We'll know the Microsoft takeover is complete when OpenAI release Ai.net.
Only for those accounts in Tier 5 (or above, if they exist)

Unfortunately you and I don't have enough operating thetans yet
Hei interesting, thanks for sharing.

Also, the idea to do it the other way around WSL -> docker image is really nice.
That sounds like the GPU labor theory of value that was debunked a century ago.
Maybe CriticGPT could be used here [0]. Have the CoT model produce a result, and either automatically or upon user request, ask CriticGPT to review the hidden CoT and feed the critique into the next response. This way the error can (hopefully) be spotted and corrected without revealing the whole process to the user.

[0] https://openai.com/index/finding-gpt4s-mistakes-with-gpt-4/

Day dreaming: imagine if this architecture takes off and the AI "thought process" becomes hidden and private much like human thoughts. I wonder then if a future robot's inner dialog could be subpoenaed in court, connected to some special debugger, and have their "thoughts" read out loud in court to determine why it acted in some way.
It sounds like GPT-4o with a long CoT prompt no ?
Why don’t you open source it and let devs know they can contribute starting today? Devs who don’t use HN can find it on GitHub by searching and you can begin posting call to arms if by posting your vision and goals.
There is something effectively preventing from doing it again, the experience of what just happened.
I am completely confused by your example. Buying a better knife doesn't make you a better chef. Buying a faster car doesn't make you a better driver. Buying a more powerful laptop doesn't make you a better developer.
It seems opposite to me. Imagine all the amazing technological advancements, etc. If there wasn't something like that what would you be looking forward to? Everything would be what it has already been for years. If this evolves it helps us open so many secrets of the universe.
Yeah, exposed chain of thought is more useful as a user, as well as being useful for training purposes.
Getting paid in stock might be a benefit at big tech, but in the startup world the median value of granted stock has to be zero or close to it. Most startups fail before going public or being acquired.
Popcorning: "the happy little jump that guinea pigs give when they are full of joy"

In case anyone else was wondering.
The external product is _marginal improvements_,

the internal product is _full employment_ along with a healthy dash of _intellectual pleasure_, though much of the latter is in rediscovery of principles and approaches which regularly recur as the pendulum swings. But pride of ownership and invention is strong human drive.
PRS 505, an old ereader. It was my first introduction to eink, it was such a fantastic device and so well thought through in terms of button placement, features, connectivity. This is before Kindles took over though, and Apple-ified the ecosystem.
Regex’s require you to understand what the obscure-looking patterns do character by character in a pile of text. Then, across different piles of text. Then, juggling different regex’s.

For a LLM, you can just tune it to produce the right output using examples. Your brain doesn’t have to understand the tedious things it’s doing.

This also replaces a boring, tedious job with one (LLM’s) that’s more interesting. Programmers enjoy those opportunities.
F#’s tooling is worse than C# for sure, but it’s a big step-up from Haskell and has access to the .NET framework.

I listed C# because that’s the mainstream language I know the best, and arguably has best-in-class tooling.

Of course you have to be prepared to lose some of the creature comforts when using a more left-field language. But, you still need to be productive. The whole ecosystem has to be a net gain in productivity, or stability, or security, or maintainability — pick your poison depending on what matters to your situation.

I had hoped Haskell would pay dividends due to its purity, expressive type-system, battle tested-ness, etc. I expected us to be slower, just not as slow as it turned out.

Ultimately the trade off didn’t work for us.
AlotOfReading pointed out that 'This theory is based on the hypothesis that humans pre-1500 didn't have any contact with one another.' is not correct by giving examples of how humans pre-1500 did have contact with one another.

That there were people and civilizations in the Americas before then is far from the point.
Caveat: I'm a ham, but not an expert.

The answer depends on what you want to do or explore. You have many different options, including RTL-SDR type USB dongles for use with software like SDR# and gqrx.
When the tool is statistical word vomit based, it will never move beyond cool bar trick levels.
An LLM could come up with an infinite amount of new theories and ideas, of which >99% would be meaningless, if given enough time and energy.

Human’s ability is in guiding the prompt (of AIs, and the mind) on what is worth knowing and which hypotheses are worth testing.

Choosing an action from the countable infinite number of actions is the general framework of free will.
As noted in the article, there's also a traditional serial interface.
As Amazonian rivers recede under drought, manatees are left exposed to poaching
I appreciated that, too! FWIW, I could get Claude 3.5 to tell me how many rs a python program would tell you there are in strawberry. It didn't like it, though.
As a society we're not solving for programmer salaries but for general welfare which is basically code for "cheaper goods and services".
LLMs have already beaten the Turing test. It's useless to use it when OpenAI and others are aiming for 'AGI'.

So you need a new Turing test adapted for AGI or a totally different one to test for AGI rather than the standard obsolete Turing test.
Why retain it as a potential punishment if the burden of proof to obtain it is impossible? There's always a chance the evidence is objectively incomplete and even if they confess, there's always a chance they were coerced. Putting an innocent person in prison for life is awful enough but at least there is time for them to appeal or for contrary evidence to surface some day. Condemning that innocent person to death puts a countdown on proving their innocence. Maybe someone else confesses to the crime or some other exculpatory evidence is uncovered years after the execution. What does the state do at this point? They can't reanimate the dead. How do you make family members whole when you've legally murdered their parent/spouse/child? Money doesn't replace a person.

You also could end up with situations where a particular governor is indifferent to the plight of someone professing their innocence due to their own bias where next year, a new governor comes in that is more sympathetic that would have stayed the execution or pardoned that person. The world is just too complex to be 100% certain that someone must die for their alleged crime.

The law needs to be applied fairly to everyone. You can't simply give some people the death penalty and others life in prison for the same crime just because a case seems more concrete (or more likely, they can afford a better lawyer that creates stronger doubt).

And you know what's worse than death? Sitting in a little box for the rest of your life separated from friends and family while the world passes you by. People commit suicide for less.
Personally a big fan of Flax. The way it separates the params and the compute graph is - imo - the right (TM) way. Saying this after many years of soing ML :)
Exploring different approaches and stumbling on AGI eventually through a combination of random discoveries will be the way to go.

Same as Bitcoin being the right combination of things that already existed.
One thing I find generally useful when writing large project code is having a code base and several branches that are different features I developed. I could immediately use parts of a branch to reference the current feature, because there is often overlap. This limits mistakes in large contexts and easy to iterate quickly.
No, its the fraud theory of charging for usage that is unaccountable that has been repeatedly proven true when unaccountable bases for charges have been deployed.
More NPM packages on Cloudflare Workers: Combining polyfills and native code
Easy solution - don't pay!
Isn't that part of the point?
A hard drive from 1995 will most likely be formatted FAT16/or FAT32 and the last Windows OS that reads that filesystem by default is Windows XP - I keep an old XP workstation operational as a test rig for reading data on old hard drives.

Or I have to mount them in another OS that isn't Windows. It's more than just adjusting DAW settings and updating plugins at this point you need to know that around 2000 the filesystems completely changed with NTFS and added security that wasn't present before.

By the time Vista/7 FAT hard drive support is gone from Microsoft land. There are of course add-ons and such but you still need to _know_ this happened and FAT drives look unformatted in modern Windows.