skysight-inc/HackerNews huggingface dataset:
If the models run on OpenAIs servers then surely they could still see the questions being put into it if they wanted to cheat? That could only be prevented by making the evaluation a one-time deal that can't be repeated, or by having OpenAI distribute their models for evaluators to run themselves, which I doubt they're inclined to do.
It might be true that pretraining scaling is out of juice - I'm rooting for that outcome to be honest - but I don't think it's "consensus". There's a lot of money being bet the other way.
Nature around Chernobyl seems to be doing well.[1] Maybe we have the wrong goal here. Why not use this stuff to protect sensitive areas from the most dangerous and destructive force on the planet, people. Find a few endangered habitats, post some signs, bury it shallow and then walk away.

[1] https://www.unep.org/news-and-stories/story/how-chernobyl-ha...
I have NewPipe on my kid's tablets which is pretty good too, but it breaks more often. But it's great to have no ads + sponsorblock.

Between that and pirated shows/movies my kids are absolutely puzzled by commercials when we stay at a hotel or with family.
Ask HN: With O3 progression, how should juniors plan our careers
With the advent of new O3 progressions, what are some recommendations for early/mid-career software engineers to maximize their job growth prospects in the future? What skills will be more sought after? What skills will be the most important? It seems more generic software engineer roles may become more saturated. Curious to hear other thoughts on this.
This is far too broad to summarise here. You can read up on Sutskever or Bostrom or hell even Steven Hawking's ideas (going in order from really deep to general topics). We need to discuss everything - from education over jobs and taxes all the way to the principles of politics, our economy and even the military. If we fail at this as a society, we will at the very least create a world where the people who own capital today massively benefit and become rich beyond imagination (despite having contributed nothing to it), while the majority of the population will be unemployable and forever left behind. And the worst case probably falls somewhere between the end of human civilisation and the end of our species.
The programming task they gave o3-mini high (creating Python server that allows chatting with OpenAI API and run some code in terminal) didn't seem very hard? Strange choice of example for something that's claimed to be a big step forwards.

YT timestamped link: https://www.youtube.com/watch?v=SKBG1sqdyIU&t=768s (thanks for the fixed link @photonboom)

Updated: I gave the task to Claude 3.5 Sonnet and it worked first shot: https://claude.site/artifacts/36cecd49-0e0b-4a8c-befa-faa5aa...
It's not because there is a finite amount of money at which this can't fail, which is never the case for martingale. Martingale is actually likely to bankrupt you against a casino that is much more well staked than you even if you have a small advantage.
The one nice part of the wasteland plaza is that it can hold large outdoor exhibitions in a way that no other space in downtown area can.

Inside the NBA was held live there recently, Boston Calling (the only largish music festival in the area) started there.

There’s obviously no massive outdoor parking lot in downtown Boston, and it would be a shame to have packed crowds trample over the common.
It's certainly interesting. I'm just not convinced it's a test of general intelligence, and I don't think we'll know whether or not it is until it's been able to operate in the real world to the same degree that our general intelligence does.
Well said. I spent years working with an attorney who was first an English major. He wrote the best contracts because his initial focus was making things readable and obvious. The legal formalities came second. I firmly believe that plain language will avoid legal disputes better than PUTTING A BUNCH OF LATIN IN ALL CAPS AD OSTENTATIONEM.
Is there consensus about whether gnn architectures are better than transformer based ones at this point? I am aware that transformers can be viewed as a gnn too.
If some bank calls you about compromised accounts, the recommended action should be to hang up, find the official phone number for your bank, wait one minute[1], then call back.

[1] You have to wait or call from a different phone, because the call might not terminate immediately, and the scammer might still be listening on the line.

https://security.stackexchange.com/a/100342
This is the most maddening thing about all content now. It's all platform based and every platform wants to constantly push/"recommend" things to you and your kids. Right now I use Roku and Plex but even both of those are constantly trying to break down the wall.
> truly general intelligence

Indistinguishable from goalpost moving like you said, but also no true Scotsman.

I'm curious what would happen in your eyes if we misattributed general intelligence to an AI model? What are the consequences of a false positive and how would they affect your life?

It's really clear to me how intelligence fits into our reality as part of our social ontology. The attributes and their expression that each of us uses to ground our concept of the intelligent predicate differs wildly.

My personal theory is that we tend to have an exemplar-based dataset of intelligence, and each of us attempts to construct a parsimonious model of intelligence, but like all (mental) models, they can be useful but wrong. These models operate in a space where the trade off is completeness or consistency, and most folks, uncomfortable saying "I don't know" lean toward being complete in their specification rather than consistent. The unfortunate side-effect is that we're able to easily generate test data that highlights our model inconsistency - AI being a case in point.
It isn't more expensive than what the UK has.

The reason we are in this position is because we looked at building nuclear in the mid-2010s, the cost would have been about half current prices, we didn't build because they were too expensive.

Nothing has been more costly than not investing in capacity.
The financial markets are a 0 sum game mostly. This approach would not work.
Yeah but anyone with half a brain knows to ignore them. Vapid cynicism is a lot more seductive to the average nerd.
https://archive.is/4ssJ6
In that vein, perhaps the delta between o3 @ 87.5% and Human @ 85% represents a deficit in the ability of text to communicate human reasoning.

In other words, it's possible humans can reason better than o3, but cannot articulate that reasoning as well through text - only in our heads, or through some alternative medium.
People aren’t responding to their own assumption that AGI is necessary, they’re responding to OpenAI and the chorus constantly and loudly singing hymns to AGI.
>«All the professionals evangelising Linux in the 20th century have moved to Macs now»

In other words: the professional Linux advocates -- that means, the people who were advocating and recommending Linux to non-Linux users –- that I read and knew and sometimes have talked to -- switched.

I'm sorry, but as a professional journalist surely you must know the contradiction you've introduced here with the difference between "all Linux evangelists moved to Macs now" and "all those I know and read of switched" because those two statements are not the same thing.

One statement deals in absolutes("all Linux evangelists switched to Mac") and can be supported by sources if so, the other is a opinion based on your bubble ("all that I know switched to Mac") which is just your opinion that's different than the situation in my bubble and holds just as much weight.
> to me seems like people witnessing the airplane and complaining that it isn't "real flying" because it isn't a bird flapping its wings

To me it is more like there is someone jumping on a pogo ball while flapping their arms and saying that they are flying whenever they hop off the ground.

Skeptics say that they are not really flying, while adherents say that "with current pogo ball advancements, they will be flying any day now"
Solving ARC doesn't mean we have AGI. Also o3 presumably isn't doing program synthesis, seemingly proving Francois wrong on that front. (Not sure I believe the speculation about o3's internals in the link.)

What I'm saying is the fact that as models are getting better at reasoning they are also scoring better on ARC proves that it is measuring something relating to reasoning. And nobody else has come up with a comparable benchmark that is so easy for humans and so hard for LLMs. Even today, let alone five years ago when ARC was released. ARC was visionary.
We have wolves and bears here also (and I'm frankly sick of hearing the same dusty tale).

Pumas in USA had killed 28 humans in the last 100 years (Two of this people died by rabies, not by the puma attack). Lets say 3 killed each ten years on average to be safe.

Only in 2021 there was more than 104,000 car accidents against animals in California (on a total of 2,1 million of crashes reported against big animals in USA). Deer are involved on a big part of this accidents.

(Dogs also, but dogs aren't wild animals). Deer not only kill people, also cause a lot of other going broke by healthcare bills as result of the accident. We could safely claim that deer are "a thousand times" more dangerous for Americans than Pumas.

Of course people could still crash against a big predator, but predators are scarce by definition, so the probability is much lower. As pumas and wolves reduce the deer population all the time, we should welcome its presence. The final effect is massively positive toward saving American lives.
Ok. Let's pivot to: if the captcha is properly solved without one fault, then it's a bot :)

Humans verify themselves by mouse movements, by errors with hard to read chars, or, and I think that's will do it, generating something unrealistic with a AI (like a sweet red dog with chicken feet in a cooking pot) and let the human guess what's wrong in the pictcha..

Or some sort of.. questions with a double-meaning, and only if you utilize the understanding of the world outside, that ones can be solved.

Like an old apple in a sitting bank and then present the pictures of an old apple computer within a money bank. A real, dry apple between a sitting bank's lattice..

Some crazy stuff can be done with AI, why not to utilize it?
What we're going to do is punt the questions and then convince ourselves the outcome was inevitable and if anything it's actually our fault.
I don't mind paying for hardware. I do mind my privacy, and don't want that kind of information in the cloud, or even traces from encryption I haven't audited myself.
Thats exactly it. And this didn’t start 15 years ago.

There is not much point in arguing about what kicked it off either - that won’t fix it.
Zonal also has issues with congestion, lack of locational price signals, and something called inc/dec gaming. A Nodal design (like in many US wholesale markets) fixes this, but some of the key policy makers are reluctant to go in that direction as it would be a ~7 year project and would introduce revenue uncertainty in the meantime when they are trying to decarbonize and need to make it easy for developers to get financing. Both pro/against arguments for nodal pricing have merit. There have been many discussions on this in government.

Marginal cost based pricing is extremely unintuitive to those outside of the field, but nearly all economists support it. The reason is that this eliminates a lot of the gaming in pay-as-bid designs and helps incentivize investment in more efficient/cheaper generation, while driving out less efficient clunkers that may not be profitable anymore. The reality is of course far more complicated. These markets have saved untold amounts of money by optimizing over large regions, but you can also point out "missing money" problems where there simply isn't enough investment in generators with access to firm fuel (as opposed to when we just had monopoly utilities). As a result, much of the US has an "at risk" or "very high risk" of not having enough generation during summer/winter peak.
Right, so you cobble together 48 different libraries, wrangle all of their configurations and stitch them all together yourself.

You do this 18 times because you have 18 different apps with similar requirements but deal with different data/endpoints/whatever.

On your 19th app you decide to standardize how you cobble together all of these libraries so that you don't have to start at ground zero every single time.

Now you've invented a framework.
Their discussion contains an interesting aside:

> Moreover, ARC-AGI-1 is now saturating – besides o3's new score, the fact is that a large ensemble of low-compute Kaggle solutions can now score 81% on the private eval.

So while these tasks get greatest interest as a benchmark for LLMs and other large general models, it doesn't yet seem obvious those outperform human-designed domain-specific approaches.

I wonder to what extent the large improvement comes from OpenAI training deliberately targeting this class of problem. That result would still be significant (since there's no way to overfit to the private tasks), but would be different from an "accidental" emergent improvement.
shrug whisper seems to do well on my GPU, and faster than realtime.
I think the point is to follow a creator. So if you like an artist, you'd follow them and get their Instagram, YouTube, Spotify, Twitch, etc. all in one place.
Apple isn’t a good example to use here because you can contact a human at Apple very easily:

https://support.apple.com/contact

They will even remote into your device and walk you through how to do something.
The report specifically mentions a number of other studies and claims it improves on their methodology, and seems to want to have their methodology become the "gold" standard of measuring SDV vs HDV performance:

"This methodology establishes a foundation for future research into the safety impact potential of ADS and offers a framework for assessment as these systems continue to scale and develop."

Or:

"By analyzing an unprecedented volume of autonomous miles, introducing a novel benchmark, and utilizing third-party liability insurance claims, we aim to set a new standard for ADS safety evaluation."

Which leads me to worry that they are pushing for this to be considered comparative, and they want policy to be influenced by their claims. And they do:

"Our findings have the potential to significantly impact policy decisions, insurance practices, and public acceptance of ADS technology, contributing to the broader societal dialogue on the future of autonomous transportation."

So I would like for them to compare to median non-impaired human driver with a car in good order (both in time-to-destination and safety), aggregate those over time of day, week and year (basically ensure identical driving conditions between comparison miles), adding claim dollar amount as weight for how severe the incident was. I believe that would be a great start.

With them being Google and already tracking most Android phones as they go through traffic (they use this for traffic conditions), they do have a very rich data set they could compare to as well, but excluding that, they could exclude insurance claims which have happened in conditions they don't operate in (claims will always have time of event and location).
It's good that it works since if you ask GPT-4o to use the openai sdk it will often produce invalid and out of date code.
That’s crazy. I live in New York in an old drafty house. My highest bill is about $400-450 and it’s much colder here. Those rates are brutal.
Looks pretty obvious to me, although, of course, it took me a few moments to understand what's expected as a solution.

c6e1b8da is moving rectangular figures by a given vector, 0d87d2a6 is drawing horizontal and/or vertical lines (connecting dots at the edges) and filling figures they touch, b457fec5 is filling gray figures with a given repeating color pattern.

This is pretty straightforward stuff that doesn't require much spatial thinking or keeping multiple things/aspects in memory - visual puzzles from various "IQ" tests are way harder.

This said, now I'm curious how SoTA LLMs would do on something like WAIS-IV.
A structural overhand is viewed by homeowners as something utterly different from everything else you've mentioned there.
That's commonly called engineering notation.
I'll believe such companies with my life and life of my family when they can handle usual South east Asia capital city for example also during monsoon, say Bombay. Because this is in stake, nothing less.

This aint some web app deployment where flaws are annoying but thats about it.

I've just finished 2 day 1500km drive back home across Europe in various storms, heavy rain, a bit of snow, a lot of it in the dark, tons of properly dangerous idiots, weird non standard marked repairs (we talk about germany), some serious accidents along the way, few nearly misses. Our F11 BMW 5 series took it on effortlessly, complementing my and wife's skills but not interfering. I am not stellar but definitely above average driver. I've done this drive maybe 30x over past 15 years, without kids normally in one 16-18 hour push. Nr of clueless idiots is definitely rising and roads are definitely more full.

No way in hell I'll trust something fully with most precious stuff in my life until its properly battle proven by billions of miles in harsh complex situations, which what you describe it isnt. Till then there is no self driving, just steps towards it. I don't need this for driving around some short distances that much. Good luck with beta testing to ya all.
I work in this field and I would bet that in 5-10 years the situation will not be much different compared to today in terms of employment unless we invent AGI all of a sudden, which I don't see any signs that it'd even remotely happen. Job definitions will change a bit, productivity will improve, cost per LOC will drop, more underserved niches will become tractable/profitable.
Because TikTok is a Chinese company most people just consider the app to by spyware. It's probably not any more spyware than other similar apps, but being owned by China makes it a little more worrisome to some people.
I was impressed until I read the caveat about the high-compute version using 172x more compute.

Assuming for a moment that the cost per task has a linear relationship with compute, then it costs a little more than $1 million to get that score on the public eval.

The results are cool, but man, this sounds like such a busted approach.
In (1) the author use a technique to improve the performance of an LLM, he trained sonnet 3.5 to obtain 53,6% in the arc-agi-pub benchmark moreover he said that more computer power would give better results. So the results of o3 could be produced in this way using the same method with more computer power, so if this is the case the result of o3 is not very interesting.

(1) https://params.com/@jeremy-berman/arc-agi
You should look up the terms necessary and sufficient.
o1 is pretty good at spotting OWASP defects, compared to most other models.

https://myswamp.substack.com/p/benchmarking-llms-against-com...
This is actually mindblowing!
Their conclusion that these utensils are dangerous depended on the results that showed these utensils are at or about the FDA's maximum safe dose. But after correcting the math error, it turns out they're actually about 1/10 of that. That invalidates their conclusion.
I think the paper's authors contribution to common knowledge here isn't the risk/hazard/exposure calculation - it is that there is a risk at all.
How does it not apply to students? It's rare to read about a college student being imprisoned and tortured for being a member of a church while a soldier moves into her apartment.
It does feel like the security protocols necessary to secure $100k to $Ms of crypto which transfers instantly and non-reversibly is a challenge for the average user.

Even as a fairly tech enabled GenX, I have forgotten passwords and had to reset them (usually accounts I haven’t used in a while), had files corrupted without a good backup, lost a Yubikey somewhere in the house (I think at least).

From what I can tell I would need to have my crypto seed laser etched into titanium, and then treat that talisman as if it was made of pure platinum as far as securing and tracking it.

Versus keeping my money in SIPC and FDIC protected accounts.

I will say, the BTC appreciation is a big attraction of course, but long term I don’t see how it becomes widely adopted with so much logistics risk, and appreciation… well who knows about that.
They also identify outages and let you do automatic efficiency stuff — you can enroll in a plan where your car charges during off-peak periods, for example.
They aren't deliberately turning power stations off. We have a renewable system so when those sources aren't generating then we need gas.

All the BBG article proved is the inherent problems with the system that we have. Rather than anyone being blamed for creating that system, rather than anyone being blamed for regulating that system, we get the same politically-motivated nonsense about "evil companies"...this is why we have high electricity costs. There is literally zero political incentive to lower them and, as we have seen, very high political gains from proposing fictional solutions. Guess what? We are still going to be in the exact same place in a few years because we haven't changed the thing causing this (over-reliance on renewables).
That's sort of the point of this isn't it? Instead of having an app for every damn thing, you have one app.
My experience is that YouTube recommendations are heavily weighted based on my watch history. If I watch a single video on, say, videogames, all of a sudden my recommendations are all gamer stuff.

Fortunately, you can easily edit your watch history. I just go through mine periodically and remove any kind of video that I don't want recommendations related to. Doing that has given me a very dialed in recommendation feed. If anything, it's too dialed in, and I rarely get serendipitous recommendations.
...these companies are among the most disruptive of the year -- and possibly of a lifetime...

...

Tennibot

Funding to date: $200k

What the startup does: Makes a robot that picks up tennis balls

----

It's truly both exhilarating and terrifying to contemplate the profound impact this will have on our lives. It will likely alter our way of life in ways few of us, if any, of can even begin to fathom
These results are fantastic. Claude 3.5 and o1 are already good enough to provide value, so I can't wait to see how o3 performs comparatively in real-world scenarios.

But I gotta say, we must be saturating just about any zero-shot reasoning benchmark imaginable at this point. And we will still argue about whether this is AGI, in my opinion because these LLMs are forgetful and it's very difficult for an application developer to fix that.

Models will need better ways to remember and learn from doing a task over and over. For example, let's look at code agents: the best we can do, even with o3, is to cram as much of the code base as we can fit into a context window. And if it doesn't fit we branch out to multiple models to prune the context window until it does fit. And here's the kicker – the second time you ask for it to do something this all starts over from zero again. With this amount of reasoning power, I'm hoping session-based learning becomes the next frontier for LLM capabilities.

(There are already things like tool use, linear attention, RAG, etc that can help here but currently they come with downsides and I would consider them insufficient.)
Well I knocked something together thanks really to @Lemaxoxo for his blog post inspiring me to do this.

https://scanningmvp.netlify.app/ - it’s more for my needs of an AT keyboard for scanning than your backlit one but you get the idea.