skysight-inc/HackerNews huggingface dataset:
The primary fallacy in your argument is that you seem to think that humans produce much better products on some kind of metric.

My lived experience the software industry at almost all levels over the last 25 years leads me to believe that the vast majority of humans and teams of humans produce atrocious code that only wastes time, money, and people's patience.

Often because it is humans producing the code, other humans are not willing to fully engage, criticize and improve that code, deferring to just passing it on to the next person, team, generation, whatever.

Yes, this perhaps happens better in some (very large and very small) organizations, but most often it only happens with the inclusions of horrendous layers of protocol, bureaucracy, more time, more emotional exhaustion, etc.

In other words a very costly process to produce excellent code, both in real capital and human capital. It literally burns through actual humans and results in very bad health outcomes for most people in the industry, ranging from minor stuff to really major things.

The reality is that probably 80% of people working in the tech industry can be outperformed by an AI and at a fraction of the cost. AIs can be tuned, guided, and steered to produce code that I would call exception compared even to most developers who have been in the field for 5years or more.

You probably come to this fallacy because you have worked in one of these very small or very large companies that takes producing code seriously and believe that your experience represents the vast majority of the industry, but in fact the middle area is where most code is being "produced" and if you've never been fully engaged in those situations, you may literally have no idea of the crap that's being produced and shipped on a daily basis. These companies have no incentive to change, they make lots of money doing this, and fresh meat (humans) is relatively easy to come by.

Most of these AI benchmarks are trying to get these LLMs to produce outputs at the scale and quantity of one of these exceptional organizations when in fact, the real benefits will come in the bulk of organizations that cannot do this stuff and AI will produce as good or better code than a team of mediocre developers slogging away in a mediocre, but profitable, company.

Yes there are higher levels of abstraction around code, and getting it deployed, comprehensive testing, triaging issues, QA blah blah, that humans are going to be better at for now, but I see many of those issues being addressed by some kind of LLM system sooner or later.

Finally, I think most of the friction people are seeing right now in their organization is because of the wildly ad hoc way people and organizations are using AI, not so much about the technological abilities of the models themselves.
Bench was a terrible service in my experience. I used it for one of my startups a few years ago. I personally invested a lot into said start-up. I checked my books and everything seemed ok. Eventually I found that they had hidden my Gusto payroll line items from view and they were no longer taken into consideration in my books. This led to a $300k shortfall from where I thought I was vs reality. Their team just shrugged when I brought the issue to their attention. The impact was immediate layoffs affecting real people who depended on me.

Sure ultimately everything falls upon me the founder. But something so common as GUSTO payroll should never be miscategorized and hidden from view.
Agentic AI Workflow: PRDs at the Core of Smarter Development
I mean you're largely correct. What I'm saying is that there may be a standing order to destroy all ballistic threats, and to exercise a very high degree of caution (high level approvals for each engagement) with any potential air breathing / subsonic threats. I imagine this is a common order given to Patriot batteries in hot, but not war, areas.

BTW, this is a bit similar to why the US very clearly advertises the Tomahawk as not nuclear capable - so that a few subsonic blips do not trigger a strategic exchange. The Russians do not do that, many of their commonly used missiles are dual purpose.
Thank you for an interesting categorization and highlighting dynamics between the categories.

Reading the article, I get the sense that being more specific in communication is a sign of better performance capability, whereas generalized communication hints at the opposite.

> "One way to tell a weak engineer in a discussion thread about some problem is to see who is bringing in specific facts about how the system currently works, and who is making purely general recommendations that could apply to any system."

This observation of specific versus general communication seems to surface also when being asked a question. If the question is specific, it hints at competence and effort.

> "One tactic is to avoid time-asymmetrical helping. Don’t do work for them that takes much more time than it took them to ask about it."

The general question offloads, to the one being asked, the work of getting down to concrete details. A specific question on the other hand relies on the questioner having done the cognitive work of comprehension before asking.

A general question deserves a counter question that encourages the questioner to become more specific. In a real world environment, I can imagine—just as the article also points out—there being many factors like stress etc, influencing the communication.
Somehow, fallible humans create robust systems. Look to "AI' to do the same, at a far higher speed. The "AI" doesn't need to recite the Fibonacci sequence; it can write (and test) a program that does so. Speed is power.
> which I recognize as impossible to productize

How so? I also lucid dream and could never say anything about that.
Not Claude
Unfortunately nothing really new on Power 11 or OpenPower.
Start of the article:

> OpenAI’s Board of Directors is evaluating our corporate structure in order to best support the mission of ensuring artificial general intelligence (AGI)1

Footnote 1:

> A highly autonomous system that outperforms humans at most economically valuable work.

A very interesting definition of AGI.
> radio talk show

Well, there's the first problem.

> were most happy about was how it had now learned

on tomorrow's episode, those same hosts learn that once their chat session ended, the same conversion gets to start all over from the beginning.
emo lower-case eulogy.
How are humans relevant here? As example, we operate at different speed.
Why is that? Video streaming isn't the behemoth it was last decade.
Sounds like a feature request for spotlight to add aliases or tags. Personally I always keep a terminal open.
Bench just forced me to switch to an annual contract a few days ago too. What a bunch of crooks.
> "We made it slightly more convenient, but it might be wrong now and also burns down a tree every time you use it."

Sounds like early criticisms of the internet. I assume you mean he should be doing those things with a search engine, but maybe we shouldn't allow that either. Force him to use a book! It may be slightly less convenient, and could still be wrong, but...
> After reading about o3's performance

I heard that people still believing in OpenAI hype exist but I haven't met any IRL.
Fortunately, open-source projects can have more than one contributor. 2024 commit log includes:

Bump CI to macOS 13
Add AVX512 support
Update LuaJIT
Document AMD, Intel, Nvidia GPU support
Add support for LLVM18
Support SPIR-V code generation
Whenever a new technology emerges, along with it always emerge naysayers who claim that the new technology could never work --- while it's working right in front of their noses. I'm sure there were people after Kitty Hawk who insisted that heavier than air flight would never amount to much economically. Krugman famously insisted in the 90s that the internet would never amount to anything. These takes are comical in hindsight.

The linked article is another one of these takes. AI can obviously reason. o3 is obviously superhuman along a number of dimensions. AI is obviously useful for software development. This guy spend 20 years of his life working on formal methods. Of course he's going to poo-poo the AI revolution. That doesn't make him right.
They are much more of a black box than AI. There are whole fields around studying them—because they are hard to understand. We put a lot of effort into studying them… from the outside, because we had no other alternative. We were reduced to hitting brains with various chemicals and seeing what happened because they are such a pain to work with.

They are just a more familiar black box. AI’s are simpler in principle. And also entirely built by humans. Based on well-described mathematical theories. They aren’t particularly black-box, they are just less ergonomic than the human brain that we’ve been getting familiar with for hundreds of thousands of years through trial and error.
I'm mostly talking about the sense of the term used in the title of the article we're commenting on, not a more philosophical sense, though I did draw the connection in my second paragraph. It's hard to get less philosophical than terminfo!
Cloudflare is getting in the way of viewing the site.

What the F?
Yeah, I used to be really into knowing a ton about this stuff because it was something I kept on having out of my control before - I used to have worsening sleep paralysis when I was young and into my 20s, and lucid dreaming would be sort of a side effect of it... I guess this is what that WILD (wake-induced lucid dreaming) thing tries to actually induce, like from what I was reading you sort of meditate your way into that state and flip your consicousness into a lucid dream. Honestly my sleep was crazy back then and I talked to a doctor who said it was grounds for going for a sleep study and getting tested to see if I had narcolepsy, but I just ended up moving way closer to the equator instead and something about the seasonality of the sun coming up and down at the same time all year round is what a doctor there said could have reset my circadian rhythm as it got more pronounced in dark winter time.
As things stand, there's a difference between purchasing for personal consumption, and for publishing. That's been standing, for a very long time. It might be as old as copyright itself.

Just because you own the game, doesn't mean you have the rights to use it in more than a personal setting. That's basically always been the case. You bought a personal license, not a broadcasting one.

The main reason for not attacking such things in the past was that it was a wasted effort at control. Too small a target, requiring too much effort. Automation, through things like audio recognition, changes that.
I think maybe this is where the polarisation of those who find chatGPT useful and those who don't comes from. In this context, the number of r's in strawberry is not a fact: its a calculation. I would expect AI to be able to spell a common word 100% of the time, but not to be able to count letters. I don't think in the summary of human knowledge that has been digitised there are that many people saying 'how many r's are there in strawberry', and if they are I think that the common reply would be '2', since the context is based on the second r. (people confuse strawbery and strawberry, not strrawberry and strawberry).

Your apples question is the same, its not knowledge, it's a calculation, it's intelligence. The only time you're going to get intelligence from AI at the moment is to ask a question that a significantly large number of people have already answered.
No commentray but I find it interesting that this is considered research by a scientific journal...

>That was the conclusion of a recent study by Mallett, who surveyed 400 posts on Reddit to identify exactly when and how lucid dreams are helpful for improving mental health.
With respect to hallucinating, I never read about training LLM’s to say: “I don’t know” when they don’t know. Is that even researched?
Money. I am sure it is pretty lucrative. Childrens books and cookbooks are typical gifts. You can't really go wrong giving a child a book.
Cloudflare is getting the way of viewing the site.

WTF!!?
It's quite frankly more than that. They think we are all idiots into believing that so-called "AGI" is going to make the world a better place, whilst investors, employees are laughing all the way to the bank with every new fundraising round.

First being a non-profit, then taking Microsoft's money, then ditching non-profit status to a for-profit organization and now changing definitions of "Open" and "AGI" to raise more money.

It is a massive scam, with a new level of newspeak.
Your idea for a "single button to immediately restore the video" system would violate the takedown process outlined in the DMCA. The way it works is that after someone sends a takedown request, the content can't be restored until 10 business days after the person who uploaded it sends a counter-notice. This gives whoever sent the takedown request enough time to decide whether they want to file a lawsuit and keep the content down.

Agreed that Youtube's system has problems with how long it gives claimants to respond to the initial content ID dispute and with people spamming false claims. It's definitely not a perfect system.
Yeah, I've had some success with WILDs, which I'd typically practice on a Saturday or Sunday morning, but ultimately it was a crapshoot compared to just meditating myself to sleep.

I'd also recommend lying on your back when attempting to lucid dream, it always made it much more likely to occur for me, along with sleep paralysis. There is some unexplored physiological reason for this, but it's real and many sleep experts have found the same results.

I would also recommend not getting caught up in losing track of the count, simply starting over from 0 the moment you realize that you lost your place.

I have severe chronic insomnia, ADHD, tourettic OCD and a predisposition for body-focused repetitive behaviors so if I can meditate myself to sleep, I'm pretty sure anyone can with some practice. The reason why I can't do it every night and "beat" insomnia is two-fold:

One, because it paradoxically aggravates sleep avoidance for me. It's an excruciating experience spending 30-60 minutes lying completely still while alarm bells are going off in my head, paired with intense urges commanding me to scratch an itch or adjust a limb, and it's anything but meditative until the very moment sleep paralysis kicks in and my body goes numb.

Two, it usually puts me into sleep paralysis which is just rolling the dice for a metric ton of adrenaline to get released during a paralysis hallucination. I've had sleep paralysis encounters keep me up for the next few days, shit gives me the creeps.
The footnote is aligned with what Sam Altman has been saying in most interviews up until recently. I was actually surprised to see the footnote since they have shifted how they talk about AGI.
> JVM exploits from the last years

Entirely user/library serialization or fine cryptographic issues, and not the VM, no?
Maybe legal threat for the company operating it? Would that help?
looks really cool! i'm gonna play with it, thanks for this awesome project
The design is superb! Seriously such an incredible looking site
A brown rat's brain is also a lot more energy efficient than your average LLM. Especially in the learning phase, but not only.
f-18 is built by boeing for the last 30 years
Is YouTube playing a bit loose with the DMCA's requirements here?

* YouTube has no liability for incorrect take-downs (17 USC § 512 (g) (1))

* That limitation on liability only exists if they restore access to the disabled material within 10-14 business days of receipt of a counter-notice (17 USC § 512 (g) (2) (C))

https://www.law.cornell.edu/uscode/text/17/512

YouTube doesn't have to host any content they don't want to. However, it seems likely that a court would say "that doesn't absolve you of complying with the counter notice provisions of the DMCA. You can't just say that you don't want to host any content that goes through a counter notice." There are always limitations on the whole "we don't have to host things we don't want to." I doubt a court would let them use that as an excuse to ignore an explicit mandate of the DMCA, but IANAL.

I think the problem is more likely that creators don't want to sue YouTube or have the resources to go up against Google.
Makes me wonder how they detect market manipulation and fraud. Trivial activities, like marking the close, probably aren't hard to detect, but I imagine that some kind of ML thingy is involved in flagging accounts for manual inspection.
Local CPA isn't a bad option though it could be pricey. (CPAs don't want to do book keeping either). I think one of the things that makes the bench.co shutdown challenging is that they used "proprietary" software so now you're going to have to migrate soon. Switching to accounting software that CPAs are more likely to be familiar with would be a step in the right direction and working with someone to aid that and help do catch up would be a good idea. I can help with both so that this isn't the catastrophe that it's shaping up to be
> I don't know what you're trying to accuse me of

Apologies, I could have been clearer in my last comment. I found that your reductive approach to assumed polarization is a thought terminating cliche, which signals an end to any interest in conversation by the invoker of the cliche.

> the near absolute fixation on partisanship and identity politics in the US is well known.

It really isn't, unless, I would argue, we believe cable news or propaganda sources are a good source of daily information and infotainment. I think they do a poor job communicating complex nuance and people not being solely engaged in wedge issues, forcing a narrative of existential danger in exchange for more clicks and views.

> People do tend to normally behave as stereotypes where American politics is concerned

Again, I think this is reductive.
"It would be against our fiduciary duty to not build the torment nexus."
Even the employees, I think, would probably be fine with being told “we just want to make shitloads of money”.

I usually feel like these statements are more about the board members and C-suite trying to fool themselves.
> I would call ‘LLM-functionalism’: the idea that a natural language description of the required functionality fed to an LLM, possibly with some prompt engineering, establishes a meaningful implementation of the functionality.

My boy. More people need common sense like this talked into them.
In my experience Claude can do this, but can we rephrase it in a way that's more testable?
You seem to believe that humans, on their own, are not stochastic and unpredictable. I contend that if this is your belief then you couldn't be more wrong.

Humans are EXTREMELY unpredictable. Humans only become slightly more predictable and producers of slightly more quality outputs with insane levels of bureaucracy and layers upon layers upon layers of humans to smooth it out.

To boot, the production of this mediocre code is very very very slow compared to LLMs. LLMs also have no feelings, egos, and are literally tunable and directible to produce better outcomes without hurting people in the process (again, something that is very difficult to avoid without the inclusion of, yep, more humans more layers, more protocol etc.)

Even with all of this mass of human grist, in my opinion, the output of purely human intellects is, on average, very bad. Very bad in terms of quality of output and very bad in terms of outcomes for the humans involved in this machine.
Inflation over the amounts needed to stimulate the economy (2-4%) is unauthorized taxation. Kinda whack how everyone is just "okay" with working hard, paying taxes, then having the value of their work spent without their authorization.

Nice there are alternatives for saving.
I was looking at the example build configuration scripts on the project home page. Not somewhere linked from the home page, actually inline code on the page. Today! Literally as I'm writing this comment, it says:

$ python2 duktape-2.6.0/tools/configure.py --output-directory src-duktape \
-UDUK_USE_ES6_PROXY

I'm glad to hear my impression that there was no other build system was wrong!
Anything is possible and running out of money is the most probable explanation but it seems so hard for this type of business to get itself into this type of situation. Customers pay thousands of dollars per year, usually upfront. That’s the type of revenue predictability that most of us would love for our businesses because it makes forecasting so much easier. They must have known months ago that they were running out of money. Failing to become sustainable and going through layoffs is one thing, shutting down overnight with zero warning is another. But yes, you’re probably right, it’s just hard to imagine how they could have imploded like this.
> We all know what's going on

I've been looking for my broadbrush. I forgot I loaned it out.

It seems we've yet again forgotten that HN is an echo chamber. Just because the audience here "knows" something does not mean the rest of the vastly greater numbers of people do as well. In fact, so many people I've talked with don't have a clue about who makes/owns/controls ChatGPT nor would they recognize Sam's name if even OpenAI.

The PR campaign being waged is not meant for this audience. It is meant for everyone else that can be influenced.
Most interesting !
Amazing job at optimizing various parts of the task.
It seems that being an MoE with 'only' 37B active params per token would put it within the reach of CPU & RAM inference for the lucky hobbyist with an Epyc homelab and 8 or 16 memory channels on a second hand single or dual Gen2 mobo (around $2500 used). Any idea of how hard it would be (will?) support the new architecture for llama.cpp ?

I must confess that my interest in LLMs is grounded RAG as I consider any intrinsic knowledge of the LLL to be unreliable overfitting. Is DeepkSeek able to perform grounded RAG like Command R and Nous-Hermes 3 for instance ?

Thx for this amazing model and all the insights in your report !
> Login was smooth if I remember it

That's unfortunately not how it works. TLS sits at the transport layer, so it's not possible for a website to use these certificates for a "login-like flow". The site doesn't get to present to the user why and to whom they are authenticating, since transport layer authentication has to happen before HTTP even gets a single request in.

There is also no "logout" button. It shares these UX problem with HTTP "basic authentication" (even though that's technically an application layer protocol).

On top of that, TLS is these days often terminated by a load balance or even a completely separate entity like Cloudflare. Not sure if you can configure these to request client certificates at all; even if you can, it makes things pretty awkward if you want to have closer control of the authentication flow.

> Privacy should be fine with TLS 1.3

It's not fine at all. Any HTTP server can request your client certificate, and most users would probably not think twice before clicking "authenticate", which then reveals their long-time stable certificate and public key to a potentially malicious server.

Compare that with WebAuthN, which makes it intentionally impossible to accidentally present the certificate for a.com at b.com.
the original non-profit and profit structure was a mistake that was based on what they thought they knew at the time. (They thought they could be a "research" firm.)

There’s no slavery here. If Sam decided it was a mistake to dedicate his time to a non-profit, he’s perfectly free to quit and start an entirely new organization that comports with his current vision. That would be the honorable thing to do.
It works for me with cmd+shift+up/down. I had it set up with the iterm2 shell integration and it just worked.
I once had a chat with a safety analyst for an airline. He told me: flying isn't dangerous at all, but taking off and landing are.