skysight-inc/HackerNews huggingface dataset:
The problem with GPUs is they're designed to be saturated.

If you have a CPU and it has however many cores, the amount of memory or memory bandwidth you need to go with that is totally independent, and memory bandwidth is rarely the bottleneck. So you attach a couple memory channels worth of slots on there and people can decide how much memory they want based on whether they intend to have ten thousand browser tabs open or only one thousand. Neither of which will saturate memory bandwidth or depend on how fast the CPU is, so you don't want the amount of memory and the number of CPU cores tied together.

If you have a device for doing matrix multiplications, the amount of RAM you need is going to depend on how big the matrix you want to multiply is, which for AI things is the size of the model. But the bigger the matrix is, the more memory bandwidth and compute units it needs for the same number of tokens/second. So unlike a CPU, there aren't a lot of use cases for matching a small number of compute units with a large amount of memory. It'd be too slow.

Meanwhile the memory isn't all that expensive. For example, right now the spot price for 64GB of GDDR6 is less than $200. Against a $1000 GPU which is fast enough for that much, that's not a big number. Just include it to begin with.

Except that they don't. The high end consumer GPUs are heavy on compute and light on memory. For example, you can get the RTX 4060Ti with 16GB of VRAM. The RTX 4090 has four times as much compute but only 50% more VRAM. There would be plenty of demand for a 4090 that cost $200 more and had four times as much VRAM, only they don't make one because of market segmentation.

Obviously if they don't do that then they're not going to give one you can upgrade. But you don't really want to upgrade just the VRAM anyway, what you want is for the high performance cards to come with that much VRAM to begin with. Which somebody other than Nvidia might soon provide.
> Too bad they consume 25x the electricity Google does.

From the article:

"However, ChatGPT consumes a lot of energy in the process, up to 25 times more than a Google search."

And the article doesn't back that claim up nor do they break out how much energy ChatGPT (A Message? Whole conversation? What?) or a Google search uses. Honestly the whole article seems very alarmist while being light on details and making sweeping generalizations.
> It explains why the public gets angry at climate protestors instead of the oil executives who've ruined their future.

The public tend to get annoyed at these protestors because their protests are ineffective and disrupt everyday working people. If the protestors brought their protests to the oil executives, I expect they'd be much more widely accepted.

> It explains how university students have somehow wound up as the villains in so many people's eyes, instead of the governments sponsoring and committing genocide.

If this is referring to Gaza, not everyone holds the view that Israel is committing genocide. Many see it as a justified response to a large-scale terrorist attack by Hamas. And either way, the students' protests aren't effective either. It does nothing to affect the actual conflict, just annoys students who are there to study.
Impressive demo, but like half the interactions were "hello" "hi how are you doing" "great thanks, what can I help you with" etc.

The benchmark for human-computer interaction should be "tea, earl gray, hot", not awkward and pointless smalltalk.
I bet this will also cause model regressions.
and say hello to your grandma getting scammed
ShadowNav: Autonomous Global Localization for Lunar Navigation in Darkness
Dude just have them build legos or something.
https://www.gatesfoundation.org/about/committed-grants

19,364 grants serving "America", of ~40,000 grants.
Chair in the sky again...
IC cars age in miles. They get older and parts wear down. So you buy parts, new or aftermarket, and keep it going. Most parts are nothing things like relays or consumables. But they start getting bigger as pumps or fans break. Then at some point the next needed part costs more than the car is worth: a gradual slide to end of life.

EV cars age less in miles than years. They are like electronics. After 5/10/15 years it may be simply impossible to purchase any parts, or if you do they are used parts from junked cars. So rather than a gradual slide, at with every broken part you roll the dice. If the part you need isn't around, the car goes from healthy to dead almost instantly.

https://www.cbc.ca/news/canada/ottawa/leaf-nissan-owners-leg...

>> Last week, Brian Sanderson described how his 2016 Leaf died five times last year. The dealer eventually towed the car back to his driveway in western Quebec, where it remains. Sanderson spent nearly $10,000 on repairs last year, but the battery was never replaced because Nissan cannot provide a new one.

The car is less than 10yo. If this was ANY non-electric car from Nissan every nut and bolt would remain replaceable for decades.
> If your language is written in itself, and you make changes to it, you're changing the language in which your language is written. Mistakes in this can cause hair-pulling bugs.

Fair, but, hold on, shouldn't you just make sure that if you're working on version 1.2 of your language, you compile it using your compiler from a stable 1.1 build? (other than maybe in tests where you need to test new features etc)

This is not a problem specific to programming languages, but to any other system that uses part of itself
"Seymour, the house is on fire!"

"No, mother, that's just the H100s."
Upon checking your profile, I noticed that you’re related to Gridlock. Adding a disclaimer to your original comment would’ve been more appropriate in my opinion.

Also, having your product open-sourced and/or audited increases trustworthiness. You mentioned going open-source and being audited in 2021 but neither happened as far as I can find. That’s fine, just different than OP’s project.

On the flip side, I noticed this Q&A from your docs that I think is at least re-assuring:

“What happens to my assets if Gridlock shuts down?

Should Gridlock ever become unavailable, you are still safe and protected! The built-in Eject Feature allows you to access your crypto independently without Gridlock. This is the definition of self-custody where you are in full control!”

PS. I’m not the target audience for these type of products. I stay away from most crypto stuff. I just stumbled upon this thread and wanted to share my thoughts :)
It's not important from an external standpoint, looking at code quality. Like in all the arts, most of it will be bad. That is not the point. The point is from the developers perspective as a medium. To put it another way, you can produce bad art in any medium but some mediums seem more conducive to creating good art.
You can edit PDF forms by opening the files in Firefox, but maybe that's not exactly what you mean. I'm not sure about other browsers.
Looking at the demo video, the AIs are a bit too chatty. The human has to often interrupt them.

A nice feature would be to be able to select a Meyer's Briggs personality type for your AI chatbot.
You just be new here?
It will be a system, not a single model, and will depend on what programming task you want to perform

probably need routers, RAG, and reranking

I think there is a role for LLM + deterministic code gen as well (https://github.com/hofstadter-io/hof/blob/_dev/flow/chat/pro...)
This is great but we really need an audio-to-audio model like they demoed in the open source world. Does anyone know of anything like that?

Edit: someone found one: https://news.ycombinator.com/item?id=40346992
The facts that AI-generated summaries are still detected instantaneously and are bad enough for people to explicitly ask not to post them says something about current state of LLMs.
Perhaps the real message here is that just locking up the crazies works better than letting them be a problem for others.
It’s succinct, but it says nothing of how to get there from where we are. It doesn’t talk about the constraints of the status quo and its inertia, let alone how any approach plays with the human psyche.

What I’m saying is that sure, I think I agree with GP that there is much getting in the way of us doing more with less. Namely inefficient, greed, and inertia. But how do we get out of this trap?
The web took over, but as discussed on HN before, the University of Minnesota did their best to help kill Gopher when they tried to license it for commercial use.

University of Minnesota Gopher software licensing policy (1993)
https://news.ycombinator.com/item?id=38150331

The Rise and Fall of the Gopher Protocol (2016)
https://news.ycombinator.com/item?id=12269784
Yeah, the way the app currently works is that ChatGPT-4o only sees up to the moment of your last comment.

For example, I tried asking ChatGPT-4o to commentate a soccer game, but I got pretty bad hallucinations, as the model couldn’t see any new video come in after my instruction.

So when using ChatGPT-4o you’ll have to point the camera first and then ask your question - it won’t work to first ask the question and then point the camera.

(I was able to play with the model early because I work at OpenAI.)
It's pretty bizarre how these demos bring out keyboard warriors and cereal bowl yellers like crazy. Huge breakthroughs in natural cadence, tone and interaction as well as realtime mutlimodal and all the people on HN can rant about is token price collapse

It's like the people in this community all suffer from a complete disconnect from society and normal human needs/wants/demands.
> Yet the US space program existed solely and exclusively due to a large and heavily centralized governmental effort.

That's not quite the right conclusion to draw from this. The US could afford to put a humongous amount of resources into its space program because of the productive output unleashed by capitalism and a market-based economy. For example, the US spent a lot of money on a brute-force approach to fixing the combustion instability problems in the huge F-1 engines through trial-and-error; the USSR could not afford to do this, and was forced to go with a much larger number of smaller engines (which due to budgetary constraints couldn't be tested together before the actual launch, with catastrophic consequences).
> How quickly we get used to that magic and demand more.

Humanity in a nutshell.
Exactly. Falcon-180b had a lot of hype at first but the community soon realized it was nearly worthless. Easily outperformed by smaller LLMs in the general case.

Now they are back and claiming their falcon-11b LLM outperforms Llama 3 8b. I already see a number of issues with this:

- falcon-11b is like 40% larger than Llama 3 8b so how can you compare them when they aren't in the same size class

- their claim seems to be based on automated benchmarks when it has long been clear that automated benchmarks are not enough to make that claim

- some of their automated benchmarks are wildly lower than Llama 3 8b's scores. It only beats Llama 3 8b on one benchmark and just barely. I can make an LLM does the best anyone has ever seen on one benchmark, but that doesn't mean my LLM is good. Far from it

- clickbait headline with knowingly premature claims because there has been zero human evaluation testing

- they claim their LLM is better than Llama 3 but completely ignore Llama 3 70b

Honestly, it annoys me how much attention tiiuae get when they haven't produced anything useful and continue this misleading clickbait.
I think at this point the unintended consequences argument has been overused.

There are significant intended consequences that implementing something like this would have. I order for this to be negative, the unintended consequences would need to be larger, and I think it's hard to see that happening.

Reading the article between the lines though, it seems like global coordination is required, and the biggest challenge is getting the top 5 producers in line (China not being mentioned by name/exact number feels evasive by the authors).
>>>>> when OpenAI released their first model people were talking about years before seeing real changes and look what happened.

For what it's worth most of the people in my social circle do not use ChatGPT and it's had zero impact on their life. Exponential growth from zero is zero.
I cannot find the mac app anywhere. Is there a link?
That's existed forever, it's customer segmentation. It's the underlying reason people want this info, age etc is just a proxy. From an advertising perspective nobody cares how old you actually are, they want to know if you're likely to respond to an ad.
My main takeaway is that Generative AI has hit a wall... New paradigms, architectures and breakthroughs are necessary for the field to progress but this begs the question, If everyone knows the current paradigms have hit a wall, Why is so much money being spent on LLMs ,diffusion models etc,which are bound to become obsolete within a few(?) years?
> As many highlighted there, the model is not an improvement like GPT3->GPT4.

The improvements they seem to be hyping are in multimodality and speed (also price – half that of GPT-4 Turbo – though that’s their choice and could be promotional, but I expect it’s at least in part, like speed, a consequence of greater efficiency), not so much producing better output for the same pure-text inputs.
I was thinking of lately to the people holding power these days and their responsibility and involvements into the current state of affairs (politics, large corporations, climate/energy crisis, etc.)

I was wondering if having people over 60-70 or even 80yo running most of the things that matter in this world, was missing the radical optimism of youth.

That is why I like students protesting, even if misguided or not well expressed. It shakes the carefully assembled (and sometimes also misguided) world order we are living in.

Sometimes complex problems need to be revisited from fresh perspective, and have the "wizened nonagenarian Australian with little time left on this earth" to fuck off.
It's weird that the "airplane mode" seems to be ON on the phone during the entire presentation.
The demo is impressive but personally, as a commercial user, for my practical use cases, the only thing I care about is how smart it is, how accurate are its answers and how vast is its knowledge. These haven’t changed much since GPT-4, yet they should, as IMHO it is still borderline in its abilities to be really that useful
Does that imply they retrained the foundation model from scratch? I thought changing the tokenization was something you couldn't really retrofit to an existing model. I mean sure they might have initialized the weights from the prior GPT-4 model but it'd still require a lot of retraining.
Absolutely agree.

This model isn't about basemark chasing or being a better code generator; it's entirely explicitly focused on pushing prior results into the frame of multi-modal interaction.

It's still a WIP, most of the videos show awkwardness where its capacity to understand the "flow" of human speech is still vestigial. It doesn't understand how humans pause and give one another space for such pauses yet.

But it has some indeed magic ability to share a deictic frame of reference.

I have been waiting for this specific advance, because it is going to significantly quiet the "stochastic parrot" line of wilfully-myopic criticism.

It is very hard to make blustery claims about "glorified Markov token generation" when using language in a way that requires both a shared world model and an understanding of interlocutor intent, focus, etc.

This is edging closer to the moment when it becomes very hard to argue that system does not have some form of self-model and a world model within which self, other, and other objects and environments exist with inferred and explicit relationships.

This is just the beginning. It will be very interesting to see how strong its current abilities are in this domain; it's one thing to have object classification—another thing entirely to infer "scripts plans goals..." and things like intent, and, deixis. E.g. how well does it now understand "us" and "them" and "this" vs "that"?

Exciting times. Scary times. Yee hawwwww.
The time for the research, training, testing and deploying of a new model at frontier scales doesn't change depending on how hyped the technology is. I just think the comment i was replying to lacks perspective.
We still have plenty of problems we can solve but don’t. The concept of food insecurity should not exist as a problem in 20th century, but we still have it around.
> Running these models as agents is hella expensive

Because they are dumb so you need to over compute so many things to get anything useful. Smarter models would solve this problem. Making the current model cheaper is like trying to solve Go by scaling up Deep Blue, it doesn't work to just hardcode dumb pieces together, the model needs to get smarter.
Sam mentioned on several occasions that GPT-5 will be much smarter than GPT-4. On Lex Fridman’s podcast, he even said the gap between GPT-5 and 4 will be as wide as GPT-4 and 3 (not 3.5).

He did remain silent on when it’s going to be launched.
>They don’t say that.

>The benefits and burdens of our transportation system – highways, roads, bridges, sidewalks, and public transit – have been planned, developed, and sustained to pull resources from Black communities that are subsequently deployed and invested to the benefit of predominantly white communities and their residents.

>Again, we know that’s not true because highways were built through cities that had no significant minority populations.

You said this about Seattle and Portland, supposing (incorrectly) that there were no minority neighborhoods to raze. It suggests one of two conclusions:

a) There were minority (if not racial, then ethnic, or class) populations that were displaced in order to build the highways in Cairo, and Dusseldorf, and Tokyo, the histories of which we're not privy to due to some combination of a language barrier wrt applicable sources, and an incentive on the part of local historians not to include that history in the foreign-facing record (for diplomatic and face-saving reasons). To disprove this, general demographic statistics aren't enough; you would need specific information about the geographic locations in question - in particular, demographic history and qualitative information about that demography. I don't think you have that.

b) The American history of highway construction is singular, if not at least abnormal. This is the more likely option, because of America's globally abnormal geographic and population size, demographic history, history of municipal development, history of transit development, and automobile-related policy. To put it simply: most other countries don't have the incentives and means to use redevelopment as an explicit tool of displacement. (China does, and unsurprisingly: https://www.fmreview.org/destination-europe/dube).

>trying to project an America-centric racial narrative onto something that was a global phenomenon driven by people’s preferences for the low density living enabled by cars.

You're projecting an America-centric narrative about the role of the auto in the average person's life onto populations with very different circumstances and outlooks. You're categorically wrong, and it might be more useful for you to ask why you're clinging to this incorrect assessment of America's racial and political history.
4o does not stand for ‘for all’:

> GPT-4o (“o” for “omni”) is our most advanced model. It is multimodal (accepting text or image inputs and outputting text), and it has the same high intelligence as GPT-4 Turbo but is much more efficient—it generates text 2x faster and is 50% cheaper. [1]

[1]: https://platform.openai.com/docs/models/gpt-4o
Plastic appears in a lot of "convenience" products which I wouldn't expect to be popular with the poorest people in the world. Plastic bottle of water? Convenient but not an efficient or reliable option.

And while you can probably find a "bad unintended consequence" for everything if you try hard enough, it's usually the case that these bad effects are far smaller than the thing we set out to achieve.
The voice mode was quite good but the latency and start / stop has been encumbering.
> [0] IIRC the Netherlands is the only country left where you can buy SIMs without ID.

As far as I know, in Romania you can still buy and activate a prepaid SIM card without having to show your ID. There was an attempt a few years ago to make it mandatory to tie the phone number to an ID, but it was overruled by the Constitutional Court.
Two GPT-4os interacting and singing [video]
Haven't checked it in smhasher yet. just saw this, claiming to be the very best. Well, maybe a PR will come soon. The rust framework for hashes is prepared, there just haven't been any good rust hashes yet.
Amazing read, nothing to add but the silence of approbation.
GPT-4o takes #1 and #2 on the Aider LLM leaderboards
GPT-4o tops the aider LLM code editing leaderboard at 72.9%, versus 68.4% for Opus. GPT-4o takes second on aider’s refactoring leaderboard with 62.9%, versus Opus at 72.3%.

GPT-4o did much better than the 4-turbo models, and seems much less lazy.

GPT-4o is also able to use aider’s established “diff” edit format that uses SEARCH/REPLACE blocks. This diff format is used by all the other capable models, including Opus and the original GPT-4 models The GPT-4 Turbo models have all required the “udiff” edit format, due to their tendancy to lazy coding.
So they retrained the whole model on audio datasets and the tokens are now sounds, not words/part of words?
>This thing continues to stress my skepticism for AI scaling laws and the broad AI semiconductor capex spending.

Imagine you are in 1970s and saying computers suck, they are expensive, there is not that many use cases....fast forward to 90s and you are using Windows 95 with GUI and chip astronomically more powerful that we had in 70s and you can use productivity apps , play video games and surf Internet.

Give AI time, it will fulfill its true protentional sooner or later.
Can't wait for the moment when I can puta single line "Help me put this in the cart" on my product and magically sells better.
Maybe it’s the geek in me, but I don’t want a talking computer.

I have enough talking people to deal with already
.
Hmm, looks like you are doing it too then. He’s complaining about the conclusions of the article and you are taking it out on him.
related, about trust in this context: Is the Pull-Down Effect Overstated? An Examination of Trust Propagation Among Fighter Pilots in a High-Fidelity Simulation https://journals.sagepub.com/doi/10.1177/15553434231225909
That kind of is an inteligence though. Chinese room meets solipsism and all that.

It is interesting how insanely close their demo is to the OSes in the movie "Her", it's basically a complete real life reproduction.
Discord says I'm a 21ecaf26-4e27-477d-a6c0-45c2c69a7645 but I identify as a 076db833-35be-4fa8-856e-fe2fa4128cee.
> HAL's unemotional monotone in Kubrick's movie, "Space Odyssey," feels... primitive by comparison.

I’d strongly prefer that though, along with HAL’s reasoning abilities.
hmm seems you're replying as a customer, but not as a GPU vendor...

the thing is, there's not enough competition in the AI-GPU space.

Current only option for no-wasting-time on running some random research project from github? buy some card from nvidia. cuda can run almost anything on github.

AMD gpu cards? that really depends...

and gamers often don't need more than 12?gb of GPU ram for running games on 4k.. so most high-vram customers are on the AI field.

> If you could take the card from five years ago and load it up with 80 GB of VRAM, you'd still not see the memory bandwidth of a newly-bought H100.

this is exactly what nvidia will fight against tooth-and-nail -- if this is possible, its profit margin could be slashed to 1/2 or even 1/8
But that's not the point of this update