skysight-inc/HackerNews huggingface dataset:
The results never fell off significantly with more training. Same model with longer training time on those bigger clusters should outdo it significantly. And they can expand the MoE model sizes without the same memory and bandwidth constraints.

Still very surprising with so much less compute they were still able to do so well in the model architecture/hyperparameter exploration phase compared with Meta.
Agreed. I am no fan of the CCP but I have no issue with using DeepSeek since I only need to use it for coding which it does quite well. I still believe Sonnet is better. DeepSeek also struggles when the context window gets big. This might be hardware though.

Having said that, DeepSeek is 10 times cheaper than Sonnet and better than GPT-4o for my use cases. Models are a commodity product and it is easy enough to add a layer above them to only use them for technical questions.

If my usage can help v4, I am all for it as I know it is going to help everyone and not just the CCP. Should they stop publishing the weights and models, v3 can still take you quite far.
"Econometric power density"? :)
I think there's likely lots of potential culprits. If the race is to make a machine god, states will pay countless billions for an advantage. Money won't mean anything once you enslave the machine god.

https://wccftech.com/nvidia-asks-super-micro-computer-smci-t...
Physics papers often have hundreds.
Show HN: Thufir – An open-source AI VS Code extension for SREs
Thufir is an open-source VS Code extension that combines server monitoring with AI assistance. Manage servers via SSH, integrate with Prometheus, and get real-time metrics, AI insights, and SRE tools—all in one place. Open source and built for developers
The code is much more readable and modular than you typical verilog dump, so it's probably the best CPU for microarchitecture experimentation. Source: did my master thesis prototyping a specialized cache. Started on Rocket Core, which turned out to be a total mess with all of the pipeline in a single module, basically impossible to introduce a new datapath without rewriting everything. Vex was a breath of fresh air. Spinal is also awesome, lots of QoL features for separating concerns between modules in a way that's impossible on Verilog and fixes lots of rough edges of Chisel.

Performance on FPGA was better than most open-source RISC-V cores out there as of 2020. Rocket might have been better on silicon, but that's it. I haven't looked much into it since then through.
Starting a company is all about initiative. Taking action after talking to customers and learning about their problems. AI is about responding to prompts. No LLM takes action on its own. It doesn't define or find target customers or interview them or figure out new solutions that save them time and money. AI can help with each of these things by summarizing and generating content, but it doesn't do them without the right, human-generated, prompts.
You broke the site guidelines very badly with this comment and other comments you posted to this thread. Could you please not do that? It's not what this site is for, and destroys what it is for.

If you'd please review https://news.ycombinator.com/newsguidelines.html and stick to the rules when posting here, we'd appreciate it.
TikTok is your friend.

Oh wait...
Of course optimizing for the best models would result in a mix of GPU spend and ML researchers experimenting with efficiency. And it may not make any sense to spend money on researching efficiency since, as has happened, these are often shared anyway for free.

What I was cautioning people was be that you might not want to spend 500B on NVidia hardware only to find out rather quickly that you didn't need to. You'd have all this CapEx that you now have to try to extract from customers from what has essentially been commoditized. That's a whole lot of money to lose very quickly. Plus there is a zero sum power dynamic at play between the CEO and ML researchers.
>final nail in the coffin of the open World Wide Web

Reports of the web's death seem much exaggerated for something I use daily. OpenAI et all compete against the open web as do facebook, twitter and all the walled gardens but the web is still there and pretty indestructible by design.
have you tried asking chatgpt something even slightly controversial? chatgpt censors much more than deepseek does.

also deepseek is open-weights. there is nothing preventing you from doing a finetune that removes the censorship. they did that with llama2 back in the day.
What do you think of Cotton's statement (that is, why in particular are you posting it)?
> useless executives

Thank you for saying it. After decades in software, I've developed the perspective the people at the top (of medium to large companies, not small ones) are actually bottom feeders and most software companies could function without having a c-suite.

In all the years of this profession, it's as if software engineers cannot be optimized enough. Yet, when has a company taken a look at middle to upper management and tried to optimize that? They don't, or do it in name only. Those people are the financial and emotional ticks of an organization.
Certainly not. Throwing Ukraine under the bus would open the door for Latvia Estonia Lithuania to be subjugated next, followed by Moldova then Romania. Once a war economy is started, there are many reasons and interests at play that make it very hard to stop.
Yes you're right, in their paper I think they say the process of sampling multiple traces then taking relative rewards is supposed to monte-carlo approximate the value network? I don't really have the intuition for that, but it does make sense that rather than simply nudging probabilities in the direction of the trace with the highest absolute reward, you want to favor the trace which had the best reward relative to current state. E.g. for quick intuition if absolute rewards for traces were {0, 0, 0, 0.01} then using absolute rewards would only give a weak signal (nudge weights proportional to 0.01 * logprob) for the last trace, but using relative rewards (based on z-score) of 1.5 * logprob.
In developed countries it's not quite that straightforward, but "his" plane will be owned by Totally Not Musk LLC, not Musk himself, so any putative drugs on it are not his problem.

In developing countries, anything goes. I've paid $15 for "VIP" clearance in Indonesia, which means I sit in a lounge sipping tea and my passport comes back to me stamped without me ever seeing an immigration official.
and why should we believe anything an openai employee says.

openai tried to hide their financial relationship with frontier. why do that if you're honest?
Ye, but "acting in perfect unison" would be a superior trait among people that care about these things which gives it a way as sarcasm?
I can't tell if this is a "link in bio" ad disguised as engagement bait. But, I've opened at least a hundred bank and credit card accounts over the last couple decades as part of "churning" community. Got tens of thousands of dollars in sign up bonuses and it sent my credit score to 830+. I started when I was broke, and kept doing it, since a couple scheduled 5 minute tasks for hundreds of dollars each time is still a fairly easy side hustle. So I could ask the opposite question — why are banks giving money away? Fees and rewards are carrots and sticks. Some use both at once, like high reward cards with annual fees. There are desirable customers and there are undesirable customers and they treat them accordingly. Since you say your customer base is specifically "luxury buyers" (meaning you can skim profit off them in a lot of ways), you probably avoid the stick entirely and treat them pretty well. If your customer base was the lower-income segment with bad credit scores, maybe you'd pivot to fee based relationships. If you're a huge bank, you probably have departments for each style.
On the contrary, it shows that the OP has a very good understanding of the type system. Plus, it’s an understandable approach given the code golf nature in which the original problem was presented. (Line length and character limits screams “code golf” to me)

I give lots of interviews and I try very hard to resolve ambiguity in the expectations and requirements. Up front I explain what the purpose of the interview is and what I intend to evaluate. It’s silly to assume everyone is equally able to read between the lines and coding interviews are already a very poor approximation of what a day to day software engineering job looks like, so I try my best to set expectations up front.
You’ll also need sufficient storage, and fast IO to keep them fed with data.

You also need to keep the later generation cards from burning themselves out because they draw so much.

Oh also, depending on when your data centre was built, you may also need them to upgrade their power and cooling capabilities because the new cards draw _so much_.
Cyber security is a huge concern irregardless of if the grids are national or international.

IMO the novel challenges are:

(1) too many borders with too many opportunities for corruption

(2) Coronal mass ejections become an even bigger issue than already (impact goes from regional to global)

(3) Literally only China is making enough aluminium for this, and nobody's making enough copper or superconductor
> There is a little bit of industry usage, with the biggest user being SiFive

do ask sifive how much they regret that decision though <shrug>
WOW! Thank you so much. Would you be willing to let me interview you directly? Regardless, I'd love to email you about the expert volunteers if that's ok.
Thanks for digging out those links. I think Imran came up with it in 2006 or 2007, he was certainly experimenting with interview questions around that time, so the blog post was not long after. I had no idea at the time he had a blog and didn't see the post myself until a few years after that. In fact, I think I saw Jeff Atwood's first!
>in 5 years

Or much much quicker [0]

[0] https://timelines.issarice.com/wiki/Timeline_of_xAI
Of course the question is how to leverage those monitors. Either games have to render 480 frames per second (which is impossible on average hardware in most cases other than Subpixel Snake), or the monitor just displays 7 black frames after every rendered frame, which would cut down the games to 60 (rendered) frames per second. But the latter would of course greatly reduce the maximum screen brightness to 1/8, possibly below CRT level, because OLEDs aren't very bright in the first place.
Seeing what china is doing to the car market, I give it 5 years for China to do to the AI/GPU market to do the same.

This will be good. Nvidia/OpenAI monopoly is bad for everyone. More competition will be welcome.
Ketamine is diverted from official sources, usually where it is produced(India, Denmark are common). It's not worth it to make in a lab given how plentiful it is. Its everywhere, in every city. He probably has a guy that gets it online. You can easily buy ounces on darknet markets. I don't recommend you do this-- I just find it interesting how this stuff is all going on as a second blackmarket economy in absolutely huge quantities.
You can of course play however you want (and I certainly think this is clever), but imo this is likely against the spirit, and perhaps letter, of the rules.

The rule on giving clues is:

"If you are the spymaster, you are trying to think of a one-word clue that relates to some of the words
your team is trying to guess. When you think you have a good clue, you say it. You also say one
number, which tells your teammates how many codenames are related to your clue." (emphasis mine).

The rule states that the number should be the number of words related to the clue. There is later provisions allowing you to use zero and infinity, but outside of these carve-outs (and imo the "allowed" language is telling here, since it implies any other number not equal to the number of words is not allowed) I don't think this is legal.
Our school used macOS classic, so I used Quitter (https://www.pliner.com/oursoftware/quitter/) to wreak havoc by closing programs on remote machines.
The absurd thing about it is that IO bound code IME often is way easier to optimize than CPU bound code. Adding batching or pipelinining is often almost mechanical work and can give huge speedups.
> a pretty mellow drug

Like all drugs, it depends on how much you take.
Tyler Perry isn't a billionaire because he's made a bunch of blockbusters. He's a billionaire because he owns the entire film production chain, from studio to productions, and puts his name on all of the credits for good measure such as writer, director, producer, etc.[0].

[0]https://en.wikipedia.org/wiki/Tyler_Perry
in 2019, the FDA approved a self-administrated esketamine nasal spray. it's available by prescription.
Then again "You are more likely to remember something if you read it out loud"...https://www.sciencedaily.com/releases/2017/12/171201090940.h...
NP. Email me (see profile) or post a video conference link.
I would argue there is too little hype given the downloadable models for Deep Seek. There should be alot of hype around this organically.

If anything, the other half good fully closed non ChatGPT models are astroturfing.

I made a post in december 2023 whining about the non hype for Deep Seek.

https://news.ycombinator.com/item?id=38505986
Why would Deepseek lie? They are in China, American export laws can't touch them.
> Then, they'll track your eyes and require you to watch the screen :)

Check out the recent controversy over Activision and Call of Duty. Evidently the game code requires access to your webcam and the company has patents on using your gameplay to train a bot that play like you do, then uses the idle time to make the bot play while you are away. This bulks up the pool of players ensuring you are matched with people resulting in a good experience for you. There's also a patent detecting your emotional state while you view their in game store, possibly for adjusting price to influence you to buy or spend more.
I'm skeptical about Elon Musk being the richest person on the planet. My mantra these days? 'Show me the money!' If Wall Street banks are preparing to sell $3B in X loans, maybe he should buy them back—with real cash this time.
Perhaps value judgements should be left up to humans?
Yes, I notice the benefit of having the same faces around regularly at the $sport I do few times per week. You don’t say much more than hello and goodbye, but after a while you appreciate each other's company and really miss it when you can’t go.
This feels like a question you would ask if you were a level designer for a zombie video game.
Yeah I find this part not clearly expressed as well. My best guess is that it's not simply binary "correct/incorrect" but rather the reward is made up of multiple parts (e.g. format + correctness) and structured in a way such that "close enough" answers still get some reward. From there I would expect that a base model might at least be able to "autocomplete" the format/style, at which point RL machinery would kick in to tune it to properly obey the format, and once that's mastered eventually correctness.

They did mention something about tuning on an un-SFT'd base model being much slower 'warming it up' with some existing reasoning traces.
They put him in a building across the street and he has to talk to chief of staff now.
I had it just today, it's slightly worse than I was remembering! The "Cancel" button is way less prominent.

Still, remember this is MFA - at least I'm pretty sure you can't have this as the only way to access your account. An attacker typically needs your password plus you to misclick here.
But Musk isn't short of money (in theory), he could buy the debt in the event.
Indeed! I started with rules to turn on or off my individual heaters based on the temperature of the room but that's too many rules to setup and update. Then I found the generic thermostat helper and my appreciation for HA immediately went one lever higher!
I stayed at the Hyatt Centric in Boston. Take a look at the photos - no desk or chair.

Moxy (Marriott) hotels tend not to have desks either.
I'm running the 7b distillation on my laptop this very moment. It's an insanely good model. You don't need reddit to judge how good a model is.
Someone else asked for more info, so I wrote a much more detailed post here: https://news.ycombinator.com/item?id=42823507
I worked on a NodeJS app that was cpu bound long before I got there. Wages of making a zero code SaaS and thinking they could dynamically render all pages and charge customers a premium for it forever. Started losing out to simpler alternatives.

At one point I shaved almost 20% off TTFB just by cutting the setup overhead of all that telemetry, tracing, logging, feature toggle, etc code that had become idiomatic and set up first thing every time you instantiated an object. By my estimate I cut the overhead by less than 2/3 (couple big dumb mistakes, and less than 1/2 of what was left after), so that’s still over 1/8th of the overall page weight. All before a single service call. But at least I think I pulled the slope of the perf regression line down below the combined trend of faster VMs and faster EC2 instances.

Bookkeeping and other crosscutting concerns add up, particularly when every new feature gets it. It’s that old joke about how you can put so many gauges on an engine that you know its precise speed at all times, but that speed is 0.00.
With an intercontinental power grid, the sun is always shining.

Of course, The Economist isn't worried about that, they really only care about how they can make money from it.

But for a technologist, there's always that messy real world to deal with. And for solar power, that reality is night. A world wide power grid could be constantly transmitting power to the dark side.
Huh, this sounds interesting. Maybe I'll give it a shot. Thanks!
The point was that 15-30% is a very big speedup for a change that impacts what is supposed to take a very small part of the total time. See also Amdahl's law [1]

Suppose that the CPU-intensive part of the program was actually speed up by 3x. This means that previously it took 1.5x as much time as the one that was saved, so thoughly on the order of 22-45% of the total time of your program. In practice the speedup was probably lower than 3x, which means the 22-45% figure would be even higher.

These are really really high number for a system that was supposed to spend most of the time (>90%) blocked in IO (which was not speed up by YJIT).

For comparison, if the CPU-intensive part actually took 10% of the time, then 10% would be maximum speedup you could get (which is way lower than 15-30%) and a 3x speedup would only result in a ~6% speedup.

[1]: https://en.m.wikipedia.org/wiki/Amdahl%27s_law
React has become the 'Linux kernel' of modern software development
> I'm sure Uyghurs in China would not give a damn about AI.

Which reminded me of "Whitey On the Moon" [0]

[0] https://www.youtube.com/watch?v=goh2x_G0ct4
No, the current goes back through the neutral. This is the normal operating mode with single phase power.

This is different from ground. Neutral is the way the return is meant to go. Ground is a safety feature. Connected to the enclosure in some devices.

So sockets here have 3 wires: Live (Brown), Neutral (Blue) and Ground (Green with Yellow stripe). But the switches only have Live and the Switching wire (Black, the wire that goes to the light). The light then receives the switching wire and has Neutral. Because the power is consumed in the light and returns via neutral.

The switch doesn't need the neutral normally because it doesn't use any power. It just switches it on and off on the way to the light. But the wifi switchboxes do need it because they need to remain connected even when the switch is off.
that's the McCarthy era red scare nonsense still polluting the minds of (mostly boomers / older gen-x) americans. it's so juvenile and overly simplistic.
No, I don't mean it from like a licensing or legal perspective at all. I'm thinking more of vibes and ethos.

There is something strange where you are sort of making your product better through open source contributions and yet all of the capital gains from your company's eventual sale will only go to you.

There is something really weird and nasty about that.
It's impressing that browser can display many levels of recursive iframe.
"Reasoning" will be disproven for this again within a few days I guess.

Context: o1 does not reason, it pattern matches. If you rename variables, suddenly it fails to solve the request.
That's really clever! Kudos. I'm gonna set aside some time this week to dive into the implementation
Am I thinking right but say the coupon is 9,6. And now they are selling it at 10% loss. But they got this coupon for 2 years over say ~4 of more safe loans. They are not even now losing too much and certainly not at 5?
Yay!!! Been waiting!!!
-These loans have nothing to do with twitters value though.

Of course it does! Why do you think Elon was complaining about the article so much?
> especially when they don't seem to make significant profits from them.

Where do you get the idea they don't make significant profits from them?
ive been demaking html elements by removing them from my custom browser so it evens out
> Morgan Stanley and others, such as Bank of America and Barclays, lent Musk money to complete his $44 billion buyout of X, then known as Twitter, in 2022.

Hm. So who owes the debt? What's the collateral?
Supposedly Musk bought Twitter personally, but this looks like a leveraged buyout in trouble.

The banks involved had a "sell-down letter", an agreement between the lenders that any sell-off gave the other lenders a right to participate at the same price. That expired ten days ago.[1] The interest charge is about $1.5bn/year now.
A big question is how long those loans run at the old interest rate. Rates are higher now.
Total revenue from X is said to be about $1bn/year, not that anybody outside really knows.

[1] https://finance.yahoo.com/news/exclusive-banks-funded-elon-m...
Longer cables need to be thicker if you want resistance to be fixed.

If you want 1Ω then by coincidence a 40,000 km cable has a cross section of almost exactly one square meter. If you're putting a significant fraction of global power through that at any sane voltage, you can't use most cutting tools (or indeed electrical motors) near it because the magnetic field is too strong.

China is making enough aluminium that this is a plausible thing they may do as a "belt and road" type project.
Vectors and added value
Not necessarily if you are pushing against a data wall. One could ask: after adjusting for DS efficiency gains how much more compute has OpenAI spent? Is their model correspondingly better? Or even DS could easily afford more than $6 million in compute but why didn't they just push the scaling？
When it comes to banks, there is also a tier where fees are waived for high value customers. For example, checking, atm, wires, are free if you have $50k or more in your checking account.
The other side of this is that if this is over investment (likely)

Then in 5 years time resources will be much cheaper and spur alot of exploration developments. There are many people with many ideas, and a lot of them are just lacking compute to attempt them.

My back of mind thought is that worst case it will be like how the US overbuilt fiber in the 90s, which led the way for cloud, network and such in 2000s.
China is doing what it's always done and its culture far predates "the west".
Hey, cool idea. Would you be able to tell me the tech stack for the whole app? I want to build a similar application for some other use case. I have built a static map with all my labels using leaflet in Python. To turn it into something like you have, what technologies will I need?
Well, will it still be valuable at bond maturity, the main question
Bonds do have a rating, but Microsoft is the only AAA corporate bond I know of off the top of my head. Even Amazon is AA, while I would guess Twitter more like AMC at CCC- if it were rated.
Logistically, space is not euclidean.