MurphyA/hacker-news huggingface dataset:
by mind-blight | 2025-08-08 21:30:27+00:00
I'm not even considering the coding use case. It's been fine in cursor. I care about the days extraction and basic instruction following in my application - coding ability doesn't come into play.

For example, I want the model to be able to take a basic rule and identify what subset of given text fits into the rule. (E.g. find and extract all last names) 4o and 4.1 we're decent, but still left a lot to be desired. o4-mini was pretty good at not ambiguous cases. Getting a model that runs cheaper and is better at following instructions makes my product better and more profitable with a could lines of code change.

It's not emotionally revolutionary, but it hours a great sweet spot for a lot of business use cases
by cornholio | 2025-08-08 21:30:40+00:00
Inference is essentially a very complex matrix algorithm run repeatedly on itself, each time the input matrix (context window) is shifted and the new generated tokens appended to the end. So, it's easy to multiplex all active sessions over limited hardware, a typical server can hold hundreds of thousands of active contexts in the main system ram, each less than 500KB and ferry them to the GPU nearly instantaneously as required.
by general1726 | 2025-08-08 21:30:50+00:00
Palestinians causing civil war in Jordan - https://en.wikipedia.org/wiki/Black_September

Palestinians causing civil war in Lebanon - https://en.wikipedia.org/wiki/Lebanese_Civil_War

Egypt building 12km buffer zone to stop spread of militants into Egypt and smuggling of weapons - https://en.wikipedia.org/wiki/Egypt%E2%80%93Gaza_border

So your whataboutism can't change the subject I have raised - why no local power is trying to help Palestinians much. Because they were already burned by them in the past.
by darth_avocado | 2025-08-08 21:30:54+00:00
If you vaped the same amount as a cigarette then yes. Unfortunately vapes get you to vape way more. On top of that Social norms around cigarettes create barriers for smoking that don’t exist for vaping. Somehow smoking a cigarette indoors is extremely frowned upon but you could be sitting on a couch vaping all day and no one will bat an eye. The fact that you have to physically go outside to smoke is a sufficient enough barrier to reduce consumption.
by conductr | 2025-08-08 21:31:02+00:00
Mine tend to be entrepreneurial and I find that if I sketch out the solution and then think about marketing and sales and the customer service and what not, that I quickly realized I have no desire to run this company even if it seems like an interesting problem space to scratch my engineering itch.

So I just won’t start building as many. If I find the engineering part interesting enough I might just build the part that’s interesting to me. That typically means I skip the front end and only focus on the core business logic.
by idohft | 2025-08-08 21:31:15+00:00
> Its a finance firm - i.e scam firm. "We have a fancy trading algorithm that statistically is never going to outperform just buying VOO and holding it, but the thing is if you get lucky, it could". > Scammers are not tech people. And its pretty from their post.

It would be great if you included any sort of evidence or argument.

Reading on to the other comments, it looks like you're throwing out a lot of accusations and claims. I don't know what you think you know, but from the looks of it, you don't really know HRT's business. I don't really these days, but I knew it years ago, and it's not from taking client money or arbitrage or some weird scam. It's not magic but the world of algo trading isn't a ponzi scheme.
by cmdtab | 2025-08-08 21:31:25+00:00
Temporal required re-architecting some stuff, their typescript sdk and sandbox is bit unintuitive to use so would have been an additional item to grok for the team, and additional infrastructure to maintain. There was a latency trade off too which in our case mattered.

Didn't face any issue though. Temporal observability and UI was better than DBOS. Just harder to do incremental migration in an existing codebase.
by jp57 | 2025-08-08 21:31:30+00:00
700M weekly users doesn't say much about how much load they have.

I think the thing to remember is that the majority of chatGPT users, even those who use it every day, are idle 99.9% of the time. Even someone who has it actively processing for an hour a day, seven days a week, is idle 96% of the time. On top of that, many are using less-intensive models. The fact that they chose to mention weekly users implies that there is a significant tail of their user distribution who don't even use it once a day.

So your question factors into a few of easier-but-still-not-trivial problems:

- Making individual hosts that can fit their models in memory and run them at acceptable toks/sec.

- Making enough of them to handle the combined demand, as measured in peak aggregate toks/sec.

- Multiplexing all the requests onto the hosts efficiently.

Of course there are nuances, but honestly, from a high level last problem does not seem so different from running a search engine. All the state is in the chat transcript, so I don't think there any particular reason reason that successive interactions on the same chat need be handled by the same server. They could just be load-balanced to whatever server is free.

We don't know, for example, when the chat says "Thinking..." whether the model is running or if it's just queued waiting for a free server.
by throwaway2237 | 2025-08-08 21:31:59+00:00
If you will please allow me an oversimplified example:

A restaurant is limited by the amount of food a person can eat in a meal. So this hypothetical restaurant can sell a $20 meal to 100 people (with $20 expendable income to spend,) and thus will generate $2,000 revenue for that meal service.

However, if fewer and fewer people have $20 expendable income, then obviously it becomes harder and harder for that restaurant to generate that same $2,000. Especially as the cost of bills, ingredients, and employee wages increase, for example.

So they are left with a dilemma of: 1) Raise prices knowing that there will be fewer customers with enough expendable income to buy the expensive food, or 2) Lower prices and hope that more customers will offset the lower prices (this usually does not happen.)

Also it's important to point out that 1 person with a lot of money won't come in and order 100 people's worth of meals. The human stomach doesn't scale in that way.

This example is obviously oversimplified for the sake of showing the point
by simonw | 2025-08-08 21:32:13+00:00
The gpt-oss-20b model has demonstrated that a machine with ~13GB of available RAM can run a very decent local model - if that RAM is GPU-accessible (as seen on Apple silicon Macs for example) you can get very usable performance out of it too.

I'm hoping that within a year or two machines like that will have dropped further in price.
by potatolicious | 2025-08-08 21:32:21+00:00
Would anybody buy the hardware though?

Sure, datacenters will get rid of the hardware - but only because it's no longer commercially profitable run them, presumably because compute demands have eclipsed their abilities.

It's kind of like buying a used GeForce 980Ti in 2025. Would anyone buy them and run them besides out of nostalgia or curiosity? Just the power draw makes them uneconomical to run.

Much more likely every single H100 that exists today becomes e-waste in a few years. If you have need for H100-level compute you'd be able to buy it in the form of new hardware for way less money and consuming way less power.

For example if you actually wanted 980Ti-level compute in a desktop today you can just buy a RTX5050, which is ~50% faster, consumes half the power, and can be had for $250 brand new. Oh, and is well-supported by modern software stacks.
by flax | 2025-08-08 21:32:27+00:00
Perhaps this is a good analogy, in which case I'd prefer they stop advertising it as a better/faster/cheaper bird. Speaking as a metaphorical bird, it clearly cannot do well what I do. It does do it poorly at a remarkable speed though.

So what is the software development task that this plane excels at? Other than bullshitting one's manager.
by threatripper | 2025-08-08 21:33:13+00:00
Your model keeps the weights on slow memory and needs to touch all of them to make 1 token for you. By batching you make 64 tokens for 64 users in one go. And they use dozens of GPUs in parallel to make 1024 tokens in the time your system makes 1 token. So even though the big system costs more, it is much more efficient when being used by many users in parallel. Also, by using many fast GPUs in series to process parts of the neural net, it produces output much faster for each user compared to your local system. You can't beat that.
by arn3n | 2025-08-08 21:33:20+00:00
In my experience it tends to be the opposite — I am not a quant (QD) but having worked with a few teams there’s a negative selection for technical expertise. QRs who are good at programming are usually pushed into maintaining infrastructure, datasets, or just tooling for less technical members of their team, who then get to use those tools to further their own alpha generation. Orgs incentivize the final step in making alpha — spend too much time helping others or building reusable research, and your coworkers steal the thunder.

That, or stop helping your coworkers/accommodating them… risky, as a career move. Only seen that work once.
by insane_dreamer | 2025-08-08 21:33:33+00:00
I don't care if it saves me money, I'm not giving my hard earned cash to Walmart/Kroger/Albertsons/OtherMegaCorp. Not to mention that what they sell at Walmart is mostly unhealthy garbage that I would be a negligent parent if I fed that to my family.

I do make an exception for Costco because from everything I've read they're a pretty standout company as far as treating their employees go, and they curate well. We get some stuff there.

But mostly we shop at small(er) local grocery stores when possible, especially for things like meat, eggs, milk, all of which we can get local (better quality and why ship stuff like that around the world; luckily we're in the PNW). And of course that includes local Asian stores for rice, fresh fish, etc.

We don't buy "fancy" food (no wagyu steaks and the like; we eat a lot of legumes) but we do try to get the healthiest local option when possible (not fanatically so, we're not on Portlandia), and we could probably cut our monthly food bill by at least 30% if we bought the cheapest food at Walmart/Safeway.

So we cut back spending in other ways - rarely eat/drink out, entertainment, etc.

I'm sure we could do better. But I guess what I'm trying to say is that it really costs in both time and money to eat healthy. We're lucky that we can, but it's also a matter of what we prioritize in our spending.
by tracker1 | 2025-08-08 21:33:49+00:00
The only other protocols I'm relatively familiar with are email and nntp protocols, mostly smtp and pop3. HTTP as a protocol is pretty similar at it's basic constructs. Compared to HTTP/2 or newer, it's insanely simple... IMAP and FTP, those are much more convoluted and difficult to get right even on a practical level.

HTTP is also pretty universal at this point with simple client and server libraries for pretty much every language or platform under the sun that could use them. You don't have to roll your own, unless you're rolling your own language, and even then, you can probably burrow an existing C implementation/library.
by quickthrowman | 2025-08-08 21:34:23+00:00
Cooling BTUs already take the coefficient of performance of the vapor-compression cycle into account. 4w of heat removed for each 1w of input power is around the max COP for an air cooled condenser, but adding an evaporative cooling tower can raise that up to ~7.

I just looked at a spec sheet for a 230V single-phase 12k BTU mini-split and the minimum circuit ampacity was 3A for the air handler and 12A for the condenser, add those together for 15A, divide by .8 is 18.75A, next size up is 20A. Minimum circuit ampacity is a formula that is (roughly) the sum of the full load amps of the motor(s) inside the piece of equipment times 1.25 to determine the conductor size required to power the equipment.

So the condensing unit likely draws ~9.5-10A max and the air handler around ~2.4A, and both will have variable speed motors that would probably only need about half of that to remove 12k BTU of heat, so ~5-6A or thereabouts should do it, which is around 1/3rd of the 16A server, or a COP of 3.
by kayodelycaon | 2025-08-08 21:34:40+00:00
Well, yeah. Anyone can own a small plane if they have the money. There’s plenty of uncontrolled airspace and uncontrolled airports.

Good God! What would happen if someone rented a box truck and bought some fertilizer?

Oh, and civilians can own muzzle-loading black powder cannons. Imagine what someone could do with a 32-pound cannonball.

The reality is anyone with the proper skills can crash a plane into anything they like. Unless you have someone on the roof with a MPAD, no one is going be to stop them in time.
by temp0826 | 2025-08-08 21:34:46+00:00
It's complex, other compounds in tobacco do potentiate the effects but (as an ex-smoker, n=1) I never found even the high-octane gas station vapes to be as satisfying (felt like a completely different experience). And the taste is definitely subjective (I know people who say the dirtier the better)- I definitely prefer a good tobacco to any vape, flavored or unflavored. (Oh how I miss it sometimes...why do we love things that want to kill us?)
by PaulHoule | 2025-08-08 21:34:48+00:00
The current situation might not be stable in the long term.

Ukraine now regrets giving up their nuclear weapons.

The "red line" of Ukraine attacking either Crimea or internationally recognized Russia was crossed and revealed to be a bluff.

The wall against proliferation will eventually break. For the life of me I can't figure out how Iran had a much harder time refining U235 with centrifuges than Pakistan did in the 1970s; I mean, I can understand how the US has been unable to deploy centrifuges to make enriched uranium for reactors (the same reason we can't have so many other nice things) but why Iran?

The lesson of Gaza may be that a people in a similar situation might want to have nuclear weapons at all costs. There definitely are paths left out of the conventional analysis of proliferation paths (accelerator-based methods, AVLIS) and even though they could reduce the footprint of producing the fuel they are also all super-high tech.)
by jaggederest | 2025-08-08 21:35:09+00:00
Fun fact about the name Lloyd, it's an anglicization of the Welsh word Llwyd, meaning grey, and was originally an epithet applied for being the color grey, or grey haired, or in some cases brown and related colors. So it's a bit like the last name Brown or White, but has become first name, oddly enough.

And the welsh word is pronounced with that particular welsh "Ll" sound, a fricative around the tongue, but the word Lloyd lost that pronunciation - but not before we had alterations like Floyd and Fludd.

https://en.wikipedia.org/wiki/Lloyd_(name)
by dataflow | 2025-08-08 21:35:31+00:00
If only life was so simple.

So file A gets saved... a rebuild starts... and now file B gets saved a few seconds later.

What do you do? Do you kill the build and start a new one? Do you wait for it to finish?

What about the race conditions - what if half the build process sees the old contents and the other half sees the new contents of some of the files - do you contaminate the output/cache? Do you even detect it to tell the user?
by Dr4kn | 2025-08-08 21:35:48+00:00
A decently powerful Server is nice, when you need it. Having some modern APU for decent en- and decoding performance is great.

There are tasks that benefit from speed, but the most important thing is good idle performance. I don't want the noise, heat or electricity costs.

I'm reluctant to put a dedicated GPU into mine, because it would almost double the idle power consumption for something I would rarely use.
by jstummbillig | 2025-08-08 21:35:53+00:00
> Many of the best models are open source, just too big to run for most people

I don't think that's a likely future, when you consider all the big players doing enormous infrastructure projects and the money that this increasingly demands. Powerful LLMs are simply not a great open source candidate. The models are not a by-product of the bigger thing you do. They are the bigger thing. Open sourcing a LLM means you are essentially investing money to just give it away. That simply does not make a lot of sense from a business perspective. You can do that in a limited fashion for a limited time, for example when you are scaling or it's not really your core business and you just write it off as expenses, while you try to figure yet another thing out (looking at you Meta).

But with the current paradigm, one thing seems to be very clear: Building and running ever bigger LLMs is a money burning machine the likes of which we have rarely or ever seen, and operating that machine at a loss will make you run out of any amount of money really, really fast.
by filleduchaos | 2025-08-08 21:37:11+00:00
...which betrays a lack of knowledge of aviation beyond the inconveniences of scheduled passenger flight services.

There is an entire world of aviation outside of commercial airlines flying airliners out of large, towered airports with fancy terminal buildings. An aircraft is a vehicle like any other, and operating one is regulated in tiers like any other type of vehicle. It's about as inane to gripe that an untowered recreational airport is not regulated to the same extent as the airports you fly commercially out of, as it would be to gripe that you driving your car out of your home is not regulated to the same extent as driving a school bus.
by doctorwho42 | 2025-08-08 21:37:30+00:00
My guess it would be around the time the first boomers are getting out into markets, and their parents are settling into their forever home - never selling until death or forced retirement home.

Tie that with what people of that generation would consider a tolerable commute and the cultural idyllic home size/lot size. I think you start to form the basis of the answer, and like all answers it is simple in it's complexity
by ignoramous | 2025-08-08 21:39:27+00:00
> Another great resource to look at is the unsloth guides.

And folks at LMSys: https://lmsys.org/blog/

Large Model Systems (LMSYS Corp.) is a 501(c)(3) non-profit focused on incubating open-source projects and research. Our mission is to make large AI models accessible to everyone by co-developing open models, datasets, systems, and evaluation tools. We conduct cutting-edge machine learning research, develop open-source software, train large language models for broad accessibility, and build distributed systems to optimize their training and inference.
Early-stage lending fintech founders –> how did you fund your first loan book?
by CocoIemon | 2025-08-08 21:39:29+00:00
Hi everyone,

I’m building a B2C lending fintech. We’ve raised some initial equity funding and have already originated a couple of securities.

Here’s the challenge: To get a proper debt line in 2025, we need repayment data to prove our underwriting model. But to get that data, we need to run a pilot and actually fund a meaningful number of loans, a chicken and egg problem I’m sure many here have faced.

For those who’ve been in this position before: 1) What capital sources did you use to fund your very first loan book? 2) Did you rely on equity, angels/HNWIs, impact capital, friends & family, or something else entirely? 3) What worked best for getting from “no track record” to institutional debt-ready?

Would love to hear what worked for you and what you’d recommend for a 2025 pilot.

Thanks!
by ritz_labringue | 2025-08-08 21:39:30+00:00
The short answer is "batch size". These days, LLMs are what we call "Mixture of Experts", meaning they only activate a small subset of their weights at a time. This makes them a lot more efficient to run at high batch size.

If you try to run GPT4 at home, you'll still need enough VRAM to load the entire model, which means you'll need several H100s (each one costs like $40k). But you will be under-utilizing those cards by a huge amount for personal use.

It's a bit like saying "How come Apple can make iphones for billions of people but I can't even build a single one in my garage"
by potatolicious | 2025-08-08 21:39:43+00:00
NAND is very, very slow relative to RAM, so you'd pay a huge performance penalty there. But maybe more importantly my impression is that memory contents mutate pretty heavily during inference (you're not just storing the fixed weights), so I'd be pretty concerned about NAND wear. Mutating a single bit on a NAND chip a million times over just results in a large pile of dead NAND chips.
by dan-robertson | 2025-08-08 21:40:18+00:00
I mean, I don’t mind the Claude 3 funeral. It seems like it was a fun event.

I’m less worried about the specific complaints about model deprecation, which can be ‘solved’ for those people by not deprecating the models (obviously costs the AI firms). I’m more worried about AI-induced psychosis.

An analogy I saw recently that I liked: when a cat sees a laser pointer, it is a fun thing to chase. For dogs it is sometimes similar and sometimes it completely breaks the dog’s brain and the dog is never the same again. I feel like AI for us may be more like laser pointers for dogs, and some among us are just not prepared to handle these kinds of AI interactions in a healthy way.
by UtopiaPunk | 2025-08-08 21:41:07+00:00
I visited Switzerland recently and loved the train network. One really awesome feature was that the train stations basically doubled as shopping malls. Which makes a lot of sense, imo!

We'd leave our room for the day, have breakfast at a restaurant or coffee shop in the train station, then jump on the train to whatever outing we had planned. At the end of the day, we'd take the train back, pick up some groceries at one of the grocery stores in the station (I saw at least two major grocery stores in our station), and then head to the room and make dinner. I also needed to visit a pharmacy at one point during our stay, and the only pharmacy open at that sleepy hour was at the train station.

The train stations are really major hubs for the towns. Even if I didn't need to take the train that day, I was still likely to make a trip down to the train station for something. It's smart.
by Detrytus | 2025-08-08 21:41:23+00:00
I do not understand why such news would cause gold price to go up... What Trump is clearly aiming for is to replace gold with cryptocurrencies - that's why his recent moves to have his own coin, allow crypto in 401(k), push toward "crypto dollar", and now trying to limit gold inflow to US with tariffs. If this goes as planned then gold will become less relevant in global monetary system, so it should go down, right?
by ActorNightly | 2025-08-08 21:43:00+00:00
As codebases get larger, inefficiencies are bound to happen. Having explicit imports that I can go look up and see where they are is better for resolving this, because you can trace how every module gets imported and time them. Having interpreter code that runs with lazy loading that is all hidden is not the way to solve this.

As for external libraries, you import them in places where you need to use them only to avoid the same pitfalls. Its also pretty easy to analyze the import process within those libraries, and then again import specific submodules only that limit what actually gets loaded.
by jaggederest | 2025-08-08 21:43:32+00:00
Leaving aside the value for humans, it's actually very valuable for the AI to provide indexed summary documents of what code goes where, what it does, and what patterns it uses, and what entry points and what it's API conventions are.

This is useful because if you just have Claude Code read all the code every time, it'll run out of context very quickly, whereas if you have a dozen 50 line files that summarize the 200-2000 lines of code they represent, they can always be fresh in context. Context management is king.