skysight-inc/HackerNews huggingface dataset:
We built several LLM-powered applications that collectively served thousands of users. The biggest challenge we faced was ensuring reliability: making sure the workflows were robust enough to handle edge cases and deliver consistent results.

In practice, achieving this reliability meant repeatedly:

1. Breaking down complex goals into simpler steps: Composing prompts, tool calls, parsing steps, and branching logic.
2. Debugging failures: Identifying which part of the workflow broke and why.
3. Measuring performance: Assessing changes against real metrics to confirm actual improvement.

We tried some existing observability tools or agent frameworks and they fell short on at least one of these three dimensions. So we built our own: https://github.com/PySpur-Dev/PySpur

1. Graph-based interface: We can lay out an LLM workflow as a node graph. A node can be an LLM call, a function call, a parsing step, or any logic component. The visual structure provides an instant overview, making complex workflows more intuitive.
2. Integrated debugging: When something fails, we can pinpoint the problematic node, tweak it, and re-run it on some test cases right in the UI.
3. Evaluate at the node level: We can assess how node changes affect performance downstream.

We hope it's useful for other LLM developers out there
And here's the more straightforward documentation from the Nomicon that takes the initial blog post and explains it thoroughly and straightforwardly. (It doesn't mention pooping your pants though)

https://doc.rust-lang.org/nightly/nomicon/leaking.html
Is there a list of correct uses of s3 we can all follow?
Gemini 1206 is the new hotness in my books. I've moved my day to day LLM needs over to Google's tab for the first time. I'm not sure what they changed, but it deserves a good look. Claude 3.5-Sonnet (New) is fantastic as well, but the 2M token context window offered by Google allows you to suck in an entire code repository and reason effectively across the whole thing. Google is catching up...
Indeed. My take is that there's just not enough context in TFA to conclude anything whatsoever.
Along those lines (synthetic data) I would keep an eye on the chinese labs given that they are probably quite data and compute constrained, in English at least.
Tesla Cybertruck: The Most Polarizing Thing on Wheels
> the jar is at a very unnatural acceleration (like it pops up).

It does pop up. Look at where his hand is relative to the jar when he grabs it vs when he stops lifting it. The hand and the jar are moving, but the jar is non-physically unattached to the grab.
"Don't make the mistake of anthropomorphizing Larry Ellison."
By then I had moved on to FreeBSD, I think starting at 3.?. The main reason for that move was internet access. FreeBSD CD set came with additional CDs of their ports collection. No more downloading via my flaky and very slow phone connection (via kermit).

I went back to Linux around 4.8 because FreeBSD started having issues with my hardware. By then I had "real" internet thanks to the company I worked for.

I remember seeing items about this, but the BSD people tended not to care about these things. But a nice look back
My 5 year old Surface pro gets 4-5 hours. I find 10th gen and later Intel to get about 8.
I'm sorry, Dave.
It's a ouroboros of bureaucracy getting in the way of security and security causing the bureaucracy itself.
It comes from an economic theory. The problem is that people who say that kind of thing ignore one of the basic assumptions of that theory: that all actors have all the necessary information to make the best choice (and also that they act "rationally"). The problem is that it is very rarely possible to obtain that information, and even if it is possible it might take a lot of time and effort.
I wrote a faxing queue because our fax server at the time was horrible at converting pdf's to fax appropriate tiff's. I didn't think to block an area code that was 911. The cops showed up when a user accidentally fat fingered 911 as an area code.
How many apps do you have installed to control everything? And how is the stuff integrating if you have equipment from different vendor that need to talk to each other, like AC units with PV inverter to start and shutdown based on electricity net production and real temperature in the rooms (using external thermometers, not the one in the AC)? And how do you consolidate and monitor power consumption in a single place, are you using a different solution for that?
This reminds me of the bug in the stealing script in Office Space. The script was supposed to steal fractions of a cent, but, instead, it was stealing dollars.

In the end, it turned out OK. The joint burned down, and Milton ended up on a beach, sans Swingline.
That's comparing apples to oranges though isn't it? Generating videos is the output of the technology, not the tech itself. It would be like someone asking "this computer that takes up a whole room printed out ascii art, what is this useful for?"
> The main benefit of LLMs was already abundantly clear: literally just chat with it

“It is valuable because you can talk to it” is the same idea that drove a tidal wave of sales for Furby in 1998
On what grounds? He changed the statement the box makes by the user. It's silly compliance but O think you really have to go further than this for contempt.
When disabled, the app icons are greyed out, similar to when screen time restrictions have kicked in.
> Might the author be overreacting a bit to this new law

As i have read it, no, it's worth a read to see for yourself though.

> it doesn't put that much of an onerous demand on forum operators.

It doesn't until it does, the issue is the massive amount of work needed to cover the "what if?".

It's not clear that it doesn't apply and so it will be abused, that's how the internet works, DMCA, youtube strikes, domain strikes etc.

> Then again, maybe he's just burnt out from running these sites and this was the final straw. I can understand if he wants to pack it in after so long, and this is as good reason as any to call it a day.

Possibly, worth asking.

> Though, has no-one in that community offered to take over? Forums do change hands now and then.

Someone else taking over doesn't remove the problem, though there might be someone willing to assume the risk.
The Unreasonable Effectiveness of Skip Locked in PostgreSQL
Recovering enterprise SaaS PM here. I don't necessarily know that a lot of enterprise SaaS will disappear, but I do think that a lot of the companies that build it will go out of business as their customers start to build more of their internal systems with LLMs vs. buy from an existing vendor. This is probably more true at the SMB level for now than actual enterprise, both for technical and internal politics reasons, but I expect it to spread.

As a direct example from myself, I now acquire and run small e-commerce brands. When I decided to move my inventory management from Google Sheets into an actual application, I looked at vendors but ultimately just decided to build my own. My coding skills are pretty minimal, but sufficient that I was able to produce what I needed with the help of LLMs. It has the advantages of being cheaper than buying and also purpose-built to my needs.

So yeah, basically the tl;dr is that for internal tools, I believe that LLMs giving non-developers sufficient coding skills will shift the build vs. buy calculus squarely in the direction of build, with the logical follow-on effects to companies trying to sell internal tools software.
He probably should worry about our brain atrophy sooner than our muscles.
I'm not quite following. His argument appears to be: The replication system requires a backwards seek, Postgres does not support that operation, things break when that operation is attempted.

I don't understand why replication would need a backwards seek - are you saying it doesn't and he is mistaken on that?
https://archive.is/ArVrG
Thanks for letting us know, we've updated it now!

Although we're currently only supporting text models, we do definitely have image and video generation models in our roadmap as these are very compute intensive models meaning they would benefit greatly from optimizations. We'd love to hear more about any specific models you're hoping to run! Please feel free to message us with further details (diederik.vink@ncompass.tech).
I hope you have other private benchmarks running that you don't talk about or publish, just in case a model maker intentionally targets one of your benchmarks, or some fuzzy "find things people have mentioned as potential LLM benchmarks" process scoops up your ideas and/or outputs.
A stored program computer refers to the computer architecture where program instructions and data are stored in the same memory. This is also referred to as the Von Neumann architecture.

In contrast, a lot of early computers were built with separate instruction memory like punch cards. This is called the Harvard Architecture. If the instructions were immutable, which they usually were, then things like modifying the program at runtime were not possible.

Concrete examples of this difference is the Harvard Mk 1 and the Manchester Mk 1, the former being a Harvard architecture computer and the latter is a stored program computer or a von Neumann architecture.
That also doesn't make much sense to me. Money existed long before energy was a significant commodity. Like, thousands of years before.
Schneier on Security Shooting Down Drones
> Ask an LLM to generate you 100 more lines of code, no problem you will get something. Ask the same LLM to look at 10000 lines of code and intelligently remove 100... good luck with that!

These two tasks have a very different difficulty level though. It will be the same with a human coder. If you give me a new 10k sloc codebase and ask to add a method, to cover some new case I can probably do it in a hour to a day, depending on my familiarity with the language, subject matter, codebase overall state, documentation, etc.

New 10k codebase and a task of removing 100 lines? That's probably at least half a week to understand how it all works (disregarding simple cases like a hundred-line comment bloc with old code), before I can make such a change safely.
you do realize that these so called "one-off" AI programs would need to be maintained? Most people paying for Saas are paying for the support/maintenance rather than features, which AI can't handle. No one will want to replace any Saas they depend on with a poorly generated variant that they want to maintain
Not parent poster, but I imagine it will be a bit like the horror stories of companies (ab)using spreadsheets in lieu of a proper program or database: They will use an LLM to get half-working stuff "for free" and consider it a bargain, especially if the detectable failures can be spot-fixed by an intern doing data-entry.

I think we'll see it first in internal reporting tools, where the stakeholder tries to explain something very specific they want to see (logical or not) and when it's visibly wrong they can work around it privately.
>> > LLM’s never provide code that pass my sniff test

> This is ego speaking.

Consider this, 100% of AI training data is human-generated content.

Generally speaking, we apply the 90/10 rule to human generated content: 90% of (books, movies, tv shows, software applications, products available on Amazon) is not very good. 10% shines.

In software development, I would say it's more like 99 to 1 after working in the industry professionally for over 25 years.

How do I divorce this from my personal ego? It's easy to apply objective criteria:

- Is the intent of code easy to understand?

- Are the "moving pieces" isolated, such that you can change the implementation of one with minimal risk of altering the others by mistake?

- Is the solution in code a simple one relative to alternatives?

The majority of human produced code does not pass the above sniff test. Most of my job, as a Principal on a platform team, is cleaning up other peoples' messes and training them how to make less of a mess in the future.

If the majority of human generated content fails to follow basic engineering practices that are employed in other engineering disciplines (i.e: it never ceases to amaze me how much of an uphill battle it is just to get some SWEs just to break down their work into small, single responsibility, easily testable and reusable "modules") then we can't logically expect any better from LLMs because this is what they're being trained on.

And we are VERY far off from LLMs that can weigh the merits of different approaches within the context of the overall business requirements and choose which one makes the most sense for the problem at hand, as opposed to just "what's the most common answer to this question?"

LLMs today are a type of magic trick. You give it a whole bunch of 1s and 0s so that you can input some new 1s and 0s and it can use some fancy proability maths to predict "based on the previous 1s and 0s, what are the statistically most likely next 1s and 0s to follow from the input?"

That is useful, and the result can be shockingly impressive depending on what you're trying to do. But the limitations are so limited that the prospect of replacing an entire high-skilled profession with that magic trick is kind of a joke.
Since you're calling out your support for underserved models, can I request you support some SOTA embeddings models? Support for embeddings is poor from other providers with only a handful of outdated models and poor latency.
Skying too! Way fewer taps required.
Makefile in Lisp
I understand and appreciate your clarification. However would it not be the case some synthetic data strategies, if misapplied, can resemble the feedback loop scenario and thus risk model collapse?
How good is GitHub Copilot at generating Playwright code? [video]
This is why spicy honey is a great addition to pizza, esp. peperoni.
Laws are meant to be dynamic. So you iterate on them as you get feedback from their implementation
You seem to readily agree that my use case is inappropriate for LLMs, but not ToucanLoucan?
> LLMs are useful for coding now

*sort of, sometimes, with simple enough problems with sufficiently little context, for code that can be easily tested, and for which sufficient examples exist in the training data.

I mean hey, two years after being promised AGI was literally here, LLMs are almost as useful as traditional static analysis tools!

I guess you could have them generate comments for you based on the code as long as you're happy to proofread and correct them when they're wrong.

Remember when CPUs were obsolete after three years? GPT has shown zero improvement in its ability to generate novel content since it was first released as GPT2 almost ten years ago! I would know because I spent countless hours playing with that model.
> If this is "just another tool" then my question is: does the output of someone who has used this tool for one thousand hours display a meaningful difference in quality to someone who just picked it up?

Yes, absolutely. Not necessarily in apparent execution without knowledge of intent (though, often, there, too), but in the scope of meaningful choices that fhey can make and reflect with the tools, yes.

This is probably even more pronounced with use of open models than the exclusively hosted ones, because more choices and controls are exposed to the user (with the right toolchain) than with most exclusively-hosted models.
If anything, they may regress due to being trained on lower-quality input.
I don't see why, unless you think they're lying and they filmed their demos, or used some other preexisting model. I didn't ignore the JWST launch just because I haven't been granted to ability to use the telescope.
As the person who wrote the original reply I agree, and can ensure you that I have no skin in promoting a product - my intention is to promote the outcome.

If someone makes this happen with custom rfid chip I can tape to my fridge I’ll be happy enough.
Lifelong Texan, and the attitudes here are really tiresome. After the fall, the neoneolithic tribes will worship whataburger and buccees.
I have both machine-made and hand-knit sweaters. In general, I expect handmade clothes to be more expensive than machine-made, which kinda proves my point. I never said machine-made things had zero value. I said we will tend to devalue them relative to more human-intensive things.

Asking for directions is a bad example, because it takes very little time for both humans and machines to give you directions. Therefore it would be highly unusual for anyone to pay for this service (LOL)
My conspiracy theory is that layout is broken on purpose, to filter the audience.
As an aside, the UK is a great tourist destination, especially if you leave London right after landing.

Beautiful landscape, the best breakfast around, really nice people, tons of sights to see.
>A hopelessly out of touch political class

Orwell pointed this out in England your England which was written during the Blitz. Many of the problems he described have only got worse in the decades since he wrote about them in my opinion. While the essay is a bit dated now (it predates the post-war era of globalisation for example which created new axes in UK politics) I still think it's essential background reading for people who want to know what's wrong with the UK, and it's an excellent example of political writing in general.
the hype bubble is nearing it's top
The affordable housing shortage is reshaping parts of rural America
> go out of business as their customers start to build more of their internal systems with LLMs vs. buy from an existing vendor.

there is going to be so much money to make as a consultant fixing these setups, I can't wait!
The most you can get with that kind of "extrapolation" is some grounds for suspicion that something similar might be happening in the current day.

Meanwhile, my concern is with what is actually happening, you know, in current physical reality. Based in turn on some form of observable evidence. Not with what "might" be happening.

Which for some reason, seems to thoroughly flummox people in the "multipolar" crowd. It's like they've bought into this idea that events 40+ years ago establish a magical "pattern" that overrides any need to gather and ascertain the actual truth in the current moment (for example, in terms of what is happening in Syria).

So when we ask them why they believe US is, in actual fact, doing a certain bad thing (like causing regime change, say) in country X, despite no visible evidence of such -- it's become quite common for them to simply point out past shenanigans in other countries, and the resulting "pattern" that they see arising from such, as their "basis" for this belief.

"Iraq/Vietnam/Chile/etc, therefore everything the US does is also bad, therefore whatever is happening in country X today must be simply the US up to its usual imperial shenanigans, and if you can't see this you must be willfully blind or worse" is the apparent line of thinking here.

Which as a way of looking at things I find to be, well, rather weird.

Do ya wanna have some freedom fries with that?

No thanks, I've had enough junk food for today.
"glad that EU overregulation doesn't hamper the freedom of the United kingdom any more."

what can we do about this creep up of totalitarian surveillance plutocracy?

sweet were the 1990s with a dream.of.information access for all.

little did we know we were the information being accessed.

srry

very un-HN-y.. maybe it's just the time of the year but this really pulls me down currently.
I don't know why they say the model understands physics when it makes mistakes like that still.
Related:

All about Modelica: An equation-based language for modeling physical systems - https://news.ycombinator.com/item?id=23690788 - June 2020 (44 comments)

Modelica - https://news.ycombinator.com/item?id=16013179 - Dec 2017 (12 comments)
+1 to this sentiment for now, I give them a try every 6 months or so to see how they advance. And for pure code generation, for my workflow, I don't find them very useful yet. For parsing large sets of documentation though, not bad. They haven't creeped their way into my usual research loop just yet, but I could see that becoming a thing.

I do hear some of my junior colleagues use them now and again, and gain some value there. And if llm's can help get people up to speed faster that'd be a good thing. Assuming we continue to make the effort to understand the output.

But yeah, agree, I raise my eyebrow from time to time, but I don't see anything jaw dropping yet. Right now they just feel like surrogate googler's.
How do you do the indexing? Could you share how you do it?
Sure, they're the wrong tool for drawing a pelican - but testing their SVG output is a useful way to get a feel for how good they are at step by step reasoning, coordinate systems, spatial awareness and generating valid SVG/XML.

There are genuinely useful applications of SVG-generation from LLMs - outputting simple infographics or charts for example.

I use LLMs to write HTML all the time, of which SVG is a useful optional component.
Can you give some more specific examples? I am currently learning Julia…
So, just like everyone else with resources and political opinions.

(Endorsing and funding challengers has been a standard political tactic for decades.)
At least we're not being read to from a book called Ulysses?
From that link:

If your CCTV system captures images of people outside the boundary of your private domestic property...

Most European countries have laws for recording spaces not your own. They typically predate the GDPR by decades. AFAIK, they are not harmonized, except for a tiny bit by the GDPR.

If I understand it well, this is a big difference between the USA where you can mostly record the public space and create databases of what everyone does in public. IN Europe (even outside the EU), there is a basic expectation of privacy even in public spaces. You are allowed to make short term recordings, do journalism, and have random people accidentally wander in and out of your recording. Explicitly targetting specific people or long-term recording is somewhere between frowned upon to flat out illegal.
It depends on whether you think they are a paradigm change (at the very least) or not. If you don't then either you will be right or you will be toast.

For those of us who do think this is a revolution, you have two options:

1. Embrace it.

2. Find another career, presumably in the trades or other hands-on vocations where AI ingress will lag behind for a while.

To embrace it you need to research the LLM landscape as it pertains to our craft and work out what interests you and where you might best be able to surf the new wave, it is rapidly moving and growing.

The key thing (as it ever was) is to build real world projects mastering LLM tools as you would an IDE or language; keep on top of the key players, concepts and changes; and use your soft skills to help open-eyed others follow the same path.
The point isn't that it can't be done, EV conversions have been done for decades at this point, but that you'll get a car that's a whale compared to the original.
I'd love to see something like that as well.

One purchase we made 25ish years ago was some Global knives[1]. They have held up extraordinarily well and if somebody stole ours, I'd buy them again the next day. I think they look great, they feel really good, and they are very nice to use. They might be my favorite thing in the kitchen.

I don't know where our flatware is from. It doesn't have any branding on it, but the pieces are all pretty heavy. IMHO, a little heft feels pretty good.

If I had to buy new bowls and plates today, I'd probably get them from a smaller maker. I learned about East Fork Pottery[2] in North Carolina after they were hit hard by hurricane Helene and have been contemplating an upgrade ever since. I like their stuff, I like how it's made in the USA. Some of it is pretty expensive, but with care it should last a long time and it looks like it would be nice to use and live with.

[1]: https://www.globalcutleryusa.com/knives/collections/classic-...

[2]: https://www.eastfork.com/