MurphyA/hacker-news huggingface dataset:
Ask HN: How to Extract Shell Commands from Raw PTY Sessions? (Rewindtty)
by debba | 2025-08-04 13:41:43+00:00
Project URL: https://github.com/debba/rewindtty

I've been working on rewindtty, a lightweight terminal session recorder and replayer written in C. It works like script/scriptreplay, but outputs structured JSON and includes a browser-based player for replaying terminal sessions with timing, scrubbing, bookmarks, and more.

Until now, I was recording sessions command-by-command, capturing each shell command and its output separately. That made it easy to analyze sessions and index them by command.

However, I just introduced a new interactive mode, which behaves more like traditional script: it records raw terminal I/O in real-time via a PTY, capturing every character typed or displayed, including control sequences.

This is great for realism and full session fidelity (e.g. interactive tools like htop, vim, REPLs), but it makes command detection much harder — I'm no longer intercepting input at the shell level.

My question is: how can I extract actual commands from this raw PTY stream?

I'm aware it's tricky, but I'm wondering:

Has anyone tried parsing the ANSI stream to reconstruct command boundaries? Is it possible to hook into the shell (bash, zsh, etc.) in real-time to intercept commands? Are there shell options or audit features that can be leveraged in parallel to raw capture? Any prior art or libraries I should look at?

I'd love to hear how others have approached this — either for recording, analyzing, or replaying shell sessions. Any insights or directions would be super helpful.
by mpyne | 2025-08-04 13:42:21+00:00
As another example, the post exchanges / Navy exchanges on base (which are retail shopping centers) also contribute from their profits to the local MWR funds. They even advertise it to the shoppers. "Hey, the money we make stays with you all, not some random PE investors"

But no one is confused that Big Morale is somehow pulling the strings behind the scenes to force people to shop at the exchange.

In fact, in general the military can't really "make money", to the extent they manage to run activities involving cash receipts with military personnel, it becomes a chore to figure out something productive to do with that spending. MWR is a very minor activity on most bases but it is something that can absorb money on servicemembers' behalf so it often ends up in this role.
by jmull | 2025-08-04 13:42:34+00:00
HTMX is the same approach as old-school server-side templating, except you can replace parts of the page with HTML returned from the server, rather than always the whole page.

This allows for some nicely responsive interactions, but introduces complexity.

I'm not the previous poster, but it's a fair question whether the maybe faster responses justify the complexity. In many cases it probably would not.

(Actually, I suspect it's rare; if you know how to make partial page responses fast for HTMX, you know how to make full page responses fast and don't necessarily need HTMX, up until your page just gets too large overall.)

The general problem with HTMX is that, by default, the page state, as a function of the initial page plus the accumulated user's interactions with the page, live only in the user's browser tab. This can seem fine for a while, but opens up some fairly fat edge cases (this article covers some of them). There are ways to handle this, but it's additional complexity and work. Maybe someone has or will create an HTMX-friendly server-side templating framework to take the grunt work out of it, but you still have to wonder if one of the numerous existing full page templating mechanisms might not still be superior, overall.
by xcrunner529 | 2025-08-04 13:43:09+00:00
It’s interesting how I don’t hear the same fearmongering over “price controls” when Kamala dared mention a plan to stop grocery store price gouging from the same people when Trump demanded drug companies lower their prices [1].

[1]: https://www.ft.com/content/5e935b63-5a58-4feb-859b-e50cbfddc...
by tovej | 2025-08-04 13:43:29+00:00
This paper centers "novelty" but also finds that human ideas are more feasible, and that LLM-generated ideas are not diverse and that LLMs cannot reliably evaluate ideas. None of the ideas were actually evaluated by performing experiments either.

Pretty much what I would expect. The paper also seems to be doing exactly what I described, I don't understand how the technique is better than that?
by rayiner | 2025-08-04 13:43:52+00:00
Sorry, this is Orwellian doublespeak. I don't know exactly what "democratic institutions" you're referring to, but you seem to be referring to administrative agencies and adjuncts that are the exact opposite of "democratic institutions." They're anti-democratic checks that are permanently in the control of one party, regardless of who wins elections.

You mention "checks and balances" but which ones are you referring to? All three branches of government are controlled by the same party. Perhaps you can clarify if I'm mistaken, but you seem to be referring to anti-democratic putative "checks" within the executive branch. Those are nowhere in the constitution.

What's the big news right now? Republicans defunding NPR, which spent the last five years calling republicans and white people "racist." Sorry, that's democracy in action!

> In practice, one party dismantling democratic institutions and checks and balances, or stacking the courts, or accepting bribes in public, or drawing districts in a way to benefit them are normal,

California's "independent redistricting commission" drew a map where republicans have 17% of the seats despite getting 40% of the vote. That's worse than Maryland's quite deliberately gerrymandered map, where republicans got 16% of seats despite getting 35% of the vote. "Independent" redistricting commissions get taken over by democrats in practice, like every other putatively non-partisan political body.
by MOARDONGZPLZ | 2025-08-04 13:44:17+00:00
Just kind of summarizing from the comments in this topic (I also couldn’t be bothered to Google Palantir’s thesis): sticky/useful tools to combine and enrich data sources with a focus on a sector where there is a lot of compliance-driven security, data sources that aren’t easily queryable outside of their direct users, and high barriers to entry.
by philipallstar | 2025-08-04 13:44:27+00:00
> Only through class consciousness can we build a new society in which the fruits of labor are enjoyed by us all.

They've done this a few times already, and it's very violent each time.

And this isn't an inherent contradiction of capidalism. Capitalism only makes people rich if they provide loads of value to others. Lower prices are good for every one. Raising everyone's wages out of proportion to the value they create only raises all the prices.
by mtlynch | 2025-08-04 13:44:32+00:00
I'm arguing for reading the doc ahead of time, identifying topics about the doc to discuss, then coming to the meeting with a specific agenda.

>Ok, but without a doc, collaboration in a meeting can become inefficient in many ways. Folks can talk past each other, bouncing between unclear options and losing clarity on what they are even debating. Only one person can talk at a time, so folks are sitting and waiting for their turn, etc.; Strong personalities can dominate and filibuster.

I don't understand. These are just problems with meetings in general no matter what you do in advance. What are you arguing for?

>While reading the doc, everyone can comment in parallel, so you aren’t serializing the conversation for small things or letting some verbose person waste the time.

>Reading during the meeting helps folks manage time outside the meeting. It means: you don’t have to prepare for meeting you don’t own. It avoids “random people” assigning you work beyond what they are asking you for in your calendar. For busy managers this is a huge benefit.

I don't understand how this can be possible. It means that letting someone else manage your time is more efficient than managing your own time.

If I need 5 minutes to read a document, how is it helpful for someone to force me to sit for 10 minutes to read it at a specific time? It's obviously more efficient for me to read it in the block that I choose and not waste the extra 5 minutes.

The Amazon way sounds like how you'd manage time for a child, where you can't trust them to manage their time, so you have to schedule time blocks for coloring, recess, and eating lunch.
by octo888 | 2025-08-04 13:44:59+00:00
Pre COVID, IKEA had a lot of decent value stuff (prices were low, relatively better built items too, relative to the other stock). There were also plenty of staff on the tills and on the shop floor to ask questions or get assistance.

You genuinely felt they passed on the savings

They also had decent online shopping.

These days though they're just like everyone else and have cut cut cut and prices have risen. The restaurant has gone to crap too
by projektfu | 2025-08-04 13:45:04+00:00
I don't think any of those uses a ligature. Ü, é and Þ are distinct characters in legacy latin-1 and in Unicode. It wouldn't surprise me if non-scandinavian websites do not like Þ, however.

It's probably not PDF's fault that parsers are choking on the ff ligature. Changing all those parsers isn't practical, and Adobe can't make that happen.

Finally, if you run based on metadata that isn't visible, you open up to a different kind of problem, where a visual inspection of the PDF is different from the parsed data. If I'm writing something to automatically classify PDFs from the wild, I want to use the visible data. A lot of tools (such as Paperless) will ocr a rasterized pdf to avoid these inconsistencies.
by spacebanana7 | 2025-08-04 13:45:27+00:00
I guess my point was that we have different OS's precisely because people want to do things in different ways. So we can't have generic ways to do them.

OS generic filesystem permissions would be like a OS generic UI framework, it's inherently very difficult and ultimately limited.

Separately, I totally sympathise with you that the OS solutions to networking and filesystem permissions are painful to work with. Even though I'm reasonably comfortable with rwx permissions, I'd never allow untrusted code on a machine which also had sensitive files on it. But I think we should fix this by coming up with better OS tooling, not by moving the problem to the app layer.
by lolive | 2025-08-04 13:45:35+00:00
I will comment only on the Foundry stack that we work extensively with, at my company. Given the complete havoc that the other IT ecoaystems has become, we are constantly struggling with proper data access data exchange data transformation and data alignment. To a point where a political layer has appeared on top of that, which looks like the middle age baronies.

Foundry has been in the company for the last ten years, and I will be frank: this is the only source of truth that I believe in the company. The integration of data, its lineage, its semantics, its consumption stack, the community who makes the enterpriseData work for real, all of that is simply much more efficient and sane than going for yet another war with data barons and IT (so-called) enterprise architects.

And now my personal comment on this: Foundry is definitely the vision I was expecting from the Linked Data initiative. And it is [stupidly expensive but] simply SOOOOO good !
by alias_neo | 2025-08-04 13:45:43+00:00
> Bluey is the best children's cartoon of this generation

Bluey is the best children's cartoon _for adults_ of this generation.

Our kids are about the same age and age difference as Bluey/Bingo are portrayed, and I can't express how much relief it gives to watch it see your own life played out line for line.

Some days, as a parent you wonder how (badly) you're doing, and Bluey takes some weight off as you realise (and talk to other parents about Bluey) that some things are just universal and you're possibly doing better than you think.
by egypturnash | 2025-08-04 13:46:16+00:00
Is a payment processor operated by the Federal Reserve good enough? https://en.wikipedia.org/wiki/FedNow

Well, it probably would be, except guess who killed it in favor of a crypto scheme? https://www.jitumaster.com/2025/06/us-president-signs-execut...

I agree about the PO though. Social media shouldn't be a for-profit enterprise either.
by deadbabe | 2025-08-04 13:46:19+00:00
If you’re on FANG compensation, and you earn roughly $200k after taxes in one year, and you spent it all of it on Starbucks coffee, you could buy roughly a century worth of coffee if you drink one a day.

If on the other hand, you spent the $200k on leasing an omg.lol domain in perpetuity, you could hold the domain for 10 millenniums.

If we were in the Dune universe, that means your omg.lol domain would expire roughly around the same time as the Butlerian Jihad starts and the thinking machines are overthrown.
by strken | 2025-08-04 13:46:36+00:00
Is this a hypothetical or is it based on real events? My dad has pieces of retired distillation apparatus from a plant that does chemical research and also manufactures codeine. Definitely the condenser, and maybe also the round-bottom flask. He's left them sitting above his TV for years, in plain view. My mum has a binocular microscope and a box of precision scales sitting near the kitchen table. Nobody has ever commented on any of this except to say how neat the glassblowing is, etc.

I'm sure some people do live in modern-day Salem equivalents, but I bet a lot more people think they do.
by user____name | 2025-08-04 13:46:44+00:00
> The attitude of young people have nothing to do with that.

“Whither are the manly vigour and athletic appearance of our forefathers flown? Can these be their legitimate heirs? Surely, no; a race of effeminate, self-admiring, emaciated fribbles can never have descended in a direct line from the heroes of Potiers and Agincourt…”

Letter in Town and Country magazine republished in Paris Fashion: A Cultural History 1771
by IX-103 | 2025-08-04 13:46:58+00:00
They have clear numbers for things, but it's not obvious how those numbers would map to what you're trying to run.

If I charged compute based on the number of micro-ops executed then that would be a clear definition, but the actual cost would not be something you could predict, as it would depend on what architecture of CPU you ended up running it on.

AWS is even more complicated and variable than that as for cloud storage you have to deal with not only the costs of the different storage classes, but also early deletion fees, access charges, etc. Combined it makes it impractical to work out how much deleting a file from cloud storage will save (or cost). Sure you could probably calculate it if you knew the entire billing history of the file and the bucket it is in, but do you really want to do that every time you delete a file?

While I don't know enough to say if this is intentional, as it could result from simply blindly optimizing for profit, this sort of pricing model is anti-capitalistic as it prevents consumers from truly making informed decisions. We see the same thing is the US healthcare system, where no one can actually tell you how much an operation will cost ahead of time. That creates a very inefficient (but very profitable) market.
by gchamonlive | 2025-08-04 13:47:14+00:00
Sure if AI could make small communities autonomous and provide everyone with everything they would ever need, there would be no need for money.

But we are far away from this utopia, this utopia will require a ton of energy to be produced just to run the AI supervision layer, so hopefully by then we'd have fusion energy or something else figures out, and to achieve this utopia there will be a transition period.

I am actually worried about the transition period in your fictional world. Some people will be replaced long before the deprecation of money. It's a lot of people that is going to suffer from extreme poverty if we don't think this right, which I believe is what the OP comment was about.
by jcgl | 2025-08-04 13:47:15+00:00
That is preposterous. Thanks for sharing. Really disappointing to hear that people who presumably care a lot about health and safety can be so blind on this stuff.

I wonder how successful it would be to have a hypothetical campaign like "Create a city-wide emergency roadway network." It would basically result in building out separated bike lanes across a city, with the explicit purpose of creating an expressway for emergency vehicles that can be used by cyclists when otherwise unused. Seems like a way to bring sides together and possibly get greater funding and scope.

That Toronto reality is depressing.
by aiaikzkdbx | 2025-08-04 13:47:23+00:00
> If this is the case, then two possibilities follow: first that its intentions are entirely benign and second they are malign

Even framing this objects actions using human concepts (benign, malign) is very short sighted. It’s possible any alien life experiences complexities were fundamentally unable to comprehend (there’s some good sci fi short stories that explore this).
by PaulHoule | 2025-08-04 13:47:41+00:00
You have choices, especially all the choices that 1999-style web applications have.

The shopping cart can be kept on the back end and referenced by an id stored in a cookie.

You can keep partially filled out forms in hidden form variables and can send them back in either GET or POST.

Not all requests require all the form data, for instance my RSS reader YOShInOn is HTMX based -- you can see two forms from it here:

https://mastodon.social/@UP8/114887102728039235

in the one at the upper left there is a main form where you can view one item and evaluate it which involves POSTing a form with hidden input fields but above that I can change the judgements of the past five items by just flipping one of the <selects> which needs to only submit the id and the judgement selected in the select. I guess on clicking one of the buttons in the bottom section I could redraw the the bottom section, insert a <select> row at the bottom of the list and the delete the one at the top, but it just redraws the whole form which is OK because I don't have 200k worth of open graph and other meta data in the <head> and endless <script> tags and any CSS other than bootstrap and maybe 5k of my own, which all caches properly.
by ralferoo | 2025-08-04 13:48:32+00:00
That'd be one way of doing it.

You don't technically need 4 reads per pixel either, for instance you can process a 7x7 group with a 64-count thread group. Each thread does 1 read, and then fetches the other 3 values from its neighbours and calculates the average. Then the 7x7 subset of the 8x8 write their values.

You could integrate this into the first pass too, but then there would be duplication on the overlapped areas of each block. Depending on the complexity of the first pass, it still might be more efficient to do that than an extra pass.

Knowing that it's only the edges that are shared between threads, you could expand the work of each thread to do multiple pixels so that each thread group covers more pixels the reduce the number of pixels sampled multiple times. How much you do this by depends on register pressure, it's probably not worth doing more than 4 pixels per thread but YMMV.
by MrBrobot | 2025-08-04 13:48:48+00:00
> Candidates tell Fortune that AI interviewers make them feel unappreciated to the point where they’d rather skip out on potential job opportunities, reasoning the company’s culture can’t be great if human bosses won’t make the time to interview them. But HR experts argue the opposite; since AI interviewers can help hiring managers save time in first-round calls, the humans have more time to have more meaningful conversations with applicants down the line. “This gives me a bad feeling about your company” “But you’re wrong”
by stego-tech | 2025-08-04 13:49:03+00:00
I came into the comments to see how the discourse is shaping up in light of the author’s claims, and I’m seeing the same old boosterism trying to dismiss it wholesale without providing conclusive evidence.

So without further ado:

* If LLMs can indeed produce wholly novel research independently without any external sources, then prove it. Cite sources, unlike the chatbot that told you it can do that thing. Show us actual results from said research or products that were made from it. We keep hearing these things exponentially increase the speed of research and development but nobody seemingly has said proof of this that’s uniquely specific to LLMs and didn’t rely on older, proven ML techniques or concepts.

* If generative AI really can output Disney quality at a fraction of the cost, prove it with clips. Show me AI output that can animate on 2s, 4s, and 1s in a single video and knows when to use any of the above for specific effects. Show me output that’s as immaculate as old Disney animation, or heck, even modern ToonBoom-like animation. Show me the tweens.

* Prove your arguments. Stop regurgitating hypeslop from CEBros, actually cite sources, share examples, demonstrate its value relative to humanity.

All people like us (myself and the author) have been politely asking for since this hype bubble inflated was for boosters to show actual evidence of their claims. Instead, we just get carefully curated sizzle reels and dense research papers making claims instead of actual, tangible evidence that we can then attempt to recreate for ourselves to validate the claims in question.

Stop insulting us and show some f*king proof, or go back to playing with LLMs until you can make them do the things you claim they can do.
by tossandthrow | 2025-08-04 13:49:07+00:00
It always gets a bit fascist to go down this path - but it is at least intellectually interesting.

One way to subsidize parents is to give free child care. As a bonus, the government get to decide exactly how these kids are being raised.

But yeah, it is a material investment for the society to take - which is likely why we have seen the cost of kids being transferred to the parents (and lower birth rates).
by TacticalCoder | 2025-08-04 13:49:12+00:00
Oh that's interesting: it uses two similarly shaped Git repos with one being transparently encrypted.

I take it that way you can deduplicate backups.

My own backup procedure uses Git too: but I don't deduplicate. For every Git repo my backup procedure (which runs from an OCI containers and mounts the Git repo volume read-only) does:

- git clone the entire repo - clean any mess - run git fsck (you never know you know) - tar + compress - encrypt - add a cryptographic sum to the unique resulting encrypted file - deduplicate (but this only works if zero change were made to the Git repo since the last backup) - send if not a duplicate - on the remote end: verify the cryptographic hash, decrypt, uncompress/untar, re-run git fsck (you never know you know)

This way I'm reasonably sure the backup is not corrupted.

FWIW the cryptographic hash allowed me to catch a bit flip, once.

So my files are encrypted too but my main issue is any change to a Git repo means a new full backup has to be made.

I can see how something automated like myba could help.