MurphyA/hacker-news huggingface dataset:
by clone1018 | 2025-05-25 21:05:53+00:00
Timely posting! I've been inspired by some recent... large gaps in data at work (silent analytics processing failures) to build a service called QueryCanary. It's a surprisingly powerful but simple tool that lets you define scheduled SQL checks to run against your database, and then checks those results for anomalies, variances, and other issues.

Really hoping to get some early feedback on this tool, I've been using it for two production sites for about a week now and I've already discovered (at work) that we've had the 2nd largest user signup day, and that we deployed a change that inaccurately tracked a specific metric. Check it out at https://querycanary.com
by mystified5016 | 2025-05-25 21:06:19+00:00
It's more profitable and predictable for scalpers to immediately buy all tickets. The ticket seller doesn't care if the tickets are sold to fans that will attend, just that they're sold quickly and reliably and non-refundably. It's even better if tickets are sold to scalpers because some of those tickets might never be resold, which means the venue gets the ticket sale but pays none of the cost a real guest would incur.

What matters is selling the ticket, getting a guest in the door is just expense.
by plankers | 2025-05-25 21:06:45+00:00
modeling the heat transfer modes in Enceladus' icy shell that rests above its liquid water ocean. previous modeling has assumed that all heat transfer is conductive, but using dynamical simulations i've shown that under certain conditions convection can occur at in the shell. specifically, these conditions are having a thick enough ice shell, the right amount of porous fluffy ice deposited from the plumes at Enceladus' south pole which jet water into space through fissures in the crust, and the right thermal conductivity of this porous layer.

now i'm starting on adjusting the model to include the liquid water ocean underneath the shell and observe the effect of changing viscosity gradients in the equilibration of the ocean and ice shell, as well as adding in compositional impurities (chloride brines) and tidal heating effects.
by zzo38computer | 2025-05-25 21:07:07+00:00
Consider DER format. Partial parsing is possible; you can easily ignore any part of the file that you do not care about, since the framing is consistent. Additionally, it works like the "chunked" formats mentioned in the article, and one of the bits of the header indicates whether it includes other chunks or includes data. (Furthermore, I made up a text-based format called TER which is intended to be converted to DER. TER is not intended to be used directly; it is only intended to be converted to DER for then use in other programs. I had also made up some additional data types, and one of these (called ASN1_IDENTIFIED_DATA) can be used for identifying the format of a file (which might conform to multiple formats, and it allows this too).)

I dislike JSON and some other modern formats (even binary formats); they often are just not as good in my opinion. One problem is they tend to insist on using Unicode, and/or on other things (e.g. 32-bit integers where you might need 64-bits). When using a text-based format where binary would do better, it can also be inefficient especially if binary data is included within the text as well, especially if the format does not indicate that it is meant to represent binary data.

However, even if you use an existing format, you should avoid using the existing format badly; using existing formats badly seems to be common. There is also the issue of if the existing format is actually good or not; many formats are not good, for various reasons (some of which I mentioned above, but there are others, depending on the application).

About target hardware, not all software is intended for a specific target hardware, although some is.

For compression, another consideration is: there are general compression schemes as well as being able to make up a compression scheme that is specific for the kind of data that is being compressed.

They also mention file names. However, this can also depend on the target system; e.g. for DOS files you will need to be limited to three characters after the dot. Also, some programs would not need to care about file names in some or all cases (many programs I write don't care about file names).
by cyberax | 2025-05-25 21:07:21+00:00
It'll keep getting worse, as more and more people concentrate into large cities. And large cities are family-hostile. The birth rate in cities is strongly anti-correlated with the city size. People just don't feel content and happy to have children in dense cities.

Japan is very illustrative in this regard. Young people are forced into unaffordable cities, where they often live in cramped shoebox-sized apartments. While beautiful spacious traditional homes are decaying just 2 hours away by car.

What can help? Promote smaller cities, by focusing on making smaller cities more attractive for employers. This is already happening with the remote work, and it helps: https://eig.org/families-exodus/

How to make smaller cities more attractive? Put a stop to densification of larger cities, and tax (or cap-and-trade) the office space in dense areas.
by crazygringo | 2025-05-25 21:08:30+00:00
Working with protobuf has been a huge pain for me, personally -- sites that make API data available only in older versions of protobuf where there's no available library to decode in a popular language. JSON and zip are easy, universal, accessible standards in ways that protobuf simply isn't.

So that's why I'm saying, there's really something to be said for zipped JSON. You point out that you "usually need to decompress the whole thing in memory", and that's precisely what most of my comment was about -- handling it efficiently so you don't.

And it's not like protobuf is inherently any better at that anyways -- if you want to access a record in the middle of the file, you've got to stream the whole file up to that point to get to at it. It doesn't support random access in any native way.

So I'm kind of bummed out my original comment is being downvoted -- I'm making an actual serious proposal in it. I think zipped JSON should actually be taken seriously as a file format in its own right, especially with much more memory-efficient decoding.
by jopsen | 2025-05-25 21:08:33+00:00
You absolutely don't need to work 37 hours per week if you are willing to live 200km from the big cities in an old house.

You buy something dirt cheap for less than 80k USD. It's not great, it's old run down. But if you do minimal repairs and don't live the high life, you can get by with very little.

But if you want a new car and nice comfy house in a decent location, with all the modern amenities, well, that's expensive.
by tamacun | 2025-05-25 21:08:49+00:00
I think what this experiment has done is get me excited about bringing my ideas to life again. I always fall deep into the Dunning Krueger curve and usually let my interest die when I realize that things are infinitely harder and more complex/nuanced than I initially noticed.

But now it feels like a lot of technical complexity that has stood in my way is just not there (at least on this scale, I still fully acknowledge professional level software development is takes enormous skill, persistence and knowledge).

My aim is to take all the ideas for apps I've had over the years, and try to make one new one per week. We'll see if I still feel this way in a few weeks.
by cmiles74 | 2025-05-25 21:09:12+00:00
Firstly, can't speak for Chomsky.

In this article he is very focused on science and works hard to delineate science (research? deriving new facts?) from engineering (clearly product oriented). In his opinion ChatGPT falls on the engineering side of this line: it's a product of engineering, OpenAI is concentrating on marketing. For sure there was much science involved but the thing we have access to is a product.

IMHO Chomsky is asking: while ChatGPT is a fascinating product, what is it teaching us about language? How is it advancing our knowledge of language? I think Chomsky is saying "not much."

Someone else mentioned embeddings and the relationship between words that they reveal. Indeed, this could be a worthy area of further research. You'd think it would be a real boon when comparing languages. Unfortunately the interviewer didn't ask Chomsky about this.
by Etheryte | 2025-05-25 21:09:23+00:00
Whenever speech to text apps come up, I get curious how people use them in their workflow. I've tried to integrate it into my daily work a few times, but have found myself dropping it not too long after. If I'm already at a keyboard, I just don't seem to find any case where I don't prefer that as an input. What are other people using these for?
by bishopsmother | 2025-05-25 21:09:45+00:00
Expanding the functionality of Wabbit S2[0], e.g. Sesame AI[1], and improving existing features based on feedback over the last few weeks of Mabel's testing.

[0] https://blog.walledgarden.ai/2025-05-20/wabbit-s2-welcome-to...

[1] https://www.sesame.com/research/crossing_the_uncanny_valley_...
by ghaff | 2025-05-25 21:10:08+00:00
I'd probably argue that most people don't have (personal) connections starting out. Maybe, if they didn't go to the right school or school at all, someone they know is the foot in the door.

>but in time you can and will build a reputation for your quality of work, interactions with (and then capability to manage) others, and achievement of results.

That IS their network for a lot of people. OK maybe there are smaller companies that are 50% $SCHOOL grads. And there are other companies that tend to bias towards a certain group of schools. But I actually think that going to, say, Harvard is a secret handshake is overstated in a lot of cases.

I sort of suspect that my undergrad may have had something to do with a job at one point but the fact that I got in through a senior person who liked my work played a lot bigger role.
by acenturyandabit | 2025-05-25 21:10:52+00:00
Love the aesthetic! Also your handwriting input is super smooth, amazing!

I've been building something similar for Chinese, just for myself: https://hazel.daijin.dev/ It's got PWA, let me know if you want my presets for working with PWA with Vite.

Will definitely be taking a few pages out of your (app) when I get a chance!
by throwaway314155 | 2025-05-25 21:11:20+00:00
The only negative comment that is perhaps overly negative is the one mentioning "killing babies" (but they make a good point still).

Everyone else's criticism is effectively entirely warranted even if it's just a hobby project if only because the criticisms are mostly pretty mild and reasonable things like "don't try to automate parenting with tech that still isn't ready yet".
by 317070 | 2025-05-25 21:11:28+00:00
But relatively few countries follow common law with its focus on precedents. In fact, civil law is a lot more common, and that one cares less about precedents. [0]

> The primary contrast between the two systems is the role of written decisions and precedent as a source of law (one of the defining features of common law legal systems). While Common law systems place great weight on precedent, civil law judges tend to give less weight to judicial precedent. For example, the Napoleonic Code expressly forbade French judges to pronounce general principles of law.

[0] https://en.m.wikipedia.org/wiki/Common_law#/media/File%3AMap...
by frainfreeze | 2025-05-25 21:11:39+00:00
I remember seeing the comment you linked few years back, and back then comments were already locked so I couldn't reply, and this time I sadly don't have the time to get deeper into this, however - I recommend you to research more about sqlar/using sqlite db as _file format_ in general, or at minimum looking at SQLite Encryption Extension (SEE) (https://www.sqlite.org/see/doc/trunk/www/readme.wiki). You can get a lot out of the box with very little investment. IMHO sqlar is not competing with ZIP (can zip do metadata and transactions?)
by waciuma | 2025-05-25 21:11:42+00:00
If you're an experienced engineer that wants to give back to learners, OSSU is a great place to do so. This can look like:

- Setting a regular time that you'll pair (or mob!) program on a side project of your own with OSSU learners. - Developing familiarity with one or more courses in the curriculum and responding to students who have questions or are stuck. - Attending weekly check-in meetings, sharing what you are working on and listening to what learners are working on.

To do so - Visit our Discord server: https://discord.gg/wuytwK5s9h - And ping me @waciuma or the @tutor role

I'm one of the leaders of OSSU and we agree that community, networking, and projects are part of a complete education. That's why we celebrate not only the professors and universities creating free courses, but also the many engineers and practitioners that have volunteered with OSSU learners over the years. I hope some of you will join that group!
by neilv | 2025-05-25 21:11:53+00:00
I think Google has ethics in their self-image. Albeit more flexibility in practice.

But decisions might've come down to what keeps the ad revenue machine fed. Or it might not have made it to decisions.

If they were slow, wasn't it only in being prepared for when the boat got rocked? Until the boat got rocked, they had a monopoly, so why rock it themselves?

I wouldn't write in my promotion packet, "Disrupted our golden cash cow, and put our dominance and entire company at risk, before someone else did, because it would happen eventually, you'll thank me later, pls promo kthx."
by fake-name | 2025-05-25 21:11:54+00:00
I have a rather ideosyncratic opinion here:

For Open-Source projects, human readable file formats are actively harmful.

This mostly is motivated by my experience with KiCad. Principally, there are multiple things that the UI does not expose at all (slots in PCB footprint files) where the only way to add them is to manually edit the footprint file in a text editor.

There are some other similar annoyances in the same vein.

Basically, human readable (and therefore editable) file formats wind up being a way for some things to never be exposed thru the UI. This actively leads to the software being less capable.
by robocat | 2025-05-25 21:12:06+00:00
Humidity in London is also awful as the temperature gets closer to freezing. I found the damp cold in London to be common over the year and truly horrid (a reason to never live there).

At home (Christchurch, NZ) we often get dry cold which can be pleasant: however when we do get the occasional vile damp cold I personally call it "London cold" because it made so much impression upon me in my 20s.
by ars | 2025-05-25 21:12:17+00:00
> The centigrade number is how far you are from water freezing

The Fahrenheit scale is how far your are from your own body temperature. It was designed that 100 is the temperature of a human. (Adjusted later to 98.6 due to inaccuracies.)

0 was designed to be as cold as you can get with ice and salt (also ended up being slightly inaccurate).

> Maybe because I was brought up with centigrade it makes more sense to me.

Yup. People brought up on Fahrenheit think it is superior. For temperature neither argument is objectively better. (In contrast to imperial distance measurement with non-powers of 10 and factions, where there are good arguments against it, with temperature both scales are ultimately arbitrary.)
by tkzed49 | 2025-05-25 21:12:53+00:00
Thanks, it's an interesting perspective. I may have underappreciated the code volume/time requirements to execute especially in a startup setting.

I think I see where you're coming from. I'd imagine in some applications, there is very niche/specific implementation on Github that is both expensive to reimplement and not permissively licensed. I can see the "AI laundering" angle there.
by nextn | 2025-05-25 21:13:03+00:00
An option I'd like to see implemented is to make the customer put down a bond. Besides charging the customer for the ticket, also charge them another higher amount that gets refunded x days after the concert. If the customer is found to be a scalper don't return the bond.

Scalpers can't pay high bond amounts at scale combined with the risk of not having the bond returned.
by jand | 2025-05-25 21:13:47+00:00
The community edition is not robbed of its value by this move. They provide a CLI tool (mc) for those admin tasks which previously could be solved via dashboard.

I have several minio instances deployed to k8s for small to medium, and non-profit projects. Easy to deploy, no problems or outages, yet.

But anecdotally i remember multiple occasions, where a quick tour of the dashboard convinced peers, that minio was the right tool for the job.

From my point of view it is much more questionable, that they "dare" to advertise the paid version with a 96.000 USD p.a. "platform fee" plus additional cost if you use more than 400TB. Small fish need tools, too.
by roughly | 2025-05-25 21:13:56+00:00
> But what exactly is understanding?

He alludes to quite a bit here - impossible languages, intrinsic rules that don’t actually express in the language, etc - that leads me to believe there’s a pretty specific sense by which he means “understanding,” and I’d expect there’s a decent literature in linguistics covering what he’s referring to. If it’s a topic of interest to you, chasing down some of those leads might be a good start.

(I’ll note as several others have here too that most of his language seems to be using specific linguistics terms of art - “language” for “human language” is a big tell, as is the focus on understanding the mechanisms of language and how humans understand and generate languages - I’m not sure the critique here is specifically around LLMs, but more around their ability to teach us things about how humans understand language.)
by Scubabear68 | 2025-05-25 21:14:46+00:00
I am most likely ADHD, probably in the spectrum to some degree.

Tried college three times and dropped out every time due to expense, boredom, and personal issues like my father passing away from cancer when I was 21.

I went into software development via tech support for a C compiler company, and worked up from there.

Worked for the NY stock exchange, two top tier brokerages, several prominent Fintechs and ultimately consulting into banks and payments companies.

It worked for me because I am largely an auto didact and do terribly in a school environment.

The lack of degree came up only a few times, and no one has cared.

At least in software development careers, degree matters very little to not at all.
by ramoz | 2025-05-25 21:15:09+00:00
Context and “memory” (not really a fan of this term and how industry uses it) are actually complex to manage for power users including humans and agents.

While it may sound counterintuitive, the agents of today aren’t truly autonomous in that you need to really guide them and plan their actions well.

I believe this is true today, and will be even more true when agents are guiding agents.

We need new infrastructure for dynamic context management.

The answer is not as simple as “hook up your agent to an MCP that pull docs from the web” … also MCP needs its own revolution. I tend to use no MCP and prefer raw agent performance.

I’m evolving the simple concepts I built in my VS Code extension to address this. Nothing public now, but I and a few others use this everyday to feed parts of large codebases into Gemini (to build plans for Claude code, other coding agents): https://github.com/backnotprop/prompt-tower
by RobinL | 2025-05-25 21:15:26+00:00
I've been experimenting with whether I can use LLMs to produce interactive maths explainers for kids. There are a few examples here: https://rupertlinacre.com/

Unless I'm missing something, it's amazing how few free, _high quality_ materials are online.

Ultimately I'm interested in two things: genuinely fun games that make you do some maths, and quality visualisations that help make concepts easier to learn
by quesera | 2025-05-25 21:15:49+00:00
I once worked at a place where the marketing team explicitly removed dates from corporate blog posts for SEO purposes.

The idea was that some content is, or benefits from being perceived as, "evergreen". Always relevant.

Maybe -- but it's still deceptive, I think.

On the other hand, even here on HN we see people talking about "unmaintained" GitHub repos where the last commit was more than a few months ago. So, recency-bias is a real thing, and marketers certainly don't want to be penalized for honesty. :)

This was a while ago. Nowadays, the obvious opposite extreme is common. Blogspam that is "updated" with a current date, but no changes to content.
by zer00eyz | 2025-05-25 21:16:19+00:00
> due to the end of ZIRP and AI productivity gains.

I think you're missing the mark with this analysis.

If you go back to the original dot com bubble it was as much of a hardware bubble as a software one. Same thing with the mobile bubble. The AI bubble we are in has NOTHING to do with productivity and everything to do with hardware. I, as a software engineer am not going to come up with a product that can compete with any of the major players without a massive capital investment.

Meanwhile, the price to play as a software engineer is also driven by high costs. AWS, for better or worse is the model and the go to, and it is NOT cheap by any measure. Its pricing model looks more like the iPhone and less like an efficient market. AWS is MOST of amazons profit margins. It makes tech companies more like franchisee renting the location for their fast food joint and less like independent entities.

The thing is there are TONS of gaps in the software marketplace that need filled. These are companies that are going to be in the 2-3 million a year range and capable of being run by a small team (think ~5 people). Nothing that would appeal to the ycombinator set. You don't need Kubernetes, Kafka, or high performance bleeding edge Rust or massive Autoscaling to run these services. They are never going to get huge, and in fact they offer enough room to start another company of the same scale if one is ambitious and wants to diversify.

Does your average 18 year old know this? No, because most people who write code for a living don't seem to know where these gaps are. Do the math on what it takes to make 100k a year at 10 bucks a month... add a zero for a million, multiply by 3 for "small team"... The number is shockingly small.

Does your average 19 year old have the chops to figure this out? No, because 20 and 30 something laid off software engineers can't seem to figure it out either, even ones with "top degrees".

That doesn't mean that there isn't a path for the sharp young kid to "skip school" and go directly into industry. That path is open source. A history of strong contributions and being smart is going to build a better network than any CS degree ever would/will... However if you can do both, open source and a degree (from anyplace) you're even better off! The same could be said for working at Fedex, Walmart or Costco while you get a cs degree from anyplace and seeking a job in a corporate office after. You have a set of experiences that make you invaluable as a contributor.

Lastly, no one talks about the bad guys. There are plenty of scammers and thieves abusing technical skills who lack formal education and do well for themselves. If we're going to remove all the options and only have a narrow path, will we end up with more criminals and fewer contributors? This is sort of why "Russian hackers" is one of the givens in the industry (crime did/does pay well).

I still think software engineering is a good career choice for a smart kid, but you have to bring more to the table than just code if you want to prosper!
by RhysU | 2025-05-25 21:16:27+00:00
When there's a stable supply of food, medicine, and housing people reproduce. More aggregate people implies more aggregate demand.

Suppose only 5% of the population breeds like rabbits in the presence of food, medicine, and housing. It won't take long for that cultural subpopulation to dominate.

Humans are not unlike microorganisms in a petri dish when one adds nutrients. All life is.