MurphyA/hacker-news huggingface dataset:
by squigz | 2025-06-14 13:00:32+00:00
> I agree would be a great thing to be able to do but seems to be a much further along than the words cure or prevent would indicate.

If the words "cure" or "prevent" don't indicate that, why did you bring it up? :)

I think this is being needlessly pedantic. Keeping in mind that one of HN's guidelines is to respond to the strongest plausible interpretation of a comment, I think we can assume that GP meant, essentially, "learn enough about these diseases so as to detect and prevent them from causing irreparable damage"
by gyomu | 2025-06-14 13:01:08+00:00
> They're not all-perfect gods. In reality, most successful people are one trick ponies. They caught lightning in a bottle once early on that boosted their careers but that doesn't mean they're still relevant and correct with their decision making today.

I don't think that characterization is quite right either. I'm a big fan of Brian Eno's "scenius" phrasing:

> A few years ago I came up with a new word. I was fed up with the old art-history idea of genius - the notion that gifted individuals turn up out of nowhere and light the way for all the rest of us dummies to follow. I became (and still am) more and more convinced that the important changes in cultural history were actually the product of very large numbers of people and circumstances conspiring to make something new. I call this ‘scenius’ - it means ‘the intelligence and intuition of a whole cultural scene’.

Extremely successful people benefit from the scenius within which they get to operate. But as that context changes and evolves over time, they fail to recreate their earlier wild successes - not because they lost any of their skills (although that can also happen), but because the skills aren't sufficient, and the deep, layered conditions that enabled those wild successes just aren't there anymore.
by echelon | 2025-06-14 13:01:13+00:00
It's 2005 again!

These tools were pretty cool and an enormous amount of work was put into them.

The ontologies were extremely extensible. There just wasn't enough of an ecosystem putting them into practice and demonstrating their utility.

Their examples are nice:

https://github.com/Netflix-Skunkworks/uda/blob/9627a97fcd972...
by fidotron | 2025-06-14 13:04:15+00:00
> Sorry, I meant that was unit testing.

So do we. If you change the schema it doesn't matter if it's unit testing, you either have a validation stage for all data coming in beforehand to check these assumptions (the way many python people use pydantic on incoming data, for example), and that must be updated for every change, or every unit must be defensive against this sort of thing.
by MyPasswordSucks | 2025-06-14 13:04:20+00:00
The Son of Sam claimed his neighbor's dog was telling him to kill - better demand dog breeders do something vague and unspecified that (if actually implementable in the first place) would invariably make dogs less valuable for the 99% of humanity that isn't having a psychotic break!

Articles like this seem far more driven by mediocre content-churners' fear of job replacement at the hands of LLMs than by any sort of actual journalistic integrity.
by jitl | 2025-06-14 13:04:33+00:00
> This policy data contained unintended blank fields. Service Control, then regionally exercised quota checks on policies in each regional datastore. This pulled in blank fields for this respective policy change and exercised the code path that hit the null pointer causing the binaries to go into a crash loop.

Another example of Hoare’s “billion-dollar mistake” in multiple multiple Google systems:

- Why is it possible to insert unintended “blank fields” (nulls)? The configuration should have a schema type that doesn’t allow unintended nulls. Unfortunately Spanner itself is SQL-like and so fields must be declared NOT NULL explicitly, the default is nullable fields.

- Even so, the program that manages these policies will have its own type system and possibly an application level schema language for the configuration. This is another opportunity to make invalid states unrepresentable.

- Then in Service Control, there’s an opportunity to prove “schema on read” as you deserialize policies from the data store into application objects, again either a programming language type or application level schema could be used to validate policy rows have the expected shape before they leave the data layer. Perhaps the null pointer error occurred in this layer, but since this issue occurred in a new code path, it sounds more likely the invalid data escaped the data layer into application code.

- Finally, the Service Control application is written in a language that allows for null pointer references.

If I were a maintainer of this system, the minimally invasive chance I would be thinking about, is how to introduce an application level schema to the policy writer and the policy reader that uses a “tagged enum type” or “union type” or “sum type” to represent policies that cannot express null. Ideally each new kind of policy could be expressed as a new variant added to the union type. You can add this in app code without rewriting the whole program to a safe language. Unfortunately it seems proto3, google’s usual schema language, doesn’t have this constraint.

Example of one that does: https://github.com/stepchowfun/typical
by jawns | 2025-06-14 13:04:48+00:00
For all the benefits, there is a large problem with this approach that often goes unacknowledged. It is fundamentally a business problem, rather than a technical problem, but it has impact on development speed, so it's secondarily a technical problem.

The business contract with a consolidated data definition is that everyone in the business, no matter which domain, can rely on it. But think about the red tape that introduces. Whenever you need to define or update a data definition, now you don't have to think just about your own use case, but about all of the potential use cases throughout the organization, and you likely need to get sign-off from a wide variety of stakeholders, because any change, however small, is by definition an org-wide change.

It's the data form of the classic big-org problem, "Why does it take two months to change the color of a button?"

Granted, in most cases, having data definitions duplicated, with the potential for drift, is going to be the more insidious problem. But sometimes you just want to get a small, isolated change out the door without having to go through several levels of cross-domain approval committees.
by bostik | 2025-06-14 13:06:48+00:00
This was the bit that I spotted as potentially conflicting as well. Having managed (and sanitised!) tech&security policies at a small tech company, the fail-open vs. fail-closed decisions are rarely clear cut. What makes it worse is that a panicked C-suite member can make a blanket policy decision without consulting anyone outside their own circle.

The downstream effects tend to be pretty grim, and to make things worse, they start to show up only after 6 months. It's also a coinflip whether a reverse decision will be made after another major outage - itself directly attributable to the decisions made in the aftermath of the previous one.

What makes these kinds of issues particularly challenging is that by their very definition, the conditions and rules will be codified deep inside nested error handling paths. As an engineer maintaining these systems, you are outside of the battle tested happy paths and first-level unhappy paths. The conditions to end up in these second/third-level failure modes are not necessarily well understood, let alone reproducible at will. It's like writing code in C, and having all your multi-level error conditions be declared 'volatile' because they may be changed by an external force at any time, behind your back.
Ask HN: AWS cdk, serverless setup advice
by jemiluv8 | 2025-06-14 13:06:50+00:00
I'm doing this part time work with another "senior/staff" developer who got tired of writing lots of code with nodejs and decided the way to go was doubling down on aws - and the various tools provided. His new stack is as follows

- dynamodb - nodejs lambda functions - s3 - cdk to setup infra, attach lambda's to various resources

nearly all our features rely on the following "common/reuseable" existing code. This is thoroughly tested because they are reused in multiple projects. -> we have a core set of infra/lambda code that does common things like -> authorization/authentication/accounts - using auth0 behind the scenes - for user management -> billing - using a stripe setup. we have ddb table, webhooks and various lambda to support billing based on accounts above -> managing uploads - presigned and direct multipart/form-data uploads and saving metadata in dd -> creating and tracking jobs - via sqs -> have api as a service module that helps us track useage for api's being sold to public

For a typical project, we spin up the above existing infra Add custom infrastructure to do things specific to the project This typically involves -> adding an api gateway and attaching lambda handlers -> and so on.

My question is Which of you do something similar How do you organize reuseable code - do you create your own custom constructs library? Do you have one giant gateway that custom apis get attached to or you create new gateways to do whatever.
by actsasbuffoon | 2025-06-14 13:07:08+00:00
In the context of the topic (could a rogue super intelligence break out), I don’t really see how that’s relevant. Clearly someone who is clever enough has an advantage at breaking out.

As for the bit about how limited it is, do you remember the Rowhammer attack? https://en.m.wikipedia.org/wiki/Row_hammer

This is exactly the kind of thing I’d worry about a super intelligence being able to discover about the hardware it’s on. If we’re dealing with something vastly more intelligent than us then I don’t think we’re capable of building a cell that can hold it.
by frankdejonge | 2025-06-14 13:07:26+00:00
A bit unfortunate they used the term domain model here. Domain models here are purely data-centric, whereas domain modeling focuses mainly on behavior, not underlying data structures. The data that is used in domain models is used to facilitate the behavior, but the behavior it the code focus.

From a modeling perspective, there is certainly inherent complexity in representing data from domain models in different ways. One can argue though that this is a feature and not a big. Not the same level of nuance and complexity is needed in all of the use-cases. And representational models usually are optimized for particular read scenarios, this seems to mandate argue against that, favoring uniformity over contextual handling of information. It will most likely scale better in places where the level of understanding needed from the domain model is quite uniform, though I have seen most often that use-cases are often complicated when they do not simplify concepts that in their code domain model is very complex and nuanced.
by gjsman-1000 | 2025-06-14 13:08:25+00:00
As an outsider, what I perceive is quite different:

HN likes to pretend that FAANG is the pinnacle of existence. The best engineers, the best standards, the most “that wouldn’t have happened here,” the yardstick by which all companies should be measured for engineering prowess.

Incidents like this repeatedly happening reveal that’s mostly a myth. They aren’t much smarter, their standards are somewhat wishful thinking, their accomplishments are mostly rooted in the problems they needed to solve just like any other company.
by belter | 2025-06-14 13:09:05+00:00
- You do know their AZs are just firewalls across the same datacenter?

- And they used machines without ECC and their index got corrupted because of it? And instead of hiding the head in shame and getting lessons from IBM old timers they published a paper about it?

- What really accelerated the demise of Google+ was that an API issue allowed the harvesting of private profile fields for millions of users, and they hid that for months fearing the backlash...

Dont worry, you will have plenty more outages from the land of we only hire the best....
by alzoid | 2025-06-14 13:09:34+00:00
The future is already here. Look at how companies behave today, AI will not change their behaviour. AI will not make them 'nicer'. People talk about the massive productivity change and how we need to think about Universal Basic Income. They don't realize that in the US and other western nations they are already living in abundance (even excess). How do we treat the unemployed and "Unskilled" workforce? Do they have UBI? When they complain about rent and food prices do the wealthy step in to help? Or are they told they should have went to school or acquired a better skill to deserve a better life. What will happen when AI makes while collar workers "unskilled"? The same thing that happens today.
by singron | 2025-06-14 13:09:48+00:00
Nearly every global outage at Google has looked vaguely like this. I.e. a bespoke system that rapidly deploys configs globally gets a bad config.

All the standard tools for binary rollouts and config pushes will typically do some kind of gradual rollout.

In some ways Google Cloud had actually greatly improved the situation since a bunch of global systems were forced to become regional and/or become much more reliable. Google also used to have short global outages that weren't publicly remarked on (at the time, if you couldn't connect to Google, you assumed your own ISP was broken), so this event wasn't as rare as you might think. Overall I don't think there is a worsening trend unless someone has a spreadsheet of incidents proving otherwise.

[I was an SRE at Google several years ago]
How do I grow this startup?
by phinophite | 2025-06-14 13:09:52+00:00
I've been building something on the sides, as a solo founder. The product is called Morzepay, which is a link in bio for payments.

The biggest challenge is I'm new to this startup thing and I can't really find co-founders because the people around me aren't invested as I am.

I'm in a third world country, not sure if that makes it even worse. Really trying to understand how do people figure out distribution.
by latexr | 2025-06-14 13:10:34+00:00
> Like what?

Like smartphones with an entire interface focused on multitouch. There wasn’t another one of those “proven in the market” before the iPhone.

Or, you know, the first mass-marketed personal computer with a GUI, and the first successful one with a mouse (Lisa, Mac).

The Kinect example is nonsensical, it wasn’t as an authentication device. Even if they used the same team and technology, so much more went into it (like the Secure Enclave) than simply repackaging what already existed.
by jitl | 2025-06-14 13:11:14+00:00
Kotlin is very easily adoptable in existing Java systems and is much safer (although not guaranteed safe)

Zig is fairly easy to adopt in existing C systems and is guaranteed null safe (although not use after free safe)

Rust, although quite safe, bears a fairly high adoption cost as existing code often cannot be ported directly.

Borgo (https://github.com/borgo-lang/borgo) is a nil safe language that compiles to go, so is easily adoptable in existing Go systems.
by jfengel | 2025-06-14 13:11:20+00:00
We cannot measure anywhere near the Planck length or anything like 100 significant figures.

My concern with the reals goes the other direction. The reals are not closed under common operations. A lot of the work is done in complex numbers, which are. The rationals are not closed under radicals, and radicals seem pretty fundamental.

You can define a finitist model despite this. But it's ugly, and while ugliness is not physically meaningful, it tends to make progress difficult. A usable solution may yet arise; we shall (perhaps) see.
by decimalenough | 2025-06-14 13:12:04+00:00
Yup, pregnancy/childbirth/rearing is particularly rife with these. Western sources suggest that eating sushi while pregnant is little short of stabbing yourself in the abdomen with a sharp knife, Japanese sources explicitly recommend sushi as a light and healthy meal. In the US giving any form of peanuts to babies is attempted murder, in Israel peanut puffs (Bamba) are among the first foods offered.
by anonymars | 2025-06-14 13:12:07+00:00
I always say that many of the things we take for granted today came from Windows Phone

At the time everything was app-based: you are looking at a photo and want to share it? Why, of course you should switch over to the messaging app in question and start a new message and attach it. As opposed to "share the picture, right now, from the photos app"

Dedicated access to the camera no matter what you were in the middle of doing, even if the phone was locked

Pinning access to specific things within an app, for example a specific map destination, a specific mail folder, weather location info

Dedicated back button that enforced an intuitive stack. Watch someone use an iPhone and see how back buttons are usually in the app in a hard to reach place. This leaks into websites themselves too

I still miss the way messaging was handled, where each conversation was its own entry in the task switcher, instead of having to go back and forth inside the app
by bmacho | 2025-06-14 13:13:40+00:00
The single most best definition I know is what is on the wiki[0]:

> A choice function (also called selector or selection) is a function f, defined on a collection X of nonempty sets, such that for every set A in X, f(A) is an element of A. With this concept, the axiom can be stated:

> Axiom—For any set X of nonempty sets, there exists a choice function f that is defined on X and maps each set of X to an element of that set.

I like this definition because IMO it is simple, close to the name of the axiom, and you might want to use it in this form, that is, having a set of sets, and taking a choice function on them.

To understand its importance and the controversies around it, you'll need some examples and counterexamples how truthness and provability and knowability (regarding structures, numbers, metamathematics) interact; also what are the views of the majority of working mathematicians and people in other fields using mathematics.

[0] : https://en.wikipedia.org/wiki/Axiom_of_choice#Statement
by piva00 | 2025-06-14 13:14:12+00:00
I don't agree that in this specific case (reading potential null data from a source) it's the kind of failure of null handling that every skill level would do, after you're experienced enough and you are reading values from somewhere with a schema you naturally check the schema to know if you ought handle nulls or not. Even if you are lazy, you want to avoid having to write null-checking and null-handling if it's unnecessary so you will go check the schema.

I haven't done such a mistake in many, many years, other kinds of null issues from internal logic I'm still very much guilty of at times, but definitely not "I forgot to handle a null from a data source I read from". It's so pervasive and common that you definitely learn from your mistakes.
by yencabulator | 2025-06-14 13:15:05+00:00
Current observations make it likely that our observable universe expands (think "stretches"), and the expansion will continue forever.

If it's expanding, then it was smaller earlier. Asking about the far past is a natural reaction, and the Big Bang theory is a pretty good attempt at explaining that.

https://en.wikipedia.org/wiki/Expansion_of_the_universe

https://en.wikipedia.org/wiki/Chronology_of_the_universe
by echelon | 2025-06-14 13:15:30+00:00
> installing this kind of red tape is great for job security

It really doesn't, and that's not the point. This is for business entities that are larger than teams.

It's way worse to have a million different schemas with no way to share information. And then you have people everywhere banging on your door asking for your representation, you have to help them, you have to update it in their systems. God forbid you've got to migrate things...

If your entity type happens to be one that is core to the business, it's almost a neverending struggle. And when you find different teams took your definition and twisted it, when you're supposed to be the source of truth, and teams downstream of them consume it in the bastardized way...

This project sounds like a dream. I hope it goes well for Netflix and that they can evangelize it more.
by smarx007 | 2025-06-14 13:16:58+00:00
Below are some links for extra reading from my favorites.

High-level overview:

- https://www.w3.org/DesignIssues/LinkedData.html from TimBL

- https://www.w3.org/DesignIssues/ReadWriteLinkedData.html from TimBL

- https://www.w3.org/DesignIssues/Footprints.html from TimBL

Similar recent attempts:

- https://www.uber.com/en-SE/blog/dragon-schema-integration-at... an attempt in the similar direction at Uber

- https://www.slideshare.net/joshsh/transpilers-gone-wild-intr... continuation of the Uber Dragon effort at LinkedIn

- https://www.palantir.com/docs/foundry/ontology/overview/

Standards and specs in support of such architectures:

- http://www.lotico.com/index.php/Next_Generation_RDF_and_SPAR... (RDF is the only standard in the world for graph data that is widely used; combining graph API responses from N endpoints is a straightforward graph union vs N-way graph merge for JSON/XML/other tree-based formats). Also see https://w3id.org/jelly/jelly-jvm/ if you are looking for a binary RDF serialization.

- https://www.w3.org/TR/shacl/ (needs tooling, see above)

- https://www.odata.org/ (in theory has means to reuse definitions, does not seem to work in practice)

- https://www.w3.org/TR/ldp/ (great foundation, too few features - some specs like paging never reached Recommendation status)

- https://open-services.net/ (builds atop W3C LDP; full disclosure: I'm involved in this one)

- https://www.w3.org/ns/hydra/ (focus on describing arbitrary affordances; not related to LinkedIn Hydra in any way)

Upper models:

- https://basic-formal-ontology.org/ - the gold standard. See https://www.youtube.com/watch?v=GWkk5AfRCpM for the tutorial

- https://www.iso.org/standard/87560.html - Industrial Data Ontology. There is a lot of activity around this one, but I lean towards BFO. See https://rds.posccaesar.org/WD_IDO.pdf for the unpaywalled draft and https://www.youtube.com/watch?v=uyjnJLGa4zI&list=PLr0AcmG4Ol... for the videos