skysight-inc/HackerNews huggingface dataset:
Though I dislike the hyperbolic tone and personal attacks, the author isn't entirely wrong. There are many design choices in Protocol Buffers that seem directly related to the scale and complexity at which Google operates, and which sacrifice safety, clarity and language integration.

The utter awkwardness of Protobuf-generated code is particularly problematic. I've had pretty good results with the TypeScript code generator, but the Go generator is just egregiously bad. There's no way to write a client or server that uses the Protobuf structs directly as first-class data types without ending up looking like a ball of spaghetti, at least not if you venture into the realm of oneofs, timestamps or the "well known types" set of types.

I think we're at a juncture where we really desperately need a better, unified way to express schemas and pass type-safe data over the network that flows through these schemas. I'm currently working on a project that involves both gRPC (which uses Protobuf), GraphQL and JSON Schema, with the backends written in Go, and the frontend in TypeScript, and the overlap between all of these is ridiculous. TypeScript is a fresh breath of air that mostly solves the brittleness of JavaScript's type system, and it's absolutely crucial to extend this type-safety all the way to the backend.

The challenge is that no language is the same, and a common schema format ends up targeting a lowest-common denominator. For example, GraphQL errs (in my opinion, wrongly) on the side of simplicity and doesn't support maps, so now you have a whole layer that needs to either custom types (JSON as an escape hatch, basically) or emulate maps using arrays of pairs, neither of which is ideal.
I will say one thing. gzipped JSON is damn efficient over the wire. It elegantly solves the problem that field names are repeated in every object. The only problem is that adding compression and a serializationd/deserialization step cost CPU time. However if you have this problem then you're often better off by using something like cap'n proto or flatbuffers which do not have an extra serdes step. If you consider this then there is only a very very narrow band in which it makes sense to use protobuffers vs JSON (which is a defacto standard) or lesser known solutions that provide superior performance.
In addition, Apple, Amazon, and even Google aren't completely open source friendly. In the case of Google, pretty much all their webservices are closed source, and they make more and more components of Android become part of the closed source Google Play Framework. I realize Google is the best major corp for open source. But somewhat cynically, I view the ultimate purpose of their open tools to be for running proprietary code.
I think both comments are sarcastic jabs at Go.
Stopping global warming seems very simple - just limit C02 production. The main thing is institutions are complex and so even simple change requires complex processes.
Didn't they go down yesterday as well? I might be blurring days together here though :)
This may come across as rude, so I apologize in advance, but... You sound very optimistic and so clearly are not a programmer.
Yes but then again, so do some Cisco switches nowadays. Arista's management team is former Cisco as well. There is a tangled legal history between the two companies, with Cisco claiming various patent infringements. So far as I know, none of the claims are hardware-related.
I would suggest you read up on Simpson's paradox.[0] The study is not eliminating their performance just categorizing them.

[0]: https://en.wikipedia.org/wiki/Simpson%27s_paradox
> Option 1 is clearly the "right" solution, but its untenable with protobuffers. The language isn't powerful enough to encode types that can perform double-duty as both wire and application formats. Which means you'd need to write a completely separate datatype, evolve it synchronously with the protobuffer, and explicitly write serialization code between the two. Seeing as most people seem to use protobuffers in order to not write serialization code, this is obviously never going to happen.

This last is not the case. I've only ever seen it used when people a) already have written serialization code that works with XML/JSON, and b) don't want to write code to serialize to a binary format.

JSON also doesn't let you use objects as object keys, or have any form of polymorphism. It seems like the author wants protobufs to be an RPC system with a modern type system, when in fact it is a data interchange format.
That said, there's the legitimately terrifying oppression of the Uyghurs, and the filtering of any information the government doesn't find supportive enough of their regime
Religion is again topping used car salesmanship :

https://en.wikipedia.org/wiki/Mark_Hofmann#Salamander_letter

Perhaps the most notorious of Hofmann's Mormon forgeries, the Salamander letter, appeared in 1984. Supposedly written by Harris to W. W. Phelps, the letter presented a version of the recovery of the gold plates that contrasted markedly with the church-sanctioned version of events. Not only did the forgery intimate that Smith had been practicing "money digging" through magical practices, but it also replaced the angel that Smith said had appeared to him with a white salamander.[28]

After the letter had been purchased for the church and become public knowledge, LDS Church apostle Dallin H. Oaks asserted to Mormon educators that the words "white salamander" could be reconciled with Smith's Angel Moroni because, in the 1820s, the word salamander might also refer to a mythical being thought to be able to live in fire, and a "being that is able to live in fire is a good approximation of the description Joseph Smith gave of the Angel Moroni."[29]
If the main argument is economic costs, then this turns the discussion from a global to a localized problem, whether the required actions or sacrifices justify the local risks.

For example, is it worth for some country to "invest", say, $100 billion in reducing emissions - which, at least unilaterally, won't be sufficient to prevent the effects of climate change but just mitigate them - or can they achieve better mitigation by a comparable investment in reinforcing their shoreline, updating or relocating buildings, etc?

If USA had to vote on whether to spend (or lose by restricting growth) $1 trillion of USA resources to prevent $10 trillion of future losses in SE Asia, then I'd assume that such proposals would be swiftly declined; spending our resources on other things now and having someone else pay a huge price later would be considered a quite good deal by many governments, as the countries that most need to take action are not the same countries that will bear the harshest consequences.
If I lose my wallet, I am an "undocumented worker" - until I get a new set of papers. If I cross the border of a country in contradiction with its laws, I'm an illegal immigrant.
> But in the end, the author probably does not understand PB solves the inter-language data sharing problem, which makes all its complain secondary and inconsequential.

Huh? It's a serialization format. This problem has been solved many times in many different ways.

The author is pointing out that Protobufs were designed poorly, and they didn't have to be.
How to Find Hidden Cameras in Your Airbnb Rental
As Steve Jobs used to say: “if you can’t buy an iPhone you deserve nothing”
Do you have some evidence to back that up? There are some dumb rich people, no doubt of every color, however there is a clear correlation between income and performance in every reputable source I’ve ever seen.

I find it hard to believe that on average rich black kids are doing worse than on average poor white kids.
Actually, I have degree in math from a reputable university. So while you are certainly entitled to think that isn't going "far enough in math", I do think that your opinion is in the minority here. If anything, I would say the fact that I have completely forgotten everything from that class is evidence I may have actually gone too far in math... or further than I ended up needing.

And I think it's evidence that the differential equations are not taught in a way that is beneficial for comp sci students (and other types of students too, but I can't speak to that). I took other classes that have not been applicable to my career after graduation - things like finite state machines, computability, and complexity theory. But I still remember a lot from those classes - due to their focus on fundamental ideas and proving things.
How about json + jsonschema + gzipped over the wire
That remains to be seen. There's clearly a certain level of "comfort" where the majority of people become satisfied enough, and the quality of life at the bottom is rapidly increasing as technology and the world matures. The rise of the middle class in China and India seems to be the biggest indicator of this approaching steady state.
Ethics aside, yeah they should be worried. The "I was only following orders" claim usually only applies when those orders were lawful.
"a worldview built on the important of causation is being challenged by a preponderance of correlations. The possession of knowledge, which once meant an understanding of the past, is coming to mean an ability to predict the future." - Big Data (Schonberger & Cukier)

so, knowledge now is allegedly possession of the future, rather than possession of the past.

This is because the future and past are structurally the same thing in these models. Each could be missing, but re-creatable links.

Also, conflicting correlations can be shown all the time. if almost any correlation can be shown to be real, what's true? How do we deal with conflicting correlations?
This feels pretty vitriolic, I wouldn't be surprised if there is some bias here. A lot of these problems seem pretty minor and there's weird stuff like

"Unlike most companies in the tech space, paying engineers is one of Google's smallest expenses."

According to here
https://www.quora.com/How-many-software-engineers-does-Googl...
there are ~28,000 engineers in 2014, which paying at 120,00 a year (glassdoor) would be... $3.3 billion a year? And this was 2014, and doesn't include stock compensation

Don't get me wrong - there are definitely problems with protobufs. But this post is citing a lot of stuff to back itself up that doesn't add up.

Also, required vs optional - definitely only optional, there should be nothing required. So I flat out disagree with that one.
Write your own generator then?
I'm plugging my own Prolog tutorial: http://norswap.com/prolog-served-dry/

Might be useful for the folks who want to learn how the language ticks but don't have the time to go through a whole book.

That being said, the book is pretty great! I recommend it.
Seriously! I consider my political philosophy to be about as inoffensive as they come, and am surprised I would be downvoted for this.
> I do find it hilarious that as people we generally hate going to work, but at the same time we also fear losing our jobs.

People also like eating, and buying Netflix subscriptions. There are plenty of people in places like Africa and Asia that have lots of time on their hands without jobs, and I don't think they think it's a plus.
It's abundantly clear why this author lasted only a year at Google. Aside from using the non-idiomatic "protobuffers" ...

"nothing wants to inspect only some bits of a message and then forward it on unchanged"

In my experience it is extremely useful to partially parse a protobuf, or to not parse it at all and simply modify it by appending to it. Also useful is the ability to define a message type that is isomorphic on the wire with a different type, but is much cheaper to parse (example: a deeply nested structure where all of the fields of interest are defined at the top level).

Nobody even within Google will argue that no mistakes have been made with proto. Proto3 is generally acknowledged to have been a big mistake and it doesn't have a lot of traction within Google. Map and OneOf are both basically antagonistic to high performance application code, and people avoid those for that reason. But the author's arguments are pretty weak and his dismissal of Google itself as an existence proof is insufficiently supported.
I hate the clickbait way in which this story has spread across the net. "secret AI recruiting tool" sounds like Amazon did something nefarious. Instead they built a tool, found out it was broken, and didn't deploy it.

The actual newsworthy part, which is getting slightly stale, is that it was influenced by the data bias.

I am not even a fan of Amazon but I think this is unfair to them. They did the right thing here.
I went to school F - full of average kids who knew how to play the system to get top grades, as well as to get into high/elite level colleges.
I’d say the recent experience in Syria, Iraq and Afghanistan runs counter to your claims.
Forge Your Future withOpen Source: First book about contributing to open source
> Who on Earth approved this trailer?

Ha, if you're surprised about that, check out what they approved here https://www.youtube.com/watch?v=w8J5BWL8oJY
In neither case should you decide to take a nap on the lawn.
I like GraphQL schema, but it would have to have a standardized serialization aspect. Moreover, it really is designed for querying and has a bunch of stuff totally irrelevant to data.

But I agree ... I almost wish they'd finish it off and go for full on serialization.

That said, it may not be that-that hard, a serialization spec can be pretty short if need be. Certainly one could 'roll one's own' for a mid-sized project if need be.
Elon Musk must leave, I saw him smoking a joint
>Data collection

So turn it off. It has ACTUAL end-user benefits being enabled by default. You are literally asked on first boot if you want it on or off. And you can go a step further by disabling it in the registry to prevent a second user on the system enabling it.

>Forced installs

This one gets me more than anything. Literally every *nix admin on the planet bitching for the last 2 decades about MS having all these insecure zombies out in the wild. The fix? Enable updates by default, and make it difficult for the average home user to turn it off. More importantly make it difficult for malware to easily turn off. And you STILL find a reason to complain. Why on earth you'd want to make the internet less secure is beyond me, but that's just such a horrible thing to try to bag them for.
Sure, in the same way -450 degrees Fahrenheit isn't technically "absolute zero."
I'm no GraphQL expert/fan but one example is requesting only the data you need -- you can cut down on what you have to send over the wire which is at least guaranteed to help some.

Also, the nested query structure can save what would have normally been multiple requests with a by-the-book RESTful HATEOAS setup.
Make tea. I like the ceremony of it. It’s during these repeated actions that I let my mind wander and think about next tasks and new ideas.
The intention is to measure post-treatment effects. You can't include a treatment effect as a measure of itself, that defeats the purpose of measuring the benefit.
Nobody asked tho
> This problem has been solved many times in many different ways.

Mind provide examples of such tools.
In context of current protobuf design required was a mistake and optional is clearly better, but author argues about grand type system that is based on different principles, including stronger validation.

Criticizing protobufs is like criticizing C++ or Java. Both have major shortcomings, but solve practical problems and de facto lingua franca with no practical solutions to replace them.
I don't understand what you mean. Any GC will reclaim `x` after it is unreachable from a root.
My contention with the quoted text is that you probably shouldn't be using elaborate data structures in streams/files. Using DTOs as heap/stack data has bitten me enough times that I'm fairly certain that it's an anti-pattern.

It doesn't matter if you're using a quantum binomial tree in a black hole: save it as a 'stupid' map<k,v> when it hits the network. That way everyone who interacts with your service can decide how they want to represent that structure. You can compose any in-memory data structure you can dream of, you can validate the data with more richness than "missing" or "present", and Protobuf doesn't contaminate your codebase.

Option 3 is the correct choice. Serialization libraries should be amateurish by design.
Because C has automatic variables. https://en.wikipedia.org/wiki/Automatic_variable For the argument "that's a different kind of memory" see this thread from last week: https://news.ycombinator.com/item?id=18124431 It is entirely possible to write complex C programs using nothing but automatic variables.
Apache Thrift is another one.
I wouldn't be surprised of most of the western world will wait until the "Areas where more than a billion people live today will become uninhabitable to humans if they warm up" start taking drastic anti-climate-change sacrifices first. If they aren't willing to sacrifice economic growth to mitigate those consequences, then no one will.
Yes, but all the different 'types of kids' should normalize across a state-level population.
Yes, it shouldn't have been far off Google as the model in the article is Google's itself. In fact you see in some of the examples how Google Translate's output and this model are almost exactly the same.

Pretraining is a very cool idea, I have also seen some good results with pretrained embeddings after doing language modelling. Fast ai discusses this and I think even has some pretrained embeddings available in their library!
That’s quite a lot in UK, not everything is like SF.
Thanks! I haven't seen cobbler before, so that's certainly something I'll take a look at. Toyed with matchbox before, but didn't end up using it.

Not using DigitalOcean currently, but look forward to reading that someday.
Well, that surely saves the reviewer time on pulling and checking the features/designs. Nice point!
I think I get what you're saying.

Wouldn't a computer enthusiast be happy and comfortable with desktop Linux though? It's still hackable as you like and you can install a more.... I mean a less simplified desktop environment.

(I assume by "targeting the mainstream" you mean simplification of the UI, mostly)

I don't like any OS that oversimplifies, such as by removing advanced options (why not put them on another screen?) But I am a strong supporter of proper usability, including in Linux - just that things should be intuitive and simple at first glance, and aesthetically pleasing. As in I don't want the Linux UIs of ~2000 to return.
"All models are wrong, but some are useful." -- statistician George E.P. Box
The food truck laws in Michigan in general are pretty annoying. You need to have a commissary kitchen where all the food is actually made in order to be in compliance. Which defeats one of the main points of having a food truck to begin with(not having to rent or own an expensive building)
> Most of the world? Not in any official sense.

There are things that are de facto true, even if they're not de jure true.
Put me firmly in the camp of "optional fields are bad." I believe all fields should be required.

I come from an ONC/RPC background, which is the original UNIX RPC. Every iteration of an rpc would get versioned, and then you could write a conversion between versions, ex from V1 to V2, from V2 to V3, etc. This allowed for true backwards compatibility.

The idea of "forwards compatibility" is a pipedream, in my opinion. Just because something won't crash because a field is or isn't there doesn't mean that it will actually work. Code ages over time, and instead of looking at the spec, which will tell you which fields are required, you need to read the documentation, which may or may not tell you what is required. And especially when dealing with an older client, it simply may not work because what used to be optional-optional fields become required-optional fields.

It's the same argument as schemaless NOSQL database like Mongo vs MySQL. Having a schema is a pain in the ass, but it's better over time and ages much more gracefully than schemaless ones, because things won't just break.
Furthermore, in the case where I'm using protobufs as a necessary evil of grpc, what's the best alternative here?
> Surely everyone has the same problem.

it doesn't mean that there is a single solution that will satisfy everyone
Blue Origin wins contract to power top secret Air Force space missions
But the author is only talking about data types. I don't see any reason it shouldn't be designed with composition in mind.

The only restriction I can think of that JSON imposes (in terms of composition) is that keys must be strings. And I don't necessarily think JSON is a great serialization format either.
DeepL had a lot of good press when it came out last year. Some saying it was better than Google.

https://www.deepl.com/en/translator
The Sudden, Shocking Growth of Hurricane Michael
In most of the country, per passenger, per person transported a mile, cars are more efficient than public transportation. Electric cars take less than half the energy per mile driven. So they definitely have a role.

The reason why is that a bus has to run whether or not it has passengers on it. A lot of empty buses without passengers are running around all day and night in most metro areas.

So yes, public transit makes a lot of sense where it makes sense. But it only makes sense within a relatively small number of very dense cities.
I can not really take someone who calls Jeff Dean an amateur seriously...
There's Avro: https://avro.apache.org/docs/current/

Not sure if the best, but maybe has different tradeoffs
Ah right... when you can't attack the ideas, attack the author. Sandy is a Haskell enthusiast, and it is not a surprise that his criticism of protobuf is inspired heavily by that.

The argument presented is that protobuf does not attempt to even replicate the best practices we already have in terms of data representation. It makes sense why languages like C are constrained in their data representations -- they want to be close to the machine. However, a language meant literally to specify data really ought to be able to handle the basic competencies of its field (like sum and product types and polymorphism) with ease.

There is no excuse in 2018 for a data language without polymorphism and without product and co-product types. Compatibility with legacy or substandard languages is not an excuse for poor design. Compatibility with needed legacy languages should be the thing that's tacked on, not basic functionality.
People should be able to provide constructive and useful criticism of a technology without having to build their own solution.
Steam is in a great position since games aren't geographically as limited as music and video. This is why my Netflix sucks and someone elses doesn't, but my Steam selection is as good as any other.
I don't work with mapping so it's all new to me, but this list of projects[0] is like a gold mine. I've known about protobufs for a while, but had no idea about pbf.
Unreachable is not the same as unused

It could still be reachable, just never read from. Static code analysis can detect some forms of this, but not all e.g.

if (someUnknownCondition) {
// access x
}
This may be a slight tangent, but I always took the interpretation of Metropolis as one of humans being _reduced_ to the level of machines (and in some ways subservient to them), less in replacement; the "moloch" scene comes to mind. The only robot in the movie seemed more an analog for questions of humanity, blurring the line of human and machine, and additionally touching on the whole classic "reviving a loved one" tropes.

(Not to distract from your point, other works such as Rossam's universal Robots definitely covered the subject matter you refer to, I'm being pedantic about Metropolis largely because the subtle difference I'm emphasizing feels to me as what makes it _so much more accurate and predictive_ to the actual relationship between man, machine, and the owners of production.)