skysight-inc/HackerNews huggingface dataset:
We're using Citus for almost a year, 8 months in production. Currently running 22 workers [32 core, 3TB space, 120GB RAM] containing almost a trillion (yep) rows in one single table.

Citus has a lot of quirks and one could write a book about it. But for shortness here is just couple of the top things you should know.

- Citus is not PG. It runs on PG, it looks like PG but it doesn't behave like PG. By using Citus you lose majority of advance strategies, especially ones that manipulate data (UPDATE from SELECT, DELETE, etc.). In very short, you have to forget about everything you know about PG.

This means that you have to rethink your table schemas, distribution strategies, etc. This is common in every distributed database (No/SQL).

- Citus deployed in a cloud is a bit sensitive. As the master node keeps track of each worker in almost realtime fashion, any breaks in communication results in corrupted shards. If you don't use just one user (postgres) but have different rights for different users, fixing the shards can be a pain (if you don't pay for the enterprise version, which provides you with helpful scripts).

- Adding new worker is easy, but rebalancing them to spread the shards evenly is not (again different for enterprise version).

- PG's planner can be a bit stubborn while using citus. It sometimes behaves differently from the native PG. The result is that you have to rethink you strategies and rewrite queries up until you don't get the desired result.

- Writing functions and automatic triggers can be challenging. The usefulness of them is determined by how well you designed you original tables. Once the tables are colocated and sharded on the same column you get impressive results. Writing the functions can be challenging though as you have to deploy the function to each table. They offer a script to help you with it https://github.com/citusdata/citus/blob/a2f6b29a6d4537b626f8...

- You do want to consider how many shards you need. More automatically means that Citus gets slower, but you want to be able to grow so chose more than you have workers.

There is much more, but those are the things that you should consider. As for us, we learned to live with the quirks and now we're able to run aggregation queries through billions of rows in under 600ms.

I would suggest to consider Citus only if you're considering something like Cassandra or Hbase.
Show HN: IoT and Device Analytics
Exit rights are extraordinarily important for maintaining negotiation power with the powerful institutions in your life. If you don't like the way England is doing things, you can leave - and this puts a limit on what England does, since it doesn't want you to leave.

If you don't like Facebook or other global institutions, there's no effective way to opt out.
As in, a threat to their wealth and earnings? Sure, that's an issue, but is not an issue with globalization in the abstract. In fact, the U.S. benefits enormously of globalization in total material terms (U.S. produced goods and U.S. brands are pervasive through the world and have been for half a century, and that generates tons of wealth for the nation). "Yeah, but that gain gets mostly concentrated at the top while working class people go unemployed and risk homelessness" one might say. But that is the issue, not globalization. A more progressive taxation scheme, a coordinated investment into infrastructure, improved access to education and even things like UBI can all be combined to solve that problem. Better wealth distribution without negating all the wealth created by globalization.

Globalization-in-current-practice has a lot of specific problems: wealth inequality, environmental and labor laws race to the bottom, etc. But those are things to work on without throwing globalization away (the same way that propaganda on social networks can be tackled without either: "ban social media" or "internet sovereignty: internet propaganda for the national government where you live and no one else"). There are big issues with specific models of globalization. But nationalism flaws are inherent, a nationalist utopia is strictly worse than a globalist utopia, and historically nationalist reality has been worse than globalist reality (e.g. for all its problems, the E.U. beats what came before it).
Why would I want people sharing news?

If I want to read about what's going on with the world, I'll read a news site. If I want to know what's going on with my friends' lives, I'll read facebook.
> It took a small team a year to make a product that is apparently taking the market by storm. Their secret sauce is apparently not some special algorithm, but common-sense automation

They're taking a part of the market by storm.. specifically Indies with a "do it yourself" mentality.

Speaking as someone who works for a music distributor who has one of the "Big Three" Record labels as a client (Warner, Sony, UMG), I can say that in general we haven't been a great fit for really small time acts (though we're getting better). But the breadth of Distrokid's offerings wouldn't be able to cover that of a major label (and although they claim they could, there's no proof that they could handle that kind of scale either).

For instance, the base Distrokid plan doesn't even allow you to specify a release date (presumably it just releases it right away). Major labels often run promotions in different countries (necessitating different release dates per country). There's also pre-order dates, some of which are tied to instant-grat tracks for certain release dates in certain countries. The fine grained control necessary for a big act is missing from most DIY offerings.

Distrokid is serving a different market than the incumbents, and that's cool. I'd definitely recommend it over the other flat rate model distributors. But there's a reason Distrokid isn't delivering the next <insert big act here>'s album.
"In fact, it's totally fine to eat a product even well after its so-called expiration date."

Not so, unless lost nutrition and bad flavor are totally fine. Americans eat a lot of moldy nuts, rancid olive oil, and stale coffee. Just Google for the expert opinions on these foods, and compare to what you read on the label of any mainstream, high-volume product on the shelves.
China: Baidu Acquires Y Combinator-Backed AI Startup Raven
I'm glad they don't. If a tipping option existed, there would be social pressure to use it. Tipping is an embarrassing relic held over from old class hierarchies and would all be better off getting rid of it.
IDE integration is a big theme of this year, so this will change rapidly over the next few months. We aim to provide plugins for everything, pretty much. Specifically, we're implementing Microsoft's "Language Server" interface, so anything that can interface with it will work well, http://langserver.org/ has the full list, I believe.

Glad you liked the book! :)
Why is it a disease, honest question.
You can click the "print" button in the upper right hand corner, which gives you http://rust-lang.github.io/book/print.html
To all those complaining about collection of stats: Pretty much _every_ SaaS company is collecting stats about your behavior. It seems a bit off that there is so much rage about a project whose code is out their in the open to collect stats.

Point me to atleast a single popular SaaS product that does not have analytics in it's page.
High quality fitness resource – free access, forever, for everyone
Absolutely. The right marketing people will take that entire lead gen/relationship building/RFP process to a whole new level.
Coincidentally I'm about halfway through writing a long Vue.js tutorial series where I build one project in this manner. Would love to hear more ideas about what people look for with this type of book.
I've never heard that interpretation of that before, but it makes so much sense. Stealing as copying done better. thanks
I'm still not used to it, but I can see the appeal. Try working with Germans or especially Israelis
I thought the problem was that this also heats ocular fluid and furniture staples causing potential problems with things like macular degeneration or house fires, respectively.
This is kinda why I avoid so many packages, even probably-safe stuff like iterm. I try to go vanilla as much as possible and avoid having to think hard of which service or app may be transmitting my data.
Cockroach is just a creature. How is it arrogance. It reeks of arrogance to call yourself "Lionhead" or "Jaguar" or "Oracle". Some form of apex predator or all-knowing entity.

Also "Is Male Anus" is not rooted in tradition at all.
>>you can look at a C statement and be pretty sure what the ASM will end up looking like.

Perhaps 20 years ago, perhaps more. Optimizing compilers have gone long way.

Compiling an execution plan is a significantly easier task than actually guessing how exactly an optimizing compiler would schedule the code, down to assembly. (and cheating w/ hints in SQL- [looking at Oracle]just reinforces that.)
Ok, more interesting than you are.
Why Liberals Are Wrong About Trump
Seek First to Understand
We were working mostly in the months before 1.0 so it was right in the time of rapid changes.
Theres no discernable progress towards their mission.

Call it what it is, PR puff.
> All of these abstractions, when rarely they are actually needed to express a program, can be naturally expressed within the type system, in a proper functional language.

If you think FP is mutually exclusive with OOP, you understand neither.

OOP is very powerful to organize the architecture of your code and capture very common patterns such as reusing an existing piece of five functionalities while overriding one (something that's still awful to achieve in FP, regardless of the language you pick).

FP operates at a more granular level, basically how you implement your methods.

And by the way, FP has design patterns as well.

FP and OOP are extremely complementary.
In my very limited experience, the libraries still aren't stable. I ran into a bunch of errors a couple of days ago because my distro was on 1.14 rust compiler but the latest was 1.15. I'm not sure exactly what the issue is but it looked like the standard libraries had changed between versions.
It helps that globalization is helpful to the fortunes of Zuck and his business.

Call me an asshole, but I'm a nationalist. The amazing advances in Asia and Africa's standard of living have hurt my countrymen... why should I be ok with that?
I'm going to respectfully disagree. Part of why I like HN is that meta-discussion of the "why did you downvote me?" and "downvoting because ..." are actively discouraged. Better to silently downvote and move on, which I do frequently for all many of reasons. This has next to no impact unless multiple readers do the same thing, at which point greying out comments serves the intended purpose of conveying that the community as a whole doesn't think the comment is valuable.
If you do sane things with your DB then you really don't need to mess with all the edge case crap that breaks ORM's.

Great DBA's are rare, but vastly more important than most people think.
I wasn't thinking that we'd loan them aircraft, I was thinking we could send a carrier group of our own.
There is far too much conviction in the world. One of the reasons HN is so enjoyable is that the discussions tend to avoid the heat that you can find just about anywhere else.
A presentation from 1969: https://youtu.be/S76_thtZ6k8

An interview with Charlie Rose: https://www.youtube.com/watch?v=vZLVs5weric
My point is that the name isn't benign. And my post was deliberately somewhat hyperbolic. I actually have read a bit about Cockroach, and if I saw a growing ecosystem that seemed relevant to my work, I would pursue it more.

But as others have pointed out, the name creates unnecessary conflict and discomfort. In a world of perfectly rational humans, it wouldn't matter. In the real world, it makes it harder to get stakeholders on board. That's why I made the comparison to ethnic slurs. "F_U_DB" might be the greatest storage system imaginable. It doesn't change the fact that the name alone is going to create extra unpleasant work. (EDITED to fix an error)
No, because you can use linq without even having a server or even a database. ORMs are one application of linq, but it's a more general concept.
You have the wrong link to the original article, the one you link to is about a double murder. This is the translated original: http://www.sydsvenskan.se/2017-02-13/this-is-how-we-let-hans...
Ah another reminder I am old, and thankful I got to learn the lost art of using my senses, and appreciating food that although not useful for winning photography awards, it was still yummy. Yes, I'm one of those that gets frustrated when I see someone throw away a banana because it had a softer spot on one end, or "ew, too many freckles". I am from a past that will never return. While I'm here, I'll miss it.

EDIT: I should mention that I prefer buying non-packaged food, so I still trust my senses better than any "sell by" or "best before" labels.
Not sure if I should like that Zuck supports globilization or dislike how out of touch he seems. (At least he thinks he's trying?)
Algol, like Ada, actually requires you to specify your bounds and is perfectly happy to have 0,1,87 or -62 as your first index.
>> Not so, unless lost nutrition and bad flavor are totally fine.

They're fine in the sense that they won't poison you and aren't worth wasting.
You honestly think that quitting Facebook is harder than changing countries? Opting out from your country is currently near impossible for most people, and the only reason it is possible for the few who can is globalism. The place where moving between countries is the easiest is the E.U. and that's precisely because of the supra-national structure.

I am not a centralist when it comes to the official international institutions precisely because the no opt-out you describe (which is true for official institutions, not so much for Facebook). But at the same time, without international cooperation and with "every nation for itself" we end up much worse off at a personal and species level.
Clearly Excel isn't always the best tool if you are free to choose between R or Python or C or Java. But if you're not a developer then Excel may be the only tool that you and all your colleagues are guaranteed to have. Even if I persuaded ICT to install R on my work PC, my colleagues wouldn't have it so work wouldn't be reproducible.

I find that approaching Excel as if it were a database or a programming language tends to reduce the replication problem.
For example to analyse a daily file:

* set up a data connection to an exemplar file

* refresh the connection with the new file daily

* set connection option to copy formula down so each row is identical

* point to a separate spreadsheet for lookups

* use a recorded macro to paste the calculated values into a cumulative spreadsheet

* do pivots and change over time graphs in the cumulative spreadsheet

I don't think Excel is the best tool for this workflow, but a logical approach makes it "good enough".
Goofys v0.0.10 – cheaper, better, and faster (tests)
I saw Zuckerberg walking in Palo Alto with wife, child and dog and no security visible just over a year ago. If he has 24/7 security, either they're very discreet or it started recently.
I can tell you, with 100% certainty, that 10 years ago that was not the case.

And there's a really good reason everyone switched.
I don't think the victims are unsuspecting and I also don't think they have a choice in the matter.

> Accept it and go on with your life.

Isn't that exactly what you say to someone when you're not giving them a say in what's happening in their life?

> You are aware of how technology removes economic, cultural, lingual, even physical walls.

Tech does remove walls but it also allows you to construct walls. That's a lot of what this letter is about isn't it? Facebook has become a tool for people to organize themselves into groups that have their own political beliefs and beliefs about reality. These groups were shielded from the ideas on the other side. Why? because these walls were more effective than the ones we had in the world before. Zuckerberg has decided this was bad because people voted the "wrong" way and wants to prevent it from happening again. You might agree with that stance, but don't delude yourself into thinking this is just some inherent feature of technology. There's human morals behind these decisions.
I've just been through this section. I'd love a separate tutorial just for the borrowing lifetimes and error handling that make rust different.

It's to easy to go though the current tutorial and think "wow, this is easy and makes sense" only to come completely undone when you try to do something.
jOOQ helps with both 1 and 2. With jOOQ you can write something like (totally off-the-cuff code incoming, written outside of an IDE with only basic syntax highlighting, be warned, etc)

List<Condition> conditions = new ArrayList<>();

if (notNull(lessThan)) {
conditions.add(MY_TABLE.MY_COLUMN.lt(lessThan));
}

if (notNull(greaterThan)) {
conditions.add(MY_TABLE.MY_COLUMN.gt(greaterThan));
}

if (notEmpty(name)) {
conditions.add(MY_TABLE.NAME.eq(name));
}

dslContext.selectFrom(MY_TABLE).where(conditions).fetch();

(This is obviously really contrived but I hope it illustrates the ideas)

We use constructions like this with jOOQ relatively frequently, especially for cases where we offer all sorts of arbitrary filter knobs (useful for internal tools especially).

In general, you can do some pretty fancy manipulation with `Condition` in jOOQ, like you can build a list of conditions and then call `DSL.or(conditionList)` to OR them all together. Stuff like that.

Source: use jOOQ in production code -- been very pleased so far, other than sometimes poor documentation
Those are far-off examples. Granted, perhaps in some cases people should be prevented from doing something utterly stupid but that shouldn't keep us from exploring opportunities in general. Just because kids occasionally hurt themselves during play they shouldn't be forbidden to play.
You're probably looking for advice on graver issues, but I've been lucky with border crossings and can only offer this: when flying to the US, you technically cross the border at the origin airport. I lost a banana and an orange that I thought I could eat on the plane.

On a more serious note, I never try to pick a fight with officials to prove any point. They can stress you out, make you miss your flight, or straight out refuse entry. Most of them know this and will abuse their power just for sport, and your defiance will only trigger this sense and give them more pleasure in hassling you.
To do OOP good requires a lot of forward thinking and careful planning. Though most of the two are lacking supply when you're dealing with large teams of programmers and you end up with some IFactoryAbstractInterfaceInvoker eg... the kingdom of nouns game.

The other main concern was it was over-hyped with consultants running around to all businesses during the dot net time with xml with no empirical evidence to backup their claims.

The whole OOP was a giant experiment.
What about Sonar?
I think the confusion is due to the fact that some products have both types of date marking, some have only use-by, and some have only best-by. Some consistency would go a long way to fixing the confusion. If you have both on the product it's clear.
there is at least one way to write plain sql and get compile time checking, with f#
Is it possible to read the first chapter for free so that I can decide if I want to buy the book or not. There almost no reviews of your book.
So the logical thing is to at that point delete your facebook profile forever and never make another one and little of value is lost
Sonetimes the dates are inaccurate, but I've always been impressed by the ability of milk to go from totally fine to weird-tasting on precisely the use-by date.
We’ve definitely felt the bane of forgetting to turn off some really expensive GPU instances. Efficient scheduling and spinning instances up/down as required is one of the first things we built to cut down on the costs.

Git is an apt analogy. The search space of hyperparameters is usually fairly large for most DL algorithms, so a good amount of experimentation is required to tune them. Things can start to get haywire without end-to-end version control of code, data, parameters, results, environments, etc. Definitely one of the core problems we solve.
Are you using the commercial distribution or the OSS one?

If the OSS release how have you implemented rebalancing?
It seems to me the APIs are there in the extension, just user defined functions are missing to actually do the rebalance.

I have been looking at integrating Citus into Flynn as an option for more scale-out workloads but needing non-OSS features to recover from workers going missing or adding new workers put me off implementing it for now.
Show HN: GoGeneric extension for Chrome
I constantly pick up the metaphorical phone to call the DBA then remember that I am the DBA...
It's one of the superpowers I got from being poor back in the day. The "sniff test" always makes my wife and her family laugh :-)
I was being sincere. Seriously.
My point is architectures like TrueNorth are very impressive from the point of view of a computer engineer, and they are very efficient when running their intended applications (neural network algorithms). The fact that they are not "brain-like" does not make them any less impressive.
Mark Zuckerberg's answer to a world divided by Facebook is more Facebook
This is awesome.

'"Would you like to meet President Trump? I asked.
"I would like that not be the focus of this. I don't really have much comment on that. It somewhat detracts from the focus of what we are trying to do here."'
Hit Makers: The Science of Popularity in an Age of Distraction
Your demo is down - http://ambardemo.rdseventeen.com/
It used to be possible. [1]

Jeffrey Skilling, former Enron CEO is serving 24 years.

However, there is a strong argument to be made that ever since Enron, the SEC has been defanged.

Taking a brief look at their list of accomplishments, it seems that these days, they just levy fines, instead of jailing criminals. [2][3] (Contrast the punishments for the financial crisis, vs low-level insider trading. The companies involved in the former pay fines, the people involved in the latter go to jail.)

[1] https://en.wikipedia.org/wiki/Enron_scandal

[2] https://www.sec.gov/spotlight/enf-actions-fc.shtml

[3] https://www.sec.gov/spotlight/insidertrading/cases.shtml
Very interesting read. Shocking how G+ moves from "hey, you don't need to manage 100 logins" to "Hey shady people, come siphon data from people that don't understand privacy".

The Android ecosystem had already gone down that rabbit hole years ago. I think we're at the stage where it doesn't seem like a commercial identity provider is going to look out for our privacy at all. Definitely a challenging space, but one where we can wrestle back a lot of privacy we've lost.
Microcule ( open-source faas server ) releases compiled language support
1. "If I can save 1000 lives by having a border patrol guy reading my email..."

So we're going to train all the border patrol people to be intelligence analysts? That sounds really expensive.

If we're not going to do that, then they're not going to have the training required to make sense of that material. It's a pointless invasion of privacy. They're going to put innocent people in jail and be completely oblivious to actual bad actors who know how to hide their tracks.

Remember, September 11th was avoidable but because the information channels were clogged with too much garbage the real threat information got lost in the noise. The real priority here should be to collect less information and ensure it's of a very high quality.
Success? A Mindset Approach You’ve Never Seen Before