skysight-inc/HackerNews huggingface dataset:
It's much better to just use a table of phrases/keywords, even if it takes a little more effort. Phrases are specific and have clear intent - using only the most common words leaves out critical context!
Nice!

I have one Go book of famous games played in Shoguns’ palaces, way back when....

Go is a beautiful game that I have enjoyed since I was u years old.
> The shortage isn't about the amount of available housing at all

Yes it is. That's literally all there is to it. A housing shortage by definition means that there's not enough housing to satisfy the demand.

> it's about people's ability to live there too

People can't afford to live there because the shortage of housing has caused the price of housing to go up.

> This stuff is super complicated!

It's not complicated at all. Onerous zoning and land use laws have made it impossible for developers to build to the level of density necessary to support NYC's rapidly growing population.
Unfortunately that doesn’t change the equation much. The lasting impression of China in comparison with the west isn’t so much their development as the lack of our. Shenzhen might be the exception, but we don’t even have that.
Not the only kind of surgeries going on there...

https://en.wikipedia.org/wiki/Organ_harvesting_from_Falun_Go...
That is correct. I am originally from Israel and moved to California 7 years ago. I'm very healthy but on the rarest of occasions, when I do need to get SOMETHING done, it has proven to be an absolute nightmare, and I always walk away feeling like I've been scammed.
The two first sentences of the respective Wikipedia article answer that:

https://en.wikipedia.org/wiki/Extreme_learning_machine
"Extreme learning machines are feedforward neural networks"

https://en.wikipedia.org/wiki/Reservoir_computing
"The reservoir consists of a collection of recurrently connected units"

So, no.
> “It offends me,” Mr. Macklowe said, “because we created a very nice building that fits into the skyline perfectly.”

Oh, please. 432 Park is one of the biggest eyesores in the city. I wonder if that's one of their selling points: "Come live at 432 Park, the one building without any views of 432 Park".
> It's supply and demand. More available housing means lower prices.

Does it? I don't understand this, at all.

More available housing would also create more demand, no? More people online will see the lower prices and move to Manhattan, immediately increasing the price again, no? The situation is complicated. You can't just say "supply and demand" and think you've solved NYC's problems in housing.

ECON textbooks teaching students that every person is a financially rational actor and that supply + demand plays out real nice in the real world is supper dangerous. My econ textbook was wrong in a lot of ways and it's upsetting to see continuing oversimplification and massive condescension for these antiquated and not-real-world-tested perspectives on how to solve complicated urban problems.
We Don't Have a Talent Shortage. We Have a Sucker Shortage
Wonder why after all this time nobody managed to replicate erlang concurrency mechanism successfully. I’ve heard about actors framework in other environments ( such as akka in .net and java), but i feel that the approach that really became mainstream is node style single thread with callbacks and async / await mechanism.

Especially, erlang seems like the only environment where the concurrency model has such a big implication on how you build your whole system ( as opposed to just letting the framework spawn actors underneath without you even noticing, like i’ve seen in some recent dart web framework)

For example in swift, chris lattner set actor as a long term goal for concurrency, but async seems to be going to be implemented first ( probably for very good practical reasons). Yet the way erlang does it seems to be so core to its design that i’m very skeptical we’ll reach something similar.
RIP Joe
Why would you avoid Packt?
My premiums and deductible are north of 10k. This is why I laugh when some one tries to scare me with the old "with Medicare for all, your taxes will go up!" I'd gladly and enthusiastically pay more in taxes to rid myself of the current system.

I used to think employers would love to have to be able to stop offering insurance. But then I realized for the most part, employers love the current situation. Offering health insurance keeps employees chained to their desks. They'll moan every now and then about costs going up, but the current system ultimately serves them well.
I don’t understand the need to “give” anything. It’s a government owned building leased for $0 to the Catholic Church.

If neither of those ridiculously wealthy parties had the foresight to insure the building they can afford to repair it regardless.

You want to “help”? Donate (either your time or money) to a charity or organisation that actually needs it.
I think it means 'if you have a hacked DNS server that has all kinds of custom code to talk to databases, or other resources, then you will find that off the self DNS servers with DNSSEC will not support your hacks'.

Note that DNSSEC implementations started out with offline signing, both for performance and security.

I guess you can do geo-routing with offline signing as well. But some people find it easier to complain about new technology then to spend some effort making it work.

There are now also DNS servers that do online DNSSEC signing.
Is there a state or country that you think is doing an excellent job of taking care of their mentally ill citizens?
... it's offensive when the country has a housing crises and the rich are building houses...

Is it offensive to eat anything besides bread and water while there are so many hungry people in Africa?
Agreed, I think too many people assume their code will live in their small team forever and ever. When you have a few thousand people committing daily it's a different story you need to keep consistency high and the jumping between past and present tense is gnarly.

Also the capitalising your sentence isn't a stupid nitpick. Broken window theory is real and if you're too lazy to capitalise your sentences in published work I'd hate to read your emails.
Prerequisites to Become a Software Architect – IT Hare on Soft.ware
> Don't ruin it by becoming that cliched new hire that sees all their problems and knows how to make it all better.

I don't, which is why I'm asking around. I'm also scheduling chats with people to understand the background and the history to see if it's worth changing anything. I will make a move if that make sense. In the meantime, I'm just gathering information to not make a stupid decision.

> And you're a perm now, in a big corp, doing data science. Relax, you got it made, right?

> So enjoy playing that game, because you are happy to be in a big corp, so the sort of benefits that can bring you is what you want, right?

I don't think this was necessary. I've only worked for startups before, and I was hired in part to see if we can do a better job with the resources we have. Buy in from management is not an issue. I am not asking for life advice.
In my experience at three different employers the lower deductible plans are more expensive by more than the difference in the deductible. That is, in my experience, it has always been more rational to choose the higher deductible plans because even if I assumed I would always meet the deductible the cost of the deductible plus the premium was always less on the HDHP.
> We can always replace old institutions with newer ones.

Can you? We’ve become a country of factions and echo chambers, and our new “institutions” (Facebook, YouTube journalists, podcasters) reflect that. Institutions are easy to tear down, hard to rebuild.
Regular databases such as SQL Server and Oracle have had columnar compression built in as an option along with the row stores for years now. I use it in SQL Server a lot and it works great.
Running MinIO in Gateway Mode to Create an S3 Interface for Hadoop HDFS
People who won't understand it, like many literalist commenters here, and the author of the article who took it as an excuse to continue acting like a solipsistic child
1. Most of the buildings in NYC are already illegal according to modern zoning laws

2. It's not always profitable or practical to tear down a building. Tearing down a 3-flat and replacing it with a 4-flat, for example, would not be economically feasible even if it were allowed. Replacing a 3-flat with a 12-story building, on the other hand, would result in a much more productive use of the land, but it's typically not possible to upzone a parcel to that extent.
I think it may come across as trolling :)
I believe some of the thinking has shifted toward virtio-fs[1] instead of the 9p/network oriented models. Though this requires hypervisor locality for sharing among VMs.

[1] https://virtio-fs.gitlab.io/
No, it's obviously not, and I do not understand how you got there. I clearly don't think that because that would be insane.

Does my eating of an apple actively hurt people who are hungry in Africa? No, it doesn't. In fact, over the long run it will help them, because I would live longer to buy more to buy more of their goods and support them economically in the future with trade, etc.

You know what would actively hurt the poor in Africa? Me eating bread and water and dying and being unable to help the world in any future way ever because I'm dead. Me choosing to have apples and peanut butter and other vegetables and proteins is so that I can live longer and improve the world more, with either open source software, general economic contributions, studies of the weather and climate to reduce overall risk, etc.

Choosing to live on a diet that would kill me is not helpful to the poor in any part of the world.

But does this building's existence with enormous effort put into cheating the rules, skirting the system, and catering to the exclusively epicrich actively harm the middle-class and poor living nearby?

Yes it does.

The rich who live in this building are not doing so purely to help others, they are not there to improve society, they are there and are specifically wasting resources with no benefit but their own view of NYC.

It's obviously a very different situation.
This piece while interesting does not seem to differentiate mainstream articles vs highly specialized fields when it compares both in terms of accuracy.

I would bet it's not only profitability but also ROI that dictates how long a paper can part with a journalist for his investigation in a story before it becomes too much time in the eyes of management; the highly comprehensive piece does not bring that many more readers and possibly even less if the topic at hand is very specific.

As well politics on the other hand are dealt with on a daily basis and accuracy can more easily be measured even by the mainstream reader unlike a highly specialized field.
Yes, what exactly happened to him?
> With guesses or context you can fill those in with high probability. In an extreme case, the redaction might be a name of let's say a member of Congress

Something like this actually happened somewhere in the report. A name was at the end of a line, causing a part of it to break unto the next line. The few, black characters present on the second line lead readers to theorise that the name ends with “jr”.
Currently I am in the same boat. Few things that we have tried/figured out are -

(a) Deprecation - Get your butcher hat on. start looking at existing things, and see how many of these are used by who all. Start deprecating (or at least archiving) the offerings that no body uses (or you are not able to find a user)

(b) Simplification - Try to find the infrastructure components (compute engines, storage frameworks) that serve the same use-case, and see if you can converge into one. For example you can converge from HDF5 and S3 to just S3. Similarly from Hive and Spark to just Spark. Don't bring another infrastructure component in the mix, otherwise someone new in your place will make another HN post in future :)

(c) Documentation - Start building a place to document all the offerings that you have. Some wiki style solution or if it works for you something as simple as google docs. Or it could be some solution like Superset/Redash that is atleast bringing everything at one place

(d) Governance - Get some power users in the system, take their help in (i) identifying important datasets, (ii) adding information about existing datasets, (iii) can review a new code/dataset/production deployment

(e) start checking in Transform code/table DDL, all metadata into some git repository. This will automatically build some documentation overtime and take care of duplicate logic overtime
Hiring a data engineer would be your best option. The second best option would be to outsource the infra to a company like datacoral.com or pachyderm.io.

Be aware that a single database or tool is probably not a solution. You may get more rope to hang yourself with.

Any solution is going to be a function of data engineering, systems design and project management. If you lack one of those abilities then you'll need more of the other two to make up for it.

(Feel free to reach out to me on https://www.linkedin.com/in/iblaine/ if you like. I'm not selling anything...I have been in the DE industry for a while...)
How Venice Works [video]
Software engineers made 51% of real estate transactions in San Francisco in 2018
Could someone elaborate (or link to a reference) on Joe's point about how having many processes can help build fault tolerant systems? I know next to nothing about Erlang/Elixir or the actor model, but I'm curious about the underlying principles. My primary interest for now is not necessarily in scalability, but actually in engineering fault-tolerant systems.

EDIT: To give some more context... I'm thinking about software systems that must interact with the real world (eg: robotics) where it is very hard to give guarantees that a certain action would necessarily have the desired effect. Also, it is very common for agents to function with an incomplete model of the world (which essentially encodes their assumptions on how the world ought to respond to their actions) and keep adapting their responses and changing their model based on observations. (BTW, thinking along these lines lead to an approach called behavior-based robotics)
The one time you aren't being scammed is when you get something done and actually get to bill the insurance company.

Paying monthly installments on your insurance plan, only to have to pay for your healthcare costs anyway (first the deductible, then this new maximum out of pocket they added to me this year), that's the scam.
Now is a good time to look up Anders Zorn. He was perhaps the most esteemed Swedish painter during the late 1800s and early 1900s.
>I would like to hear what you propose, as an alternative for them.

https://en.wikipedia.org/wiki/Assertive_community_treatment
Upvoted. For some reason more recent skyscrapers (not just the ones mentioned in the article) lack the modesty of older constructions. They feel completely out of place when viewed from afar. Totally not like some of the pre-ww2 skyscrapers which are monumental or even flamboyant (think of Chrysler Building) yet work together to create a consistent and harmonious feeling.
It looks like the methodology was to try and match buyer's names to public profiles, so I suspect there will be some bias towards software engineers since they are more likely to have public profiles, but none the less an interesting find.

Also interesting is the fact that the number of public software companies in SF is expected to more than double this year.
That was my first thought when seeing Sanger's initial tweets about this. Someone else replied to the tweet pointing it out, and his response was along the lines of "that's not an off switch, it just blocks it". I'm not really sure why blocking the camera isn't strictly _better_ than a physical off switch, at the very least from a trust perspective
Disclaimer: I’m a cofounder of Segment [1], we build a product to help with these problems.

Given what you’ve shared here, it sounds less like your problems are related to scaling for data volume, and more related to all of the complexity that comes with a data pipeline. Instead of adding a bunch of new components, it sounds like you need just a few.

My concrete advice:

- Standardize and document the collection point for your data. Create a tracking plan which documents how data is generated. Have an API or Libraries which enforce the schema you want. If the sources of data are inconsistent, it’s going to be hard to link them together over time.
- Load all of the raw (but formatted) data onto S3 into a consistent format. This can be your long term base to start building a pipeline. And the source for loading data into a warehouse.
- Load that data into BigQuery (or potentially Postgres) for interactive querying of the raw data. For your dataset, the cost will be totally insignificant and results should give your analysts a way to explore your data from the consistent base.
- Have a set of airflow jobs which take that raw data and create normalized views in your database. Internally we call these “Golden” reports, and they are a more approachable means of querying your data for the questions you might ask all the time. The key is that these are built off the same raw data as the interactive queries.

We use Segment to manage all of the top three bullets (collect consistently, load into S3, load into a warehouse). Then we use airflow to create the golden reports that analysts query via Mode and Tableau. As other commenters have mentioned, there are a number of tools to do this (Stitch, Glue, Dataflow), but the key is getting consistency and a shared understanding of how data flows through your system.

This is a pattern we’ve started to see hundreds of customers converge on: a single collection API that pipes to object storage that is loaded into a warehouse for interactive queries. Custom pipelines are built with spark and Hadoop on this dataset, coordinated with airflow.

[1]: https://segment.com
The next generation keyboard (I'm guessing) will be a sealed set of raised keys that uses the taptic engine to simulate a keypress, similar to how the touchpads on MacBooks work now. This will be a good change because not only will it fix the issues of dust and crumbs and outright key failure, but it could also provide some protection against small spills onto the keyboard.

I just purchased a 2018 MacBook Pro and I actually kind of enjoy using the keyboard, but I'm worried about keyboard failures in the future, and on a $2500 laptop I shouldn't have to worry about that. I'm considering returning it and waiting for a 2019 model with a redesigned keyboard, but I haven't completely decided yet...and it's possible that Apple will just refresh the 2019 MacBooks with new processors, and not address the keyboard issues this year.
Don't think this analogy really applies. I mean you have complete control of the knife - it's a physical tool you hold in your hand. Can't compare this with any electronic device at all, which always does lots of stuff behind your back.
A running example would be really nice. I'd be much more likely to try it out if it was one click rather than a bunch of node stuff.
Well, I think a book to motivate you and get you out there, making something the quickest, is RE:WORK. It was written by the Ruby on Rails creator and his co founder at Basecamp. It’s pretty different than most books and most ideas are 1-2 pages long and I found that it got me motivated to build faster than any other book.
A Rapid Development for React Web Applications
Ahh gotcha — yes that makes a lot of sense! Would love to chat to you more about this stuff - will email you :)
> I’m talking about what should be.

What about journalism to you means it should have some kind of a higher calling? The people working in that industry have jobs and families to feed. If they simply did the right thing they wouldn't be employed very long if their competitors didn't follow the same path.

When 'journalists' write either left leaning or right leaning pieces are they being circumspect and covering all angles with fairness? Or are they just making sure they have food on the table and/or ensuring that readers continue to patronize the paper (or broadcast) that they work for?

It's nice to think that someone else who has to earn a living and feed their family should be 'above it all' but in reality that is never going to be the case.

Maybe a better idea is to teach children in school over and over that what you read and what is reality are two different things. Don't believe what you read.
As a workaround if you're on Android you can use Tasker to toggle GPS based on selected apps opening or closing, works rather well
I have largely stopped trusting dentists, unless I have pain. When my lifelong dentist fell out of network I went to a new dentist and they pushed all kinds of procedures and nonsense, saying I needed 3 fillings, etc. I've been to another dentist and they said things completely different. I don't understand how two professionals can draw polar opposite conclusions. It really seems they recommend needless procedures to fill their books of business and fleece costumers.

It's visible cavities and decay, it's not like you're diagnosing based on symptoms someone is describing to you, which I can understand for general doctors or specialists drawing blanks or different conclusions.
From a customer perspective I couldn’t disagree more. Calling on the phone to order something is often infuriating —- dealing with thick accents and loud background noise it could be awful to communicate.

With DoorDash and similar apps, you can place the order online and be precise about what you want, and get updates on where your order is.

It’s not perfect to be sure, but the “nightmare” is ridiculously exaggerated in the article. P
> “Some of the attacks target the DNS, in which unauthorized changes to the delegation structure of domain names are made, replacing the addresses of intended servers with addresses of machines controlled by the attackers. This particular type of attack, which targets the DNS, only works when DNSSEC is not in use,” ICANN stated.

Isn't that a bald-faced lie? The attacks in question are "someone stole my credentials and told my DNS provider to change the records to point elsewhere", as I understand them. DNSSEC merely protects against DNS spoofing on the internet backbone (not everywhere, because you're not supposed to turn out in when querying your ISP's DNS server, so the last mile isn't protected).
Thanks for raising - will fix!
What if someone suggests your taxes would rise by 10k? Are you concerned because that much money is an expenditure you can’t afford, or is it just behavioral and you’d rather prepay in the expense in your taxes than choose to spend it?
> extremely competent team of chip designer
> decades of experience in manufacturing
> plenty of money to snatch top talents

You understand that Blackberry created their own mobile driver because the 850 released in 1999 predated chip companies providing firmware, right?
I can’t imagine having so many mental issues I can’t get any benefit from shelter.

Secondly, most mentally ill people are not violently psychotic, you asshole.