skysight-inc/HackerNews huggingface dataset:
Hello guys. It’s me, Hanif. Just a response to some of the comments here:

1. I think it’s absolutely fair to criticize this aspect of the analysis: the relative frequencies of the backgrounds of data scientists have been presented as suggesting the success rate from each field. Many of the comments in the post itself made a similar critique. As I’ve acknowledged in my responses to these comments, what we need are the relative frequencies of applicants from the different backgrounds, not just hires. However, one can justify the inference about the success rate of, say, Statisticians and Actuaries if one has the prior belief that the relative frequency of statistician applicants to DS positions should be higher than the observed relative frequency of statistician hires (<1%!) to DS positions. I don’t think this is unreasonable.
2. I make a similar argument with regards to MOOCs/bootcamps: my prior belief is that the relative frequency of bootcamp-only applicants should be higher than the observed relative frequency of bootcamp-only hires. Hence my statement about necessity vs. sufficiency.
3. It’s somewhat more complicated for applicants with both degrees and MOOCs/bootcamps. I haven’t done this, but what I can do is to look at the education distribution for hires with and without MOOCs. If the education distributions were similar, it would suggest that MOOCs have negligible impact. If, however, there is a higher relative frequency of say Bachelor’s degrees in the MOOC category, that would suggest that MOOCs/bootcamps have some value-added impact.
4. An ideal prospective study for the above would be to extract a sample of individuals from a precursor role, say, data analysts (hence naturally controlling for education). Note which of them have MOOCs or bootcamps, then follow them up in time to see how many end up as data scientists in each category.
5. I might actually change that profile picture. It’s 3 years old, in more innocent times.
6. As it happens I have landed a data scientist position in Singapore and will be starting in September.
Pretty sure the microbes will shortly figure out how to survive in the gut together with the red seaweed. This party won't last long
The founders of Project Wren (https://projectwren.com/) went from building HR Software to building software to combat climate change.

We need profitable enterprises focused on this large global problem! I think the tides are starting to shift and would love to see the smartest minds of our generation focused on this issue!
Caveat emptor: if you process too many returns, Amazon will outright ban you from shopping there ever again. I know this might be anecdote, but it is true.
The original domesticated carrots were purple, and these remain the default carrot in parts of the world (eg. much of northern India).

https://en.wikipedia.org/wiki/Carrot#History
It is, considering the majority of the internet-connected population is not on fiber.
So you are saying that complicated conversations are so cognitively demanding that it is not possible to do any sort of math while engaged in one. On the other hand, math is not so cognitively demanding that you can't have a simple conversation concurrently.
Very interesting article that shows how far we've come. But I was completely turned off by the undertone of "GMO is OK, we've been doing it forever". Actual gene editing has so much more potential, not just for beneficial changes but for inadvertent catastrophes. What happens when GMO creates a mistake on the order of Thalidomide?
> Theoretically, this is what testing is for. How many companies do that?
Then I can just put ads on in the background while I'm cooking my dinner to save up ad bucks.

I guess the solution is to use some technology to ensure that the viewer is watching the ads, like in Black Mirror S01E02.
My point is more that if they can do all of those things and "game" the interview they seem like a good hire.
Yes, rareseeds.com is a great place for heirloom seeds and funky seeds/plants you can’t find elsewhere. Most of their stuff has been “tamed” but they will mention if it’s not ready for farming type cultivation.

They have an ancient watermelon that’s been breed to taste good. I’m going to start growing Paw Paws this year or next.
Looks like something you don't want to touch with a ten foot pole as an employee or an investor (or user for that matter). Avoid vexatious litigants.
Instagram is building a tool called Pyre (https://pyre-check.org/).
It’s always better to have multiple sensor modalities available.

This is the main takeaway. Unsurprising but interesting nonetheless. I'm working in the field and it confirms my experience.

However they have a big bias that need to be pointed out:

[...] we must be able to annotate this data at extremely high accuracy levels or the perception system’s performance will begin to regress.

Since Scale has a suite of data labeling products built for AV developers, [...]

Garbage in, garbage out; yes annotation quality matters. But they're neglecting very promising approaches that allow to leverage non-annotated datasets (typically standard rgb images) to train models, for example self-supervised learning from video. A great demonstration of the usefulness of self-supervision is monocular depth estimation: taking consecutive frames (2D images) we can estimate per pixel depth and camera ego-motion by training to wrap previous frames into future ones. The result is a model capable of predicting depth on individual 2D frames. See this paper [1][2] for example.

By using this kind of approach, we can lower the need for precisely annotated data.

[1] https://arxiv.org/abs/1904.04998

[2] more readable on mobile: https://www.arxiv-vanity.com/papers/1904.04998/

Edit: typo
When I was doing it they claimed that $65 per hour was a "good" rate for the Bay Area, and constantly pressured me to lower my rate to that. There were a few clients with names anyone here would recognize, but they were looking on Toptal specifically to get cheap labor for poorly specified work. Most of the clients were more like the stereotypical E-lance "small business" (read: one-person operation looking for cheap developers to build them the next Facebook for the equivalent of a $500 fixed fee).
Enjoy those modern bananas while you can, they'll be extinct soon enough.
> A complex system such as the climate (a huge understatement) is full of non linear of feedback loops that we don’t understand or are even aware of. It’s very hard to model such a system accurately.

I agree on principle, but OTOH you have the IPCC working by consensus and being too conservative in their conclusions. Also working with data that is years old since their process is slow.

For example, we've known about feedbacks for a very long time.

> Mario Molina, who shared the Nobel prize in chemistry in 1995 for his work on depletion of the ozone layer, said: “The IPCC report demonstrates that it is still possible to keep the climate relatively safe, provided we muster an unprecedented level of cooperation, extraordinary speed and heroic scale of action. But even with its description of the increasing impacts that lie ahead, the IPCC understates a key risk: that self-reinforcing feedback loops could push the climate system into chaos before we have time to tame our energy system, and the other sources of climate pollution.”

Source: https://www.theguardian.com/environment/2018/oct/09/tipping-...

Big media and some governments mostly listen to what the IPCC says so their responsibility is immense.
The bar for "working" is limited to perfect-path tests then?

As someone pointed out the other day, we do seem to be getting better about security -- less issues overall than there used to be. Likely a sign people are taking this seriously and frameworks have gotten better.

* Full Disclosure Mailing List || https://seclists.org/fulldisclosure/

Ultimately, a big issue with security is that nobody thinks of it as their job. "Oh cool, I made code my non-technical boss said was good enough! Time to clock out!" And that's really flawed. Security is everyone's job. Everyone. The best system is still susceptible to bad passwords. (=

What's security training look like in your org? Most likely there isn't any. So... pitch it. Here's a good template to get started:

* For Everyone - PagerDuty Security Training || https://sudo.pagerduty.com/for_everyone/
Water vapor is by far the biggest green house gas
Nobody made a rule that says cars can't have cameras in more than one location.
Yup, this fake corporate and civil socialism implemented by FDR and LBJ through farcical social "advancements" and the culling of our rights to live free (meaning not taxed by virtue of being born - Federal Reserve Act) and to choose our local politics over Federal infringement (17th amendment) is very much like the racist Uncle Tom days of plantations. Utterly laughable considering the self delluded state "Liberals" believe themselves to live in.

Slaves outnumbered Slavers scores to 1, and it wasn't the guns that kept them in check - it was psychology.

Racism is psychological and the people who convinced you Socialism is the way are the same ones looking to short circuit the capitalist system that allowed them to advance. Cause that's how you win this game of thrones, by being the only one who gets to choose.
I meant that adding seaweed to a cow's diet seems unlikely to increase the demand on our limited fresh water supply. AFAIK, there's no risk in running out of seawater.
Want to hire me? I’m a reasonably competent programmer already, pretty good designer. Most recently I wanted to familiarize myself with the basics of C++ so I wrote my own big integer library.
Didn't he say that of the code for a particular Lisp interpreter/REPL?
> Lets say we needed to deprecate a function named ‘fn’ for a better named function called called ‘add’.

wondering why they would deprecate fn instead of renaming everywhere with 'add'. I thought that was one of the main big sells of the monolith.
One time I was having trouble with my PayPal MasterCard and called them from Mexico. I was sort of stranded because I needed the card to work to get an Uber and it was really hot so I didn't want to walk home. I don't remember if it was the 800 number or the local area code in the US. Probably 800 number. But I was in Mexico and dialed it the same way as I would in the US.

Someone from Mexico answered and initially it sounded very similar to what PayPal employees really say. It took like two minutes before I was really suspicious. I was really tired that day and didn't want to walk though so I kept trying to give them the benefit of the doubt. But a few minutes after that they actually asked me to go into 7/11 and buy a Google Play card for $150 or something. At that point it was just really obviously a scam and I decided to walk home. Called the real PayPal from my computer using the US area code.
How were they going to charge him for 9 FTEs for 6 months to uncheck a checkbox?
So you are implying that caring about the environment is a matter of natural selection and genetics?
Yes I did. And a couple of days later, 2 detectives showed up, verified the item was in my possession and went on to the next person on their list. Incidentally they also marked the item by etching a number on it to make sure that they didn’t see it again, i.e., to rule out my cooperating with the person they were looking for.
Having a redundant system is the key here.

Also provides a reliable source of data, if humans have a LiDAR in their system then we would use it to improve our decisions.

I don’t see why we should limit the AV.
Ha! Stephen Dolan had to add this to jq quite sometime back, making it preserve object key input order on output. And yes, it's infuriating, but it's also somewhat convenient, and yes, there really is software out there that cares about object key order (sigh).
Feed is an efficient way of transferring water from wet places to dry places.
Related: https://news.ycombinator.com/item?id=20704138
Again, though, the practice is pervasive. Are the "best value!" items you see at the ends of grocery aisles really the best value? How often are things "on sale" from prices for which they were never offered?

I mean, I guess it's OK to be outraged about this. But doing it specifically about Amazon seems maybe a little unobjective.
> If this were true, there wouldn't be a single device on the planet, including phones, servers, etc. with remote update functionality that would be secure, ever.

You're so, so close to waking up from the matrix. Very excited to have you join us in the real world soon! Follow the white rabbit.
There's also an interesting comparison to Australian "bush tucker" (native foods), which have been eaten by Aboriginals for thousands of years but not systematically cultivated and bred:

https://en.wikipedia.org/wiki/Bush_tucker

Surprise surprise, more or less all bush fruit are small, have large seeds/stones, and taste inoffensive but rarely delicious. I wonder what a bush plum or lilly-pilly would look & taste like after 4,000 years of systematic breeding?

https://en.wikipedia.org/wiki/Carissa_spinarum (bonus: poisonous when unripe)

https://en.wikipedia.org/wiki/Syzygium_smithii
Companies have been doing this for at least the last 8 years, just based on my own experience working for a company that provided a solution in this space. Amazon isn’t unique or even remotely the first. They’re just using one of those social media reputation dashboards to manage complaints and support issues before they go viral online.
If you don't like YouTube demonetizing videos, don't upload your videos to the site then. Personally I don't have a problem with people discussing or making videos of whatever they want to.

I don't like the fact that YouTube with the recommender and Autoplay is stochastic mind control. It's programmed to make people watch loads of YouTube, and the mechanism that it has discovered to do that is to nudge people to obsess over fringe things, causing them to watch videos about these things for hours per day.
I agree, the lack of social responsibility of corporate managers at all levels is terribly anachronistic.
I'm an early adopter to flickr, more than a decade ago.
I can say that I did rely on Camera Roll and felt it was a much needed, but somewhat incomplete in scope, tool to flickr.
I would be one of the users who relied on it, after years using Organizr. So, there are some of us old timers who used it regularly. Luckily, a strong experience with Organizr allowed me to cope with its downtime.
I look forward to its 100% bill of health return.
Might I suggest as a request, that Camera Roll be incorporated in some way to view and manage our Favorites Album. There is no longer any way to quickly access a list of Favorite images from a decade ago without trying to scroll through countless webpages...only to find the browser and flickr server crashes. That's been a very old bug on this interface, btw.

Thanks,
WNG Image and Design
Isn't that just like an average KickStarter campaign?
Seems like they are either incompetent or cooking the data. When converting from the image to a top view shape / outline, one would design and or train the system to adjust for perspective. Clearly they have not bothered to do that.

And the title is inflammatory. Nobody who understands the discussion is talking only about camera versus lidar. It’s more about camera+radar versus camera+radar+lidar, and other comparisons between other hybrid or standalone sensor combinations. It’s not as simple as one versus the other... surprised we still have to point this out to them.
"Someone else" (Amazon) was always getting an extra cut. That's exactly the value proposition of a retailer. I mean, there's some complexity here that you don't see with physical retail due to the affiliate relationship, but the principle doesn't change. The whole point to going to a store for an item is that you know they'll stock it in an easily accessible form and quantity that you would have a hard time sourcing from an original manufacturer. And we've always been willing to pay for that service with a markup.
A third static analyzer for Python! I wonder what this landscape is going to look like in a few years... (There are probably others, but this is the first I'd heard of Pyre, and I'd only known about mypy and pytype up until today - if there are others I'd love to hear about them!)

On a more substantive note, static analysis is one of those things that sounds like you shouldn't even need it but in reality is a surprisingly powerful tool, for the same reason that all code has bugs: humans honestly suck at writing code. (For those of you that have seen Bret Victor's "Inventing on Principle", I feel like static analysis is one of the first steps in getting to the kind of feedback loop he envisions.)

We have all these ideas around how things should work, but so frequently our mental model diverges slightly from what's actually happening (which dynamic typing only makes worse, and NPEs even more so) and that's on top of the mistakes we make (like this one in the JDK, which was caught using ErrorProne, a static analyzer for Java: https://bugs.openjdk.java.net/browse/JDK-8176402).

Also, a fun tidbit about using static analysis to apply automated code fixes: this is basically when you realize you need auto-formatters, because you still want your code to be indented like a sane person would after applying a change like this. (And imagine how much more complex the autoformatter/autofixer have to be when you think about how they have to do things like preserve comments, etc!)
How can you not have a demonitized gap? Advertisers aren't going to want to have they ads on content that hasn't been vetted.
Yes you’re right, they do.
Errors to avoid in OKRs?
> The central bank is 100% confident that they are on the brink of deflationary spiral; much like Greece or Japan.

Is this a good thing for people who hold cash assets? Meaning, would you be able to buy say a $40,000 car for $20,000 in case this happens?
This. Console access is one of the handiest user-level features of VMware in my opinion.
This is probably a really sensitive topic but.. Donating your time to help get politicians elected who are serious about doing what's necessary may be the best thing you can do.
I'd say that culture, like religion, is passed down by the parents. There are a lot of calls and attitudes that are passed down.
What a crazy life ! Great read. So what know the guys is out and will scam again?
We didn’t explore causal graphs because doing so would require manually creating a causal graph for each relationship that you wish to explore. Our goal was to create an automated approach that could provide an estimate of the treatment effect for any page/event within your app.
I think Noah's point was that homebrew wasn't a simple tool. And, not only was it evidence of the developer's capabilities but it was also used extensively by all of the employer's own engineers. A tool they all relied heavily on to get their own jobs done. Making it clear he could create software they found important for their own production. And, made it possible for them to deliver service to billions of users.

He got nervous on the whiteboard when asked a relatively simple question. They proceeded to shut him out of the position, completely disregarding his ample qualifications signified by their extensive use of his product. Which is a kind of tragedy.

Comparing an A/C unit to rocket science isn't an equivalent comparison. I can't think of a good analogy to be honest, but that isn't even close.

No other field I can think of really has the problem ours does. You can either make A/C units and rockets, or you can't.

But, in software engineering, it is highly possible to be able to say..... develop high quality GPS mapping software from scratch used by the Navy. Then turn around and have a company making word processing software reject you because you couldn't write a Mandelbrot fractal generator from memory on a whiteboard with a marker.

The fractal generator has zero to do with their word processing product and will never be a useful test of your abilities. Clearly the guy who made GPS mapping software can figure out how to write a word processor. Not to mention, they already have the word processor written so he'd only be assisting them in maintaining their existing code. In other words, he's already demonstrated his ability to write and maintain code of a high caliber. None of it makes any sense.

I really don't think any other field has this issue. It is weird as hell, and unusually pervasive in our culture.

Edit: It won't let me hit reply on your comment below. I think we've hit the maximum length for this comment thread. But, I'll respond by saying I don't know what position he was applying for. I'm not sure he ever explained beyond that tweet. The first I heard about it was Noah mentioning it. But, if it made their lives easier so they can deliver products on time to their own customers it makes sense they have a position somewhere in their company that would be a fit for him. At the very least it proves he is a good developer. Massive systems like the ones Google creates are made by many good engineers working together to make it possible.

The point was they knew he was a good dev.
Ups Testing Self-Driving Trucks in Arizona
I highly recommend this related podcast discussing the project to build a synthetic cell
https://www.preposterousuniverse.com/podcast/2019/07/22/56-k...
I was waiting to see how long it will take for media to find the tweet thread and publish as news. Man what a world we live in.
How the cloud is more complex than you think
No, I wasn’t experimenting, but I haven’t had any car accidents in my life, and I find that more valuable
Lame. Is there any real reason to do this? Does the code take a lot of maintenance, aimed as it is against a protocol from 1971? Is there a reason to cut people off from easy interoperability with links on the older parts of the web, many of which surprisingly do still work?

FTP sucks, sure, we get it. No reason to use it now. Still, Google seems to have a mission of deprecating the old Web, from their search results that push that kind of content down, to their browser deprecations of FTP and Flash and Java applets. How is one supposed to even see the old parts of the web anymore?
Indeed, solutions for all needs. For example, presidential elections. Tweetattackspro, buyaccs.com anti-captcha.com

https://www.thedailybeast.com/i-bought-a-russian-bot-army-fo...

Anyone can start this up. Just get some bitcoin ready.
Is this comment historically accurate? How often are lands conquered? I think not, to what you say. Also, this only accomodates the last ~3000 years of human history. Before the agricultural revolution, humans lived a very different life as apex predators. It was not until we can grow and store crops that armies could exist.
In China, Apple Isn’t the Privacy Advocate It Claims to Be
Write some software that reordered it probably.
Know your customers’ “jobs to be done”
Tibetan monks have to memorize pages every day, starting as little children, and be able to recite any if it any time for the rest of their lives.

When they start they don't know what any of the words mean.

I don't understand how it is possible.

But I also don't understand how a Harlem Globetrotter can loft a ball backward, without looking, from mid-court, and sink it first try.

There is a lot I will never understand.
But at the same time people don't think much about getting in their cars and driving to work or the grocery store.

So it seems that a truly accurate 3D representations of the world are not necessary, at least for driving. Perhaps it's the resolution? Looking at the samples in the article, they are just terribly fuzzy, with a narrow field of view. If I had to drive and only see the world through that kind of view, I don't think I would be doing very well.
I think this article from bellingcat earlier today is better overall: https://www.bellingcat.com/news/americas/2019/08/15/amazons-...

Here's the list of account names using Sprinklr that was mentioned in the NYT article: https://twitter.com/AlexBNewhouse/status/1162036960027742209