skysight-inc/HackerNews huggingface dataset:
I really hate the H1B issue. It should be open to "high skilled, essential/lacking labor". We have many universities in the US that will grant an MS or similar degree without much substance for a nominal (but not negligible fee).

A simple revamp to the program would be simple:
- Absolutely zero visas (or very few) to any of the outsourcing / bodyshop companies - Tata, etc.

- A company found bringing workers in on a Visa to be trained and then lay people off, the company should face a significant fine.

- Companys have to show if there is on one side a significant downsizing and then sudden rehiring that includes H1Bs, that they didn't offer training and mentorship before laying some of the original people off. IBM and Disney come to mind.

- Keep or impose caps by region. Sorry, we are a melting pot, but certain countries have significantly higher numbers trying to come in than others. The best companies and groups I worked with were diverse rather than monoculture work places.

Personally, I do like the score based system that Canada and Australia generally have.

EDIT/Addition:

- The original mentality of cost being the overarching factor has become less en vogue. Many companies started insourcing again. That said, it's still not wholly reversed most of the outsourcing going on.

I know outsourcing does not equate with H1B wholly, but there are companies that abuse the visas to hire cheaper, have locals train, and fire the locals.
My impression of IEEE and ACM are that they're more oriented toward academia (e.g., publishing peer-reviewed journals) and less oriented toward the average working programmer. We might do better with an organization that's primarily focused on improving working conditions rather than propagating technical information. I think there's room for both kinds of organizations.
Also, if he ever needs to check if a variable is a number, there is another one-line-of-JS project on npm that does just that!
try RTFM, doesn't work too!!
Right, that was poor diction. I have since updated articulating my viewpoint more correctly: that startups have needs which are different from a child's need but are just as pressing and demanding due to the love you've invested in both.
It's a good idea to install elsewhere, because there are other packages that install into /usr/local (TeX, for example) that aren't in homebrew. I like having my homebrew installation totally separate -- I feel significantly more comfortable upgrading my software to bleeding edge versions if there's no chance of some non-homebrew brain-dead configure script grabbing them. I install to ~/.homebrew, and everything works fine for me. Of course, I don't use ruby at all, or install Python or Emacs packages via homebrew, so I haven't had any of the problems that the homebrew folks warn about on their installation page.
Ah Russian math lecturers in first year courses. Always making it perfectly clear that as far as they're concerned they might as well be lecturing to a bunch of 8 year olds.
I know some engineers who work or worked there. Aside from the normal engineers (tools, backend, etc.), they are essentially high-priced tech consultants sent to companies and orgs that have no internal software talent at all, to help them analyze their (atrociously stored/managed) data. None of them work on anything shady. They essentially build web apps and backend apps to analyze and visualize data. Nothing vodoo military about it, although the military is a customer.

The pay actually isn't actually particularly high for SV standards. I think Facebook outpays them, for example - and Facebook is less selective than Palantir. But their reputation is that they're full of top engineers and you'll learn a ton there, and that's why the people I know work/worked there.

The downsides I've heard about are no work/life balance and fratty culture. Those are real downside, not the FUD about ethics.
> If you want metric byte storage sizes, then
> first redefine a byte as 10 bits.

I think this is the funniest rationale for using base-2 that I have come across. It makes no sense.

Memory works conveniently with base-2 because of the address lines. n address lines can select 2^n addresses. The size of the bytes stored at those addresses is completely irrelevant.

Your logic would suggest that US gallons, which contain exactly 256 tablespoons, should use binary units, as in "%1010 gallons of gas please" or "that tanker truck can carry 11.3 KGallons (where K equals 1,024)".

Base-2 definitely works most neatly for memory capacity, because memory chips are exactly a power of two, always. There's no need to invoke the (irrelevant) size of the byte.
Was really hoping someone would share a comment like this, thank you. I feel like I just got completely screwed by Kimono...
> face the income and career insecurity that comes with quitting

Get a firm job offer before you give your notice. You won't face any income uncertainty, and you'll have much more leverage while negotiating with potential new employers (unless let you it show that you're really desperate to leave your current gig).
https://bitbucket.org/snippets/jdubray/
Can also share more code privately
Is there a reason why people downvoted this? It'd be nice to get feedback on why people thought it was not helpful or constructive.
I'm always disappointed when I read of Palantir acquihires.

Technically talented employees receive a pittance and are sold into the Palantir machine, a morally unscrupulous high-tech consulting shop that takes in talented engineers and churns out one-off software black boxes designed to lock in customers and extract a long-term tax.
> All STEM graduates are not automatically qualified for all open positions.

What does this tautology offer to the topic? Common sense obvious true, not every STEM graduate is universally qualified for every open position. So ... ???

> And even then, STEM graduates are not automatically going to move to any random place in the US just because there is an open position there.

Again, common sense true. So ... ???

> Lastly, if an employer has an open position, they are not going to wait indefinitely for a candidate to show up.

OK, true. So ... ??? Is your assumption then that they would have to wait indefinitely and that no candidate would ever show up? cited evidence of this is appreciated. Links to studies, large amounts of unfulfillable postings are welcomed.

> Furthermore, because of the additional legal costs involved in hiring H1-B candidates, they can be more expensive than a local citizen.

So ...?

Nothing stated above bears any light to the argument (flawed or otherwise), it's just a sequence of common sense things that one can say about a labour market.

This topic's referenced article is specific in its position in claiming there is no shortage of STEM qualified applicants and presents cited statistics and studies. If you have counter data and studies, please present their facts and cite links.
This is off topic (sorry), but I'm an animal rights activist and I have no problem with hunting or hunters. As long as the animal is consumed as food, no problem. Trophy hunters are a different story.
Er, no.

#include <stdio.h>

static char buffer[80];
static char *bufferp=buffer;

int main(int argc, char *argv[] )
{
char localbuffer[80];
char *localbufferp=localbuffer;
printf("buffer: %ld\n",sizeof buffer);
printf("bufferp: %ld\n",sizeof bufferp);
printf("localbuffer: %ld\n",sizeof localbuffer);
printf("localbufferp: %ld\n",sizeof localbufferp);
}

Output:

marcel@sarek[tmp]./a.out
buffer: 80
bufferp: 8
localbuffer: 80
localbufferp: 8

As you can see, it depends entirely on whether you give sizeof the array or the pointer.
If you need to load everything up front then you might need a loading page like Gmail has
Very cool!

I've been working for a while on extracting "semantic" from naked text (mostly news). One of the big limits of word2vec is that the semantic should be related with words proximity in sentences, which works in some cases, but not always.

I'll give an example for all: travel/geography. If you query something like Italy [1], the results are other European states. But if you're looking for news in Italy, or to plan a vacation to Italy, or for some Italian food... or anything related to Italy itself, you probably don't expect "Spain" to be the first result.

It would be nice to have some sort of easy way in word2vec to define domains and their relationship with words proximity in sentences, to overcome situations like this one.

[1] https://sense2vec.spacy.io/?Italy%7CGPE
On Go (2009)
Well, sure, but SIMD more than compensates for all of that, given how hard autovectorization is. In fact, I think with things like AVX and NEON becoming ubiquitous, you can get more benefit out of writing in assembly (or intrinsics) than any time I can think of in the past 10 years.
The first name to show up when I search 'evil man' is 'Mr Rogers'...what the hell, reddit.

(first name, not first word, had to scroll down a bit, past words like 'Nazi soldier' and 'evildoer')
Apple and Uber are not as much of a resume boost (in Silicon Valley, to a SV recruiter). Palantir is sort of on the level of quant finance / Dropbox 2 years ago.

Apple and Uber are hiring like crazy and so it's quite a lot easier to get in there as an intern than at Palantir (which is more selective).
Indeed accurate: https://sense2vec.spacy.io/?turd%7CNOUN
First he dismisses React, then he says his code is looking a lot like React without the ceremony, then he shows code samples that look a lot like React except they use string concatenation (hello XSS) to build the DOM (goodbye DOM diffing) and instead of following the modern React idea of small composable components they are huge and bloated.

Is this satire?
The fact that you're blaming me for sticking up for people who are dismissed as inferior due to the tech the use, and after reading the comment chain, speaks volumes about the kind of people who frequent this site, and the kind of person you are too. Could you just ban this account please
I heard it as: when you are brutally honest with someone, they will remember the brutality but not the honesty.
> people have been angry for a while and they're angry for reasons

The reason they're mad here, evidently, is that Stephen made a joke to his friend. It doesn't seem like it's Stephen who failed to assume good faith.
Building a Startup in 45 Minutes per Day While Deployed in Iraq
I don't think wordnet (and similar) will ever be obsolete. From my point of view, they are a "curated" subset of the semantic that you can obtain with word2vec. The latter can fail, also on very trivial cases (I discussed geography in another comment), and curated+structured knowledge bases such as wordnet allow to overcome the limitations of any machine learning algorithm.
Except that some % of PhDs go on to be professors (who do a real service), and every once in a while you get a PhD student whose work probably contributes hundreds of millions of value to society. And every couple years or so, you have a PhD student who contributes billions to society in value.

Not to mention PhDs usually have to TA (teach students), which accounts for some of their pay.
Software developers already have the best benefits, a union will not be an improvement.
> Can install software to a home directory and so does not require sudo
> Install software not packaged by the native distribution
> Install up-to-date versions of software when the native distribution is old
> Use the same package manager to manage both your Mac and Linux machines

From the "Features" section of the Github.

It could also likely be used so make a "safe" Linux distro, where the base system is mounted read only, with a writable home partition.
Sorry here's another link:

http://www.forbes.com/sites/andygreenberg/2013/06/07/startup...

Second to last paragraph.

As for the sex accusations, who knows what really happened. Though the article doesn't exactly paint him- edit- either of them- in a flattering light. It does show poor judgement for the dude to get involved with a student he is mentoring. Then again, he also stumped on the campaign trail for Rand Paul of all people and also pals around with Mitt Romney.
Would love to see this as an official part of YouTube! YouTube has large amounts of music that is hard to find anywhere else online (vinyl only, etc). For listening I usually make playlists on YouTube, but this is incredibly clunky.
If you are serious about this question, you should begin by specifying what "data scientist" means in this context. Otherwise, whatever answers your query engenders will be lost in the noise.
And you'd now have a 3-way split of resources (both learning materials and developers) between 2, 3 and (an imaginary) 4.
That was me ;) Atom's great until you save and goimports locks up the whole UI. Think that is the protobuf interface used by the client and server in our tracing tool, so handy having them all open! https://github.com/mondough/phosphor
I never met the man, but wasn't Steve Jobs famous for this too?

I've read supposed accounts where he sounded worse than Nest's management.
Is that because you personally have such enormous bargaining power (inspite of a flood of STEM workers)?

Or is it because employers always spontaneously pay the highest wages they possibly can and never, ever treat employees unethically let alone break the law?

Or is that because the current political system is so very, very good at protecting the interests of employees or because market forces do it so very, very, well?
Look, my point is, it's not valid to bring up the sex accusations, because they were proved to be false.
Yay! Glad you enjoyed it.
> how do you grow your business into a multibillion-dollar company?

Many people will allow themselves to be abused at work without doing anything about it. They'll commiserate with their coworkers, or vent on Glassdoor, or fantasize about leaving, but without ever following through.

So some good ones leave, and the rest get ridden hard and work on weekends and stay late to make their managers happy and rich.
Its a two edged sword, sometimes you're doing great - but the fact they're shutting down means they're not doing great but have a team of experts in parsing that will benefit Palantir since they're likely into analysing surface and deep web data and just data in general :)

basically its like hiring a bunch of regex gods because you have regex problems on large scale
....now we know
Yeah, but is it easy to find jobs in those places? Most economic activity happens in the big cities, so most people will want to live close to those cities in order to be in a reasonable commuting distance from work.

Of course, you could do remote work. But those kinds of jobs are not yet the norm. Plus, a lot of jobs require daily face-to-face interaction with clients, coworkers, or higher-ups.
Encoding Statistical Independence, Statically
Caps by employment region would be "interesting" since it seems H1B is clamored for uniquely in Silicon Valley, where (as an American engineer) there are really few Americans, which contrasts entirely with other regions of the United States. It seems that is because Americans are averse to the unaffordability of SV, and recognize that unaffordability is a trap for foreigners.
Here's an upvote, because I was thinking exactly the same as someone who has used most of these "scraping" tools.
Viking Ships (1999)
Sounds like a safe place :) Being outraged is still within the range of self expression. You are close to your limits of harming others, but not automatically.
Personally it's because every union seems to be either so big as to be corrupt, highly inefficient and far too busy with petty power struggles to ever care about my insignificant problems or so small as to be essentially powerless.
Is it? There's plenty of tropes about dumb students who make it through Ivy league schools because their parents "bought a new wing" or somesuch. Same for rich children avoiding military service.
Are you still taking questions for the AMA you posted in the thread?
It's impressive how good it is.

But there is still room for improvement. Searching "Haskell" leads to Clojure and C++. Yes, these are both Programming Languages, but out of all I personally wouldn't have said C++. :)

"Scheme" leads to Haskell, witch is very fitting, and Prolog, witch seems to fit as a "university language" aswell.

For "Agda" it outputs "typeclasses" at 74%. This is a much discussed topic. But for a "truly" semantic understand It should know that both "Agda" and "Haskell" are in the same category and that "typeclasses" is a property that elments in this category have or don't have.

Still, very impressive. But not the singularity jet.
This piece appears to beg the question - is there any evidence that C students are more successful after graduation? Or that students who get a C grade are doing any of the things he talks about? Or is this a weak justification for why his own grades were poor and perhaps the average C student is actually a stressed out kid who is working a minimum wage job to help pay expenses and following every rule they can find/spending 40 hours a week drinking and watching football?
Not long ago, I took an Uber ride from a driver who's main job is as an online dating consultant. It apparently is a fairly lucrative niche.
Kanye West asks Facebook's Zuckerberg for $1B for his 'ideas'
Thanks for keeping the lights on. Please do make preparations with archive.org or Archive Team to preserve the projects on SourceForge just in case your current strategy does not work out.
Wow, what a gracious reply. Best of luck to you both!
Oh boy. Auto play vimeo video that wouldn't let me click pause? Definitely need to fix that.
Thought-provoking post. The responses below are my opinion, but I do think there is a good chance they're the opinion of the majority.

>Why do more people use Python over Perl?

Two factors, it came packaged by default on Linux distros and OSX. That was really a king maker in itself. Then add that the language itself is that much better in most people's eyes. People don't really like obfuscated code unless it's the only game in town (ie. the 1990's and Perl).

>Watch out for Go and Swift. These statically typed languages are better for large projects.

It will be interesting. But both have pretty big flaws so any extraordinary claims will demand extraordinary evidence of success. Neither language has exceptions. Go is controversial in its conservatism and this is likely due to the fact the entire team had to agree to add features to it. So it did end up odd in some ways because of that. Swift is in a constant state of change. Both are missing a lot of libraries. I've tried to use Go and see many abandoned projects like Martini and newer projects seemingly abandoned like Revel. The Go faithful will tell you this is because frameworks are not The Go Way, and point you to the Gorilla toolkit instead. I'm not sure I buy that story unquestioned, or not.
On the whole, I'm unconvinced due to their language characteristics, that Go or Swift are ideal replacements for either Python or Ruby for webapps.

>These statically typed languages are better for large projects.

Yes, but I wouldn't want to write poorly engineered, bloated and difficult to port code even with static typing. I don't write massive 500KLOC projects so it's not a problem for me, but I have a policy of modularization with my code that would stem the issue (regardless of static typing or not). When you get to 500KLOC, there is no magic fix. Static typing, modularized code, nothing is a cure and hardly even a band-aid.

As a result of all this, I'm still onboard the Python train. I honestly do not see myself hopping off with PyPy being here today and Pyjion, Nuitka, and Pyston all up and coming. I would not be discouraged if a Rubyist either.

Of particular interest is that Microsoft is working on Pyjion and if it works out, API changes will be made to so that you download a module and JIT right from CPython3.

I'm probably more interested in Rust over Go and Swift at this point. Go is an island, similar to Java that way and not C ABI compatible, this makes it a less than ideal wholesale replacement of Python or Ruby. Swift is just too young on both the server and iOS. Rust itself is the best language I will likely never need to use, but its existence unlike most languages, makes an enormous amount of sense.
I'm from New Zealand: Kakapos are one of my favourite animals in the world (along with Tuataras!), so I've got nothing but respect for the work you're doing :)

Is there a way someone like myself can help? Donations, etc.
Unfortunately, as you can see on the wiki article, accuracy and precision have a different meaning for binary classifiers.
Don't put hate on people with money and speak different language you don't understand, my constructive suggestion for you is that earn it by yourself. Guess what, these richest people also pay the most expensive tax in Canada, and you are welcome.
> Why not simply require the component to call a function to indicate it's changed a couple variables in its state

This is actually roughly what knockout and ember (pre-glimmer) do. They are known as KVO (key-value observer) systems and have implementation challenges of their own: knowing when to batch operations, dealing w/ computed properties and rx glitches (in reactive systems, a "glitch" is the name given to temporary inconsistencies that occur between stable states), and added complexity in terms of requiring the model layer to be observable-based (as opposed to POJOs in Angular/React/friends).

Also, high quality KVO systems are far more difficult to implement. To my knowledge, Vue is currently the fastest KVO-based javascript library in existence and in order to support its POJO-like model API, it's significantly larger than Snabbdom (which is one of the fastest vdom implementations currently, despite clocking at a mere 200-300 LOC)

AFAIK, most of the challenges faced by KVO system have not been as extensively explored (at least by the javascript community) as virtual dom algorithm optimizations have, so currently I believe high quality virtual dom libraries are likely to perform better at various real life scenarios than current state-of-art KVO systems.
Try "Haruhi" and "Madoka". IMHO impressive.
How long has Comcast been around for?
The vast majority of the factory jobs are automatable, and will vanish in time, and have likely been continually vanishing.

That reality would sit poorly with Americans who didn't see it coming, and certainly with those trying to use offshore manufacturing for political support, but that's been the plan all along.
Well, you can't do this in the web API obviously, but there are a few ways you could do this. This is a good idea really -- thanks for bringing this up.

One way would be to predefine the entity types or tags that you want to get in your results. So you could ask for things like Italy that are nouns.

The other way is to use the vector space. The classic demonstration of this is the arithmetic, doing like "Italy|GPE - *|GPE + food". My results for this have been very mixed. I wouldn't expect the query above to work.

I would think you'd have more luck specifying the query as a combination of constraints: first query for foods in some way, and then sort them by distance from Italy.
Not everyone has a set schedule like that, though.
I'll be the first to agree that Excel "actually works" and "solves a real problem". But if you had actually used Kimono, you'd know that it was one of those tools that was just finicky enough to be unusable by "regular people". And just weak enough to not be productive for programmers.