skysight-inc/HackerNews huggingface dataset:
In spite of assurances from business process owners that the underlying data sources are clean ... they almost certainly are not.

Multiple legacy systems with no consistent cross reference to unambiguously identify the same customer. Assured that systems have been gone through and all the names made consistent. Consistency for a human is not consistency for a computer. "Commers Ltd" is not the same as "Commers Ltd." And, isn't it lovely when a salesperson decides to add a location to a customer name. Now we have "Commers Ltd Dallas" as a unique customer. Business process discipline is often lacking and will mess you up.

Subscription data sources that change their schema with no notification to paying customers. And, when you are scraping data from websites you need to constantly be checking that your scrapers are still working properly. Source websites change regularly.

Crazy processes like entering a negative invoice to indicate a refund to customers but forgetting to zero out the cost of goods related to the invoice. We may have refunded the money but we didn't do the work twice. Arggh! Errors abound.
Late stage?
Well it allows them the set prices on per-lead basis. I suppose it is a form of price discrimination. A lot of companies do it in the open and it seems to work for them - Uber/Amazon/Expedia/Sears/etc. A lot of cloud companies are doing it as well where they charge exorbitant prices for "enterprise" grade service.
I believe it's a result of install-this-app ads being the only format that delivers measurable results.
Is the nonprofit you work for affiliated with a particular University on the south side?

I can sympathize with having to deal with VBA. I'm working in a lab that deals with lots of questionnaire data and uses Access as the main tool for gathering said data because that's the way things have been done in the past, despite the fact that nobody in the lab really knows how it works.

I can also sympathize with having your network drives go down and render everything inoperable. Everything we use in the lab is stored on an offsite network drive, probably because of HIPAA compliance, and said network drive has dropped out twice in the past couple weeks. Once for almost an entire day, and once for an hour or two.

Best of luck with the job search.
> I'm talking simple, cheap design choices in this case, that could've eliminated the whole Mirai debauchery.

This is utterly ignorant of the facts, Mirai took advantage of weak passwords to spread, but was not dependent upon them.
It is unclear to me how issuing certificates you have no intention of using in any way constitutes "other changes to increase performance or security of your website."

In fact it can only decrease security, as it increases the number of valid certificates in existence for a domain, and the number of certificates available to be compromised.

Logically, you can only claim that this behaviour falls under "other changes to increase performance or security of your website" if you actually make use of the SSL certificate after obtaining it.
I personally love this for statistical inference:
http://www.springer.com/cn/book/9780387402727

and this for statistical ML:
http://statweb.stanford.edu/~tibs/ElemStatLearn/
Visual Studio Code 1.7
Assuming users are not on both mobile and desktop, then the desktop is around 100 million (DAU less Mobile DAUs):

Daily active users (DAUs) – DAUs were 1.18 billion on average for September 2016, an increase of 17% year-over-year.
Mobile DAUs – Mobile DAUs were 1.09 billion on average for September 2016, an increase of 22% year-over-year.
India does have property tax.

https://en.wikipedia.org/wiki/Property_tax#India
> Is advertising dead on the desktop?

It could mean that the only thing keeping the ad supported internet alive is the lack of AD blockers on mobile operating systems.
Things I Wish I'd Known At 20
Half of Europe has dreamworld house prices which are unsustainable. As the boomers die out, the demographic changes are going to annihilate the fools today, taking out 450k for a bedsit. I have to say, I am looking forward to the peasants-turned-property-magnates getting a rude awakening :)
Buying the cheapest SKU of O365 that has Microsoft Teams is actually cheaper than per-person cost for Slack.
Nice little plug for your app there haha I'm kidding.

I'm skeptical that you'd be able to lose that kind of weight just by breathing exercises.

Thankfully I'm not super overweight but I want to be cut. Gotta start lifting again.

I want to run though I think that makes your mind clear or soemthing.
It's the stream; Facebook has a stream and Google does not.

After various experiments, I think people have figured out that the #1 way to make money on mobile is to have a stream of information that people willingly spend lots of time in. You just inject ads into that stream and bingo: profit. It's much harder for Google to win big here because while they have a very useful tool, people don't linger in it. They usually use it to get in, get what they need, and get out.
I saw a few comparisons to Slack and Google Wave.

But, seriously, this is likely more like Microsoft Groove (originally from Groove Networks) brought to the web. This idea has been around for a very long time.
Google should start show ads on top of Facebook. If you are not paying, you are the product ;)
I think most experienced people would agree that a technical interview could be appropriate for someone with 0-5 years of work experience.

For somebody with 10+ years of experience, who you can find that talks the talk during a real interview, CS101-level quizzes are insulting and inappropriate.
Actually yes, they are probably delivering less than 1℅, according to a power law distribution.
When black people are forced out by rising property taxes apparently it's fine but God forbid the same thing happen to old white people, Prop 13 gets passed in 5 ms then.
I don't think that's a very safe assumption.
I still want an N900 now. The web be damned: it could still browse HN.

Nowadays, the next best thing is the Pyra, but that still has't been released.

By the way, have you seen the neo900 project? It sounds like you would be interested, although it is also incomplete.
Please start with this log entry

> 2016-09-29T23:57:32Z Xach: I dropped com.informatimago because I do not want to have to discuss anything with pjb again. I have never enjoyed discussing anything previously, but his vile twitter feed drove me over the edge, and his project no longer builds, so I don't want to report problems or discuss solutions.

from here: http://ccl.clozure.com/irc-logs/lisp/lisp-2016-09.txt
I think the difference is that they are willingly trading suffering for some personal benefit, whereas you don't actively choose to suffer. While that's unfortunate, you are unfairly writing them off because you arbitrarily have to suffer. Sure, it's commendable that you don't give up, but that doesn't negate the testament to the human will that these individuals are going out of their way to test. It seems to me that you believe anyone who doesn't endure a chronic illness is a wimp.. well guess what, some sufferers don't get to choose to live and have to die in front of their friends and illness due to terminal disease. Does that make you a wimp relative to them?
>This seems to imply that these technical interviews are skills that you don't really gain on the job.

the technical interviews skills are gained, at least in my experience, as result of the technical interviews - which normally implies, and did happen in my case, high rate of interview failures initially. Taking failures hard would naturally lead to Catch-22 here.
This is mostly empirically provable [1]. Diverse teams come up with better ideas that apply to a larger swath of markets (an immediate payoff for companies).

[1]: http://www.mckinsey.com/business-functions/organization/our-...
"Clever" code. shudder It's confusing to read for anybody not used to it, particularly as you need to set the "initial" value to the "actual initial value"+1.
That's because I (the author) still work on it alone and nobody wants to contribute. I receive tons of e-mails that go like "when will it run XYZ", but practically no pull requests.
> Yup. Unless you've worked for Palantir. Then your shares are worth their weight in magical fairy dust.

So, a lot?
Logarithmic growth is a more accurate description of the actual growth and on a limited interval grows faster than linearly.

It's symbolic of the typical winner-take-all markets in the IT sphere, unlike the term exponential, which is constantly misused and inaccurate in most cases (except, perhaps, for Moore's law on a limited time interval)
In the real world, these attacks are fairly trivial to work around - you use an ensemble of independent architectures (say ResNet/Inception/VGG) and train them independently on the faces.

(Also, it's well known that if you include the perturbation in the training the networks won't be fooled by them.)
Nokia went under, and the world outside of tech cares more about the shiny than the capabilities.
That's not really comparable because banks and insurers require a huge amount of capital to earn those margins. For example, JP Morgan (a pretty well-run and profitable bank) made a ~32% operating margin in fiscal 2015. But JP Morgan's return on tangible equity (a measure of net profitability relative to capital used) was only ~13%. Facebook, on the other hand, had a return on tangible equity of 25% in the last quarter. And as time goes by, that return on tangible equity should increase quite a bit for Facebook, while it is highly unlikely to improve all that much for JP Morgan because of competition in the banking industry.
I am not sure why this was down voted. I think you touched a great point, maybe because of the tone ?

I am fine with any requirements for the interview process as long as they are clearly defined.
Yeah, but I'm not talking about 'something'. I agree that it would be a giant pain in the butt for a toothbrush. But for a code analyzer you'd have to atleast run a trial/demo on your codebase to see if its actually valuable in your particular situation. Anyway, for me, picking up the phone and talking to someone for 5 mins doesn't seem that scandalous. I'm in an industry where I have to do it often so maybe I'm just used to it.
Try our blanket today! It is best blanket, used by astronaut!!
NoPLSql and Thick Database Approaches
Technical interviews are moving in the direction of more concreteness

Herein lies the scam. (Disclaimer: It's not on purpose, they're lying to themselves too)

Just because you have a process to produce a "concrete" result, that doesn't mean it doesn't stand on a house of cards.

I'd have to sit and have a think to make a clearer explanation than by analogy, so here's an analogy. This reminds me of the way people with an insufficient anti-authoritarian streak are more willing to question a person's on the spot decision than a decision from systematic, bureaucracy or law.

Just because more time and process has gone into establishing them, doesn't necessitate that the decisions are any better founded than what some rando pulls out of their ass.

Because, at the end of the day, taking the technical portion of the interview itself too seriously is the real fraud. There is exactly one way to tell how well someone will be able to contribute to our team, with our technologies, in our environment after a few weeks of getting up to speed. And that's to hire them.

The technical portion of the interview isn't useful beyond a low-bar rule-out criteria. After that, the technical portion turns back towards the softer skills of how the interviewee handles hitting a wall at problem solving in a field where nobody knows everything.
I think what they mean by "not diverse enough" is in comparison to the pool of candidates. Under the hypothesis that race/gender/ethnicity/sexual orientation/nationality/age are uncorrelated to performance/ability of the candidates, it implies bias in the selection process. Which isn't only an ethical concern, but also a business one: it means they're not hiring the overall best candidates.

Of course, there's also something to be said about the lack of diversity in the pool of software engineer candidates, which signals a problem for the industry in general for the same reasons. That's e.g. why tech companies are funding programs so that girls aren't turned off a software career if that's what they fancy.

Those details aside, in general my impression is that the industry isn't currently in any sort of stagnation of opportunities, with employees' salaries racing to the bottom to cling on those opportunities. On the opposite, there seems to be plenty of things still to be done, and not enough hands. If you dump a lot of new good candidates in the pool, they're going to be absorbed right away, even if they're all white American males.
maybe rephrase to desk top only users is around 100 million.
You are not alone, I have two good friends that still have theirs too, and nothing will ever be good enough for them again. That phone spoiled them, and is the direction they wish phones went. Alas, they have gone the way of the masses instead, but you know if someone like a OnePlus came out with a modern powered phone, with a big battery (phone size doesn't matter to us), a physical keyboard, dual sim, expandable memory and an open source OS, there would be DOZENS of us that would snap it up. Dozens!
Flip side -- I view it as a strong negative if a company has what I consider a weak interview or vetting process.

The interview doesn't have to be all whiteboard coding but I want to at least feel like the person got some grasp of my technical ability. If the entire thing is basically a "personality interview" I start to think that it is very possible technically weak people may have been able to schmooze their way into a place in the company and I'd prefer to not work with a high ratio of bozos.
Meego was a merger of Intel's Moblin and Nokia's Maemo, so yes. Sailfish is the modern descendant.
>Color cameras only became common at the end of the 60s and beginning of the 70s

Huh? What is a "color camera"?

Kodachrome film was invented in the 1930s. It wasn't immediately commonplace and, yes, we still tend to think of the 40s and 50s largely in monochrome. But color slides were certainly common by the 60s.

What was true (and what you're seeing) is that news photo reproduction for print was primarily B&W. Therefore, most news and documentary photography was still done in B&W even after color was readily available for consumers.
I ended up watching it at double speed (thanks youtube) but it still felt about 1.5 minutes too long.
> Don't put all your eggs in the same basket.

This is a strange complaint. A couple years ago, Facebook was making 90% of its money on the desktop, and people were saying the same thing. Now they have (very successfully) diversified, and the complaint is still there?
But this can be literally true for anyone. I tried re-reading it again and again and I am not seeing the distinction. Different people have different reactions to things. In other fields, it's not uncommon for people to submit resumes to 10, 15, even 20 firms and go through 5 to 10 interviews before landing their first "real" job. I think we forget how good we have it in tech relative to the rest of the economy. It's either that, or work for shit wages and awful benefits. But we don't hear about the diversity problem for a position like, an "analyst" for example. This lends me to believe it really is a pipeline issue.
Indians of Asian descent are a minority in the US, but well represented in Software Engineering. Further, there are plenty more Indian, Chinese, and Russian female engineers, than there are American (of european, aka White) female engineers.

I am less inclined to believe there is systemic bias against women, and minorities, and instead a culture difference between groups who are well represented in engineering, and groups which are not.
How many years ago? You might be experiencing the property boom - everything under construction with cheap loans from a careless ECB. It's not a way to dodge taxes. Many of those constructions were finished, others stayed halfway. What many of them have in common is they are looking for buyers.
What kind of issues you have got into? I would like to hear from you.

Though I don't have any experience of managing containers, argument does not make a lot of sense to me. They are pitched as holy grail which will get rid of entire IT team managing the operating system i.e. upgrades, patching, and configuration management. It just looks counter-intuitive to me as any new upgrade, bug fix , or security patch still needs to be applied to base OS. Containerized application can move freely from one host OS to another host OS but they will still depend on same underlying environment.
It technically also applies to Macs -- but in practice it means Windows-based desktops and laptops.
Facebook is the least transparent ecosystem. As a marketer, I find it hard to invest dollars without being able to see or understand what percent of custom audiences are exposed to ads. For example, one can upload a file (list of emails) to a custom audience, lets say 1 M big, for FB to ingest. Voila your audience is active for targeting, but how many of those emails (users) will be targeted, who knows. FB's inventory is overpriced and untraceable.
Or more likely, if your web designer demanded a percentage you would just find a different one.

The reason employees are commodities is that they allow themselves to be treated as such.
Notes on Migrating from Soylent to Baby Food
I don't think it counts as diversification if you just move all your eggs to a different basket.
On the flip side, I wonder how much of face recognition failures by humans would be considered "huge glaring faults" to machines?

For a famous example, take the https://en.wikipedia.org/wiki/Thatcher_effect (where a face will look normal upside down even if you individually flip the eyes and mouth). Or the https://en.wikipedia.org/wiki/Hybrid_image (where you see two different images depending on how far you are standing). I bet machines can or eventually handle those cases better than us.

Is trying to emulate our own recognition system the best possible approach? It might end up inheriting all our edge cases as well.

(For another example relevant to halloween, with a bit of face paint you can fool any human: http://gfycat.com/DeliriousEdibleAustraliansilkyterrier. But you can totally imagine a sufficiently advanced algorithm that is capable of ignoring color information and reconstruct a 3d surface for comparison instead.)
Ack, please kill/remove the scroll jacking. Not good.
Well, I already have several magsafe charges, monitor adaptors, and a wifi printer, so none of these is much of a concern for me.

I'm more concerned about having to purchase several chargers (we have 5 magsafes in various rooms throughout our house) and adaptors so that we can plug in phones, hard drives, and monitors.

Will I transition eventually? Yes. Does the new MBP make me want to do it now? Not on pricing or features.
Olive oil has relatively good price and is very easily exportable. An olive grove is a profitable investment, unlike a house/flat.

Also olive trees are very easy to cultivate, very durable and last for centuries! disagree strongly on this.
The real question is, does it slice my cheese as well as the Air does? I like it thin.
> these technical interviews are skills that you don't really gain on the job

I believe that's definitely the case. That's why I shifted my interviews to be as much like the work as possible. First round was a pair programming interview plus a bit of technical discussion. Second round was kicking around a feature and its implementation with the team.

My goal was also to see people at their best, so I did as much as possible to make them comfortable. E.g., for the programming portion they were welcome to use their own device, their favorite editor, and their strongest language.

I was very happy with the results, and aim to keep moving in that "make it real" direction. I wrote more about my approach here: http://williampietri.com/writing/2015/slightly-less-awful-hi...
Absolutely agree that data cleaning should be at the top -- how someone prioritizes data cleaning is for me, the main litmus test to how effective they are at real-world data problems. I also agree with how the author summarizes the issue, but he also runs into the same issue I have: data cleaning is such a broad term that it obscures how difficult and important of a problem it is.

For example, some people think data cleaning is "Convert 12-FEB-2012 to 2016-02-12" type problems, and can't believe that such a task would be 80 to 90% of the difficulty in data work (compared to say, learning enough ggplot2 to make a nice chart).

On the other side of the equation, you have people who want to do a JOIN-GROUP-BY aggregate so they can calculate how much "evil" Wall Street money goes to each political candidate, a la OpenSecrets's calculation [0], only to find that the FEC does not classify campaign contributions by industry type or company, nor is the "employer" field filled with normalized entries such as "Evil Wall Street Company" that would lend itself to easy GROUP BY calls. For fucks sake, I've found that executive-level/professor folks can't even spell "Goldman Sachs" and "Berkeley" correctly (even on a typed form)

And that doesn't even scratch the surface of how little this person knows about the data question the purport to answer, or about how the FEC, the American political system, and real life works. Among the data cleaning problems they will have to mitigate are also the 2 hardest problems in computer science (how things are named/classified, and how up-to-date the data is).

I don't have any better ideas at the moment for how to break apart the category of "data cleaning" that reveals the many facets of the problem but also still preserves the interelatedness of the facets. But it's possible to be very good at some of the parts of data cleaning without knowing the rest.

[0] https://www.opensecrets.org/industries/indus.php?Ind=F

[1] http://www.fec.gov/finance/disclosure/metadata/DataDictionar...
Oh, did I misunderstand what he meant by "this era"?

I would assume the proper way to phrase it would be "that era" if he meant the 60's, although he did use "ran" which is the past tense. He also mentioned JavaScript, which didn't even exist in that era, so I guess I'm confused by the entire statement.

Edit: by 'he' I guess I meant 'you'. Which means you did mean to refer to the 60's.
Have you looked into sailfish? No physical keyboard, AFAICT, but it's based on the successor to Maemo.

Also, look into the neo900 project and Pyra. Neo900 is trying to upgrade the n900, and Pyra is the successor to Pandora, which is a pocket computer in a semi-similar vein to the old nokias. Neither is finished, but they're worth a look.
I think that 'non diversity' (or at least how I interpret is) is an indicator of possible unfairness, and not necessarily bad per se. If a company has too few people of a given group in comparison to the hiring pool availability, it may be something that the company is doing, maybe unconsciously, to keep them out. If the difference is with respect to the country's distribution of that group (which I suspect is the case in tech), there might be something either with the companies and/or the education-motivation given to access that knowledge.

In any case, I think it's worth to look at it and try to solve it, because maybe you end up improving the actual/future employees' life and the company itself.

And possibly race is not the only division to look at, as you say. Class is probably more interesting, but also age, family responsibilities or sexual orientation can reveal practices/attitudes that can be improved.

I agree with you in the cynical thinking and in the motivations of some of the groups pushing for it, but I think that in the long run thinking about the diversity problem and trying to fix it can lead to better outcomes for everybody.