skysight-inc/HackerNews huggingface dataset:
You can use a two-step hashing approach: for the source file, 1. use Rolling Hash [1] to find the boundaries of the variable-sized blocks of the file, 2. compute a strong hash (sha1) of each block. Repeat the same steps for the target file.

Sort the two lists of sha1 block hashes. Then do a merge-compare of the two lists of hash. Blocks of same content have the same hash. See how many blocks are the same out of the two lists. Compute a percentage of similarity. Have some threshold to declare the two files are similar or not.

Rolling Hash is good in finding dynamic boundaries of similar blocks, gracefully handling addition, deletion, and movement of content. Once the block boundaries are known, use sha1 to strongly find equivalence. Sorting the two lists allows comparing out-of-order blocks.

[1] Content based slicing using rolling hash. http://en.wikipedia.org/wiki/Rolling_hash#Content_based_slic...
I disagree and further would be concerned that a comment like this says more about you professionally than it does about the software we're discussing, somewhere in the vein of "Stack Overflow? Weekend Project".
if only new Date.getDate() was as easy as Std::get_money
Speculation on what a candid NSA response to this might be: "If helping a brutal regime find more targets to brutalize also helps us find terrorists, then the collateral damage--which are entirely Saudi nationals anyway--is acceptable."

You could also throw in the old chestnut about "Well, if they didn't get our help with this, they'll get it from someone else (probably the Chinese)."

Truth is, I have an open mind. Perhaps this calculus is correct against the backdrop of what is a cruel and violent game played by nations against each other. Much of the horror we all feel is predicated on the false belief that this is happening in a US state, or that Saudi Arabia is even really our friend. "Keep your friends close, and your enemies closer," the saying goes. Unless we're willing to take strong action against Saudi Arabia, what good would weakening the relationship serve? Just as in the game "Diplomacy", when you backstab, you better make it count.

That said, if there was an Arab Spring type movement on the peninsula that had even a small chance of winning, and the US helped to crush it, even a little, then we'd have crossed the line. This? It may be defensible as a "least evil" option.

The problem that the NSA faces, along with the entire executive branch, is that their credibility is almost completely shot. We don't (and shouldn't) believe them. They have lied to Congress and to us. We can't really know what their reasoning was or is. And in this great shadow of doubt, collaboration with brutal regimes no longer seems like a less evil option, but rather training ground for new domestic policy.
It's a tradeoff.

Some markets require large research investment up front (like pharmaceuticals) and they won't bother if others can immediately sell the same finished product without doing the research.

Without patents you also get trade secrets which can leave the public without ever knowing how something works.

Patents are an attempt to exchange a limited time monopoly for a complete disclosure of how the invention works so that the public can benefit. There might be an argument for that time being too long (and I think it's completely broken when it comes to software), but I can see why they're useful as incentives.
Chocolate Peanut Butter Surprise Cookie Recipe
Every article you've read that suggests salted hashes are adequate stored password authenticators is wrong.

I don't know what the "security community" is, but if it has any relation to "the community of people who can speak with any authority on cryptography", let me ruefully assure you that it is much, much smaller than you think it is.
Yes, for people who understand licensing stuff. For the many people who don't, they could say that 3rd party components of CoreOS are under various open source licenses, but the code they produce is Apache 2.0.
I don't know why more people don't understand that equality of opportunity is perfectly compatible with inequality of outcomes

It's an easy theoretical thing to understand, it would be good to see equality of opportunity in practice.
Statement from the President on Unlocking Phones
No. The industry must be competitive. Companies compete in the market. People must stop promoting the fiction that women are poor competitors. That's a stereotype that just happens to be awfully convenient for male entrepreneurs.
If you really want to address both women and girls, may I respectfully suggest trying the phrase "women and girls"?
Ambient noise doesn't bother me unless it exceeds a certain volume. Musician's earplugs take care of that nicely, they're nearly invisible, fit in my pocket, the nonexistent batteries never go dead, they're cheap, no worries about breaking them, there's no weight on my head, I keep several sets handy, etc.
Pretty crazy how much I agree with this.
Generally I'm against patent battles, but not in this case.
Noise and echo cancellation are very research-demanding areas, with pretty smart algorithms and hardware to be developed. Those are certainly patentable, unlike "round edges".
> Every article you've read that suggests salted hashes are adequate stored password authenticators is wrong.

Thanks for making that very clear. :)

Re "security community", sorry, I'd edited that part out before I saw your reply, and replaced it with a call for these posts to be called out for giving bad advice.
I was that girl. You were my mother in a lot of ways (my mom is a programmer) the problem becomes more endemic in high school and then college where you have to start becoming a woman.

Programming in a startup is not a great choice for peak fertility (where I am at...thankfully I consciously only want one kid, so I have time to delay)
I believe you missed my point, though. I will grant you that many libc functions do not follow what are currently considered best practices.

But you can't take the textbook example of what not to do and say that this is representative of everyone who works in the language. Just as I can't take an example of extremely bad code in some high-level language, which naively goes against what is considered sane practice, and say it's the language's fault.

Put alternatively, in many contexts an out of bounds exception is almost as bad as the trouble you can get into with pointers. [I say "almost", conceding the point that with the C you can do crap like clobber a return address on the stack.] Someone who simply doesn't give a shit about array bounds (like this author, it sounds like) is not likely to be totally saved (as if by magic) by a higher-level language.
I wound up nearly writing a book in response to this so I'll try to sum it up.

I agree with most of what you've said in this thread, but...

> Why have a centralized legal roadblock to new product development?

Rephrased: Why have a centralized catalog of discoveries and inventions available to the public that afford the contributors temporary exclusive license?

The idea behind patents was to counter very real problems created by the failure of markets to balance out the incentive to invent and innovate with the incentive to make those discoveries publicly available so they can inspire or cornerstone further discoveries. If they weren't published they were hoarded as trade secrets and were at risk of being lost forever over time.

Did the inception of patents account for or even acknowledge the kind of market control they're being used and traded for today? The lauded free market has turned patents into a commodity, possibly by expectation, but it's clear that this activity is relatively recent and seems to me due to changes in how the scope of their exclusive license grants is being interpreted/abused. Would we be better off assuming that the publication of discoveries would lead to closely aligned derivative products that would create a competitive environment or should we be generous with the licenses granted "inventors" so there's greater incentive to share those inventions?

Personally I think we're just seeing the beginning of what effects doing the later can have.
It doesn't make sense to do research on race and IQ. Except for people who are looking for scientific sounding reasons to legitimize their age-old racist prejudices.

So to be clear: there is no serious debate about race and IQ to be had. There are no insights to be gained from Race-IQ research.

I don't believe -- not for a second -- that a discussion about race and IQ helps minorities. The idea that we have to engage in a discussion on race and IQ which is based on racist premises otherwise "we are not troubled by the problems minorities are facing" is ludicrous.

Also see http://www.theatlantic.com/national/archive/2011/11/the-race...
if they live.

http://www.theguardian.com/business/2013/nov/22/moritz-erhar...
> That said, murder & paying the least amount of taxes possible are slightly different scales of morality - don't you think?

Certainly, which is why I wouldn't advocate the death penalty or life in prison for tax evasion/avoidance.

> Should you be judged morally because you have a tax accountant maximize your tax refund every year...

No, but I'm not asking them to engage in stuff like http://en.wikipedia.org/wiki/Double_Irish_arrangement.
I promise you from the bottom of my heart that I find that page embarrassing, even if I can easily rationalize why it says that.
> Using a good password KDF as a password authenticator is a fine decision.

FWIW, I've seen several people claim that using PBKDF2 to store passwords is something only idiots do. They never explain why, of course; they just laugh at the people who don't know better. "lol PBKDF2 idiot." I don't know where this particular thought comes from, but it's out there.
It works best with continuous sound, but it does have an effect on talking, yes.
The Mini-Blockchain Scheme [pdf]
Source : http://cryptocurrencylive.com/item?id=10622
We Put the Waterproof Kindle Through Hell
Maybe they are flagged precisely because those topics don't lead to meaningful debates? Take any slightly controversial (or can be spun to be controversial) gender-and-tech issue, submit it to HN: if it gains traction, it will consist of hundreds of posts with two kinds of people who have so little common ground and set of mutual opinions/world-views that they can not do anything but shout past each other. Meaningful discussion? Hardly.
Can China Best the West at Statecraft?
Beware - keeping your bitcoin in a wallet you do not absolutely control (like Coinbase), you can and almost certainly will lose them at a moment's notice, sooner or later.
I think this answer is pretty bad. You don't need to do all that CJ stuff of building a Linux just to learn systems programming. That's all domain-specific knowledge that would be irrelevant to the vast majority of people doing systems programming on Linux, and all of the people doing systems programming on other operating systems.

Assembler is also a low priority to learn. The details of x86 assembly are not important until you happen to need them for some reason -- in which case, learn it when you need it. The same goes for Linux and OS details. Most systems programming does not require the use of assembly language. If you do want to get a feel for "what assembly is", do some of the challenges at https://microcorruption.com/ . It's a lot of fun in its own right and avoids x86 minutia, and you'll get a good picture of how what a stack would look like and what function calling can look like.

What you need to do to learn "systems programming" is crack open C and C++ and get yourself pretty good at them. And get good at dealing with multithreading and the like. That's what the original poster is asking for.
I disagree. I thought the website was very nice, easy to navigate and I found the full page photos very nostalgic looking (grainy, oversaturated ...). It evokes exactly the kind of atmosphere I would associate with a Grandpa from the 60s.
Your update made me realize what's happening. When there is an exact multiple of 30 Ask HNs to display, the system doesn't decide "should there be a More link" correctly. I've added it to a list of things we need to fix. Thanks!
Compare Your Healthcare
But typing speed is never an issue when programming. It's just not the bottleneck.
My apologies, I wasn't right. Please see this reply chain: https://news.ycombinator.com/item?id=8088208
> Do you really want a meaningful discussion, or a discussion where PC opinions are upvoted and critical voices are downvoted/banned by dang?

I find it funny. Some controversial or just plain stupid opinion gets posted, dang responds to it with basically "I disapprove of this post soooo much, and I'm glad that HN does too, judging by the downvotes. Pats HN on the head". Then a whole bunch of regular users reply with how awesome his reply was for telling the poster that he was an idiot/bigot.

I mean, they can of course moderate or whatever in whatever way they want to. I just don't understand why so many are fascinated by it.
Thanks for such a great analysis! You pointed out some things I did not really think about. Especially the way Source Code Pro handles "i"s and "l"s.

I will try to add 1px of horizontal space between letters.
The goal was to make the font vertically compact, but still legible. The size is small and it feels quite clumped though.
> It fits in better with my thesis if it came out in 2004. ;-)

No doubts on that... get undercover again! quick! :)
What question have I evaded?

"How would you feel if websites colluded to prevent you from communicating at all"? They don't have that ability; a website cannot reach through my computer screen and gag me. Since the you question are literally asking is absurd, I guessed you really mean to ask how I would feel if a website removed my comment, which I answered. Now I think you are asking how I would feel if every single website agreed to a common blacklist and put me on it. This is still quite far fetched and drastically different from what I am talking about, but I will answer it anyway. I would feel annoyed and embarrassed.

What I am talking about (removing vile comments) is already done manually, so your notion that it "never comes up" is strange. If you right to force other to distribute rape threats is so important to you then I doubt we will come to an agreement.

Lastly, there is no need to call me a "piece of shit". If you stoop to this level again, I will consider you to have conceded.
I can't imagine entering into this career or hobby and not bothering to learn how to edit text.
Not to be pedantic, but shouldn't this read that proper spelling and grammar _are_ absolutely vital?
I'm guess in expensive cities, the occupancy rates will be higher since visitors are more likely to look for deals.
My medium tech solution: earplugs with holes cut in them that music plays out of, more or less. The isolation puts active noise cancellation to shame: http://graphs.headphone.com/graphCompare.php?graphType=6&gra...

Graph is of attenuation in dB: lower values are better, and positive values are worse than no blocking at all.
You're right about parts of the post being unclear. I'll be rewriting that paragraph to be more precise regardless.
Probably < 10, but it's hard to know. The software (older version) has been open source for a long time and I've had dozens of people interested in it in the past.

I should have installed a tracking pixel or something ;-)
"can" is true "almost certainly will" is pure speculation.
I'm dad, not mom (though mom shares my concern) :-)

You're saying "start becoming a woman" as it contradicts being an "IT person", whatever that means, programmer, sysadmin, etc. Why? I want my daughter to become a woman (as much as my wife is a woman, and oh boy, she is. While being a an executive director of a science foundation). But I also want her to become whoever she wants to become professionally, programmer, designer, digital artist, VFX guru, whatever.

I can understand fertility and startup, but statistically, majority of "IT people" work for established companies, not startups. Enjoying all these extended maternity leaves, good health benefits plans, etc.
And it should be noted, that based on the replies here and the feedback, it sounds like he might not be a liar anymore. Hopefully he's seeing the issues that his employees have to put up with when he puts them into this kind of situation.

So he has gained some personal growth from this event. All's well in the long run, but its difficult to make up for broken trust.
The bitcoin ASICs I'm aware of can't be used for cracking SHA256, because the function they compute is SHA256(SHA256(f(params)), where f is a function that stuffs some parameters into a bitcoin-specific block template before hashing.
It's important to note that what to put into a project like this (ie, requirements) is at least as important as the technical aspects. Part of the reason for the massive upvotes is the software has most of the right features for the job, which is actually quite rare in this arena.
First time in India, Google to directly mentor 25 startups
Sure.

https://github.com/daylerees/colour-schemes it's "Frontier Contrast"
Show HN: Swift-style-checker – showing how to write Python-style Swift scripts
When a company spends money to continue its existence, such as rent, it is deducted from its taxable profit. When a person spends money to continue their existence, such as rent, it is not deducted from their taxable income. If people were taxed as corporations the median American would pay no taxes.
I hope this way of using the word "denial" can someday be eradicated. It means that somebody else is not convinced by the evidence that you (and maybe a great many other people) find convincing, nothing more. It makes as much sense to talk about "Vim supremacy denialism", etc.

I apologize for getting meta and not responding to the content of this article specifically.
Don't forget to factor in time as well, it would be accurate to judge a high crime street as safe during the day if all of the crime happened during the night.
>I claimed that a single individual disagrees with the proposition that legally reducing one's tax burden is immoral.

You are butchering his position even worse than I thought. He never said what you claim. The best case you can argue with his support is that the bluster of morals cannot be used to legally force someone to do more than the law requires.

Never once does he come anywhere near "disagree[ing] with the proposition that legally reducing one's tax burden is immoral". He makes no moral judgement on the matter whatsoever.

And I don't see how trawling nearly 100 years back in time to find a dissenting and disregarded opinion is anything other than selective quoting. You may as well have quoted yourself from 10 minutes prior.
I suspect systems where the dataset, or at least the indexes, largely fit in memory?
Sure I think that's a reasonable question.

On a basic level I expect that Amazon will continue to grow, and that in the future the value of the stock will be increased and that I'll be able to sell it. I think an Amazon that was paying dividends could be great, or not, but it doesn't sound like a business that I understand so I wouldn't invest in that because I don't think I could evaluate it. I'm investing in Amazon because I believe in Jeff Bezo's thesis and like what he's doing. It's also a significant but not life changing amount of money invested, so I'm comfortable having my own investment ideas here instead of playing safe with mutual funds, which is what I usually do.

But more importantly: why do I expect Amazon will continue to grow? Because they have an extremely large e-commerce business that is providing a base for them to expand into other markets, and because they have demonstrated ability to do so. AWS, Fire, Prime, content etc., are all built on an extremely solid e-commerce foundation. AWS is huge, and is expected to be more and more of Amazon's total business (7% in 2015), and I am betting that they will continue to make new business lines like AWS.

My time frame is very long, I'm young and don't need the money anytime soon.
Hey, point taken - agreed, I don't mean to belittle the work in any way. The dedication to stick with a project that long (and do a nice job on it) is something I definitely admire. I haven't run the software or browsed the code, so I can't speak to the technical merits of it, but again , it looks great. And I don't think we should underestimate the enormous amount of time and effort it takes to make something like this.

However, I don't think a little bit of self/community introspection about its popularity would hurt. Might learn an important lesson or two.

Broadly, I haven't figured out when criticism/cynicism (if you want to call it that) is or is not beneficial. You and I would probably concur that modern culture is all to quick to tear down the good and pure with cynic blasts. Yet criticism at even seemingly happy things, like the upvoting of this post, can be beneficial.
Do you know which schools those numbers are from?

(That doesn't really affect your chance of startup success, but surely it affects your inclination to go into one.)
More like the latter, but you don't get to choose. The system automatically clusters you with others whose historical votes are like yours. So what you get is "true karma" where the collaborative filter you get is a direct result of the quality of your votes, as opposed to karma being a score as it is on HN. You may think you are an open and fair-minded thinker, but if you vote ideologically, or vote with things you want to be true whether or not they are, you will be clustered with the same. It would be like Apple fanboys and Android fanboys getting put into their own clusters, getting out of the way of others who engage in "agnostic" fair-minded discussions. [Typo in prior reply. Should be "A lot of people... who think like you think this item is garbage."]

Astro-turfers and bots trying to game the system naturally don't cluster with anyone except those who happen to agree with them, in which case who cares? There really is no difference between fake votes and dumb votes, so the system doesn't make that distinction. Trust is asymmetrical, so even even though your filter is influenced by those in your cluster, you may have no influence on your cluster. Trust is earned by voting on something before others do, and then those others agreeing with your vote. So it is impossible to game trust by mimicking the votes of others. The system distinguishes between leading votes and following votes.

The clustering is entirely transparent... You can "flip" your view at any time to one of the other clusters. I have a number of ideas how to surface a meaningful label for each cluster so that, for example, a self-professed fair-minded user can realize he is actually rather close minded and ideological. I call this "Mirror". One's cluster would never be publicly displayed (not in the business of shaming), but you can look in the mirror in private, and gain self-awareness.

The system would support the maximum privacy and anonymity guarantees possible (allowing people the option to register with no password recovery email, for example).

Before the system has enough users to do effective clustering, it would simply flag garbage that has widespread agreement as such. That in and of itself is a valuable service.

My original project name for this is "Garbage" (How much of the web do you think is garbage?), but I've settled on "Common Karma" as the official name. This is my first sharing of the above idea and its names beyond my friends. You heard it here first!
Show HN: iOS Notes Made Beautiful
We just released version 1 of our note app and Note is the first publicly available app we've released.
I wonder how long it will take for this to get Goxed?
If they were doing this for real, they'd be using reservations; with one-year reservations, the i2.xlarge bit (the servers) cost $905k/year.
Coincidence: I also code and I speak Mandarin, though I'm not Chinese. Let me know if any of my ideas (in the many comments I made herein) are of interest to you.