skysight-inc/HackerNews huggingface dataset:
Proposal safer email.
I am (have been for two months) thinking of starting an email service that is compatible with the current email system, but adds features.

-If server is seized by authorities they got nothing;

-RSA/public key encryption on every email. Every user has a public key and incoming mail is encrypted on the server right away. Public key is available for every email upon request, then other party encrypts and not the server. (this to integrate with other providers who can adopt this protocol)

-Probably need downloadable apps (can be simple javascript) to prevent 'man in the middle' attacks.

-Private key is generated via a standard procedure, client side, on every login. User password (such as 'correct horse battery staple') will be the 'seed' for the procedure to generate private keys.

I need help for this project.
Please respond for any suggestions.
Please look up the word "fascist" in the dictionary next time before using it.
Boundless Informant: the NSA's secret tool to track global surveillance data
Quantum ATM
Less than a fifth of US public budget is spent on defense...
Dear Mr Zuckerberg,

Feel free to deny your company's involvement, but don't you fucking dare criticize the free press's reporting as "outrageous".

Thanks.
I found this interesting news as it directly contradicts the wording used in the Facebook and Google denials - that no direct access to their servers was involved. The wording of the slide is:

Collection directly from the servers of these U.S. Service Providers: Microsoft, Yahoo, Google, Facebook, Paltalk, AOL, Skype, YouTube, Apple.
Historical accounts of both Thomas Jefferson and Benjamin Franklin make it very clear that both looked to the Iroquois for inspiration and considered them important.

Franklin in particular became inspired by the idea of combining the colonies into a federation after reading about the Iroquois. And did much to raise awareness of their culture, and get involved with them. (Including becoming an envoy to them, and being involved in treaty negotiations.)

The fact that these ideas had plenty of European precedents does not diminish the fact that, historically, it appears that key figures in the US revolution and the later writing of the Constitution actually took inspiration from the Iroquois.
I think the most straight forward solution for most foreign countries would be to insist that Google, Facebook etc. set up their servers within those countries and limit the amount of data they can transmit to US.

Small countries can't do it alone, but EU for example has enough muscle to make this happen and force all data to be stored within EU countries and be staffed by people who are EU citizens. There are already several Google data centers in in EU. They just need more.
- Use an open-source OS like Linux. Ubuntu is user-friendly. Windows isn't safe: https://en.wikipedia.org/wiki/NSAKEY

- Install the OS with full-disk encryption.

- If you use a service like Dropbox, use EncFS to encrypt everything in the cloud.

- For email, you'll need to start using a mail user agent like Mutt or Thunderbird with PGP/GnuPG. And then the problem will be that none of your friends use encryption anyway. Personally I'm telling everyone about using Bitmessage instead of email: https://bitmessage.org
We barely even need to have that debate, we know our Government are using classified information to cover their own asses and for political reasons. Transparency is pretty much the only weapon we have against Government corruption, thus we really should to be doing more to help protect legitimate whistle blowers and to stop documents being classified or redacted without some overseeing from a party without skin in the game.
What do you mean by primary sources? Primary sources typically means the original source material[0], not articles in the Washington Post, etc., and I just want to make sure we're on the same page :)

DocumentCloud[1] is a startup that helps you annotate and organize primary sources (canonical definition)

[0]https://en.wikipedia.org/wiki/Primary_sources
[1]https://www.documentcloud.org/home
Good luck doing that in the US.
Google went on record as saying that they'd never even heard of any FISA order remotely as broad as that before the Verizon story broke, let alone acceded to one, and other tech companies said something similar. That doesn't answer every question about just how much ground the orders moving through the tech companies' streamlined new FISA-order pipeline cover. But does mean that either the tech CEOs are orchestrating a brazen and elaborate lie, or they're not sharing information on anything close to the Verizon scale.
Boundless Informant NSA data-mining tool – four key slides
I became aware of them as SGI proposed extension to C++ standard library.

Good evaluation of them remains

http://www.sgi.com/tech/stl/Rope.html
Looking into the IRS scandal, it seems like a non-issue in reality, despite being hyped up politically. They were targeting tax exempt organizations abusing their status by remaining politically active. And they did nothing beyond investigate. No threats, no bullying, no arrests. So that seems like significantly less of an issue to me.

Dragnet surveillance by the NSA, on the other hand, is a huge deal.
It's interesting. There are few European countries where I can unconditionally refuse to speak to police or refuse them to search my home or person without a warrant. We don't have CCTV cameras like they do in the UK. And when it comes to being able to overthrow your government yourselves, much of Europe has the same sham democracy we do. (Berlusconi? Seriously? Twice?) We have guns, but the parts of Western Europe who really wanted to overthrow their governments within recent history were able to get all the guns and explosives they wanted too. So it's hard to really say one is any better than the other.
anyone else notice there's been none of the usual China-bashing going on ever since this whole thing broke?

which is funny because it should be at an all time high, what with the meeting of the Chinese president Xi and Obama.

maybe they're sharing tips on how best to spy on and kill their own citizens! it is the most important economic and political bilateral relationship in the world, after all.

i kid, i kid.

but seriously, wasn't one of the talking points of the meeting going to be Obama redressing Xi on how the Chinese hack/spy on american companies and citizens' communications?

LOL x 10.
100 Rules for Being an Entrepreneur
TrueCrypt for the laptop, and GnuPG for emails.
Google can't offer proper encryption because most of their business model is based on analyzing all the personal data they store. At least it's very unlikely and I wouldn't hold my breath.
A while ago, I implemented ropes based on the 1995 paper by Boehm et al in JavaScript. The interface provides methods charAt, charCodeAt, concat and substring, so it functions as a drop-in replacement for JavaScript strings (provided you limit your use to the given supported methods).

https://github.com/Mortal/ropejs/blob/master/rope.js
We're in the midst of several legal cases that are trying to argue that the NSA spying program is criminal conduct.
Meet this Robot Surgeon Side-Kick (PICS)
Thanks! We've been using appannie.com, but their data simply doesn't answer all the questions. Especially on how sales affect long-term and short-term health of the app. Often sales attract bad reviews (people complaining they paid too much before/after the sale), and often you can't really tell how much the sale actually increased the revenue.

Appshopper is just for iOS for now. Thanks for the tip though!
If the government hasn't done anything wrong, then they shouldn't have anything to hide from us :)

More seriously, some amount of surveillance is probably necessary, but they need to be a lot more open and transparent about it. Secret laws and secret courts are not the stuff a free society is made of.
CANARYEX | Joinup
>> However the USA introduced important ideas from the native Americans that had not previously been part of the European tradition.

The last part is the bit that annoyed me. Of course I don’t know how or where the founding fathers got their ideas, but saying that these important cornerstones were non-existent in Europe before is simply wrong.
The government is incapable of providing absolute security. You cannot be "secure" in the sense that sufficiently determined terrorists will always be able to kill you.

The way this issue is repeatedly framed is as a trade off between security and privacy, with the implication that people should always choose security over privacy and therefore have no privacy. Which is utterly ridiculous. Insert any other two things you please: In a trade off between speed and safety, you should always choose safety. It's complete rubbish because if you always choose safety you can never leave your house (or, ultimately, your cell).

Even the trade off itself is a fallacy, because the reason for privacy is security. Loose lips sink ships. The same logic applies to individuals and corporations. Someone who knows everything about you can easily kill you (or rape you or blackmail you or kidnap your children etc.) Creating a huge database of everybody's secrets is an enormous security threat.
Show HN: FakeSnap - create fake snapchats
I wanted to get more into ImageMagick and image manipulation in Ruby and since I recently started using SnapChat, I thought this might be a fun weekend project. Let me know what you think :)
Washington Post did revise the story. And yes, they should have issued an explicit correction. But this is still a major story:

> Sens. Ron Wyden (D-Ore.) and Mark Udall (D-Colo.), who had classified knowledge of the program as members of the Senate Intelligence Committee, were unable to speak of it when they warned in a Dec. 27, 2012, floor debate that the FISA Amendments Act had what both of them called a “back-door search loophole” for the content of innocent Americans who were swept up in a search for someone else.

> “As it is written, there is nothing to prohibit the intelligence community from searching through a pile of communications, which may have been incidentally or accidentally been collected without a warrant, to deliberately search for the phone calls or e-mails of specific Americans,” Udall said.

US Senators with classified knowledge of the program don't have knowledge of what exactly it entails or how it is being used. I can forgive the Post for getting some of the details wrong.
- I don't have the expertise to list off the progress, but a lot of questions about water seem to have been answered. We're ahead of where we started, which is more than I'd expect from a billion dollars spent on searching for a designer (well, to be fair, I'd expect it to help progress in biology; but we could just focus the money on that and do better).

- Assuming we started at 1:1 odds for long-term presence of water, and moved to 100:1 odds, that's twenty decibels on that particular question (and there are others). Again, way higher than I'd expect on designer research.

- Probably not.

I'm not sure I why you said to apply the same logic to the pursuit of life on other worlds. We're clearly discovering things. Plenty of people say the value of discovering those things is low, but I didn't mention value-of-information.
Oh hell. Looks like the anti European thing has just hit poor HN. :(
are you talking about her comment that Python has no data abstraction? Even I found that odd, maybe she was only talking about the lack of static types as opposed to java, c# which are "expert languages"
I think the idea behind immutability is as follows.

Given X, we are completely comfortable with applying functions to X which take it from its current domain to a different one. These functions are composable.

f: X -> Y
g: Y -> Z
g . f : X -> Z

The "state" is transforming from one operation to another. The TWO tricky parts are how the initial state is populated, and how the state is presented to the enduser.

Good programming practices involve isolating these two endpoints from the code logic that does this change anyway. As I see it, Haskellers will have two "dirty" modules which handle I/O with monads, and a lot of pure modules that are very easy to reason about. However, doing this requires discipline and employing a certain coding style that many are not used to. Having worked with Haskell a lot, my programming style in other languages has improved a fair bit, and I intend to revisit Haskell for possible production use in the near future.
It remains unclear to me how I was able to get an entry-level Software Engineering position in the DC/Baltimore area with nothing but a BS in CS from UMBC and no internships or relevant experience.

Also, Tom Friedman was a really boring commencement speaker.
Virtus 1.0.0 (Ruby Object Mapper object description DSL) fundraiser
For those that don't know: Ruby Object Mapper is the old DataMapper.
Phew?
Solve Math Problem — Get $1M
> If my secretary holds my briefcase, is the government allowed to seize it without any warrant or judicial approval?

if your secretary agrees to give it to them , of course they can.
Out of curiosity, has anyone's opinion of Bradley Manning, Julian Assange or Wikileaks changed over the last few days?

From the previous times the topic has come up, my impression is that this community is more or less whether they're doing good work or not.
Rather than using RSA/public key encryption, I would rather suggest using OpenPGP so that others can encrypt email before it even arrives on the server. Instead of generating the private key on each login, it might be better to either a) store it on the server with the password acting as a passphrase or b) store it somehow in the browser/app. It will be interesting to figure out how to transfer it from one device to another, then.
Yes, journalism truly has collapsed. That's why these scandals were outed by Reddit and Twitter users, not journalists working at more-than-a-century-old newspapers.

</snark>
I wish there were an easy, drop-in lib that would give me swarm/peerless networking like this, for Mobile platforms.

Even a Lua implementation would be enough for me.
Ask pg: "&#x2F" considered harmful?
TDD Proven Effective Or is it? [2008]
There are already several Google data centers in the EU. And there's also a wide range of surveillance and legal interception in the EU.

The question for users is there not whether the can evade surveillance etc. or not but which government they prefer to directly monitor them. And there is of course data exchange too with many close US allies in the EU in particular the UK, but also Germany and countries in Eastern Europe.
Dropbox CEO Revisits MIT for a Surprising Talk
Our solar system is old and by comparison our species has been here for mere fractions of a millisecond.

Say what you will, but those machines rolling around Mars, navigating terrain, taking pictures, analyzing samples, using sunlight to recharge, sending data to earth via radio, and countless other capabilities, are a celebration of human intelligence.
>She added: "The continued publication of these allegations about highly classified issues, and other information taken out of context, makes it impossible to conduct a reasonable discussion on the merits of these programs."

I can think of a few other things that have "[made] it impossible to conduct a reasonable discussion on the merits of these programs." That they're classified Top Secret. That numerous attempts to investigate the programs via the judicial process have been stymied under the "military and state secrets privilege." That whistleblowers have been hunted down and persecuted. And the list goes on.

edit: PSA: waterphone, your account appears to be hellbanned.
It's a lot, no?
There are many reporters working on this, not just Greenwald or Poitras. The most recently released slide directly contradicts the statements by Google and Facebook:

http://www.guardian.co.uk/world/2013/jun/08/nsa-prism-server...

Given that they used the same phrases in their denials, it would seem they have been fed talking points, and are are using a novel interpretation of direct access. For example if Google or FB interpret a real-time mirror as 'not direct access', their statements are technically true, while at the same time being misleading. Doubtless the truth will come to light eventually, but for now we'll have to wait for more information and denials.
As long as we are talking about jumping to conclusions, the author does the same thing here:

Declan McCullagh of CNET examined the Washington Post story independently and concluded that the Post story was wrong.

The Cnet "analysis" depends on the credibility of their unnamed source. The reader is left to decide whose source sounds more believable, Cnet's or the WaPo's, and which version of the story they find likely.

The problem inherent to secrecy (which isn't an argument for abolishing it) is that the public is left to draw conclusions from partial facts and an unverifiable amount of misdirection.
His ending of the interview is pretty critical and lays out the proposition that appears will make or break Tesla given the above (I will for this comment ignore the very real problem of loads on grids that weren't designed for this, e.g. blowing transformers by using them at night when they need to cool down):

"I have no interest in an article that debates what we consider to be an obvious point -- which is that there is a dramatic reduction in battery costs," Musk said, after a few questions. "You clearly do not understand the business. My apologies. I am terminating the interview."

I don't follow this field; is it that obvious costs will go down (ADDED:) enough for these high performance (at least as claimed in the article) battery packs?

As they note immediately after, "Elon Musk is not a guy you like to bet against.", and I note he has a bachelor's degree in physics, he is very much not a "suit" divorced from hard reality, e.g. his comments on the Dreamliner battery pack issues sounded like they had a good grounding. And I know more about "rocket science" and SpaceX really impresses.

Note also this is not exactly Moore's Law envy; current size/energy density seems to be OK, he's just talking about cost.
If the story was 100% correct then they wouldn't have to alter it, even if said companies were somehow forced to lie to the public. It would stand up on its own merits.
You're right. Assange uses the term "justice" in the documentary. It just surprised me that he and Markos were the only two people quoted in the documentary who actually referenced moral values when talking about why society needs a free press.
The only alternative to making this assumption is to treat computers as compromised non-trustable entities, forgoing their use as true extensions of the mind and leaving all individuals at the mercy of the ever-growing computing system. Instead, we treat the assumption as fact, making do with the imperfect OSs we have and incrementally fortifying them to solidify the assumption.
They had the knowledge but they could not publicly disclose a classified program. That's what "unable" means here.
Well I mean sources which are either primary (i.e. tweets, blog post which are from eye witnesses, people involved) and also secondary sources such as WashPo when WashPo acts as the sole conduit for a primary source (i.e. WashPo publishes information they got directly from a government insider).
Researchers Invent Invisibility Cloak
>You can't show that the decision to follow a security posture you dislike is objectively flawed.

To some extent you can. Cost benefit analysis, statistics and evidence point against the effectiveness and scale of our existing security posture. Obviously people can ignore all of those things if they please (cue Upton Sinclair: "It is difficult to get a man to understand something, when his salary depends upon his not understanding it"), but at some point you have to find a way to cause the people who are wrong to cease being wrong so that they stop fighting and start helping.

>This is why I think the long game is strengthening constitutional protections through amendment.

That's fine. But I'm sure you know how difficult it is, and how long it takes, to amend the constitution. We can surely approach the problem from other angles in the meantime.
97 billion pieces of intelligence gathered worldwide in a one month period! 3 billion of which are from the U.S. If that's not an example of a massive dragnet surveillance program, I don't know what is.
Agreed. These systems have been around for a while, but as we see, haven't been widely adopted. So why? What's a good theory as to why these ones haven't taken off? If we can identify why these haven't taken off we can build the next one better.
CONGRATS!
Bah. Such terrible logic. Truth is, we can do both. You're also mis-quoting your own source.

http://www.smbc-comics.com/comics/20120716.gif
She gave this talk at my university, and was an engaging speaker. I was pleasantly surprised at the turnout, especially considering that the timing was strange.
Ask HN: US this bad about civil liberties?
While I love, love, love the EFF, I'd also really like to point out that Public Knowledge are another, lesser known non-profit fighting for your rights and doing incredible work.

Unfortunately the tech community is much less aware of their existence. Public Knowledge are the folks who work day to day in DC lobbying for tech policy. While EFF lead efforts around activism and the legal side, they decided years ago that they wouldn't be involved in the DC side of things. And while being uncompromising is a great ethical position to take, actually making things happen on the ground is crucial.

I spent the last week taking meetings with staffers in DC around §1201 reform for my project, FixtheDMCA.org, and PK set up a dozen meetings with judiciary committee staffers with a weeks notice. They're well-respected in DC, and were critical behind the scenes in organizing the anti-SOPA/PIPA campaign.

Take a minute, read about them, and donate now:

http://www.publicknowledge.org
Welcome to the Web 3.0.1984. It is the read-write-rewrite one.
This is the most detailed and plausible explanation I've seen and Marc Ambinder has been writing some really great stuff on this topic over the past few days. He should be getting a lot more attention.
Yes, encryption before sending it to the server is possible as follows: Ask server for public key associated with email address, encrypt, send.
But if other people do not have this feature for their mail, the mail is encrypted on the server.

The server should never directly store the private keys. Maybe more efficient than generating every time again is to use symmetric encryption on the client side to store the assymetric keys on the server in encrypted form.
It appears that trailing slash characters on URLs
are showing up as an encoded entity (amper-hash-x-hex-hex)
in links since sometime on June 7, 2013.
Many examples in the search posted, here's a
simple example:

https://news.ycombinator.com/item?id=5845372

On a related topic, I've seen mention of people
adding a bare '#' hash to URLs to avoid duplicate
link detection. No idea if that can be handled well
in software.
Side note: seems raw.github.com no longer works, raw are served from github.com instead (they must have changed it very recently), so links in README are dead.
In the midst of the PRISIM NSA fiasco, it's amazing to see just how awesome we can be with technology when we put our minds to it. Despite big sister Curiosity showing off, it's incredible that this came from Opportunity, a rover we've had on Mars for a decade!

My favorite part:

> Opportunity is poised to break the international record for distance traveled on another world during the drive to Solander Point.

Incredible. Just incredible.
Because the IRS was rightfully reviewing a group of right wing lobby groups who were massively applying for tax breaks that they were not entitled to?
Cheers, that's a wonderful quote.
In the US conditions censorship does not come from outer pressure, but from inner servility.
To add: it is not a cage for what you call free speech, it is its skeleton.