skysight-inc/HackerNews huggingface dataset:
> I want to add something here: great mathematicians compute too.

I totally agree.
To me, there are 2 kinds of algorithms that one should get used to. 1. is computing fast enough (that doesn't mean super fast, but sometimes it means hours instead of months), and 2. is spotting out errors, both evident and subtle.

This said, I'm certainly not in the set of great mathematicians.
Obviously this doesn't apply everywhere, but gentrification is really rough on students. Between tuition increases, rent increases, and high demand for student neighborhoods because they're "hip", students moving to certain areas are being pushed farther and farther away from the actual school (because of both price and availability).

DC is particularly bad in this regard.
I don't think they're doing this to follow a particular law. They're doing it to follow their own internal ethics about child abuse.

If they start reporting sedition and blasphemy, it'd mean that Google's ethics as a group has fundamentally changed, and we'd see far greater effects than their occasionally reporting someone.
Workday Inc. - Data Scientist (full time) - San Francisco

We are looking for some smart data scientists to join my team at Workday.

In particular, we're seeking a curious, out of the box thinker and interdisciplinary data scientist to work as an applied machine learner and algorithm designer. You will bring your expertise to efficiently extract patterns and insights from millions of rows of transactional data to help optimize our workflow and come up with data-driven products using machine learning to help customers in optimizing their business.

About the Team
As part of the Data Science team, you will partner directly with the core decision makers at Workday to help optimize customer experience and help answer business questions. You will work with some of the smartest data scientists to analyze user transaction data and build machine learning models to optimize search, recommendation and personalization across the suite of Workday products including HR, Financials, Payroll and others. You will work on delivering actionable insights and predictive models to help answer business questions around organizational and employee performance.

Responsibilities
• Proficient in translating unstructured business problems into an abstract mathematical framework.
• Excited to learn and apply new methodologies in the intersection of applied machine learning, computer science and statistics and make approximations where needed to build scalable algorithms.
• Excellent interpersonal and communication skills and ability to convey concise and actionable story through data to different parts of the company.

Requirements
• MS/PhD degree in Statistics, Computer Science, Operations Research or related field.
• 2+ years of work experience with proven track record of data science and/or algorithmic development.
• Proficiency in at least one high level programming language like Java, Scala, Python or C++.
• Experience in large scale data analysis in Pig, Hive or Spark is a plus.
• Proficiency in atleast one statistical modeling tools from among R, Matlab or Weka is a plus.
• Experience in predictive analytics and machine learning algorithms especially for supervised (e.g. SVM, Logistic Regression, Boosting) and unsupervised (e.g. k-means, LDA, EM) methods.

For more information and to apply, see our job posting here
http://www.workday.com/company/careers/job_description.php?i...

Feel free to reach out to me for any questions at kevin.tham@workday.com
About the safe haven issue, I'd imagine that if there is probable cause, they can just serve me a warrant rather than serving the guard who is on duty at Fort Knox. From what I know, it is wrong to bypass me to get to my data. If my data happens to reside outside the country, well tough luck. Transfer the case over to the other country and stop being a bully.
The CIA doesn't have the authority to criminalize "protest-plans.ppt". They're part of the executive branch. To criminalize protest plans, Congress would need to pass a law, one that at the very least left room for executive branch rulemaking about which slide decks were or weren't criminal. And, of course, that law would not survive in court.

Of course, you aren't talking about outlawing documents. You're suggesting a different case, one in which hashes of documents are used not for criminal investigations but to keep tabs on citizens. But that wasn't the case I was responding to.
From the write up

>> Since 2003, NCMEC has reviewed and analyzed almost 30 million images and videos of child pornography

I deeply deeply hope that is not 30M distinct photos ... thats an industrial level output not a few ugly people trafficking in horror.

I was struck by a recent podcast from BBCs More Or Less which tried to estimate number of paedophiles in population - and came up with 1%-5%. The problem was the definition - sexual thoughts, sexual actions with pre-pubescent of pubescent children upto the age of consent. That lumped people who thought Kate Moss' early nude photos were sexy together with people who rape 4 year olds which I don't think helps the debate.

Anyway - I think the prevalence is greater than is generally recognised - but even so - 30 M images seems enormous.
Thanks for the reply. Updated the OP - my areas of interest are the kinds of things that get posted about on HN, but I'd really like to broaden my horizons and learn about things like politics and economics and philosophy, anything interesting really.

I'm leaving tomorrow so the only requirement is that I need to be able to get it before then. I had planned on downloading/buying stuff for this during the weeks before I leave but just never got round to it.

African Fractals looks interesting though, I'll check it out. Thanks
Searching for collisions in a corpus of robust hashes seems to me like the post office drug sniffing packages, and people seem to be ok with that. The same way the drug sniff dog won't give away anything other then drug/no drugs (is that how they work? I thought so?), this scheme shouldn't give anything away more then CP/no CP.

At the same time I think that to eliminate CP entirely you need to get rid of some of the freedoms we enjoy. I'm sure you can 100% get rid of CP if you track what everyone is looking at on their computers, but is that a tradeoff you want to make? Even if the filter really only can ever report looking at CP/not looking at CP, would you be comfortable with that running on everything you own?

I could be arguing to a nonsensical extreme, but the NSA tracking all data is following this to some perverted extreme - if we can track EVERYTHING that is going on, and eventually actually make actionable data out of it, we can catch all the criminals/stop crime. But I think we accept the possibility of a bit more crime in exchange for preserving some of our freedoms.
That sounds like different problems have different solutions. "Most math problems are (initially) poorly defined" would have been a better title, I guess.
When I see "Tinder + LinkedIn" I think of a tool for having no-strings-attached sex with former colleagues.
I've seen this discussion around LW an incredible number of times.

The problem seems to be exactly this one: LW may be a nice product (many experience lispers seem to think so) or not (as someone on this thread is arguing), but in all cases LW's price is seen as incredibly high for today's sensitivities, especially given that there are very fine Common Lisp compilers around that are free (although way less refined, in the GUI department especially).

LW has chosen to follow this kind of 80's-like business model and it's certainly their right to do so. They certainly have their reasons. Although personally I doubt this road is future-proof (or even present-proof).
Music Memes: The Human Imperative
This is awesome - never knew it existed. Thanks a lot
'"To my surprise," Freeman says, "it seemed to suggest that people in neighborhoods classified as gentrifying were moving less frequently."'

Well, duh. With rents rising, your only chance to stay in the neighbourhood is to cling on to your existing contract. Look at a fully-gentrified neighbourhood; the only low-income residents will be those who have been living in the same home for decades, with a rental contract that protected them from rent increases.
hahaha seriously? get some books or something. or like, talk to people in real life
How To Avoid Giving Our Your Personal Info (Legally)
And yet when this fancy boulangerie opened in a modest neighborhood in 1812, that was a textbook example of gentrification.
If this is Microsoft's hash algorithm, the one from the robust image hashing paper, it's not perceptual (like Shazam); it's coding-theoretic. If I understand it correctly, it leaves room for false negatives from drastic (but intelligible) image changes in favor of avoiding false positives from attackers trying to deliberately trip it with non-matching images.
endianess doesn't matter when you all you do with it is store it and check if it's equal to another.

In all cases where it does matter, the values are converted.

Tox has been confirmed working on big endian machines by many people.
Wikipedia archive.

https://dumps.wikimedia.org/other/static_html_dumps/

Gutenberg archive.

http://www.gutenberg.org/robot/harvest

Should be enough for anyone, and way less space than you would expect. Also, something on the languages for where you're going? And some docs for the work you'll be doing?
That's also what any non-technical person who thinks about for more than a millisecond would realize too, given the volume of Gmail traffic. Unfortunately, a millisecond's thought isn't conducive to fearmongering.
This scares me. What can we do though? Can we decriminalize/legalize possession of anything? Perhaps just make it illegal to produce illicit materials? How would it work? Would that mean bad guys would be able to legally own weapons-grade plutonium if they didn't produce it?
5 weeks is a decent time: at first glance it seems like you'll need all sorts of materials. Seriously, it depends on how much free time you anticipate having and whether or not you could use a lot of that time having experience.

All that said, I'm pretty sure you can survive on what my spouse and I view as our own travel pack: Bring a few physical books - I read 1.5 during a 5 day stay in Amsterdam and it was much more comfortable carrying around a paperback than a laptop and much less risk of people taking it. Otherwise, just download away in any category you find interesting. I use an app called pocket on my phone to save long articles for later reading, right now my it centers around long occult articles, in depth current events, and tech news. These prove to be more timeless than other things - at least in short term. Movies and shows, that's fun. I hesitate to suggest movies or shows, but if I were going to suggest, might try Six String Samurai - but I'll warn you, it is a bit odd. Also, I do suggest downloading some free games to take with you that can be played offline just for some variety.

I partially agree with those that say to not bring it, though - use the laptop lightly, if possible, but I'd not leave it at home. Go out with a camera (gasp) and enjoy :)
Beats store now redirects to Apple store
I have pretty good confidence in law enforcement caring about the law in general.

I have quite low confidence in intelligence agencies using a law for its intended purpose. I expect them to stretch and distort what they are allowed to do, and to abuse mass filtering mechanisms. So they should for the most part not be allowed to have mass filtering mechanisms.

To put it more simply: I do not expect the details of these laws to be followed, and 'type of content blocked' is a detail. But at a macro scale the law will be followed.
I wholeheartedly agree. The problem is that I won't be able to go out exploring after dark unless I'm with someone (which won't be every day) for safety reasons so I'll definitely have a few hours stuck in my room every day.
It's not really fair for me to comment since I have only ever posted in one such thread and I did get a few interesting conversations out of it, but there is a lot of strong competition on HN. I wouldn't go in with high hopes.
Over the past few years I've spent a total of five months travelling through or living in various remote corners of Africa. My advice would be to forget the Internet and all that is on it. Take a decent camera, learn photography. Take a local dictionary, learn the language. Take some art supplies, learn to paint. Take a guidebook and learn about the animals and plants. Get up for sunrise. Immerse yourself in the culture and the environment. You'll get far more out of your trip that way than if you're glued to a computer screen the whole time.
Oddly enough, Overtone running in Clojure will give you a much interesting and creative musical experience than OM will.

(In my opinion, anyway.)
Why do you target your rage at the hipsters paying £5 for a beer rather than the bartender charging £5 for that beer? Or, perhaps, his landlord, for raising is rent to the point where he needs to charge £5? I guarantee that the hipster would rather pay £2. The whole issue of gentrification seems to always come around to blaming the people who move in rather than the people who profit off of them, be they existing home owners, local businesses or real estate developers.

If the displaced want to cast this as a class war, follow through on the metaphor. Profiteering is considered criminal in times of war. Those profiting from the influx of money need to be held accountable.
I'd highly recommend fiction, history, and biographies. If you can, find things written about people and events in the region(s) you're visiting. Detailed maps are also great, because they let you go wander around and get a bit lost, then find your way back to wherever you've staying.

My usual media/gadget inventory while traveling is a compact camera, Kindle and a paperback or two, and a battery-powered speaker for listening to music at night. No headphones (too isolating), no tech books (distracting, and honestly frustrating w/o the Internet to do additional research) and only as many hours of video as I need to keep myself distracted during the flight.
Semiotics in Art  (2010)
You know, I sold a website that I created in 2007 that was
dedicated specifically to selling contemporary art.

This is a huge untapped market that is so fucking easy to
get into...

I love art. If you have a magento install and a beating
heart and the willingness to talk to art dealers you can
find a lot of people to work with to list inventory. Many
times art dealers are quite willing to work with honest
people that have tech skills.

This is a fantastic idea. Executed properly, you stand to
make a considerable sum. If you care about that, that is. I
do so it is typically how I evaluate value. Others might put
more value on the dissemination of the art itself.

Not me.
The IP you query as a client is anycast, but I'd wager recursive queries out of their caches come from geo-friendly IPs.
I am from India. Bandwidth is costly here. Internet enabled devices have surprisingly have become cheaper. Making some access 'Free' could change the world of many people.

I agree where there is food and shelter in short, giving Internet free may not help to fight these. But it will help to fight corruption and bureaucracy. Many important government services are online nowadays. In West-Bengal, government stopped online college admission just to keep bureaucratic system going. They fear free Internet, it is a killer for corruption here.
It only sounds batshit insane if you're used to oppressive misuse of law enforcement. Living in the UK it sounds reasonable to me that when a child abuser is caught with images of child sexual abuse that law enforcment:

1: tries to identify and help the victim

2: tries to identify and punish the perpetrator

3: tries to find and punish other people using these same images

Whilst I disagree with drug laws I think it is sensible (within the context of thise laws) that customs officers inspect packages passing the borders to find contraband.
One reason is that it's extraordinarily expensive to enlist Google and Microsoft in the pursuit of government secrets compared to the alternatives.

The USG (like most F-500 companies) invests a lot of money in patrolling the borders of its networks for breaches of sensitive information. For commercial entities, there's a whole class of product ("data loss prevention" systems) that do this out of the box.

Since (a) the information the USG is watching for is generated from within their network borders, and (b) USG isn't comfortable sharing that information or artifacts of that information with third parties, it doesn't make a lot of sense for them to invest a huge amount of effort (both technically and legally) to come up with some cockamamie scheme to have Google look for those secrets without knowing what the secret is.

Instead of engaging with the intractable problem of having Google dragnet GMail for documents Google is not allowed to read, the USG is much more likely to just subpoena the accounts of people it suspects to be trafficking in those documents.
Why are so many websites trying to be app-first?

Desktops went through the same fad.

Guess what?

The browser is the only app I am actually going to load your service in so why not DO BETTER?
5 hours of lectures on deep learning:

http://www.trivedigaurav.com/blog/quoc-les-lectures-on-deep-...
Download a dictionary to translate to and from English and whatever language they'll be speaking where you are.
User enters location data, habits, events, tasks, hours of sleep - pretty much all individual activity for 24 hours, spanning months, years...

Who else wants that valuable data :-)

http://www.timeful.com/founders-pledge

"We hold that information in our servers...

There’s a legitimate question people raise when asked to provide personal information: “How do you make money? And does it involve selling my information or otherwise sharing it?”

Here’s our answer: We don’t yet know how we’ll make money. We believe if we are successful in helping people make better use of their time --­­ this most precious and perishable resource -- ­­ then they will be willing to reward us for it."
FWIW: Child abuse of most forms is subject to mandatory reporting.

It is highly likely this is not as voluntary as you think.

Google would likely have liability if they discovered this and didn't report.
Worked for me in Chrome but not in the latest version of Firefox. Is there something that I need to turn on for Firefox to work?
Can't upgrade my Mac
I bought a new Mac with retina display (4gb/cheapest retina version). I used it for a month without any problems, but after that it started freezing/crashing. I tried contacting apple saying this is not what I expected (with 4gb of ram) with all the crashing's and they said they couldn't do anything about it since it's past the return date. I can't even upgrade the ram (which is the reason for the freezes). Now, they come out with 8gb ram and faster ones for the same price. I can't even sell this for as much as I would have liked either.

Shouldn't they be able to help me get 8gb of ram, since my Mac doesn't even function properly?
FWIW: Child abuse of most forms is subject to mandatory reporting.

It is highly likely this is not as voluntary as you think.
While in the US, they are not required to scan, this is not always true in every other country.
" Why the hell are our service providers becoming Internet police these days?"

Because laws keep getting passed that ask them to?
>Imagine a world in which every single human being can freely share in the sum of all knowledge. That’s our commitment.

Wikipedia can never be the sum of all human knowledge and anyone who thinks so is seriously deluded. The web in its entirety still doesn't include even a fraction of all knowledge. To think that one website -- a poorly designed one, with terrible software, design, media handling, policies and leadership -- is claiming this is simply laughable.

That goes for Wikipedia "protecting the internet". Protecting it from who exactly?

Wikipedia's leadership and its acolytes constantly use false utopian goals to justify their expansion into areas that they are ill-equipped to deal with (see: dangerous and incorrect medical data). It seems they are trying to use the same lofty ideals to carve out market protection in Africa.
Some people do use them though, and they actually work.
Are you thinking of Donald Knuth? rms replies to email.
"Why" is the big question, isn't it?

It looks to me like a feedback loop. Fear in the populace results in politicians and media responding to and magnifying that fear. Fearful news reporting generates better ratings in a fearful populace, which makes them more fearful. Similarly, fearful political rhetoric garners votes from a fearful populace, which makes them more fearful. This would imply that it's sensitive to small differences in initial conditions.

One thing that I'd bet makes a big difference is the existence and magnitude of other threats. The US and UK are pretty solidly secure right now. There are no major human threats present now or in the near future. Compare with the era you're talking about, in which our countries (and, of course, many others) were perpetually 15 minutes away from total annihilation at the hands of the Soviets, either because they decided to try to take over the world, or just because of some terrible mistake. I imagine one probably doesn't take a terrorist nail bomb too seriously when you legitimately think there's a real possibility that you and everyone you ever knew will be killed in a two-hour-long WWIII.

Perhaps we're geared towards thinking about existential threats, and if one isn't present, we'll magnify whatever we can find.
We're always interested in learning and exploring new things.

While we love Ruby and Rails and we've been fortunate to have chosen Rails very early on, as people who want to enjoy our tools, we don't want to miss out on finding the next thing we love to work with.
> Google shouldn't be looking in my email in the fiery place period for anything . The end. I'll give them permission to automate scanning to place ads but that's it. Anything past that is out full stop

All services have clauses saying what you're not allowed to do on those services; and that they monitor to ensure that you're not doing that. I'd be amazed if the user agreement you signed when you started using Google (assuming you use Google) didn't have that.
I've always enjoyed fiction in situations like yours. Iain M. Banks has some accessible but interesting sci-fi (recommendations: Consider Phlebas, Use of Weapons), and it's hard to go wrong with anything Jose Saramago wrote (recommendations: All the Names, Blindness, Death With Interruptions).
There is a Wikipedia archive now that includes images: http://www.kiwix.org/wiki/Wikipedia_in_all_languages
Sure. I think I agree with this. However: what's being described here is not a system that can be directly abused by the CIA or NSA to trawl for documents. It is coupled in at least two specific ways with the search for child pornography: (1) it only works for images (from a known corpus), and (2) the corpus is managed not by the USG writ large but by a cooperative effort specific to child pornography.

I see the slope, but the traction on it is pretty solid.
Not quite the same as what you said, but I really wish any of the offline reading apps would automatically scrape comments from HN/reddit/disqus along with the page itself. Reading articles on Instapaper/Pocket/... without being able to check out the top comments feels incomplete :(