skysight-inc/HackerNews huggingface dataset:
>However I think there is also plenty of room for improvement in the ETL process that doesn't necessarily mean a one-stop solution.

I agree with you here as well. I recently wrote a blog article about this, at least for log data > http://www.fluentd.org/blog/unified-logging-layer
Software Engineers Will Work One Day for English Majors
Holy crap. Thank you for that link. I never knew about frission before let alone that I could experience it fairly powerfully. It seems like I have to "open my mind" to it before it can happen. Very cool mind hack.
Flash Boys by Michael Lewis: http://www.amazon.com/Flash-Boys-Michael-Lewis/dp/0393244660

Flash Boys is about a small group of Wall Street guys who figure out that the U.S. stock market has been rigged for the benefit of insiders and that, post–financial crisis, the markets have become not more free but less, and more controlled by the big Wall Street banks. Working at different firms, they come to this realization separately; but after they discover one another, the flash boys band together and set out to reform the financial markets. This they do by creating an exchange in which high-frequency trading—source of the most intractable problems—will have no advantage whatsoever.
If you want to focus on something and be a happy expert on it, but don't want the insane license costs and lack of freedom, Java + Linux/BSD is a superb alternative. It gets you the same familiar, consistent platform, IDE support, performance, and do-anything-ness, while being much less restrictive. There's nothing that platform can't do that Microsoft can't, and you also get the option experimenting with interesting JVM language alternatives like Scala, Clojure, Kotlin, JRuby, etc, if you ever get bored of pure Java.
1. This would require an extended license (key being the fact that the final product is used by multiple customers)

2. How exactly would this embedding take place? Using an iframe?

3. Thanks for pointing this out, we'll get that sorted out asap
I love this. I'm so glad there's still inspiration for simple refreshing games
Wait, they can return it, opened? And it is resold as new?
If cancer research is anything like high frequency trading and time series analysis (which I used to work on), then one heinous thing about corrupted data is inter-dependencies among different data processing tasks: if data coming in "early" in the data workflow topology is garbage, its errors are propagated throughout the dataflow, resulting in multiple days of data detective work, laborious undoing of everything that went wrong, etc.

Also, I observe that the data science community does not talk about these issues nearly as much as they should either because 1) it doesn't make a very inspirational topic 2) they are shielded from this kind of issue by a data engineering team.
Yeah ditto :P
"awesome-lua" is probably not a good choice of name as the most common association of "awesome" with "Lua" is the Lua-based Awesome window manager.
It's not controversial. It is well known that triglycerides and LDL/HDL ratios are more important than total cholesterol. These are all part of the standard blood lipid panel. Some doctors are just unaware of the relatively modern (10-20yrs) advances in blood lipid analysis.
Bitlocker wasn't written by hobbyists. Ironically, Truecrypt probably was. But if you'd like a more certain comparison, look at the Linux encrypted filesystems to see how amateurish mainstream open-source crypto can be.
Discovering Python – PyCon 2014
Just HN Enhancement Suite, which does a whole heap of things like collapsible comments, an (arguably) nicer site design / layout, as well as a bunch more.

https://chrome.google.com/webstore/detail/hacker-news-enhanc...
Persistence was disabled. Nano and ZMQ are in their own group and aren't even attempted to be compared to other groups.
You gather that from exactly which crypto experts?

Yes: Bitlocker was audited. No company in the world spends more on audits, and is more sophisticated in sourcing them, than Microsoft.
It's hard to blame doctors, when patients prefer to eat mcdonalds+pills over any kind of vaguely healthy diet from any model (be it ketonic or paleo or south beach or low fat or low sugar vegetarian or whole foods or whatever)
> changing IPs is OK and accessing a website is OK, but when you've received a C&D letter and you circumvent an IP ban, that's lawsuit-worthy

I don't see how this differs from the situation of entering a store while banned (and optionally disguised). It's quite clear that visiting the website was an offense regardless of IP cloaking.

> Craigslist gave the world permission (i.e., “authorization”) to access the public information on its public website. Then, just as Brekka instructed that an “authority” can do, it rescinded that permission for 3Taps. Further access by 3Taps after that rescission was “without authorization.”

> The banned user has to follow only one, clear rule: do not access the website.

The IP ban was considered evidence that the requirement to stay off the website was properly communicated, but the C&D is surely sufficient for that purpose.

(Orin Kerr actually blasts the judge for misunderstanding the law, saying that in his [Kerr's] view accessing the website after receiving a C&D would not in itself be an offense... but the judgment is officially the law, and Orin Kerr isn't.)
After you get past the components, some computer architecture will do you good. See eg https://www.coursera.org/course/comparch. There's also the classic Hennessy & Patterson book.
There was no news coverage because virtually no expert in the field believes it to have been an NSA backdoor. The arguments suggesting it is were debunked by Bruce Schneier more than a decade ago. Among them, the obvious: Microsoft held the other key, the "non-NSAKEY" key, and could do anything that the NSAKEY can do. If NSA can coerce Microsoft into adding a whole new signing key, it could just as easily have coerced Microsoft into signing things with Microsoft's own key. Given that, an "NSAKEY" backdoor is irrational.

In promoting the notion that "NSAKEY" is a backdoor, you're lining up against the likes of Bruce Schneier and lining up alongside the likes of WorldNetDaily, which is indeed among the top Google search results for [NSAKEY backdoor].
How does hidden help? An honest response to a hypochondriac is pretty obvious to any other doctor. 100x "patient complains of y, tests show no evidence"
how does one create his own 'crowdsource' labor force to solve specific domains of problems? Where do you find workers that will be readily available to work on problems?
Interesting. To who else's work can you write a nerd message board post explaining your intrinsic entitlement?

It's not at all shocking that the creator and maintainer of a free encrypted filesystem would eventually abandon the effort, given that comments like these are how the effort is ultimately repaid.

Is there a name for the hammer you swing, the hammer not of abusing copyright but instead of not writing simulated hardware disk encryption and donating it for free? It seems to me that's a far mightier hammer than the one you described.
It is worse that the articles says -- the data cleaning is often not tracked nor reproducible. First, think about how much data cleaning is not done by a script or versioned at all. So many people just overwrite the bad data in place. Second, of the data cleaning / ETL that is scripted or partially automated, think about how much is not saved, much less version controlled. As a result, if you ever want to go back and reproduce or share your analysis, good luck!

Doing it better is simple: If you do ETL, version control your code. If you do hand edits, track the before and after.
Ultimately anyone can lie to anyone, and misplaced trust is a separate problem from letting people know. Plenty of records are incorrect, and kill people.
Yes! They'll never know what life is really like in the data trenches, watching your best buddy get renormalized just one standard deviation away from you. And maybe they can sleep at night.

Semper chi^2, bro.
Comma separated values (or CSV) [1] format is dead simple; more simple than JSON, you just need a print function.

1. http://en.wikipedia.org/wiki/Comma-separated_values#Basic_ru...
that's awesome you were able to do that. but i think you might have been one of the lucky ones who got in when the dynamics weren't quite as unbalanced. uber, lyft, et al. needed initially to entice drivers to the platform so paid more to jumpstart the labor supply.

the disproportionate power i'm talking about is the ability of these companies to unilaterally decide the terms of the contract with labor as a whole (and really, they don't have as much choice as this makes it sound). yes, you can choose to accept or not accept, but your decision has no bearing on the market dynamics because you have no power to dictate the terms for which you'll work (e.g., you couldn't demand an extra $5/hr). part of this is because the fluidity of fractional employment means that labor supply can instantaneously fill demand--there's no longer a penalty for an employee leaving.
Url changed from http://www.gizmag.com/harvard-multifunctional-electrochemica..., which points to this.
Very cool! I guess my particular ETL complaint above is solved for log data. :)
Extraordinary claims require extraordinary evidence. Please share.
The problem is a part of the "public" part of it, not the public part itself. The problem is a weak government that could be corrupted by private interest. That doesn't mean you throw the baby out with the bathwater.

I'd be fine with red light cameras if they were owned by the government and it allowed police to worry about more pressing matters.
I'm aware that there are distributions without finite first moments; that's why I specified it. But a real electrical signal cannot have infinite expected value, so that's not an explanation.

Note that your archer analogy implicitly requires infinite expected energy input.
Was anyone else confused by http://www.hexraystudios.com/ vs http://www.hex-rays.com/
Is this the same or different project from the TrueCryptNext one?
After Code, I recently discovered Brian Kernighan's D is for Digital, published in 2011, which gives a much more concrete description of how modern computing works.
Dell especially seems to be targeted a lot by NSA - easy target? Or something more?
There was plenty of solid evidence of those countries spying on US citizens long before there was solid evidence of the US spying on US citizens.
This is a totally valid point which I considered, but I think the association of awesome-<language> for lists like this is more valuable, since part of the purpose is to make it accessible for people outside the lua world (who probably won't know about awesomeWM anyway).
Lorem Ipsum: Of Good and Evil, Google and China
> Your making the point that because they weren't outright communist, they must have been right-wing.

That wasn't the point I was trying to make.

And after re-reading, I'm pretty sure that wasn't the point I actually made. In fact I tried to make it pretty clear with the last sentence.

But perhaps it would be simpler to note that the Nazis were not simply a left wing group without confusing substantiation.
Yes, I disagree. If there's an almost guaranteed chance that my org will be spied on, I would rather have the US do the spying. Unlike a BRICS country, at least the US has the occasional Snowden to shed light on things. The resulting scandal may do little to change the status quo, but it'll lead to a discussion and maybe a civil suit my company can win.

I have never heard of a similar public leaker from Russia or China (I imagine those people just quickly get disappeared).

As for BRICS country spying, we have some very solid evidence that China is engaging in cyber espionage of both US military and corporate targets.
http://boookshelf.com/

Like Goodreads, Shelfari etc... but emphasis on shelf and following other's shelves.
I feel your sentiment. I’ve had a love hate relationship with the medical community. It took two plus years for a diagnosis of Celiac’s. Along the way, I met a number of doctors who took the attitude that if they couldn’t figure out what was wrong, it must be in my head. And while these docs made me want to crack skulls, I also came across a number of great doctor’s with empathy, knowledge, and an understanding that their knowledge is not infinite. It’s a tough job, and after that many years in school, it’s tough to be humble.

It was my step-brother, a Cardiologist, who recommended against my second endoscopy that led to the diagnosis and the transformation of my health. Like my XO in the Navy said, ‘Only you are responsible for your career.’ It applies to medicine as well, ‘Only you are responsible for your health.’ Especially with anything chronic that may be outside the 99.9% bounds that modern medicine is designed to handle well.
Please don't astroturf on HN.
When I went back to the States recently I was shocked by just how bad Starbucks coffee has gotten. And, it was not predictable at all - the three drinks I got there (all the same drink) were each terrible in their own special way. The only reason I went back a second and third time was sheer morbid curiosity. I didn't bother going to a McDonald's, but then again McDonald's was already like that when I still lived in the US anyway.

Starbucks coffee in Japan isn't great, but it isn't fucking gross either. And it is very, very consistent.
The regexp must have taken some time to write and debug:

s/^."n":"(.+)","i":"(.+)","p":\[([\d\.\-]+),([\d\.\-]+)],"s":"(\w+)","c":"(.+)".$/\1,\2,\3,\4,\5/;
Ah I really love videos like this one. I recommend http://www.reddit.com/r/ArtisanVideos/ for those who feel the same way.
Speaking for the data janitors of the world, I can't believe there wasn't a mention of hiring some of us to do this so-called dirty work, if for nothing else than to save these data scientists from using the word sexy.

After twenty years of financial operations, including a decade in the back-office of a top hedge fund, I eventually accepted that I'm a bit backwards for my desire not to jump straight to a pivot table when encountering a new data set. Like a farmer reaching down to touch the soil, my first step is exploring the rows and columns with little tests here and there to find weaknesses within the information at hand rather than paving over them with instantly flawed reporting.

Anyway, for all the big data scientists out there too sexy to clean up their own data, I'm looking for work and don't mind pushing a broom. Check my profile for more info.
Elevated rail, even monorail, is considerably more expensive, but also provides far higher capacity. You can also increase capacity on a train line relatively easily with shorter intervals and longer trains, while a gondola's carrying capacity is essentially fixed.

Rough comparison between three recent 3rd-world systems:

- La Paz gondola: 3000 passengers per hour per direction (pphpd), ~$10m/km

- Mumbai Monorail: 10,000 pphpd, ~$20m/km

- BTS Skytrain, Bangkok: >30,000 pphpd, >$100m/km (on recent extensions)

Gondolas make sense in hilly locales where hurricanes are not a threat (like La Paz), they'd be a non-starter for both reasons in Miami.
that's certainly true, but there are no economies of scale to that model, so those small, disaggregated market players will have little to no effect overall. at best, collectively they may have a minor bounding effect on wages but that's unlikely to change the overall market dynamics (even the US, as the largest oil consuming country in the world by far, cannot dictate the price of a barrel of oil).
Is that really the case? You've always been able to simultaneously charge and sync your iDevice with your Mac.
What information have Microsoft released about the Bitlocker audit(s)? I couldn't find any, although my search wasn't particularly thorough. It seems to me that the value of an audit, for third parties, is that the auditor puts their reputation behind it.
Works for me at least. Try this shortened URL though: http://preview.tinyurl.com/lm57wd6
I think that a stronger government will only be more corruptible. The only reasons private interests try to influence government is that government has the ability to benefit then.
That happens in a statistically negligible number of audits, and most especially in the best cryptographic audits. Which systems has Cryptography Research audited? Answer: you have no idea.
i'm afraid that with the ever greater concentration of wealth in the US that there will never be a strong enough economic cycle to deplete the fractional labor supply. i'd love to be proven wrong though.
Heh, this is timely. Just showed up on Reddit:

http://www.vice.com/read/undercover-in-americas-privitised-p...

The money quote:

"I didn’t believe that a “concentration camp” could exist in America until, in Phoenix, Arizona, I saw one with my own eyes."
Meet the new, reversible USB
Evolution of Docker and Its Impact on AWS
But the government doesn't optimize for profit, corporations do, and this is one of many cases (like with prisons) that optimizing for profit is bad for the world.
Solzhenitsyn's "One Day in the Life of Ivan Denisovich", about a bricklayer at a Soviet gulag camp, has a short section about how getting into the flow of laying bricks (well) is pretty much the only remotely enjoyable part of existence in this frozen hell.

http://en.wikipedia.org/wiki/One_Day_in_the_Life_of_Ivan_Den...
Can you clarify what you mean by 'empowering'? Something like the warning labels on cigarettes, or what?
Uh oh, it looks like @tptacek's calling people message board nerds again.
OP will surely deliver
We're finding out that fat [1] isn't actually bad for you, now salt, too. I wonder if we'll find out sugar is good for us, too, before long.

[1] http://online.wsj.com/news/articles/SB1000142405270230367840...
SQL Server Radio Show 2 – Death to UDFs
Of course I'm not saying expanding income inequality is better for the have nots. I'm just saying this may lead to some self-correction of relative earnings from labour vs. capital as wealth becomes more concentrated, although overall things are still worse for the working chump.
I'm @tqbf, not @tptacek. @tptacek is some other person on Twitter.
I'm pretty sure a gondola can hold its own in a hurricane (as long as it isn't running). Most urban systems use the same technology as alpine systems, which have to withstand harsh blizzards.
I get the impression that most violent terrorist groups are rather low-tech and are not using _any_ server hardware. If you're aware of terrorists routinely operating DCs or co-locating servers, feel free to correct my understanding.
Tracking time
There is another awesome-lua list that predates this one: https://github.com/forhappy/awesome-lua

The "master list": https://github.com/bayandin/awesome-awesomeness
He could also have had that sitting around from a recent project.. it doesn't really seem worth second-guessing.
Have you looked at luafun? If so, what's your opinion?

https://github.com/rtsisyk/luafun
Robin Williams Dancing To the Happy Song
If you are working on US based consumer web site, spying by NSA could be much more harmful than spying by China or Russia for vast majority of users. NSA is known to provide leads to police that then reconstructed evidence to exclude any links to NSA and still arrest/harass the person. There are people who are not national security risk in US jails because of NSA (mostly drug busts). BRICS countries have zero interest whether you smoke dope in your free time and won't rat you out to local police.

PS. I am not arguing that they should not be in jail. Just that NSA used semi-legal (waterboarding!=torture kind) methods to obtain initial leads.
I'm pretty sure we will. At the end of the day it's the amount of each that's consumed that matters. As the old adage goes, everything is fine in moderation.
Ask HN: Good books on robotics
Hi,

I'm starting to get interested in robotics and I was wondering if you guys could recommend any good books in the field. It's fine if the books only cover a subsection like vision, actuators or sensors. I just want to understand more about the field. Thanks
Building a Modern Computer from First Principles:
http://www.amazon.com/The-Elements-Computing-Systems-Princip...
> Like a farmer reaching down to touch the soil, my first step is exploring the rows and columns with little tests here and there to find weaknesses within the information at hand rather than paving over them with instantly flawed reporting.

This needs to be done by more people, more often. I've worked with more than one (often SQL, but it doesn't matter¹) database whereby someone will claim that X is always true. It X is something that can be put into a DB query, then you should do that, and run the query. Every time. In my experience, if it isn't enforced by constraints in the database software, then it isn't true.

¹The nice thing about SQL is that you can but constraints on the data, and use the database to keep various invariants invariant.

Often, you'll find people making assumptions about the data based on the business logic, not the constraints in the database. Such thinking is flawed: you cannot reason about what you think the data is, you must reason from what the data actually is.

Worse, you'll find people making assumptions about what they think the business logic is.
Is anyone actually doing that? I wonder if there's a profession, or if there ought to be, specifically for "data wrangling". On the other hand, maybe you can't separate it from the work of the data scientist... perhaps the data scientist needs to be involved in the data wrangling in order to understand the source better?
It depends on how you measure.

Per capita and income adjusted there are far more corrupt countries.

But in terms of sheer amount of money spent on distorting politics at home, combined with countries invaded or otherwise screwed around with for economic reasons abroad, the USA is definitely hands down number one since the second world war.

This is not because the US is particularly nasty, it just has more resources to throw at this kind of stuff since becoming the dominant superpower. Though you do not become the dominant superpower by being all that nice either.
New Version Available (app for Android with regular updates)
Perhaps more interesting is the actual report this article is based off of: http://www.mass.gov/ig/publications/reports-and-recommendati...
I think my kids are having the same problem (boredom) as you did.

What do you mean by "paying attention"? One of the thinks I really like about Montessori, and I'm guessing each school is different, is that the children are able to develop long attention spans and learn to stay focused on something for as long as they like; half a day is not uncommon, and an entire day is possible. Normal school allows children just long enough to become absorbed in something, then interrupts them with "change class!". Montessori allows that child so keep focused on their activity (drawing a map, writing a story, maths, etc) for as long as they want.

The up-side is that I've found Montessori to be a very efficient means of learning. My kids have no homework and an enjoyable relaxing day, but they're far ahead of their friends in normal schools.

Down-side is, like you say, low tollerence for droning teachers and being bored
also, http://js1k.com/2010-first/demo/778
At Kenhub[0], we built an anatomy learning platform that uses spaced repetition. The algorithm might not be as elaborate, but the basic concept is the same.

In our case, it's a little different from a flashcard system, because anatomy structures are typically grouped and asked together (e.g. choose the right term from the following images, choose the correct term in the picture from the following list etc). There's also more complex topics such as origin, insertion, innervation, function - which covers the way anatomy structures 'interact' (e.g. this muscle flexes the arm...).

[0] https://www.kenhub.com