skysight-inc/HackerNews huggingface dataset:
This guide is a nice start, but it misses the easiest way to analyze Python performance -- using the %time, %timeit and %prun magic commands in IPython, the better python command line interface (and which also has a killer web notebook interface).

The line and memory profiler modules he mentions can be installed as IPython extensions, too, and you don't even have to modify your code with decorators to make them work.
I always thought trawling involved dragging a net whereas you trolled with a lure (enticing fish out to chase it).

e.g., you might trawl for prawns but troll for snook.
Hell yeah!!

Bob Lazar!
Lets call 115 Lazarium... or maybe Ununpentium itself!
Hey, I'm working on a similar project for scientific writing. You can find the link in my profile along with my email.

We have had success with large collaborations working on it, 15 authors should not be a problem.

Check it out and feel free to contact me.
Building async Unit of Work with MVC 4
I edited my post to add some stuff....

But, yes - it is pretty clear that they have the MSM on their freaking sqwak-box right now against Snowden.

Funny how it was also revealed that they (the UK) were lying about being "leaked" info from Snowden to the "Independent" -- where Snowden came out and said he never communicated to them anything, and the "leak" was a lie and information that Snowden was specifically avoiding getting out because it was too pointed at actual personnel...

This info needs to KEEP COMING -- as we need to get to a tipping point where change is a reality.
I think you could say the same thing about anything else in the software world.
I like http://freshbooks.com it can also track business expenses, invoice, etc. It's nothing ground breaking but I believe they are a strong company and will be around for a long time.
I'm curious; what's the reasoning behind this?
I did say "requiring quadratic memory from every word." I'm not sure you can really call that linear - it's O(n m^2), where n is the number of words, and m is the average word length. It's not the O(n m) that Ferret achieves.

For even a medium sized dictionary of a few dozen MB, you'll find yourself quickly running out of memory. The 4MB dictionary running on the demo would jump to 328MB. Both a Trie and a binary search tree (both of which I've coded and tested) take significantly more memory, and the binary search tree is somewhat slower (try doing a tree traversal between two points as fast as the same traversal over an array).

EDIT: I'll just reply to your edit here for simplicity and because I really don't want to start another thread as I have to get to sleep. I view m as about as constant as n - adding or subtracting words generally changes them both. Try thinking of m as c/n, where c is the total number of characters in your dictionary. O(n m^2) -> O(c^2 / n), whereas ferret uses O(c). The 1,000,000 most frequent English words might have an m of 6-7, but the 100,000 longest English words might have an m of 12-15, taking more memory in a trie, but less memory in Ferret.

But the point is really irrelevant, tbh. We both know how m and n work, and how much memory the trie and Ferret cost for different dictionaries, which is all that should really matter.
Hi xvedejas -- I so wish we had adequate resources so we could consider better formats for the course. I completely agree with you that this is not the optimal format.
btw. he says 0.1% not 0.01%

If you try things that have 0.1% change one after another, the change of succeeding has binomial distribution. The probability that you get at least one success is just 63%
I worked at a firm in 1999 as IT manager... the director required our syslog to be piped out to a line printer printing a real-time physical copy of log/messages and logwatch output on our linux machines.

I was required to manually go through each 24-hour output with a highlighter and look for attacks....

This was not a fun part of my job.
This is a really fantastic idea. I can also imagine writers will get a great deal of use out of git cherry pick.
Hi anonucbstudent -- actually my plan is to personally grade all the quizzes, so that is a burden on me -- not on them.

I completely agree with you that the undergrad students are not at all at fault here. You pay a lot of tuition and deserve better.
Uber launches in Bangalore,India
Telebears is certainly shit, but it's not to blame for this particular fuckup.
Ahh Magic. That's the game that inspired me to build my own TCG engine.

It uses websockets and modern javascript to let anyone create their own TCG/CCG in the browser. Unlike Lacky or similar all the cards and actions are automated so there's no work for the player. It's currently in beta.

This is one of the games that have been built on it:

http://dulst.com/TouhouRSO
Samza - distributed stream processing framework.
Five Incredible—and Real—Mind-Control Applications
Bummed I missed it. We can usually see the contrails from Vandenberg launches. The Delta4 Heavy is something of a beast as I understand it, an engineer that works for ULA described it once as 5 rockets flying in a very tight formation :-) Perhaps we'll see some SpaceX use of Vandenberg which would be fun, then perhaps I'd get enough notice to drive down and watch the launch.
Interesting. When we set this up we looked at maybe adding an automated build (transform) step on commit via our CI (Jenkins) server.

While the software developers are comfortable learning and using Markdown or other markup syntaxes, I eventually would like to spread this approach to other technical people in the company. In previous attempts at teaching basic scripting (Awk, Ruby, Python) to engineers who are comfortable writing complex logic operations in Excel and who can dabble in VBA - I found a lot of resistance to the idea of learning a new syntax. So I expect the evangelism required to get these colleagues to write in Markdown will be greater than showing them Lyx, which at least looks like one of the editing modes in Word.
I went through a period of pretty unproductive time when I was so mad at how crappy things were designed that I couldn't use them to make progress against my projects. That is a bad place to be.
I have a horror story to share with regards to my experience within the LA ecosystem, but unfortunately now is not a good time and I don't have an anon account handy right now.;)

So accelanon your deal was in LA for $40K. There's only a handful of them. Hmm. My situation was pretty bad and I drove 3200 miles and rented a place for 4 expensive months. It was stressful, fun but not what I thought I was getting myself into. BUT the whole ordeal fired me up and I haven't lost a bit of focus. I am much better off now. If anyone wants info on what LA is all about hit me up.
You... what... you built what I considered building (I worked on the OCTGN project, which has a similar idea [build your own TCG]).

I had no idea this existed, can't wait to check it out (awww... closed source? Still cool!).
sudo vim means vim gets root privileges. Meaning vim can do anything it wants if it's bugged or somehow compromised. Not likely on an otherwie secure system but not impossible.

sudoedit (according to the manpage) makes a copy of the file first, lets your editor edit it, then copies back when you're done -- so even a compromised editor couldn't do much damage beyond corrupting the given file.
what if it's not on the left or right. something to think about
The black budget is far far higher than 52B... I recall previous exposures on the black budget - and they were much higher... let me see if I can dig anything up..
Are we blaming him that so many of our actions are embarrassing? Or that they come back to hurt us later?
I was at berkeley for physics and computer science, and while some physics classes had problems with the number of students enrolling, they were nowhere near as bad as the CS classes. It's well known at Cal that there's way too many incoming CS major. They recently started capping the major to people above a certain GPA, but apparently that isn't enough.
This attitude likely resulted from the lack of TAs rather than the other way around...
Thanks, I wanted to build a really really easy engine that truly anyone can use, not just developers. This is the first time I've heard of OCTGN too. Checking it out.
I talk about it any chance I get :)
> btw. he says 0.1% not 0.01%

it was originally posted as 0.01%.

http://webcache.googleusercontent.com/search?q=cache%3Ahttp%...
I honestly thought that it had something to do with http://en.wikipedia.org/wiki/Graphite when I looked at the title... I was almost going to go to the store and buy all the pencils I could get my hands on. But then I clicked the link.... :/
What's so damning here?

It's typical stuff that happens in political or voluntary organizations. JA is far away and don't know the people well enough nor does he have experience. He tries to take control and people give feedback.

There is reason why these discussion are not public. You want a channel where disagreements and internal issues can be sorted out.
Don't you need to trust a certification authority in order to be sure that the browser running SRP hasn't been tampered with?
I'm a CS sophomore at Berkeley, and I actually had the same choice. Sure, it would have been better to go to CMU based solely on the school, but the difference in tuition and fees between the two was too great to not go to Berkeley. (I'm in-state and received little financial aid.)
And it will always be happening, as long as data gets stored.

The only way to prevent it is to not store data.
Never heard that in UK english.

I just assumed that with an american accent the words sounded the same so people stopped using 'trawl'... I've been wrong before though.
Sure, for a bridge. Which isn't an entry or exit node.
Why not just arrange to pay the missing TAs out of your own pocket or a research grant? I'm sure you make enough money. If the issue is not financial but an inability to find other qualified TAs, then double the hours of the ones you already have and it will work out. Offer them 1.5x or 2x pay if they say they are too busy.

Asking students to drop the class just ends up making those who choose to stay in the class feel guilty. And employing draconian measures to encourage people to drop out in order to meet administrative needs is passive aggression, plain and simple. Your beef is with admin, not with students, so don't take it out on them, don't involve them in it, and don't use them as pawns.

Worst case, sacrifice research and do all the TA work yourself. You can use this as a bargaining chip with admin next semester.
There's a section on "running effective attack simulations" starting on slide 104. Where are you supposed to find the (simulated) attackers, within your own organization ? External contractors ?
> I'm not saying that there wasn't oppression in California but if you are going to fight the war wouldn't you got to the front lines?

The Black Panthers drove around with guns shouting legal advice to blacks who were pulled over by the notoriously racist Oakland PD. Is that not "on the front lines" enough for you?[1] Can't people fight racism in their own communities without having to move to the South--which, incidentally, King never did, he was born there?

> There are many ways to accomplish the same goal. How the history is remembered is decided by those who were the most successful in their pursuit.

Not true. History is remembered by what the majority of people choose to believe, not by what actually happened. This is especially true when it comes to assigning credit or blame.

> It's hard to deny that white folks - the ones who's support was needed to get the CRA passed - were not more interested in MLK's message than the Black Panthers and Malcom X.

Of course white folks were more interested in the nonviolent civil rights movement than in black nationalism. But if they weren't scared as hell of the black nationalists, they wouldn't have given enough of a shit to care about anyone seeking to end the oppression of black people. Up until then, white folks were most of all interested in continuing to oppress black folks! That's what I mean when I say MLK gave white folks an out. He made it look respectable and noble for them to fold in the face of a black population that was increasingly refusing to accept being oppressed any longer, because he helped them uphold the illusion that nonviolent, democratic change was still possible. And this illusion is the history that we teach our children.

[1]"...groups of armed Panthers would drive around following police cars. When the police stopped a black person, the Panthers would stand off to the side and shout out legal advice."

http://www.theatlantic.com/magazine/archive/2011/09/the-secr...
5 Common Mistakes When Adopting Agile
Digg Reader debuts on Android
Game-ready 3D Model Challenge
I thought the main technical hurdle (for not dev crowd) in my current workflow was git as there exist a lot of good Markdown editors on all platforms ;)
What prevents TA's from teaching more than 1 or 2 discussion sections?
If each of them taught 2~3 discussions, wouldn't everyone be happier? As students, it would be nice to know how the TA hours are managed. TA's earn a lot of money for teaching. Does a 20-hr TA do 2 hours of discussion, 1 hour of office hour, 1 hour of discussion prep, 1 hour of making homework, etc? How does it add up to 20 hours? Do they have to redesign the entire class and grade homework problems?
If necessary, can they take one more hour of discussion from one of the other responsibilities?
It would be great if the teaching staff is clearer and more open about this.
It looks really good but there is a little bit of a problem with your data structure. Technically rarity, flavor text, number, and artist can all be unique per set. You would be better off including that information in your "prints" sub-array with each set.

The big one with rarity is Serra Angel and Hypnotic Spectre as both of those have switched between uncommon and rare, as well as the original Planeswalkers were just rare in Lorwyn, not mythic as that rarity did not exist at that time.

Another suggestion is to remove the empty fields. It doesn't really matter too much normally, but it could be a slight boost in performance for mobile apps taking advantage of this API as they would essentially be pulling down less data, especially if it is a multi-card search. (100 bytes isn't too much for 1 card, but 100 cards suddenly makes that 9.7kb of extra data that isn't necessary.)

Looks really great though, very responsive.
Foobar -- thanks for your suggestions. I've been trying to hire TAs for this course for two months. It is not a money issue.
Yep, just remove the symlinks and rename the .bak files.
Opened an issue to cover this. Thanks! https://github.com/square/maximum-awesome/issues/37
I spent plenty of time with caching and optimization, CDN all of that. Octopress dropped my load time by a huge margin, simply because I wasn't hitting a database a bunch of times for no reason.
That's not entirely true. Take for example, passwords. If your requirement is "store passwords securely, so they can only be used to check logins", then you implement that as salted, one-way encryption. But if you add the shadow requirement "and be able to retrieve anyone's password when the government requests it", then your password storage will inevitably be much more vulnerable.
> Some of us don't view that as a wanted feature.

No doubt! But someone who was describing his or her fondness for Haskell's type system---the person I was initially responding to---probably does.

I'm familiar (though not intimately familiar) with the idea of dictionary passing and with the fact that type classes are implemented that way. I'm not sure what the point of the remark in the present context is, though. I gather that you aren't a fan of the compiler doing this for you. (I'm not precisely sure what you mean by first-class type objects here, given the mention of Data.Typeable; concretely, I'm aware of three monad libraries that offer something like a polymorphic return and I know how two of them do it; one with symbol macros and regular old maps, and one that (in the presently released version) uses a run-monad function that threads through a regular old map. You presumably are thinking of something else; if so, I'd be glad to know what it is.)

> There is no reason that a user-level macro can't add enough inference to perform polymorphic returns.

Well, if you say so. I can't quite picture how this would work as a macro, but I'm not going to deny it can be done on that ground.

FWIW, regarding your last sentence, I find the idea of a program separate from a particular type system kind of hard to grasp.
I think your approximations are focusing solely on keystrokes, whereas the parent specified just "data", which makes me believe that network traffic, application logs, etc are included in this. I can believe the 1PB/day number, it's not that far fetched when you consider the above.

I also agree with the parent. There are so many possible scenarios and things to log that eventually you're playing a "logging" version of whack-a-mole. Even just managing these files (as the parent talks about) is really no trivial task. Honestly, I wouldn't even know how to begin managing a petabyte worth of daily data.
> When I get new staff, the most difficult thing to teach them is that if they have completed a task that had been given to them, that they have to hand in the work and ask for a new task. But it gets even worse. Many times, if they run into a problem in the task they just give up and don't tell anyone. When you ask them what happened, they give you a blank look and say something like, I couldn't open the file. So instead of asking someone for help they spent the next three hours on facebook until a manager noticed. Very frustrating.

I'm sure that's completely down to a lack of thinking for oneself, and not, say, a stronger desire to be on Facebook than to do one's job ;)
There are two "Card by ID" endpoints, I believe the second one should be "Card by name" or possibly "Search card by name".
That 99percentinvisible article is great reading. OTOH, the site puts some kind of idiotic music player at the bottom of my window. I wouldn't mind so much, except that it covers up the bottom of the FF scroll bar.

Icky design. :-(
Related is the concept of "agency":

• http://lesswrong.com/lw/id2/to_what_degree_do_you_model_peop...

• http://lesswrong.com/lw/5i8/the_power_of_agency/
A slight OT anecdotal - most of the best MtG players I personally knew are now stock brokers.

The game itself, meta-game and the entire ecosystem around it taught me a lot about value, perceived value, sales, psychology... I don't regret any of the thousands dollars spent on it more than a decade ago.
I asked how this could be implemented online 2 days ago: https://news.ycombinator.com/item?id=6284610

I believe Operational Transforms are the way to go, but I have no basis for that.
Another_anon -- that's a great question, but I cannot answer personnel questions in a public forum -- if you want to drop by 739 Soda, I'll be happy to try to answer your question.

Our course has what Berkeley EECS calls "30 hours of TAs." I can assure you that they'll end up doing a lot more than 30 hours worth of work each week.

I'll also remind you that our teaching assistants are unionized, and we are bound by contract rules.

But even if I had the power to change that (and I do not), you can see that it would have to change for every single EECS class -- why would a TA want to teach 4 discussion sections for CS 161 when s/he could get the same pay credit teaching 2 discussions sections for another CS class?
Part of my research is to semi-automatically generate client libraries for web APIs. For funsies, I made this site a definition and ran it through, producing bindings in Racket, Python, and Java.

They won't be 100% correct, but they'll probably work okay. https://github.com/RealTimeWeb/MagicTheGatheringService
Show HN: What can you do after processing millions of tweets and posts.
Well if you have a lot of money available, maybe it's a money issue in that you aren't paying enough. As a thought experiment, what would happen if the TAs were paid $100,000 each? Would you attract more qualified candidates?

Good plumbers make a lot of money because it's a dirty and disgusting job that nobody wants to do.
i built Masel with it (as a demo for our local javascript meetup) i.e.: http://replycam.com/m/ pretty much serverless encrypted message sharing. its MIT L. and unfinished https://github.com/franzenzenhofer/masel
Short of a one-time pad, there's no such thing as truly secure, only theoretically secure according to time and processing constraints.

I'm not trying to be pedantic, I'm just saying, it's not mathematically impossible to crack a salted encryption without the cipher key.
Beginning NES Assembly
How I Smuggled 'Porn' Out of North Korea
Assuming you meant "any good reason you can come up with not to give" then, yes, here's a reason:

People are voting for the State to take my money by force. Why would I compound that by voluntarily giving money to the very same people voting for it? It'd be like giving money to a mugger to ensure that he has enough for his needs.

FWIW, I do give to charities. The Heart Research Institute and the UN refugee camps organisation are two. But they're charities that I value, & they're not putting a gun to my head demanding money.
It has its pro and cons. Let us see how this goes.
Naive legal question - is this API considered copyright infringement? Could they charge for it?

Similarly, are you allowed to create an endpoint for "products for sale at Whole Foods" assuming you collect the data legally?
The "budget PDF" referenced "media exploitation" and my first thought was manipulating the mainstream media into basically reporting what they wanted and if some of the taxpayer funds given to the NSA goes into the pockets of various reporters and journalists.

Maybe the "media exploitation" is what you mentioned in your original post; in effect, "NSA officials", "sources inside the NSA", etc. all "anonymously" making off-the-record comments to journalists.
Women who apply for engineering studies have significantly worse spatial skill than males.

They made experiment where women did some spatial skill exercises few times at the beginning of their studies (total time measured in hours, not in days). Difference between males and females disappeared.
The 'unfriendliness' doesn't come from the names, it's that for it to make sense you have to absorb some conceptual framework of how the thing works. People adapt to odd names quite easily, just as you have, once they get used to them - both syntactically (ls doesn't actually stand for 'list') and semantically (find doesn't really find, maybe 'ftraverse' for filtered traverse).

To use the unix command line you have to get your head around a bunch of key concepts, say, globbing, i/o redirection, pipes, etc. After that, whether it's 'grep' or 'brian' isn't going to get in your way much.

This sort of jargon is pretty typical, though, and I think it's a fairly small barrier. Want to learn some LISPy language? Here come the entirely meaningless car, cdr, cons, s-expression. Functional processing of sequences? Well you can map, you can reduce, you can zip, you can fold. Building your first RoR app? Don't forget to railtie your merb to the phusion gem of the sinatra rack, but not before you've raked your capistrano with a cucumber. You can probably safely rename every JS framework to an MMM (Marklar, Marklar, Marklar) framework given the liberties they take with the meaning of the things M, V and C stand for.

You can give, say, 'cat' or 'tee' magically and perfectly clear names but than in itself won't make anyone understand things like 'unix utilities have one input and two output streams' any better.
The NSA has its own team of elite hackers
Is not Rich Hickey in opposition to typing systems?
http://www.reddit.com/r/programming/comments/lirke/simple_ma...
SoapSox: Adorable children's bath aids by dads