skysight-inc/HackerNews huggingface dataset:
there is no way to find out the relative significance of the input variables and how they affect the output response

"Calculate partial derivatives of inputs w.r.t. outputs."

Yes. There are other ways to interpret the output. See, for example, "Visualizing Higher Layer Features of a Deep Network." by Erhan et al 2009:
http://www.iro.umontreal.ca/~lisa/publications/?page=publica...

It is true, though, that more powerful models are less explainable. But in return you have more compact modeling and training via gradient descent. This is faar faster than the combinatorial optimization involved in ensembles of trees. (Trust me, I've implemented both.)

there isn't a way to cluster similar cases together to analyze what rules bind different decisions together

"PCA analysis of activations of the hidden layer. Could also do k-means clustering of activations of hidden layer."

I concur. However, for 2-d or 3-d visualization, you should use the more recently developed t-SNE algorithm instead of PCA or other alternatives. t-SNE does waay better.
Software (in Matlab and Python) is available at:
http://ict.ewi.tudelft.nl/~lvandermaaten/t-SNE.html

Also, you should look at the JMLR paper (http://ict.ewi.tudelft.nl/~lvandermaaten/t-SNE_files/vanderm...) and supplemental material (http://ict.ewi.tudelft.nl/~lvandermaaten/t-SNE_files/Supplem...), to see the visualizations produced by t-SNE and competing methods. This qualitative evaluation by looking at pictures speaks for itself.

Also, one cannot find the partial dependence of the output on a given input variable.

If your model assumes that the output is a non-linear combination of inputs, then yes, it is hard to express the output in terms of a linear decomposition. But that was your choice of modelling assumption, presumably because linear models are insufficiently powerful to fit the underlying variations.

"The random forest classifier looks interesting, I'll have to investigate that. Any suggestions for papers/tutorials?"

Random forests were developed by Leo Breiman (RIP).

"Ensemble methods in machine learning" by Diettrich (2000) compares different tree ensemble methods. He concludes that boosting an ensemble of decision trees is better, except when the data are very noisy, in which randomized trees are better. (Boosting is when you focus on the examples that the model is currently doing the worst. Randomized instead works on random subsets of examples.) The main reason boosting is worse than randomized trees in the noisy case is because the AdaBoost exponential loss is sensitive to outliers. Which is to say, AdaBoost boosts the wrong loss function. Boosting an appropriate choice of loss function (perhaps a regularized log-loss) is probably superior to randomized trees in most circumstances.

"Improved boosting algorithms using confidence-rated predictions" by Schapire and Singer (2000) is a great introduction to boosting.

Around the same time Llew Mason and Jerome Friedman independently demonstrated that boosting is essentially fitting an additive model using gradient-based methods to select the features that have the steepest loss gradient. So you should follow up by looking at their work.
and why do people disagree with me ? downvotes but no argument?
Wouldn't it make more sense to not autosuggest certain names in the first place in that AJAX drop-down? This kinda feels like they're building a feature to defend against another feature. Perhaps these guys need something else to do?
I believe in free markets too (I'm an economist + engineer). But Google, which professes to "not be evil" & is making peanuts on these services, could have set a great example (and not risked damaging publicity)..
Ask HN: Startup Critique Request for Qwanz.com - polling site with a twist
Now is the Time to cash in on your passion
Hi HN,

I'm working with the founder and dev team of Qwanz.com, and I'd love to hear your thoughts on the site. Qwanz is a polling site, with one key feature: once you've answered a poll, it gives you some next steps you can take -- including political / media / business figures who can actually use the information.

For example, someone who voted in this poll (http://www.qwanz.com/poll/245/) about Olympia Snowe could then forward the results to Snowe's press secretary (or call their office directly).

Qwanz is a startup with a thesis, and the thesis is simple: people fill out polls because they care about issues. If they really care, they'll welcome the chance to make a bigger difference.

Your feedback is appreciated! We're in the very early stages, so HN's comments will play an important role in what Qwanz does next.

Here are some Qwanz polls on what Qwanz should be doing:

http://www.qwanz.com/poll/248

http://www.qwanz.com/poll/247
Take this thought experiment to its extreme and the unlucky should have gone extinct. The fact that both lucky and unlucky personality types persist, suggest both have advantages.
A theoretical case is always made against the risk of deadlock, and it is a very real risk that I have seen lead to disastrous results at times.

With small-business type clients, where buy-sell agreements are the norm, you would normally want to have some special provisions for breaking deadlocks as well.

With startups (i.e., ventures that are intended to scale comparatively quickly over time), though, the norm is not to have such agreements and, indeed, those agreements will get in the way of normal funding, etc. and prove counter-productive in most cases.

In practice, then, an overwhelming number of startups do 50/50 type arrangements and it is very rare in my experience that a problem arises with this. I would be more cautious if it is an opportunistic teaming than if you know your co-founder fairly well and have a good sense about his character. This also assumes that the contributions are equal. 50/50 is a bad idea just to achieve a sense of comity if one founder is in fact more of a contributor than the other.

If you do have doubts of any kind, then you can usually achieve formal voting control even if the equity interests are otherwise equal by using a simple voting agreement to assure this. In such a case, there is a delicate balance between formal control mechanisms and the "spirit of the deal," which you shouldn't violate lest you do damage to the important relationship you have with someone as important as a co-founder.
The search engines are the parasites. They are making billions of $$$ using other people content.
I suspect it's not Apple so much as the carriers. Joe Telco isn't going to be happy supporting users who randomly brick their phones, install viruses, etc.
Maybe I'm a little slow, but I finally see where they were going with the Sketchup acquisition.
Three20 is really useful. It's also quite powerful. But I wouldn't recommend it to someone trying to learn/study idiomatic Cocoa.

Joe tends to do things his own way, which works out fine, but does tend to depart from most people's Cocoa/Touch code.
+1 on the meetup. We should pick the venue in the loop (or not too far from it) to make it easiest for everyone to attend. I do love the hopleaf, however...
Count me in either way.
Good idea but I wish the article also mentioned the printing technology and production. How did they solve these problems?
I'll preface this with the fact that I've not read those papers (but today was thinking through the concept of write hooks in information graphs), but I'd assume from the common formation of the pattern that it's only really set up for one-dimensional publishes.

The problem in large scale information networks, more in the Facebook way than the Twitter way, is that you you potentially trigger a cascading effect in information updates if you go pub-sub. Specifically, applications that do interesting things with social graphs have to go beyond basically doing message passing.

I naturally look at things from a recommendations angle, but if you've got a new edge that enters the graph that may affect other edges that are connected to the end points. Those may in turn affect the edges that are connected to those nodes and so on. You want to avoid something that effectively becomes a breadth first traversal of the graph doing updates since that's well, slow, to put it mildly.

This is why large-scale graph algorithms like PageRank work on constantly regenerating static matrices rather than doing regeneration of the ranks dynamically, but that naturally is problematic when you're working on data sets where the most recent data is the most important and is being generated at very high rates.
I was with a bunch of nurse and doctor friends and they launched into acronym-speak and jargon. Every profession has its own jargon.

I think non-geeks don't know or particularly care what technology/acronyms they rely on/are using as long they like the experience (e.g. teenagers with iPhones - do they really care it's Objective-C or openGL ES?)
I've noticed that there's a simple formula many bloggers use to come up with titles for posts.

1) Take famous other blog post title, book title, or meme

2) Negate

3) Apply DeMorgan's law
Having had both central vac and a Dyson luggable, I vastly prefer the Dyson. Carrying around the 30 feet of tubing for the central vac was hardly convenient at all.
I'm guessing you're going to get bit of traffic from people who meant to go to QwanTz.com
Just a thought: I don't see the implementation of the algorithm here, but if you're using the values of individual cards scaled to 1, then you are saying that v(A) - v(K) = v(8) - v(7) which seems absurd

Instantiating the features correctly is crucial to getting good neural network accuracy.

It might be the case that a thermometer representation for the card value is more appropriate.
The thermometer representation is that there are thirteen bit-values, and the first n values are on for the value n.

Using this sort of input representation, the network can model the non-linear relationship between different card values, instead of assuming a strictly linear correlation.
More of crazy Whitney code available at http://aplusdev.org
I've heard this book was good http://www.amazon.com/Beautiful-Code-Leading-Programmers-Pra....
It asks 30 different developers for pieces of beautiful code. I read an excerpt or two back when it was released and put it on my to read list. Unfortunately haven't gotten to it yet, but still plan too.
It seems easy, right? That's what I thought too.

I purchased 7M+ NL200 hand histories from a guy who mined them all day every day. I filtered that down to hands where someone was playing with 15 - 25BB stacks and then for every person that played more than 5,000 hands in that range, calculated their profits and winrates.

Here's the breakdown:
http://www.mattmazur.com/images/ShortstackingNL200.png

It's important to remember when looking at these numbers that the vast majority of people who try shortstacking never make it to 5,000 hands so this is heavily skewed toward the survivors. It would be like analyzing the revenue of startups that are over five years only and then using that average for the average revenue of all startups. There's only 80 people in the full ring group and 19 in the 6-max group. Just looking at these folks, only about 1 in 5 win more than 2bb/100 and in my experience that's not easy to do. The people that are profitable shortstackers would likely win a lot more playing with a normal stack.

I don't think there are many long term profitable shortstackers following rules. Again, I think the majority of the profits come from a few strategic plays that would be impossible for a spreadsheet, or a bot, to emulate.
Well, that would be nice. Not exactly targeted traffic, unless we start to do dinosaur-related polls.

(I didn't pick the name / URL. I would have gone with something like "pollsthatwork.org" or "pollsbutbetter.com".)
That's because to non-geeks, this and "techie" are the same.
Why not, worked for the image service .. I played a few times it was kinda fun in a waste-a-few-minutes way. Pretty good vocab tester, think of different ways to describe a scene, think of ways that your "opponent" is likely to use, think of the most complex way that is still likely enough (more points).
Facebook Headquarters by Studio O+A
I think the main thing to realize regarding the Fundamental Attribution Error is that when committing the error you are judging others and yourself on differing criteria.

For example, if you, when experiencing 'bad luck', tend to recognize its source as your environment rather than qualities of your person, it would be because of the Fundamental Attribution Error that you would incorrectly assume that the same 'bad luck' experienced by someone else is because of some qualities of that person rather than their environment. In many cases, especially with regard to 'good luck' it's just the opposite: For yourself you see the 'good-luck' as coming from YOU, for others you see the 'good-luck' as coming from their environment.

It's been helpful for me to remember these things especially when someone ticks me off or whatever... It's incorrect to assume that my neighbor being a douche is because deep down he is a douche-bag, rather than because of his crappy morning that I know nothing about. I would want to be given the same courtesy.
I'm curious how you deal with someone else running your company now. Do employees email you with horror stories of what's going on now? Do you try to keep an eye on the company from a distance or try to keep it out of your mind? Do you try to help the current owner?
> The fact is in spite of the heavy retweeting, although it gained the attention of people for a short time, its actual impact is not much more than #MUSICmonday or #Followfriday

Normally yes, but in this case it nudged other news outlets such as the BBC to report on the story and prompted a debate in Parliament on if these gag orders are too easy to abuse. I would say that's some positive outcome, even if it does little to help those who were affected by the toxic waste.
Sweden’s Entire Internet Goes Black for an Hour
It's a neat idea, but I doubt it would work well because the hand value in this context is so qualitative (whereas in HU LHE it would be more quantitative.)
but it's well worth asking how a small project becomes a big project with no noticing, or noticing but not taking appropriate action.

Simple, nobody pays you to notice how much of a problem the software that works perfectly has become. They pay you to improve feature x by a very small amount, or to add feature y.

So, sure Agile works fine, in the same world that most every other methodology works fine.

It's not the methodology that makes good software, it's the people behind it.
What makes Twitter hard is that it is a semi-real-time communications medium that is no longer particularly web-based - a good portion of Tweets come from phones.

But you're right with Digg, I don't see it having any excuses
It's not like my life is ALL cherries and no pits. But because of the personal challenges I have had, I simply couldn't make it if I didn't have some serious "saving graces" going on.
Worse, usually once this kind of companies get deals with record companies, they close their sites to non-US users. It's what happened with Pandora and Last.fm, also Hulu on the video scene.
Normally yes, but in this case it nudged other news outlets such as the BBC to report on the story

The BBC reported about the Trafigura case in depth on Newsnight* many weeks ago. Inevitably journalists would have had to mention it as it's something affecting their profession.

*One of the few good news programmes around in my opinion along with World Service news.
Completely out of mind. For me, it was total detachment on that day described. Like moving to a new home, your mind and time is best invested in your new place, not your old.

That was part of my condition of sale I told the 3 companies before we even talked: on the day of sale, I'm gone. No consulting or anything. I had to make a clean break.
When people start being afraid of one company, that's the best indicator that the said company really needs some serious competition in the market.
Okay, I think I should accept that I am probably outdated or misinformed about the things one can do with neural nets. Re, randomforests, I was referring to the Leo Breiman implementation on the univ berkeley website. Andy Liaw of Merck also maintains the R version of Leo Breiman's algorithm (http://cran.r-project.org/web/packages/randomForest/index.ht...).
I know neural nets have been successfully used for really compute intensive applications like face recognition or vehicle autopilot, but for applications like poker or predicting car sales or data mining used by walmart etc, there has been increasing use of simpler regression based / decision tree based approaches. There is an impression in the machine learning community that NN is like a black box which is probably the reason for its falling popularity if that is the case.
See the løveli lakes, then.
When trying to register at GWAP only a stacktrace of an exception was shown.
Tell HN: Atlassian now offers $10 licenses for most products. (for small teams)
I think that's the most twisted definition of "dogfooding" I've ever heard.

Also, this article seems to be about 98% speculation from a site that has no previous history of "inside sources" at Microsoft. Boo on that.
Isn't this typical behavior for an email client? Thunderbird and Outlook do this (auto-suggest). I don't know about other web mail clients, since GMail is the only one I use...
The TetriNET bot was a blast. You can download the full source code on the site as well.
If you normally mail Bob X and Alice, instead of Bob Y and Alice; however, you frequently mail Bob Y. How would it know if you always type Bob's name first?
How come Nordic cities like Oslo, Copenhagen and Helsinki is so well built already, but not Stockholm and Swedish cities?
What a massive inferiority complex! One person does not like Memphis.

I thought the very same after the first paragraph! But reading on and between the lines, it becomes clear that the letter is less a jibe against @keyinfluencer and his company, and more an open letter to the FedEx management responsible for hiring a fancy, presumably rather costly PR outfit in times of shared hardship. The crux of the letter does not concern the slight against the fine city of Memphis, but the oversight of ignoring in-house talent. Mr @keyinfluencer is merely collateral damage, and if Ketchum are a communications company worth their salt they will recognize this.
It works both ways... why do you think Google offers free 411 service?

(hint: voicemail transcription)
Oracle OpenWorld - Interview with Calvin Rowland
Sony Ericsson W660i Walkman Red Unlocked
Sony Ericsson W660i has the beautiful and slim look which attracts all its lovers from internal as well as external. music player and the FM radio entertain You all the way with different playing tracks with digital and stereo sound.
The Unix philosophy is the only one I have ever seen work. If you can't componentize it enough that you can pretend that the rest of the system doesn't exist\ is magic then you can't focus on a smaller human sized chunk to make that work properly. The second you have to break the black box complexity has just won.
Beat me to it. I wonder how long this has been in development in relation to that story breaking. Unfortunately, I don't believe it would matter in that case as the sender wasn't using Gmail.
Thanks =) I should have explained perhaps that the first comment on the page had noticed that the graphics that sould look like a finger pulling the lock looks like a.. well.. penis..
There are facebook pages with millions of fans. If you fan a page, you'll "subscribe" its updates too. It is similar to asymmetric following.

The internal representation is quite elegant.
Sir Tim Berners-Lee Looks Back: the "//" in Web Addresses Was Unnecessary
New Bladeless Fan - Introducing The AirMultiplier
The whole article is flawed. The article, minus the title, implies that he is arguing you shouldn't Release Early, Often but rather Release Ready.

The title, however, implies you should miss deadlines regardless of your readiness and you should never update. Wtf?!
I'd agree. The problem is five stars is easier to understand... but if you explain that you have an algorithm, users should understand.

My problem (and other's) with yelp is their discovery system is muddled in with their revenue model. That is you can't really trust the algorithm because they're also putting restaurants who paid at the top. So it no longer is a user revenue but an Adword. Not cool.
My startup has used twitter really effectively the past few months to do a few things. 1) Drive traffic to our blog, 2) drive people with a need for our service, to our service

Of course, we're a b2b targeted at engineers / sales guys, so we have a strong early adopter / seeker basis for our guys. But yeah, we've driven a worthwhile number of users to our site through judicious (and not even obsessive!) use of twitter. Even had to write a twitter bookmarklet to manage how we broadcast everything.
because 1:49PM on Oct 13th is before 11:59PM EDT on Oct 13
Idk, those numbers seem pretty reasonable to me and don't indicate any real level of difficulty. 58% of people in it are even or worse, which means if there is any sample bias at all, it's very small. The fact that losers tend to drop out over time is probably mitigated at that level by the fact that winners will quickly rise from low stakes to at least middle ones.

2-4 bb/100 is great even for a human since you can do huge volume that way. The spreadsheet method can easily be applied to 8 tables at once, maybe more. 2 bb/100 (thats $4 as calculate by PT right?) could be $30/hr. 4bb is obv $60, which is more in line with what people I knew were making.

Also when short stacking, opponents at higher level games don't necessarily respond any more appropriately. In fact a lot of times they play worse against it because they have more idea what to do vs. normal opponents. Low level players like to call big bets too much, which makes them accidentally better vs. a short stacker. My friends claimed they were getting higher bb/100 at the $3/$6 and $5/$10 games than the $1/$2 because of that.
Unsolicited suggestion: What it's like to get rich

Everyone talks about how to do it, but most people are very reluctant to talk about what happens if you do. I think it would be very motivating, especially since you've been honest in saying that you started Viaweb with that as the primary goal.

It's considered unseemly to even talk about it (or ask you to talk about it!) in public. Right up your alley.

Just a suggestion...
The additional storage cost of InnoDB can be quite significant if you have a lot of data. We have about 4 billion rows in MyISAM, we simply wouldn't have space to do that with InnoDB.

Hopefully compressed tables will improve matters.
Xmonad is nice code indeed. For an introduction to a piece of it, check out Simon Peyton Jones's talk at OSCON 2007 - he uses Xmonad code as an introduction to Haskell, which gives you a taste of how the program is constructed.
Programmers: Stop Whining, Be Productive
Google Wave’s Creepy Feature?
Yes but really really bad post title. Is nothing to do with techie talk, that was the first thing that bugged me.

I think the use of the term "geek" is reasonable: I am seeing that term apply more and more to anyone with a deep/obsessive interest in a subject.
No, that's the sad part. But when you have 10 people tromping around the same code in a hurry it's hard to keep things clean and consistent even with code reviews... and there's been such a push for new stuff that there hasn't been time for refactoring.
According to 3 out of 4 articles I've read about the thing today said it wasn't quiet at all. Cnet compared it to an Xbox 360 in full cooling mode and Engadget compared it to a (!)vacuum. Dyson vacs are pretty good and I think the Airblade hand dryer is amazing, but this seems like Sharper Image BS to me.