skysight-inc/HackerNews huggingface dataset:
> One of the strengths of having most of the vectors contain 0 values is that it makes computation much, much easier, even if the vector size is so large. I can quite quickly dismiss any vectors that have 0 dimensions in common.

I agree, and depending on the data, it might totally be worth sticking with this for simplicity. One has to think if it is good or not to dismiss these pairs which have zero dimensions in common. Say if one url has "people, tech, jokes", the other has "ppl, technology, fun", maybe they should not be missed. On the other hand, maybe because of the tag suggestions, as you said earlier, the tags might be quite homogeneous.

Your intuition about PCA is very correct. The way it works is that it splits your matrix into a product of two matrices. Say you have a large matrix with n rows (urls) and 60000 columns (tags). Let's call this X.
PCA will give you X = A*S (approximately)
Matrix A will have n rows and say 20 columns, matrix S will have 20 rows and 60000 columns (notice that there are much less entries in total, this is a compression of the data)

Matrix A will tell you how the original urls's are represented in the new features, matrix S will tell you how the new features are represented in the old features (a linear mixture of the old features a.k.a the tags). Depending on how you do this factorization into two matrices, you can have different methods but PCA is optimal in certain important ways.

Lot of nice intros if you search for "pca tutorial" and there are ready implementations in most languages. The basic ideas are simple but then the rabbithole goes deeper and deeper - just like this thread, so I'll stop here :)
He sold a vibrant, viable business to...yahoo. Enough said.
We're based in Reading, Berkshire, UK. On the one hand London is great, really great for certain things. It is up there with NYC. On the other hand it's easy to end up paying a massive premium. Reading and the surrounding area (known as Thames Silicon Valley) is a tech incubator. You've got Oxford Uni to the north, Reading Uni in town and Holloway to the East, all good for tech grads. There's cheap office space going all the way up the M4 and you're 20 minutes from Paddington.

Is it the best outside the US? Possibly not, Cambridge might be better, but you get all the advantages of London and Oxford without the failings.
True, this example of doctors and patients is just the wrong sort of example, because that world has already been standardized into oblivion. However, in general, for your relatively small app for company X, you wouldn't go this far. In general, the fact that a supplier can sometimes also be a client is usually not very interesting.
achieving anything practical using purely functional code remains extremely difficult ...

Excel is pretty much a functional language (though it is a little disabled), and people do amazing things in it.

The problem isn't that functional languages are hard. They are a bit different, but good code is often fairly functional anyway. The problem is that most of the community seems to be obsessed with showing that functional languages are both better, and more difficult, than mere procedural languages.

Take monads. The only way to "get" monads is to realize that they are just sections of the program that are imperative. But most Haskell programmers introduce them in the most incredibly obscure double-talk, just to avoid admitting that Haskell needs the ability to do imperative things in order to be useful.
The Transparent Toaster Corollary
Did anyone notice how many grammatical mistakes that article has?
I would argue that most people who use C++ in anger don't fully understand that language, and get by because they do what they know has worked before. The accumulated experience doesn't imply intelligence.

That's not to say that I think that it's intelligence that's lacking with respect to Haskell etc. I think the problem there is the mode and style of thinking, and that most programmers are too old to easily pick up the different way of approaching problems that it becomes second nature. That, and functional programming's chosen compositional breakdown of fixed common data with an open-ended set of applicable functions is not always as applicable as object orientation's breakdown of open-ended data with inheritance of fixed sets of functions; you often need to introduce an extra level of indirection (=> abstraction) with functional programming to get access to the dynamism that OO gives you in the box.
Perhaps because the data is interesting and we can draw the contrary conclusion: that there is no reason to suppose the data has been tampered with or fabricated?
Plug Computer Developers Camp
Startup 1 (HavenCo): I regret not being "incrementalist" enough -- should have had a plan for $200k, $500k, $1m, etc., not a plan for $5-10mm in spend. I'm still not sure if it would have ended up any differently, but it might have been technically more interesting along the way.

Startup 2 (Iraq): I regret not vetting employees, and on relying on dishonest/criminal people. This killed a $50-100mm business.

In both cases I regret not properly separating my personal and business fiances, such that I was broke (or ~$200k in debt) after each. I ended up having to do boring consulting for a couple years to stay above water, after each.

For Startup 3, I've set aside a year of personal money which I won't touch, and am only working with people I trust -- I pretty much picked this of 2-3 other opportunities based on the skills of people I know and trust. I'm going for outside investment, even though it isn't absolutely essential, to enforce greater financial/reporting/fiduciary discipline, and to get really smart people involved -- I don't think advise from someone who won't invest is worth anywhere near as much.
If I were involved in the high stakes game he is, I'd be a bit more emotional, a bit more eager to convey the fact that I'm a human being who makes decisions that have consequences outside the purvey of my own foresight.

But Lamo continuously seems like an emotionless machine. He gives speeches about damning a kid to life in prison and the moral dilemma of discretion when faced with illegal but possibly necessary civil disobedience with all the passion and articulation of a Hall of Presidents automaton.

It's hard for me to get the impression that his actions are anything but calculated.
But "not necessary" has never been the problem here. People are complaining because the idiomatic way to explore a directory structure in Haskell isn't familiar to them—it's functional and mathy and strange (because exploring a directory structure in a pure-functional way is mathy and strange.) They want the language to be "as easy to use" as, say, Python, for directory-diving—but that basically means that they want to do it the same way they'd do it in Python. The only part that's hard is figuring out how to do it "the Haskell way"—a.k.a., the unfamiliar, math-filled way.

What people need to learn is that you don't have to do that for everything. You can have all the advantages Haskell brings while not trying to make everything in sight into arrows and co-monoids; if it's easier to do it the non-idiomatic way (i.e. the way that doesn't involve turning your easily-expressible business domain into difficult-to-express Mathematics), then do that part that way, and get back to work.
If you enjoyed this film and interpretation, I really recommend watching The Prestige, then reading the following:

SPOILER ALERT

The Prestige has a much better "alternative ending" theory, because it is not well known and very unexpected.

The Prestige ends with all the questions supposedly answered, i.e. the twin thing, the machine that Tesla made, etc. Except that, there is one alternative: the machine never existed! Throughout the entire film, Michael Caine's character keeps insisting that there is only one way to do a transportation act, and that is with a lookalike. We already know that Hugh Jackman's character has a lookalike.

We also never see more than one "replica" of Hugh Jackman. The only time we actually see replicas being created is during the memories written down by Hugh Jackman, except, they could be fake!

In short, it's possible that Christopher Nolan has managed to fool pretty much every audience of The Prestige into believing in Science Fiction, when there is a perfectly logical explanation for the entire film that doesn't require magical explanations. But like the film repeatedly says, "Audiences want to be fooled".
Over seven nines of charged cases in Japan most years. 100 pct of all jury trials ever. Do we win? (I do not think it is a very useful metric. Japan has a problem, but it is forcing confessions as the sole investigatory tactic, not corruption. The conviction rates for any Western democracy should be ninety plus, or prosecutors are bringing cases they should know are insufficient, to the detriment of the public and innocent alike.)
Google Says Regulators Are Okay With Yahoo Japan Search Deal
You actually give a buried but excellent reason to see Bangalore as the perfect place for your start-up. Amongst the large number of 'talkers' there have to be a number of 'doers', and a newcomer might just be the catalyst required to unlock some of that potential by giving these people the chance they've been waiting for.

Sometimes attitude is more important than on the spot capability.

Tax breaks are not that relevant for a start-up, most of the first couple of years you'll be making relatively little profits.
Four Stages for Team Effectiveness
You are on the right track, but no, it is more subtle than that. The test could still be meaningful depending on what probability of false positives you are willing to accept.

[1] http://en.wikipedia.org/wiki/P-value

[2] http://en.wikipedia.org/wiki/Type_I_and_type_II_errors
The best way to stay out of this is by not talking to the media.

No comment is far more effective than any story.
True; and according to NS Savannah's reports, it actually would have been viable during the 1970s.

I wonder if there would be a business in either operating nuclear civilian ships, or outsourced reactor build and management. The irony being that some of the best prospects would be nuclear Very Large Crude Carriers.

The other reason the Navy doesn't want 100% nuclear (and kept ships like the Kitty Hawk around) was some countries are weird about nuclear warships specifically transiting their waters; some are anti civilian nuclear too, but are generally more anti-nuclear warship.
America's Worst Job Market: What 27.6% Unemployment Looks Like
Really cool post, Greg - thanks. That's a neat application of Twilio.

I'm almost more intrigued by this write-up as a study of advanced planning for multiple triaged failure modes and remote correction. It can be difficult to break out of the "ship code, find a bug, ship a patch" cycle long enough to realize the power of a few well-placed remote management hooks that can keep an app on course with minimal intervention.

(Also, as one who's been to my share of swim meets, this sounds invaluable for such a large competition. I'm sure folks appreciated it!)
I immediately thought of Know-Nothing Bozo the Non-Wonder Dog from THHGTTG.

http://tinyurl.com/33n6wy6 (Wikipedia page, URL too long and HN truncates it so it doesn't work)
I don't think one should speculate about what the outcome would be if you had done something differently. Instead, it's best to determine what you are going to do next time which is different from the last time.
Neeleshs' reply is what I meant. In Postgres, it's like a namespace.

But you have a point: adding customer_id is not unreasonable. In fact, this is my preferred logical approach, but I'll typically only do this on Oracle. It's DRY, meaning that, in the abstract definition of schema, you're modeling your relations once. If you have widgets belonging to customers, you have a one widgets relation containing a customer id, rather than one widgets relation in each customer schema. It's an elegant design, and it takes even greater advantage of the benefits I described earlier.

I'd advocate against it when building a multitenant SaaS application on Postgres if the following hold:
1. You have many, many customers
2. Each customer has a lot of data

The reasons:

1. You'll quickly hit a place where it's hard to throw more vertical processing power at it. You'll need a serious SAN with lots of spindles to be able to service many request to a lot of data, and plenty of CPU and RAM to handle the grunt work. Let's face it: you're on Postgres because you're broke :) I really love it, but sometimes I'd rather buy a solution to a problem (e.g., VPD). However, if you're rolling out schemas, you can build a new database and move them discretely. This isn't automatic or transparent, but it's very clean and very easy.

2. You're likely dealing with clients with special needs. It might not be appropriate to roll out new features to every customer all at the same time. If you ALTER TABLE widgets... it's all or nothing. Sometimes that's great, but sometimes it's not, and in building multitenant SaaS apps, I find it's most often not. In repeating your design, you're buying the flexibility to deal with your customers on a more granular level.

3. Others have already mentioned this: sometimes it's easier to articulate the integrity of your system to technical auditors with a hard separation like independent schemas. It's a little easier to show that a particular database account has access only to a particular schema than it is to convince him that your code does a sufficient job of enforcing the rules. I've done a lot of work where regulatory compliance is a concern, and this is always an easier sell.

The thing is that most vendors have solutions to these problems, and if Postgres doesn't have already built, you can probably build it from their primitives. This is just my general thinking under this particular set of circumstances.
Purchase software online
The best online store to purchase software online has been launched by treuemax, login to buy software in dealer price.
Generally, I refuse to put my email down for an invite. But rule.fm looks BEAUTIFUL, I couldn't stop myself and submitted my email address. The YouTube video is very well done as well.

On their tour page, http://rule.fm/tour/, I like how selected section on the right changes as you scroll through the page.
Adrian Lamo is a violent psychopath. His ex girlfriend got a restraining order on him because he had attacked her with a stun gun.
For well over a year I've been looking for that 'better hammer', but to tell you the truth I haven't found one yet.

I've built some non-trivial application in a number of environments just to see what the grass is like on the other side of the street. Each and every solution I tried has its own warts and requires you to work around a new set of issues.

PHP is a 'bad' programming language, it smells of band-aids all over. But for all the elegance and brevity of some of the other contenders there isn't a single one that is better on all fronts. So far. Still looking, still trying new stuff. If I find something that makes me really happy I'll feel so much better because the P*/Javascript/CSS/HTML/SQL hodge-podge that we call web applications feels like the absolute wrong way to develop software to me.

We're doing something terribly wrong.
Bryan Cantrill is joining Joyent
Can you elaborate on viable?
Is there many bookmark websites that are making the cut nowadays?
Spotify 'will launch in US by end of 2010'
> Except it's not "just another job" at all--it's an impossible job, they have to assemble the best team, they make an elaborate plan that goes all wrong and they have to clean it up

Exepct that's the plot of every single "just another job" movie. But of course inception is a lot more than that.
Why Digital Advertising In India Sucketh So.
Most code just doesn't need to be guaranteed correct, plain and simple. If more code did, then Haskell would be more popular.
> Things like that will create a systematic overpricing of contracts.

Only if you're dishonest.
My Lai Massacre
Typeclasses don't solve the same problems as objects do. Typeclasses are a way of getting access to more operations in polymorphic code, which in functional languages is a compile time feature; they're also a way to implement overloading.

Objects are a way of implementing protocols without needing to know the details of the object that implements the protocol, even at runtime. Polymorphic code in OO systems is a runtime feature, not a compile time one; objects may be loaded dynamically and almost always are at some level in any large OO system, for configurability and testing if nothing else.
this is awesome. how'd you stumble onto it?
I like the irony of the fact that that site is down so I can't read the article titled "Redundant is never redundant enough".
A decade to separate us: Ars reviews StarCraft 2
I love how the scoreboard was in a 10pt font. SMS message on a HTC evo is probably larger.
There have been plenty of (non-aweful) movies with more unprobably plots than that, that weren't in a dream of any kind.

I think that stating that the entire plot is a dream just because the main plot is not realistic is akin to denying the film's right to being science fiction!
Similar complaints here: http://gitguru.com/2009/03/18/git-enterprise-requirements/

While I personally use and like git, it's just too low level for general use. Too easy to mess up the repository.

Git is a very good basis for version control. It just needs a better front end for slightly less sophisticated users.
A list of major changes can be found here http://kernelnewbies.org/Linux_2_6_35
The Living Robot
What does that have to do with the price of tea in China ?
Working as a permi this is certainly the case, but if you can get your living costs down and work on a more ad hoc basis (i.e. contracting or consulting) then you can take big blocks of time off to use as you see fit.

Personally I wouldn't want to work in the US right now. The worker protections are practically non-existent (advantageous if you own a company but if you work for one, not so much).

Health care has always been problematic in the US but I suspect by your comment you mean that you don't like that you have to have it now. Not having insurance would be an insane level of risk to put on your family (especially when coupled with the risk of a start up). One of the bigger mistakes the US did with health care IMO was letting companies pay it for the employees. That creates a terrible lock in incentive and removes health care from the person's budget so when you do wish to try something else (e.g. a start up) you're not prepared for the sudden jump in expenses.
I hear you, but the way he expresses what he does has the net effect of making excuses for PHP and saying "Hey, it's okay that it's an entirely inconsistent language without fundamental features of other real languages because it's just a tool to solve a problem I had at one point, and I hack in new features as the problem-of-the-minute requires them". That's not a well-designed tool. That's a hammer glued to a screwdriver duct taped to a torque wrench welded onto a lawnmower. You can probably properly tension a timing belt with such a monster, but just because you can doesn't mean that it's the right (or most productive) tool.

I'm very productive in PHP; that doesn't change the fact that I discovered I could be more productive in other languages that didn't actively frustrate my development progress due to being well-designed.
The Minimalist - 101 Fast Recipes for Grilling
If it takes 'pitching' TC to get an article there then I predict that sooner or later there will be party that goes out of their way to look for the news that will eclipse TC.

Sure, PR is part and parcel of running a tech company. But active journalism is the staple of a good media outlet and to rely on being pitched almost guarantees that you're going to miss out on a lot of important stuff.
It's a good way to put how I've approached many contracts and jobs in the past.
I wonder, could the following be true: any startup that faces the problem of unscalable implementation will by then have the necessary resources to resolve the scaling problem? Because, by the time you run into scalability problems, you'll probably have enough investor|customer money to invest in re-architecturing your code?
Ask HN: Paypal Analytics?
We sell software online using Paypal. Paypal's reporting is limited and extremely slow.

To do Sales Analysis, I export the data to CSV and use Excel Pivot Tables.

Is there a better way?
I agree, but sense of security is also important.
Has POHMELFS reached production quality yet ?

I heard a lot about it when it was first included but not much since.
iHollywoodOS
What boosted you back up? Maybe you just needed a time-out?
Shouldn't this be posted on Cracker News instead?
I use them. They've been consistently not evil for years.
The Ipad is a consumption device, unless you've found a professional niche to employ it you probably could do without it. But for me the magic is when I'm in some foreign country sitting in a hole in the wall cafetaria and my mom calls me and doesn't even realize I'm not even on the same continent.

There is plenty of magic in all this new stuff, you just have to think for a second what life was like in the 80's (2 MHz computers and bigger stereos) to appreciate the benefits of all this stuff.
i like OS but the desktop UX is where linux was years ago
That's interesting stuff that I didn't have a clue about, thank you.

How do you know Bush was aware of Goldbergs work ?
I thought it was a rather pointless puzzle. I'm lost as to the profound meaning you're alluding to; is it doubting reality, that it may be a dream, etc.? That's a long way short of original, and besides, it was done much better recently by The Matrix. The plot, however, was clearly less than awesome, with much of it seemingly existing to justify special effects and repetitive fight scenes, providing cheap tension. (The same could of course be said of The Matrix.)

The whole concept of Inception, of course, is unremarkable. Ad-men have been putting ideas in people's heads for some time with quite a degree of sophistication, and magicians have been doing similar for far longer; done well, marks are fooled by both without realizing how or when.

Reading the movie as an allegory of movie-making I think is a stretch. There are matching elements, in particular, not changing the world too much or provoking antagonism; but the whole plot theme of father / child guilt is rather out of place, as is the recursive breaking into of fortresses.
I think the main issues are that lazy evaluation makes it more difficult to reason about the performance of your code, both in terms of what actually gets computed in which thread / on which core, and in terms of space requirements (lazy evaluation can force the runtime to keep things referenced longer than you might expect offhand). I'm not saying a good Haskell programmer can't overcome these obstacles, just that this is definitely a perceived barrier to commercial Haskell development.

Check out this paper for a discussion of these issues in a real world Haskell application:

http://www.starling-software.com/misc/icfp-2009-cjs.pdf
>This mythical "programmer that can't program" myth

This is absolutely not a myth. I've seen it first hand in an interview. Having said that, it appears that you've been dealing with "interviewers who can't interview" or companies with really stupid hiring practices. These silly tests they do are designed to tease out what you've freely offered. They should just have their engineers look at your repos, verify which commits are yours and how good the code is.

But honestly, imagine working with a place that has such stupid rigidity. I seriously doubt this inflexibility would exist in HR in isolation. I think they did you a favor (accidentally).
Check out http://wiki.github.com/facebook/hiphop-php/
It's really sad how many useless jobs exist in the US because someone thinks that these people would simply starve to death if they weren't able to e.g. hold a stop/slow sign at road construction sites.