skysight-inc/HackerNews huggingface dataset:
I can't say I know how fast the web is growing. I do have some idea of how fast Yahoo! and Google's search indicies are growing, and they aren't that exponential curve.

It's never going to be the case that the PARC algorithm we use will be as fast as Google's algorithm. There is simply a lot more work to do, so they stay a fixed width apart.

But the way that the industry looks at computing resources has changed a lot since when Google is founded. They're becoming a commodity to the point where you can rent CPUs on a hourly basis. Also, there are new kinds of hardware that are starting to be tapped (for example, over the last few years graphics cards have become more and more a platform for general parallelized computation) that make the task seem feasible.
Three businessmen are sitting in the hallway of a court, waiting for their trial. After some time, one of them starts lamenting: "because my prices are way higher than my competitors, those bastards want to convict me for profiting from a monopoly!"

"Well, that's interesting" replies the second one, "because I'm also here for an antitrust case: I charge way less than my competitors and they say that I'm doing predatory pricing to kill the competition."

The third one starts laughing hysterically: "Believe it or not, guys, I'm here for the same reason as well! But my prices are exactly the same as my competitors, so they want to convict me of price collusion and profiting from a cartel!"
I am asking because I am honestly not sure, but why is this a big deal?
Hear hear! Dumb work preventing further piddling away in astrophysics classes.
of course not IMHO, but MS are desperate, and can't create things on their own any more.
Just like by not paying for a car you can't go home from the store driving it.
Read how I built Reddit River using Python at:

http://www.catonmat.net/blog/designing-redditriver-dot-com-w...

(source code included)
Just because "The People" democratically wants something doesn't mean it's right.
You're right, but that's not the point I was making.
A settlement doesn't mean either side won. You'd have to know how big it was before you could say that.
Probably because most people hate windows and consider it buggy and not very good?

Live seems more of a name for streaming video or something though rather than search.
Dragtable: drag-and-drop reorderable columns for an HTML table
i think my favorite is theo jansen's beach creatures (and a pretty relevant one to hackers, imo).

http://www.ted.com/index.php/talks/theo_jansen_creates_new_c...
Q&A with Insomniac Games chief Ted Price on cloning the golden goose
High School Should Be Voluntary
I agree. Let's get off this Luddite train.
Most phishers are not concerned with being sued.
They are still illegally "tying" by their use of the WHQL certification process.
I presume those are the Mandarin characters for "phisher heaven"?

Which makes me wonder what the Chinese equivalent of ".com" is. Hopeless American that I am, it hadn't yet occurred to me that the ICANN decision has very different implications if your native language is not English. Have the Chinese (or the Turks, or the Russians, or Language Group X) been anxiously awaiting this moment?
What would you guys/girls say are the important problems to be solved in the space Hacker News covers?
Stay on message.
It's a quick 2.35x return on investment which is not so it's more a question of risk. Take 2.34x now or risk 0 on 10x.
They will cost-reduce it some more and with the further cost reductions, plus royalties etc. will end up making money. Remember as well, they are manufacturing massive volumes of Blu-Ray readers, which saves them money when they sell standalone BR boxes to home theater folks.
Trademarks are not unique. They are unique within a given industry. And they may not be owned by the same party across different nations. This is already a problem with companies with the same name in different industries vying for the same URL.
"Crime and Punishment" (Fyodor Dostoyevsky) changed my life.
Want to Enhance Your Brain Power? Electrically stimulating the brain can speed learning.
Indeed, the last thing freed from Pandora's box was Hope.
AVG disguises fake traffic as IE6
"Eat food. Not too much. Mostly Plants."

Throw in regular exercise, and that's how you can maintain a good healthy lifestyle. Don't obsess over it; just learn to enjoy the healthy stuff. There's a lot of great, healthy food that tastes good. I'm not an expert on getting away from an unhealthy lifestyle, but once you're to the place you want to be, it's dead simple to maintain.
Upload progress bar with Passenger and Apache
Does the Invisible Hand Need a Helping Hand?
Good Resource: Joel On Software Forums
They are both right! Here's why:

Taleb describes two imaginary countries. The first is "Mediocristan" where predictive math models dominate and people and events are nicely characterized by Gaussian statistics. The second is "Extremistan" where events are difficult, if not impossible, to predict.

Here's the bad news: we think we're living in Mediocristan but we're really living in Extremistan.

Taleb says the models of Mediocristan do not predict "Black Swans" - the "unknown unknowns". Their surprising occurrence isn't captured by Mediocristan's predictive math models. So in Mediocristan the Black Swans are characterized as outliers. Should a Black Swan occur, Mediocristan's math modelers sweep it under the rug by either forgetting about it or by extending the math models to include it (for "next time"). Of course next time a _different_ Black Swan alights.

Note that there are good and bad Black Swans depending on your viewpoint.

Bueno predicts something more specific - the outcomes (e.g., wars) of political conflict. Bueno has a mathematical model of such conflict based on game theory. So in Taleb's parlance, Bueno's model is in Mediocristan - the land where mathematical models prevail and no Black Swans exist.

Bueno's model sometimes fails. And sometimes events occur whose parameters fall outside the domain of Bueno's model or any other model. Those would be more Black Swans.
Microsoft's first explicit anti GNU/linux ad
Maybe everyone already knows about this.. but this is a really good forum on building and selling software. Just discovered it today.
Knowing Taylor series can save your life
It's surprising how easy it is for me and for other people to forget that Google is not a monolithic entity. It's a collection of people, like any other big company. There's going to be good people and bad people in there. The press tends to refer to it as a single entity, however.
Industrial Robot controlled by two WiiMotes and two infrared diodes
I realized about a week ago that I'd been shorting myself on sleep, and drinking too much soda. I've cut way down on the soda and cut back on my late night activities and after just a week I feel much better.

It really does make a difference.
Object-Oriented Javascript and SproutCore (Podcast)
I can flag posts but not comments, and I'm at 118.

I do notice, however, that I see the flag link for your comment now that I've hit the reply button. I think this is a bug.
This is sweet - a visualization of the collective psyche.

It would also be interesting to see one with websites filtered out.
Perfect multi-column CSS liquid layouts - iPhone compatible
Sometimes I don't get it, is this 17-karma comment really more than 3 times as desirable than the 5-karma comment by tx?
I've used a skiplist before to implement the "Six Degrees of Kevin Bacon" program in C++. It was easier to implement than a balanced tree, and it had similar performance, so it was pretty interesting and worthwhile. It took about 30 seconds to read in over 100K actors (if I remember correctly), but after that, searches were very fast.

There are several negatives, however. One is that skiplists use about (lg n)/2 times as many pointers as a balanced tree, if you actually make your skiplist tall enough to be worthwhile. This could mean an extra MB or more of memory for a skiplist with 100K items in it. In some cases it could be that half of the memory consumed by the skiplist is the overhead involved in creating the skiplist. This is also true of hash tables that don't have many elements, of course, but with hash tables, the situation gets better as more items are added. With skiplists, the overhead remains high regardless of the number of items in the list.

Another "problem" is that of standard libraries. Even C has a tsearch function (in search.h) that implements balanced search trees, and many other languages include balanced search trees as well. Balanced search trees have less overhead, both in terms of memory and performance. With a skiplist, you have to choose a few random numbers (on average) for every item added to the tree. Then with searches, you always have to start at the top (the "skippiest" part) with comparisons, working your way down towards the bottom until you figure out which node to jump to. With BSTs, it's just a single comparison and then a movement left or right.

And of course, if your items are hashable, skiplists really lose out to hash tables, which have near-constant lookup time, very little overhead as the number of items in your table approaches the capacity, and implementations in nearly every language.
They still have two strong monopolies: Windows and Office. Neither are going anywhere anytime soon.
Apple to Offer iTunes Remote Control App for iPhone and iPod Touch
MC2: High-Performance Garbage Collection for Memory-Constrained Environments [pdf]
The fight for a cross domain XMLHttpRequest
Yeah. The great part of this story is that Google approached this developer and asked him nicely.
A constant time, yes, but that constant time may be significantly higher than expected. There are quite a few factors that can reduce hash tables efficiency.

This post summarizes several potential problems with hash tables: http://enfranchisedmind.com/blog/2008/02/25/problems-with-ha...
Down voted? Google is worth 2,000x what Powerset is being bought for, and that's money MSFT really wants. If it really could "kill Google", it should be worth more than 0.05% of Google.

If the goal is to kill Google, then the price indicates it's a risky bet. If the goal is good semantic search, then buying Powerset's tech and adding MSFT's financial resources to scale it sounds like a really good plan. Exaggeration doesn't help anyone.
I'm curious. I haven't been sold on the micro idea coming from twitter because I think it requires a fair amount of arrogance to think anyone will get something meaningful from 160 characters. But video provides a lot more expression in less time. I think I'd want to increase it to 30 seconds. Alas, they've chosen a name that restricts them in this regard...
But that's another argument of where skip list are not appropriate. Or do you have a real example?
Google enters the PC to TV arena
Any opinions on the best way to deal with this?
Summary: eval('statement', fn) lets you evaluate that statement in the environment context of fn. This includes access to the environment closed over by the function.
In terms of cache friendly and massively scalable, check out Cliff's work on non-blocking hashtables: http://blogs.azulsystems.com/cliff/2007/03/a_nonblocking_h.h...
I live in Atlanta and have essentially been sedentary since graduation a few years ago (I average 1 or 2 30 minute workouts a week). This isn't enough to keep the creep off.

I moved right into the middle of the city so that walking opportunities would be more prevalent and that's helped as a lifestyle change.

I had a hard time keeping up with going to the gym on my own and recently joined a sort of group exercise thing. There's a little more accountability with this as people expect me to be there in the morning. So far that's working well, and it has motivated me to start running in the afternoon so I don't suck so much in the class.
One pattern in Javascript is to use closures to make variables "private" in the sense that a limited set of functions can access them in a shared environment. This particular use of eval() lets a third party evaluate its own code in that same environment, thereby gaining access to those closed values.

Honestly I'm a little bit surprised by this myself. A lot of library writers have assumed that those closed variables are not visible to users of their libraries.

[Edit:] I'm not convinced that this is a negative thing. I'm a lot more worried about name collisions than perfect data hiding in a dynamic language. If someone wants to jump hoops to alter my private data, let them.
Amazon Marketplace is great as well. I remember taking 4 boxes of books to a used book store a few years back and being offered $17. I instead sell them on Amazon Marketplace used and get over $200.
Physicists Produce Quantum-Entangled Images
This is a good forum for micro ISVs in particular. Good place for general questions as well - it has a pretty experienced group of users supporting it.
Several of those (3,8,9) are among the first videos released. People (like me) that have stumbled onto TED recently and start from the beginning will view those first. Also, the iPhone, Photosynth, and WiiMote probably come up in searches because they have Google juice.
I remember using a skiplist few years ago and I implemented it so that its memory usage averaged to just two pointers and no ancillary information per node. This is as small as it gets when it comes to the binary search trees and alike.

In other words the memory usage is not really an issue with skiplists.
unfortunately for Sony, that's not the end of the format war. the bigger war is with downloads; both legitimate and illegal...

to be fair MS has also lost a considerable amount, plus this is the first time Sony has sold a console at a loss (Nintendo still hasn't done this yet).
Rogers Announces iPhone Plans in Canada
So we'll finally see apple.com vs. apple.recording-industry-sharks and both will be happy.
It is simply a matter of your cache hit ratio. If that is very low then you have two choices - increase the size of your cache, or use an algorithm that is less reliant on caching for predictable performance. The size of your cache is a pure price/performance calculation, "is it worth spending X to improve performance by Y%".

By all means stick to caching/LRU/B*tree if that works for your application. A cache is a brute-force solution, tho', and it helps to have more than one tool in your bag.
I think the better question is if the Java/corporate programmers (I am one by day) want to ride the Rails?
> eat food, not much, mostly plants.

See what you did there :-P
Here it is for the others: http://www.nytimes.com/2007/01/28/magazine/28nutritionism.t....
Great article, do yourself an article and read it.
Great work. This is a great advice for folks like us who are looking for some heads up. I know its hard, you have proven that with confidence and passion, nothing can stop you from acheiving what you want to
There aren't any. The User-Agent header has always lied, at least a little bit. What AVG wants from your site is the same thing that the users do: the content (in this case, so that they can scan it). If you insist on giving them a different page (like an error page) than a user would see, they're going to take steps to prevent you from doing that.

Remember that the desired outcome here is to see fewer AVG hits, not to "screen them out" just because they're not a human being (although they are a necessary condition for a human hit coming later). Try reducing the unique URL count on your site (basically: make it more RESTful and eliminate all the query strings on broadly identical content) so they have fewer URLs to crawl. Set your caching headers properly so that they don't have an excuse to recrawl you too often. That won't force them to hit you less, but it will "help them to help you" by reducing the total bandwidth. Remember they don't want to pull any more than they have to either.
Google Released API for Google Finance Portfolios
Open Documentation for Hardware: Why hardware documentation matters so much...
15 Career Mistakes
"I’ve made a lot of these career mistakes myself, and it’s hard for me to admit my failures. But in the interest of helping others avoid some of the mistakes I’ve made, I’ve decided to go ahead and create a list of the major career mistakes that I’ve made or that I’ve seen other people in IT make."
I think your size computation is wrong, no? A skip list is a doubly-linked list at each level, so two pointers per node at level zero (i.e. same as a tree). Each higher level has (statistically) half as many entries, so you have to multiply by the sum of the 1/2^N series, which is 2. So on average a skip list takes 4 pointers per node, which is twice as much as a tree. It's just a constant factor, there's no dependence on set size.

(It's true that balanced trees take a little more than two pointer per node, but e.g. red-black trees only need one extra bit, which you can pack into the bottom of one of the pointers if you like).

On the whole, skip lists are cute, but not really "better" than a red-black or AVL tree. And the balanced trees aren't that hard to implement, even if you decide to do your own. Add that to the fact that tree stuctures in code are kinda esoteric (almost always, you just want to be able to fetch an item by identity. Range comparisons are pretty rare in memory -- that's what databases are for) and skip lists leave me a little ... meh.
I think it's going to be ~15 years before video downloads take over 50% of the DVD/Blue Ray market in the US so they have plenty of time to milk Blue Ray.
Windows and Office are dominant but they are not monopolies. People have alternatives. I haven't used a Windows machine (at least at home) for years.
Thanx alot guys for all of the feed back. This project is a flash based Virtual World with some very unique bells and whistles, and because of those pluses I believe I need the cash to be in a fast first mover position with this. I have self funded up to this point, but definitely need the money for scaling and ongoing large scale development.

In addition there is a BIG Philanthropic angle to the revenue model, so VC's/Angels run when they here that.
I am in Kelowna, BC.
That is exactly it.
Researchers develop neural implant that learns with the brain
Google / Yahoo do this as well to check up on sites to make sure whats served to their bots is the same as whats served to real users. Google has 3 or 4 different fake user agent strings they use for the googlebot
Open Questions for The App Store
10 Audacious Ideas to Save the Planet
User-Agent information is inherently unreliable. I don't see why people are upset when a misguided practice that already provides false results with an unknown margin of error gets even worse. If they insist on using such a metric then it's their responsibility to assume the risks that it incurs.
Thanx Dani,
Appreciate the advice. Will start the condensing.
Shared-Dictionary Compression over HTTP (SDCH)‏
Funny, but the purpose of an investigation is to determine the truth. It probably sucks to get dragged into an investigation when you've done nothing wrong, unfortunately history is not on their side.

If they're looking for someone to blame, blame the monopolies, cartels, and predators that abused public trust so badly that laws had to get made.
Ice is on course to disappear entirely from the North Pole this year
Velocity 2008 Slides
Existing Drug Reverses a Form of Mental Retardation in Mice
One wonders if there's an "advantage" you could gain from this.. If your site is getting requested by AVG but NOT a valid browser, you can assume someone did a search that you appeared on but they did NOT click on you. This metric could be useful for improving your CTR in regular search results, something totally impossible to measure before.
s/Google/Microsoft/
Google doing it isn't nearly the same as 10 million users doing it. In fact, this type of activity should be limited to search engines, not client-side apps.
This is one of those where I try and put myself in his shoes.

If your business had the opportunity to take the contracts he mentioned, would you? Of course there are those silly things called laws that might get in the way of this business deal, but they may not have been clear at the time (I don't know).

All I know is that Gates leveraged his product against an unknown market and then created barriers of entry for other competitors. Machiavellian, perhaps, but brilliant.
Well, as far as I'm concerned the real, evil bastards are the ones attacking their competitors through the government using lawsuits instead of working to improve their products and services.
Most phishers probably aren't going to have $50-100k spare either.

I think there has been a great misunderstanding with this scheme due to how its been reported by the likes of BBC News - applicants will have to submit business plans and demonstrate a technical capacity to maintain DNS servers - I bet they will have some pretty strict requirements by the time it's finalised, and I'm sure they will heavily scrutinise all applications.

It's not like any old Joe can just set up their own TLD overnight.
The bad guys know how to bypass this check.

It took seconds to figure out a way that could cloak malware against this AVG check.

Malware is a problem.

This is not a solution.

This is another botnet.
Nintendo makes a profit on every Wii sold from day one
"Just like by not paying for a car you can't go home from the store driving it."

Not quite the same thing IMHO. By not signing the contract and doing what Microsoft wanted, since it was the de-facto standard at the time, you were pretty much kissing your PC sales business goodbye..
I know this is an old article, but I think it's interesting in contrast to the admission by Sony in their annual report that they've lost $3 billion selling PS3s.

The Nintendo business model makes a lot more sense to me.
I live in a fairly large city (Seattle), but it's not QUITE like NYC...

I walk a fair amount, mainly to and from the bus, and within downtown Seattle when I go to after-work gatherings and events.

I also spend a lot of time in the dojo, both as a teacher and as a student. I tend to take advantage of being a teacher to raise the level of intensity enough to give myself a workout, but I also carry that intensity to other styles where I'm a student. It makes for better training overall.

I also do a lot of hiking, nearly every weekend, usually in the mountains, and with a 50-pound pack full of, in addition to the usual supplies, my large-format camera kit. The camera gear accounts for around 30 pounds out of the 50.

Other less obvious things I do are eat a healthy breakfast (almost) every morning, do a lot of my own cooking, and drink herbal tea rather than a lot of caffeinated beverages. That is not to say that I avoid caffeine; I do like to have a cup of coffee in the mornings, but I try to avoid caffeine beyond that, mainly to avoid dehydration.
I don't know, but whether they do or not shouldn't matter. They should design it so it can scale, 100k bookmarks is not really that big of a dataset for anyone, let alone G.

I don't see why they can't just fix this instead of asking him to revoke his plugin, especially if people are using it. It's needlessly removing a use case which may gain traction.

Of course they may know something I don't and which isn't mentioned in the article which makes doing any of this infeasible or undesirable.
It's surprising to me that he's insightful when it comes to music. An area that M$ has struggled. I agree it's not surprising that he is insightful in general.
I'm probably not the typical customer. I don't use Linux on the PS3 often however. I liked the capability to do so as I was planning to use Linux for Firefox but the PS3 browser is decent it has flash support (very nice way to watch youtube videos with a group of friends).

I also have an XBOX 360 and a Wii but I've never used either for anything other than games.
#3 is bad advice. It's a good observation, but really you should try to identify your audience and be prepared to go in either direction. Some people will be turned off immediately if you presume to know how something will benefit them.