skysight-inc/HackerNews huggingface dataset:
> You're right that they are doing this at scale.

Not in my experience. My product uses >5PB data and we don't bother to do significant deduping. I've only heard of a few products at Google that do, and that's only when you can anticipate that most objects in the system are going to be duplicated many times, which is decidedly not the case in heterogeneous storage like S3's.
What if you were to symlink your entire tree to say /z - as far as I'm aware, this would not use much more disk space, and as it would be the last to go, could you not call /z/bin/whatever ?

I may be way wrong here, but I thought the symlink would still work with the original being deleted, because the file system won't actually mark the file as gone until all links are gone.

Would (or does) this work? Haven't got access to a unix box right now, so can't try myself, but thought it might be a naive failsafe (at least until you actually wait for rm -rf * to finish...)
Brand New Louis Vuitton Mens/Womens Belt - Fasttwo Outlet
I see no reason why these two are mutually exclusive. If I boycott a company due to disliking their business practices, can I not also be making a statement?

In fact, I support Google in this because it is making a statement, even if that's not the primary intention.
Absolutely correct. Drilling it down to the logical nth degree you are effectively paying to be part of a defined marketing demographic that can then be sold to advertisers. The reality of course is there is a more symbiotic relationship going on as if people did not percieve value then the circulation would drop off.

Glossy magazines like Wallpaper are the most stark example of this where it's often a case of flicking through 4 pages of watches before getting to the next article about italian leather day bags, but news media run on the same model.
Google has 1/3rd of the Chinese market, which it got after Baidu was entrenched. That is pretty much what MS and Yahoo have collectively (IIRC) in the rest of the world, but nobody says they don't factor. Google is very much aware of them, as Baidu is of Google in China.
Dr Grigory Perelman turns down $1million prize 
Dr Grigory Perelman turns down $1million prize
Does anyone here have any experience with sulbutiamine?
30% of a market the size of China doesn't seem like loosing. Companies survive off of much less, 1% of the market is often plenty enough.
Gamer has resilient brain, survives stab through the skull (graphic)
Kurt Vonnegut: I want to share with you something I’ve learned.
More fun: libc is hosed on a remote box and you can't find any working statically linked binaries with network support, but still have a few live shells accessible.

I was starting to write a UUDECODE function in pure bash (I'd saved my ass before by pasting uuencoded binaries into terminal emulators) — but then I remembered that bash has built-in tcp support!

At least when called from the exec builtin to spawn a file descriptor, you can open /dev/tcp/$host/$port psuedo-devices. It's existence would have saved me countless more times, if the spoilsports at Debian/Ubuntu hadn't disabled it at compile-time.
World's cleverest man turns down $1million prize 
World's cleverest man turns down $1million prize after solving one of mathematics' greatest puzzles
Blue Guest House website - Garden Route
They don't think they're a software company, and they're addicted to the margin on that $600 dongle to run their tools. Otherwise you'd be able to rent a VM running MacOS, like Linux and Windows.
Probably not smart to tell the government you'd like to hide your comms from them. Although if everyone did (e.g. it was built into the browser) then it might be hard to figure out who to watch.
Friendster, the first online social network

Factually inaccurate? Friendster was launched in 2002, LiveJournal, for example, in 1999.
Sounds like a good idea, I would go to this as well.
My guess is that negotiations have produced this result: the Chinese government is going to (at some point) filter the results themselves, which lets both parties save face and lets Google continue basic business functions in China.
What about students studying outside of the US? Most universities in South Africa use the .ac.za TLD...
I agree. If you genuinely feel that your product is worth the $80/month and that your customer will feel the same way, why not send them a reminder a few days before the trial stops to let them know? Maybe give them a freebie if they continue the membership.

I used to have a subscription to a UK magazine where apparently automatic subscriptions are not legal, so they have to sell you the magazine again, every year. I thought this was great, as opposed to the situation in the Netherlands where you have to cancel at the right time, or else you are stuck with another year. (There are laws in progress to change this, so clearly many people are annoyed by it)
I immediately close any site that does this - don't care what they are offering/selling - if they behave this way, I guarantee their wares aren't worth bothering with.

It's almost like looking at code, I generally only have
to look at the formatting. You don't see a lot of
lame code that's impeccably formatted (nor a lot of
useful websites that do that in-your-face
type of advertising).
Symlinks won't save you but you could use hardlinks to make two pointers to the same file. You would have to do it for each file — was it cp -l which would do that.
From the Wikipedia page on USB:

Additional USB hubs may be included in the tiers, allowing branching into a tree structure with up to five tier levels.

So by "infinite" they mean "five", unless the connectors can make use of a non-standard communications mechanism between each other which makes them appear as only one big hub.

The other downside of daisy-chaining is that you have to unplug everything to remove a lower-level device.
That's called a link (a hard link). It's just another reference to the inode on disk, indistinguishable from the original. All regular files are hard links, most just have a link count of 1.

Symlinks are symbolic links (soft links), potentially-relative references to fungible user-facing filesystem paths, that work across filesystems and respect the permissions of the original. They're a special kind of file.

It would work with hard links, but you'd have to make and maintain a whole separate real directory structure with an individual link to every file. People used to do this to efficiently maintain branches of the Linux kernel in the days before bitkeeper, depending on their editor to replace files by rename+copy instead of rewriting in-place.

Hard links to directories were removed in an early unix for being hostile to reality and fsck, but were added again in Mac OS 10.5's HFS+ implementation to implement Time Machine's storage.
This is true, but he gets props in my book for being smart enough to switch away. It seems surprisingly common that people don't do this.
regulation on the Internet is a sovereign issue

This is the key line - Google was foolish to press the sovereignty issue with one of the wealthiest, most powerful countries in the world. Now the precedent has been firmly set on who runs Bartertown, by China - my guess is that Google's stance will look increasingly stupid in the months to come as other countries follow suit.
It's slightly worrying that applying a fairly standard set of patterns should be "crazy" or "heretical." This is one of the things I worry about in Rails - people seem content to shove everything in an ActiveRecord subclass and hope for the best.

Still, at least doing the right thing is not only possible, but generally easy once you've broken your mind out of the prescriptions of the framework.
How to visualize A/B test results? (Flot-based) new charts
Yesterday: http://news.ycombinator.com/item?id=1209748

(submitted link differs by a trailing slash)
Which Intel CPUs Support XP Mode Virtualization?
> Debt can be a good thing

yes, it can be or can not be - it's an uncertainty they try to avoid.

Buffet quote:

“Leverage, is the only way a smart guy can go broke … You do smart things, you eventually get very rich. If you do smart things and use leverage and you do one wrong thing along the way, it could wipe you out, because anything times zero is zero. But it’s reinforcing when the people around you are doing it successfully, you’re doing it successfully, and it’s a lot like Cinderella at the ball. The guys look better all the time, the music sounds better, it’s more and more fun, you think, ‘Why the hell should I leave at a quarter to 12? I’ll leave at two minutes to 12.’ But the trouble is, there are no clocks on the wall. And everybody thinks they’re going to leave at two minutes to 12.”
Experience vs Wisdom
They do know, I have a friend who works at a seeding agency, and they know very, very well how it works.

Make it funny/stupid/sexy enough, and people will watch it.

A couple of good examples of this:

http://www.youtube.com/watch?v=6xfBNxNds0Q
http://www.youtube.com/watch?v=owGykVbfgUE
http://www.youtube.com/watch?v=USYa5iH3f3s
http://www.youtube.com/watch?v=JJmqCKtJnxM

Don't know if the latter was launched as a viral campaign, but my point is that you know it and the brand it was from if I'd phone you up and shout "WASSUP".

Marketing is a lot scarier that you think.
Reminds me of my first job straight out of University (1988) - there was a client who had a sysadmin who had a bad habit of logging in as root and running rm *. This didn't cause any immediate problems - but all hell broke loose when tried to reboot their system.

They had about 25 users on a 386-25 - quite a decent system in its day.
You take the system that you know and assume that this is the only way to do it.

I have just explained how it can also be done and is done and run perfectly well.

Make of it as you want.
Wouldn't it be easier to use a PUT with a Transfer-Encoding of "text/x-diff" (or some such) and an If-Match header?
Maybe he should start asking people for their permission.
فيديو نحتاج ان يراه مليون شخص فى العالم الاوربى - شركة الصباحي
The important thing you need to do if your planning to take these drugs is do the initial work.

You cant (at least not from my observations) just start taking these tomorrow and see improvements; you'll be mentally and organisationally unprepared and the effects will likely be negative. You should:

- get really organised; get into a routine and stick to it. This is key because if you drop out of the routine things can spiral out of control

- learn how to recover fast. At uni I learned quickly how to get home at 4am and be up/bright at 7am ready for work - and last the whole day (I never took drugs but coffee + OJ were consumed in large quantities).

- focus your mind with mental exercises and so forth. This will help you concentrate generally and will help the drugs do their job.

I advise you try all of that first and get a routine/pattern sorted - often that might give you the desired affect anyway (worked for me :)).
China condemns decision by Google to lift censorship
I sincerely hope that they're using the JIT from .NET for this, although I've not seen .NET so much as mentioned anywhere near it. They already have a high-quality compiler that can emit insanely fast dynamic code, why not use it? Although honestly, I'd take it one step further: start rewriting everything possible in .NET, and improve the compiler in parallel (although they're obviously already doing this -- they're making good strides).

If you limit the portions that are in unmanaged code to the absolute most performance critical components, you could honestly tout it as the most secure browser. Not to mention the benefits of their generational GC, or the development time improvements. If they could cut their development and debugging time in half, they could potentially build a browser that actually doesn't suck (which, sadly, I don't believe exists from anyone right now).
Create Your Own Cloud With Linux
Create Your Own Cloud With Linux
Take on Google, at least in a small statistically insignificant way: create your own cloud computer using Linux and Eucalyptus
Common Ovarian Cyst Symptoms
Completely inaccurate title. The article itself states: "The Clay Institute did not say whether the Russian mathematician agreed to accept his prize."

It does say "Russian news agencies say that Perelman showed absolutely no interest in his latest prize - $1 million – from the Clay Institute.", but that's not the same as him turning it down.
Chinese government is the most vocal voice when it comes to the independence of Internet from America to UN. But it has unbelievable censorship on its own Netzens.

Communism wants the world to only understand its ideologies and get criticized for not being communists. Its like "heads" I win and tails you "loose".
Follow the ghost of @kurt_vonnegut on twitter...
I voted "no" because I have the choice of making it easier if I wanted to. For me, life is artificially difficult, not inherently difficult.
Today is post-your-favorite-hn-numbers day
Today marks the 1111th day I've been on Hacker News. I haven't been a prolific poster but I owe a lot of insights to articles and comments posted on HN. I've been wanting to make an 'All-Time-Best' list of HN stories but never came to it.

Now, for no other reason than that 1111 has such a nice binary ring to it I'd like to encourage all HN users to dig deep & share the stories that they think make HN 'Hacker News'. I've been going through the first 100 HN stories sequentially and these are the ones that brought back memories for me:

X = 6 14 59 75 83 90 92 ... in ... http://news.ycombinator.com/item?id=X

A list of the Top 111 HN Stories of all time on my 1111th day on HN would make a perfect birthday gift ;-) ... So post your favorite HN numbers Today!

P.S. : I know about http://news.ycombinator.com/best ... but that's only the recent best stories.
Teach, if you like it.
So it goes.
deduping definitely not going to work on proprietary encrypted data

but for web services like tumblr, flickr, youtube or hulu, deduping must be a killer app - there's a huge amount of overlapping imaging data between them and it makes little sense not to analyze it. I'm sure google is deduping youtube stuff on a massive scale.

So it depends on the data and application of course.

Then again, purely deduping based on byte comparison is not going to cut it - you need to look into logical deduping (not storing same image in different resolutions, formats etc)
confirming. at this moment its still unclear if he will take money. my bet - he just dont care about any money and awards he is getting.

disclaimer - i am not a friend of him but graduated at same university and saw him few times there. he is typical "out of this universe math geek" type of person
He should just take it and give it to his mom.
What people paid for was for the businesses to pay for their own costs and make enough of a profit to keep going. What people got in return was content.

However, why they paid was because their was no good, free alternative. When people are offered a product that very closely replicates their current experience, but at a largely reduced price point, they switch.

Case in point 1 - the good replacement: digital downloading of music. People are willing to download from iTunes because the cost is less than that of a CD, and they don't listen to music on CD players anymore. As such, it reduces cost to them, both in terms of financial cost and in terms of time expenditure (don't have to go to the store, and don't have to rip it to their computer).

The problem here is that you need something that people are still willing to pay for. Which means something they think they'll use more than once. That's why people still buy books and periodicals. Because they're building librarys, not reading something for one-time consumption.

Case in point 2: Linux. Linux does not yet closely enough replicate the experience of Windows for people to be able to switch. "Where's the exe file for me to click to install something?" "Why isn't their a download button on Apple for iTunes on Linux?" "What does compile from source mean?" "Why can't it install the driver automatically?"

The problem here is that it's very hard to make Windows again without getting your ass sued off, and it's even harder to do it for free. Also the myth that anyone anywhere can fix a problem found in the codebase is a myth. Most people can't (lack of knowhow), and there's no guarentee that their fix wouldn't break something for other users, or that it'd be safe and not introduce security issues.

Case in point 3: real estate. The housing market collapsed because people were given access to morgages they couldn't repay, by people who couldn't afford to take the hit if they didn't. Of course if you offer people more money, for the same or lower levels of repayment they're going to take it. In this case, it was perceived immediate value, against long term reward that brought the problem, but the underlying mental process is much the same.

The problem is, how do you differentiate yourself in a market, where you can only offer a sensible product that matches the needs of your consumer, when every competitor can match you exactly? That's where market research and common sense need to come in, to find better ways of meeting your consumers needs, and balancing what the consumer desires against what you can offer as a responsible business.

Takeaways:

* Consumers pay for the value they perceive their getting

* Businesses set the prices based on the maximum ROI they can get based on their marketing efficacy

* Everyone trys to find ways to cut costs whilst maintaining the ROI, to attract more customers or increase ROI whilst maintaining price points
My friend was paying EE for their service and moderately happy with it until I showed him how they display the answers publicly after the scroll barrier of google originated requests (and not others). His response was something along the lines of "why those little..." He promptly cancelled his subscription with them.

The fact is that the only reason they rank highly at all is by cheating google's indexer and using a cheap psychological trick. Their business model is naiveté. Once you point it out to their customer base, they will quit the service.
The hundred push ups training program
Sometimes I'm getting pretty tired of apps/people who keep forgetting about non-us people, the internet is global, why put restrictions on it? Both my Dutch and Danish eduational mail addresses won't work obviously..but seems we can mail the owner to add or schools, at least that's something.
Ok, I need to remove a device in the middle of the chain...

Now what will I be disconnecting to make this happen?

I hope it's not my external HDD 2 hours into a transfer with 30 minutes left to go.
When I was at Apple, I wrote a lot of software to make it easier for Apple employees to read code that other people in the company had written.

This idea of improving the reading/understanding experience with software is something I've tinkered with, without much success, so your story intrigues me. If you can say & don't mind, what sort of approach does/did that software take?
Virgin Galactic’s VSS Enterprise First Flight
It's the old Socratean thing: "I know that I don't know".
Have you used the internet from within China?

Baidu cannot compete internationally on the strength of their technology. The quality of their results is sub-par, and subject to gaming if you have enough money to pay them.
Their market share is as big as it is because of the fact that Baidu is essentially the Chinese Pirate Bay.

Feats somewhat more difficult to reproduce when competing on a level playing field where promoting, aiding and abetting piracy is somewhat frowned upon.

Who would want to compete with the deck stacked against you like that? Sure, China's market may be big enough to mask such deficiencies, blinded as everyone is by that 1 billion+ number, but its a net loss, long term, losing access to a quality competitor.
Well-placed Pixels - a visual record of beautiful software
Just to be more specific because this seems to be a point of confusion:

HTML5 video tag was always supposed to allow any codec to be used.

It was also, from the first time it was proposed by Opera, intended to use Theora as a baseline interoperability codec which everyone could implement for free so that if you wanted to guarantee someone could see it on any HTML5 compliant platform you could use that.

Just to underline that point. No-one, at any point said "Theora is the best codec in the world lets force everyone to use that", instead they said "What are the best, freely implementable codecs" and the only good answer was Theora (and maybe Dirac at some point in the future).

This was all written up in the spec.

Apple and Nokia said "we don't care what you write in the spec, we're not going to implement Theora".

The spec author didn't want to have anything in the spec that didn't reflect reality, and didn't think having such things in the spec would help change that reality, so he took Theora out and said no baseline default codec would be added until there was a royalty free option that the main players could agree on.

Again, just to underline this twice, no-one proposed H.264 (or any other royalty bearing codec) for inclusion in the spec because W3C specs do not allow royalties.

Even Apple thinks this is a good thing:

"After careful consideration of the draft patent policy, Apple believes that it is essential to continued interoperability and development of the Web that fundamental W3C standards be available on a royalty-free basis. In line with the W3C’s mission to “lead the Web to its full potential,” Apple supports a W3C patent policy with an immutable commitment to royalty-free licensing for fundamental Web standards. Apple offers this statement in support of its position."

http://www.apple.com/about/w3c/

Yet thanks to Apple (and others) we have a W3C standard that specifies no codecs and a de facto web standard codec with royalties as a result.

(And before anyone brings up IMG, name 5 royalty free image formats, then name 5 royalty free video codecs)
Distracted by shiny things [software design]
The only guys I know like this are holed up in some pure research job at universities, working on problems that interest them so much they barely leave their offices to shower. I'm talking top 50 living programmer material though.
I don't really know how safe Russia is as a country but would it be wise for a teacher there to just accept $1 million without taking some precautions ?

Personally I'd say the first precaution would be to move with the family in the US, presumably to another large institution.

It's strange how reclusive the guy became (was he always like this?). I wonder if all this attention is getting him to lose interest in math.

On the other hand, without being in the spotlight, will the contributions really be acknowledged as fast as they should ?

I mean -- the guy had to solve one of the Millennium Problems to be noticed. What other "minor" stuff did he do that goes unnoticed because of his lack of "fame" or because he's not from an institution with a great pedigree ?
My Mum is a primary school teacher here in the UK; it's incredibly depressing because, as you say, this is the best way to teach kids... and here in the UK they have basically dropped the learn-see-do and reinforcement techniques. Everything is "approved" worksheets.....
i get the general story, but why does he say that hamlet is like cinderella ("I don’t have to draw a new line, because Hamlet’s situation is the same as Cinderella’s, except that the sexes are reversed.") and then later say hamlet is a flat line?
I'm not so sure about that. SO users are quick to answer simple/easy problems but those are never the ones I need help with. Ask a question that requires a lot of expertise and some effort to answer, and it's nothing but crickets chirping, whereas if I'm lucky enough that someone already had my problem on EX, it is usually answered fully, even if it took 3 people multiple tries and a lot of effort to get there.

Not to say that I'd ever pay for EX, but it obviously has its uses...