skysight-inc/HackerNews huggingface dataset:
RAM was not the issue. The reason for the slowness, as I understand it, is rather that different files are spread out in different areas of disk, even if they are in the same directory. This is considered a feature, and I guess it makes sense under normal access patterns. So accessing a million files (even to load them into memory for the first time) would require the same order of disk seeks, and takes forever. I might be simplifying a little bit, but this is my understanding.

Could I have re-written the code by messing around with inodes and other low-level details so that it accessed the files in physical order? Probably. Was it worth my time, rather than using an SSD? Hell no.

I agree that SSDs are still a tad expensive for the average Joe. For most hackers, considering that we spend most of our work hours in front of a computer, I feel that the added productivity from an SSD is easily worth the investment.
Yeah, that's what I figured. Be nice to see some numbers though.

Not like I have a hope in hell of getting anything I write even into 6MB .. sigh.
I presume this was developed for the Twilio contest?
Why didn't you try finding a co-founder, and giving up the code in exchange for a portion of the equity. Giving it completely away just seems wasteful.

10-20% would guarantee decent payout in case the site becomes a success. You pretty much did all the hard work already, might as well ensure that you get rewarded for it eventually.
What're you storing in files, and why is it in files versus in something more structured?
Look, it is 54 KB. That is not large. Perhaps with some effort it could have been slightly smaller but the author is probably optimising for a different metric.
Consider this aphorism every time you want to procrastinate: "Make decisions based on what will lead to happiness, not what avoids discomfort." Think it over...
I agree. Google is an advertising company -- first and foremost. Their main objective for online apps is to bring users off the desktop and onto the web. They stand to gain from any enhancements to browsers and web apps.
I think a better solution is to adjust the color slightly. You can easily get 3 or 4 bits of encoded data for every pixel in the image. Just take the least significant bit of the red, green and blue colors. You go from 16.7 million to 2 million colors, but that shouldn't be a problem. The human eye cannot see the difference anyway. Only in perfect gradients that kind of difference can maybe be noticed, so then the solution is to pick grainy pictures.

Photos (especially of poorly lit environments) should be perfect. They're very grainy by nature, so it will be especially difficult to trace where the data resides. Because you can hide so much data in the image, you can get very creative with where and how you do the encoding.
There's value in being able to tell the system "I do not want to do X. If I seem to be doing X, it's a mistake, so please stop me."

Frequently in the real world this sounds like "I am about to call out to code written at our valued partners in India. JVM, please make sure that the most they can do is touch the object I'm about to pass them. I can test the object for correctness when I get it back, I don't want to have to check to see if they redefined Array#sort as well.")

A lot of the things that irk skilled programmers about Java (I know what I want to do, why do you persist in stopping me?!) make a lot of sense when you're working with a geographically dispersed team of dozens, many with sharply disparate skill levels.
This is interesting, because every image host suddenly becomes a place to host torrent files.
Oops - replied to wrong thread :(
Absolutely, and Eric Schmidt has said this outright in recent interviews. Anytime more people are using the internet, whether it is their or products or not, Google wins.
Ultimately, I think Google Docs is an attempt at diversifying revenues. Eventually, there will be a push to charge more people for the service. Even though Google is a behemoth in advertising, it must be a bit worrisome to derive 95-99% of your revenue from just one source.
But how do you decide that a theory has turned out to be false? Once a certain percentage of qualified scientists agree? What percentage? And how do you define a qualified scientist?

Think of all the FUD around climate change research. Do YOU know exactly which studies are valid and which ones aren't? Do you just hire climatologists to tell you? WHICH ones do you hire?

How do you define "good enough in practical cases"?

I could keep going like this.

I'm not even worrying about commercial viability, I'll make the assertion that what you're proposing to build is impossible to build. My reasoning is that you're trying to fit things without a universally accepted definition - "truth" or "scientifically accepted" - into an absolute, binary system - "include" or "exclude". That is impossible to do correctly or completely.

At best, you either end up with something fairly arbitrary, or you end up with a system which only includes the "facts" accepted by everyone, and excludes anything controversial. This ranges somewhere between useless and downright harmful, in my opinion (if this system is accepted as "truth", then new ideas have that much more of an uphill battle to be accepted).
I managed to change a bad habit of mine--I've always had a problem getting up early. I changed this two years ago when I moved into a new house with large full-length windows in the bedroom. I discovered that when I kept the drapes open, I would consistently wake up around 6:30. With the extra time, I'm not rushed in the morning and even work out a few days per week.
Cool, thanks. The main motion works for me (FF 2, linux, on a eeePC 900MHz), except for black circles that appear and disappear near the center, in 3 locations (only one is visible at a time, and sometimes none are).

Here's results from your test that gives performance figures (http://animatedengines.com/svg/testp.xhtml), running with several tabs open:

fps 10 20 30 50
Average overrun -0.091 30.212 46.919 59.707

(I let it update a few times to settle down before pasting the figures, so they represent an average of the "average overrun").

Maybe it doesn't matter so much if the fps is slow for more complex engines? If the change between updates is small enough, it will look pretty smooth. Is it important to be able to make the machine rotate fast? I mean, it looks cool, but does the viewer gain greater understanding from this? (I'm not being rhetorical - I really don't know. Maybe it gives an overall feel for the engine's operation).

Sorry, I don't have any experience with JS animation to address the issues you mention. Some people here probably do though - or why not ask on http://stackoverflow.com ? It's often excellent for clear technical questions.
/Large ISPs routinely get subpoena requests all the time/

sure, but I don't think they are secret very often.
That Polaroid look
I really like getting RSS by email, because I can avoid using another product (an RSS reader) but still get the feeds I want without having to browse out to them everyday. I've found I waste less time browsing blogs ever since I switched to a once-a-day digest.

It'd be nice to set the time you get the digest, in addition to specifying how often you get it. I also found the URLs on the right side (once you're logged in) are cut a bit short, perhaps you could display the full URL on hover?
Thanks guys. Exactly what I wanted.
> Give me a second...

L1$: a second

Hard disk: more than five months
For the last two or three years perl 6 has soaked up all the concerted forward thinking energy, I suspect. Fortunately, perl 6 is on the home stretch now.
You're right. 53,966 bytes and not one larger. I wonder why it chokes up my browser.

But seriously, if you're going to use a GIF, at least make it animated.
Granted, the CSS wouldn't require zooming in, and the citation link would be usable.
Her point is pretty simple really: that Taibbi claims that Goldman is the evil wizard behind the current financial crisis and other's in the past, yet his claims don't back this up. He writes as if they were wholly responsible for the entire fiasco because they did some shady stuff, but he ignores that all sorts of entities did the same and no single person or entity is solely responsible.
http://en.wikipedia.org/wiki/Page_cache

the idea is that if you have enough ram, you only need to read the files from disk once. after that, the files are in ram cache. Once the files are in cache, at least for reads, it doesn't matter how spread out on disk they were.

And yeah, you do need to read the files from disk once, and that is slow; thus you often see a 'warm up' effect on servers. hitting a new page is often slower than it is for the second person who hits that same page.
I'm getting 403 Forbidden's on all requests at this point
looks like twitpic has shut out requests from pingwire's ip?
Nice. I'd been looking for some way to consolidate information gathering into one app (email), and tried feedmyinbox at one point (ended up giving up on it because, at the time, it could not do digests).

With regard to the UI, I'd appreciate a cue telling me when to expect the next digest. I see that it's on a 24-hour cycle, but is that on the stroke of midnight? Midnight in which time zone? Is that 24 hours since the last time I added a feed?
Our app is featured there, cool stuff.
> I do not view the property ownership as illegitimate. I do not agree with the libertarians who view taxation as theft. I think such a view is neither justified nor constructive.

Yes, there are some libertarians who naively, and incorrectly, believe they are the de facto owners of property in the US. That is, they would not even agree that USG actually owns all US property. However, I'm afraid I still don't see why you view USG's ownership as legitimate, unless you are equating de facto ownership with legitimate ownership -- i.e., you don't believe there is any difference between the two.

> USG's management structure is awful...

Agreed, of course.

> ...but it would also be in the interest of USG itself. Thus I view a management change as more likely to succeed and less likely to incite violence.

I'm not so sure about this. Yes, USG could be run more profitably, but severe agency problems exist. Would the leaders of the Catholic Church want their subjects to be told there is no God, even if these leaders were paid large sums of money? Power and influence are worth something to them as well.

> If a management structure is bad enough, an outside group could be morally justified ( though not legally justified) in trying to seize the territory itself. But that would be worst case.

I assume a group of subjects would also count as an "outside group"? If we are using common legal principles, then for the reasons I mentioned earlier, I suggest they would be legally justified as well, even if this might not matter all that much in the real world.
nice. heh. my experience has been that ram errors correlate most closely to how careful you have been with your ESD precautions.
A lot of people don't use linux, but they will in the future, just not on the desktop, cause nobody will use a desktop in the future?
demo account details:

username: demo@peekfeed.com
password: demo

...or make a free account and try it out!
Heh, ah memories, I can somewhat relate. I was a student in Japan, but spent a bunch of time hanging out with Taiwanese business men in southern China after that. The first block of time I spent there (relatively short) I couldn't speak mandarin at all, and it was pretty miserable experience. The second block of time I spent there, my mandarin had gotten pretty decent, and so I was able to flip everything into a learning experience...from learning words in Chinese like "static IP address" to how to politely refuse a karaoke prostitute my host habitually tried to force on me, those are some life and language experience I didn't get in the classroom =\
If there's one thing I know nothing about, it's football. But I think these themes are written by the same person

http://listen.grooveshark.com/#/song/Monday_Night_Football/1...

...I'm not convinced one way or the other though. I'm sticking the R&Rp2, since that's actually something drunken fans can yell out

HEY

duh duh duh duh DUH

HEY

duh duh duh DUH
Any thoughts on adding an API so that sites can use your service to offer email digests of their posts?
"She mentioned that she still wanted to go over and watch the band but she stayed put. I took a stab at failure and left her alone with the other guy at the bar"

The author did a good job and it worked out well! But in this scenario (where it is likely she's interested), be confident, stand up from your barstool, nod your head toward the band, and ask her to join you with a smile!

Otherwise you run the risk of letting her think that you aren't interested. And, confidence is kinda sexy ;-)
I'm not sure why this was voted down; here's the relevant text:

NOTE! This copyright does not cover user programs that use kernel services by normal system calls - this is merely considered normal use of the kernel, and does not fall under the heading of "derived work".

http://git.kernel.org/?p=linux/kernel/git/stable/linux-2.6.3...
Thanks for the suggestion! I'll add a timing option so users can choose whether they want digests to arrive at 8am, 1pm, etc...
Yes, see @dacort's tweet: http://twitter.com/dacort/statuses/2610766940
Thanks for the idea! I'll put an API on my to-do list. I'm thinking of allowing users to develop modules, so they can custom format whatever site they want. These modules would be available through the "Select Feed Type..." menu.
Ex-Googlers Working On Local Startup TownMe
Here's an effective way to fix Kindle books and make them better than paper
Thanks for the suggestion! I'll also include the title of the feed along with the url.
My point was while SSD seek time is 1/100th HDD seek time you don't get anywhere near that big a jump. Because, while a HDD might take 100x as long to get you the first bit, HDD and SSD take about the same amount of time to read the rest of 4kb the sector.
One-third of mobile phones to have an accelerometer by 2010
Found more info:

http://www.techcrunch.com/2009/07/06/back-to-basics-ditch-de...
interesting implications as accelerometers start to approach the same penetration rate as full keyboards
care to teach others the process from start to finish? A tutorial based on something similar would be oh so helpful.
Thanks for asking this. I've been feeling the same way lately and many of the suggestions here seem like great ideas. Time to set my homepage to dontbreakthechain.com and hit the gym!
Good heavens me! What an extraordinarily interesting article, a real feather ruffler if you ask me. I've always contended that there is no such thing as truly original thought, only complex amalgamations of stimuli - there are no "flashes in the pan," so to speak, only haphazardly arranged piles of nonsense which spontaneously combust, and produce such visionary ideas as the toaster oven or the baffling shade of green/yellow used to color tennis balls.
I'm not entirely sure whether I'm right, as I only skimmed the article, but something tells me I know more than these crackpot/loose cannon scientists.
It would include controversial topics where there is nonetheless scientific consensus that one side is correct.
Yes. I made the mistake of providing accounts for friends, and one of them had a common account name and an easily guessable password - it was bruteforced, the server rootkitted, and used as a 0-day ftp drop. I only noticed when bandwidth usage went through the roof.

I didn't trust the server at all after that, of course, so nuked it and reinstalled from scratch. Sans my idiot friend's account. Luckily it was a standalone and not part of the network or i would have had a lot more work to do; once something gets inside you just can't trust anything. Needless to say I would never have been giving away accounts on a big, fast, well connected work machine to friends though .. coughs .. anyway.

Was a good lesson about not trusting others to know what they're doing, taking steps to minimise bruteforcing (different port, limited tries) and rootkit detection, which is pretty hard. In fact it can be almost impossible to even know you've been rootkitted if you do not take proper precautions in advance. I only knew because they'd blown their cover with the bandwidth; that was a bit of a wakeup call. Nothing like having one of your boxes subverted before your eyes and you don't even notice .. that was a nice feeling, not.

Still, glad it happened. Getting hacked is like catching chickenpox; good to happen nice and early (but not too early), builds resistance for later on. Haven't been hacked since.
Great. I know my team would definitely be interested in leveraging this tool if it worked that way. I'm envisioning it looking something like AddThis or SocialTwist. You mouse-over an embedded item that gives you the opportunity to sign up for daily/weekly/monthly digests via email. Anyway, I've bookmarked this. Looking forward to seeing more.
Excel Infoways Limited IPO Information
When the population increases at alarming rates I'm wondering if there is any sustainable solution other than relocation places to areas with more water.

Import water in the form of crops. The amount required by humans to drink, bathe, etc is literally a drop in the ocean next to the irrigation required for agriculture. If you import a ton of wheat from Kansas rather than growing it locally, that is essentially equivalent to importing 1,300 cubic meters of water, in a form factor which is conveniently transportable and storable.

See: http://en.wikipedia.org/wiki/Virtual_water
I'd rather leave versions of Linux like Ubuntu and their ilk and keep the amateurs out of our way as we use the professional setups and leave us alone.
I just love Posterous, it's a great product and the pace at which they add new features is great.

One feature I just realized I would love to have is to "integrate" Dropbox Photo Uploads in a post in the same way I can add Photos to an email (they create a gallery for me). This would avoid having to upload the pictures twice which would be especially useful on the road where the Internet is often slow.

Also a very nice feature would be the ability to edit the galleries, especially the possibility to add and remove pictures.
In fact, they say it'll be out by Xmas...

http://www.oreillynet.com/onlamp/blog/2008/01/what_the_perl_...
Hey, that's neat. Maciej Ceglowski and Joshua Schachter (founder of del.icio.us) are good friends. I recall what must have been five years ago when they would hang out on Freenode arguing how to design various projects like LOAF (http://loaf.cantbedone.org/about.htm#authors) and I'm sure del.icio.us. It'd be amazing to see some of that energy again in practice. My best wishes to this project.
The articles thesis is ok, but the hook sucks. When was positive thinking ever about blind optimism? I thought it was about altering your perspective to realize your assumptions aren't clear and obvious. "I'm good enough, I'm smart enough, and doggonit people like me!" was never the point. The point was that your negative perspective is often relative.

You might think, "this cruise isn't as fun as I thought it would be." Which can either lead you to think "this sucks" or "maybe if I take this elevator below the passenger decks, hey a ping pong table... why are their Mexicans lined up along the wall... maybe I should go back up." (sorry, I have a better example from when I used to be a bit depressive, but I can't remember it.)

Also, I expect that what works has a lot to do with the personality type you match.

[addendum: It's as the guy who moved to speak to a new person every day for thirty days. I doubt he tells himself he sucks at talking to girls. More likely he recognizes the fear, chooses it is something he wants to do, and speaks. (Boy that can be hard.) Huh, I'm struggling to point out the article title and hooks betray none of the nuance.]
How about a filter that takes ordinary images, but turns them into low-light photos? Or just turns them into grainy photos? The grain can then be shifted around for purposes of the encoding.
But this format is so easy to decode, that existing search engines could crawl for such torrents very easily. In order for this to be really useful, there needs to be some steganography involved.
Somebody doesn't understand statistics. That aside ...

The ERV similarity as evidence of a common ancestor has mileage. The video doesn't give sufficient details to make the assertions that it does.

Are those ERV sequences predisposed to align to particular sequences and hence appear on particular genes? How many such sequences do we share with other primates? other mammals? molluscs? trees? What have these ERVs survived intact, what's the relative proportion of partials and do those match up too.

Even if the insertions are random, then are organisms more fit with the insertions at a particular point. Hence the virus could have affected multiple organisms randomly* but only those with the insertion at a particular location were fit enough to survive. This appears to contradict this being evidence of a 'recent' common ancestor.

The method that shows "evolution being true has a probability of 1" can also be used to show that the probability of extraterrestrial life is zero [abiogenisis observed = 1, number of stars O(10^23)].

--
* = analogy: kids draw on their bedroom walls in random places (perturbed by height; this is equivalent to some sites being favoured) their mothers wipe off the drawings where they see them (virus at some sites reduces fitness of organism). Observe all kids walls, drawings only appear behind the bed - hence kids all got together at some point and decided to only draw on the wall behind the bed (common ancestor).
It's hard to beat the home team. Microsoft (and Apple) created what we think of as a desktop computer. It's not enough for Linux to be "just as good" and not even enough for them to be "a little bit better" Linux would have to totally redefine what a desktop computer is and how it works. I don't think there's enough room for innovation in what we think of as a desktop computer experience for that to ever happen. As this article rightly states Linux is on the ground floor of the next wave and that's a good spot to be. Linux won't be visible however. How many TIVO owners know they're running Linux? 1%? less? It's a great product though because TIVO was able to focus on things like usability, UI design, predictive recording, etc. Linux is a fantastic foundation but someone else has to build the house.
Interesting.

One competitor, xFruits, has been around for a while and offers quite a few methods for transforming data from one form to another (RSS to email, mobile, voice, etc.)
http://www.xfruits.com/
This is pure speculative thought, but I feel the crux of the problem may be in the tests to determine the caloric load of foods/drinks - they literally just set it on fire, with the fundamental reason being, this is more or less how our body draws energy from what we consume. I could be wrong, but I feel as though my insides constitute something much more intricate and complicated than a furnace. As I see it, the problem lies in our methods of determining the respective energy potential/composition of a given food - the problem may be with the standard of measure, not a method of dietary restriction.
Just curious - I've seen Posterous mentioned a few other times on HN - for anyone using Posterous, what are you mostly using it for? As a true blog, with posts, or as a quick way to dump files to a public place?

Seems like a neat tool, and a very polished presentation, just curious how people are using it in the real world.
Professional in terms of Enterprise Finance/HR - like Windows XP/Vista/7? Professional in terms of Music/Design (OS X)? Professional in terms of Rock Solid Reliability and Stability (AIX?) Professional in terms of Security? (OpenBSD?)

I wonder which Profession choses Linux? Software Developers are all over the map. Professional Startups? Professional SysAdmins?