skysight-inc/HackerNews huggingface dataset:
This type of question is really too vague to answer without knowing what the data is and exactly what you want to do with it.

How much of the 20k of data items and 100k related objects do you actually have to scan/process over? If the 20k item is an opaque block like a photograph, for example, you probably aren't going to do much analysis of the block often -- you might pull the header or citation data and other related information down into 1k or less of useful data per item. On the other hand, if either the 20k or 100k are long form plain text, you might be able to distill it down to keywords to focus the data for efficiency.

When you say you want to do "multiple views on data, mostly in aggregate form", do you mean like OLAP cubes and other common reporting/analysis metrics, or more like searchy stuff where you might be interested in a needle in a haystack?

How do your users expect to interact with the data? If you were to sequential scan the entire uncompressed data set using Storagereview's current fastest disk, the Seagate 15k.5 300G, it would take over 21 disk DAYS ((min+max)/2 transfer rate), and this is contrived, and unlikely to be attainable in a real world scenario. It would take about 700 disks to hold it all, which would wall-time down to about 45 minutes. So naive walking the data for search purposes is out. It would have to be indexed.

Indexing is a tricky problem. In small databases they're usually so small and help so much that nobody pays any attention to the resources required to maintain them. With this dataset that's unlikely to be the case: you may effectively need several times the size of the data in indexing alone, which is going to have drastic implications on the computing resources required and the data storage format needed.

700 of those Seagate disks at roughly $200 street price is $140k. That's before simple redundancy, systems, cases, racks, power, logical redundancy for performance or extra space for derived data. Making a system to manipulate this much data is easily $500k, and paying a commercial vendor to provide you systems, hardware, and possibly software to do the job could be an order of magnitude more. For that kind of money I'd spend a few thousand hiring some people you trust to give you a reasonable assessment of the task at hand. It sounds like fun, though. Good luck. :)
The best sword-fighting game I've come across has been Defenders of the Crown. Very quick little fights, mess up three times and you die.

For an example of an original martial arts fighting game, Toribash is worth checking out and seems very interesting. My computer isn't powerful enough to play it though, so I can't give a review.
Kinetic Flyte 125cc Scooter Launched in India by Mahindra
I want to think that it helps to have pretty, descriptive URLs, but things like YouTube make it seem like it really doesn't matter what the URL string is – as long as users can copy and paste it and link to it, anything beyond that is a waste. On one hand, I guess it can be nice to see what the content of a page is before you follow the link (title slugs on blogs) and it would help you avoid getting RickRolled, but on the other, it could teach you to get used to trusting that the URL slug actually matches what's on the page. There could be anything there, really. It's up to whoever has control over it.

Anyway, I think YouTube's random string/hash URLs worked out fine for them. They're short, easy to copy and paste, and they make you think twice before clicking :]
Html5 canvas support matrix
agree with tom_rath (upmodded). you shouldn't give out equity if you don't know the value of what the other party is bringing to the table. And if sales do not cover their own cost you are losing money having him work for you.

Also, never forget that issuing equity takes time and money for handling the lawyer, accountant, etc. It also significantly reduces your flexibility down the road if you need to reincorporate, transfer assets, etc. And it is easy for things to get messy if there are problems. You want to keep your organization as flexible as possible.

If I were in your shoes I'd offer an excellent commission on sales and give your friend an option to purchase equity in the company up to a set amount at some reasonable valuation. Treat your friend's required investment the same way you would treat an investor at this stage. If he helps the company grow your agreement becomes a very good deal.
Might interest you: http://collabfinder.com/
How To De-Risk Your New Idea Using Google Adwords - Part 1 of 2
At the same time, his very first point deftly avoids the need for mocking up:

"The gameplay will work the same, but look a lot more reasonable."

And I really agree with him on that particular point.
Having a job doing research on campus gives excellent experience and looks great on a resume. It also helps when you need a professor to write a letter of recommendation if they know you outside of the classroom environment. Assuming you are going to a research university, there should be no shortage of opportunities. Your school website should have a list of professors along with what areas they are researching, pick something that sounds like fun and go an ask.
sometimes I'd just rather work it out for myself.
Donate Plasma or Sperm.
While novel, they have a hard enough time providing 3G to SF. Infrastructure here is immensely weak, if they fix this... then i'll be on board for most anything else they are so called doing.
Poll: What kind of cell phone do you have?
What kind of cell phones do hackers use?
Landline
I think the author needs to have a look at the latest Prince of Persia. The elaborate acrobatics plus his suggested system are what make the game interesting.

However, the other thing that needs to be remembered is that if you want to play real sword fighting, take up fencing. Better graphics and level of immersion than you'll find in any game.
I'm not an expert -- more of a hobbyist who is actively trying to improve my statistical chops -- but I don't think there's ever a general metric for this sort of thing. For any statistic it's important to demonstrate that it's being used as intended so that its lay translation means what it seems to mean.

I mean, r^2 isn't all that misleading if your relation really is linear. R^2 plus a graph of the regression would be sufficient. Just taking a look at the residuals of the Anscombe sets easily shows that linear fitting (thus r^2) is inappropriate for sets 2-4.

(btw: if you fit the second Anscombe data set with a 2nd order polynomial, r^2 = 1. Of course, it's pretty clear that there's hardly any error in that set.)
However, the other thing that needs to be remembered is that if you want to play real sword fighting, take up fencing. Better graphics and level of immersion than you'll find in any game.

I don't know. Fencing is way more effort, harder to master, and I'm not so sure I agree about "better graphics". I mean, they're higher definition, sure, but not nearly as awesome.
@brm: TransFS.com is my startup, and I seriously considered posting a response to this thread a couple of hours ago... but I didn't want to spam HN with self-promotion.

Indeed we are new, having launched in beta about 6 months ago. However, we have a number of aggressive processing bidders on our site... and I honestly believe that our marketplace offers some of the best pricing out there, especially for new merchants. We are always improving the product, and have plenty of work to do, but our customers have been very happy overall with the comparison shopping experience that we offer.

As my co-founder put it in an email to one of our potential customers today: "TransFS is the second startup for both the cofounders of our business (Sean and Josh) and we are radically passionate about helping business owners get a fair shake and reduce the way-too-high profits of the credit card processors."

Anyway, thanks for the mention!
Whoa...

I don't know much of anything about Canvas, but it looks way too involved for different implementations to ever have any hope of reliable compatibility.
HN should put it in the head so browsers will pick it up and show it in the location bar.
http://www.aei.org/issue/20303

This was easy to Google up. There are more recent figures.
Why work on administrative trivia for no money for the FSF when Google will pay you to actually write free software?
The constitutional system of school finance in the Netherlands is more than a century old.
I can tell you're British by the way you put together your BS.

It's distinctly different than American BS.
Yes. Instead of the "less enjoyable" experience of paging, we have the totally appletastic experience of disappearing applications, without warning.
I have a prepaid from T-Mobile. A hundred bucks gets you 1,000 minutes that expire after 1 year. While the iPhone and Android are awesome, I don't really talk on the phone very much (was averaging 76 minutes a month) and would rather avoid monthly fees. I guess I'm cheap. I do have an iPod touch, which gets me many of the benefits of having an iPhone, but without the monthly fee. Most of the places I hang out have wifi and I save content for offline reading w/ Instapaper for bus trips.
No idea... I'm hoping the upvotes are mostly because of the pseudoirony of me submitting this.
10 Things You Should Know About the Nation’s New Technology Czar
Aneesh Chopra was selected as Obama’s CTO, and since he’ll be in charge of developing new technology policies that will ultimately change the way we operate socially and economically, here are 10 things about him that you should know.
Check out GlusterFS: http://www.gluster.org/
He had some great comments on school:

http://learninfreedom.org/Nobel_hates_school.html
Portable 3G-to-WiFi Router Coming This Month
When you talk about that amount of data, you'll have to make some decisions about handling disk failures. At that scale they will happen as a matter of course. There are a number of different distributed databases out there and some of them have very different ways of handling consistency and availability.

I work on a database called dynomite which currently only supports a strict key/value interface to the data. It follows a dynamo model of eventual consistency. It trades the consistency the data stored in the case of failure and race conditions for greater availability. What that means is you might get more than one version of the data back for any particular query. Dynomite is primarily designed around serving live data to power a site. You can see it in use at powerset.com for the images in the republished articles.

From the sound of your requirements the strict key/value lookup might not work so well. If you'll need to occasionally crunch over the numbers something like HBase might be a better fit. HBase is optimized for scanning over the entire table for things like hadoop jobs. It's built overtop of the Hadoop File System (HDFS) which is a distributed file system built after Google's GFS paper. One of the drawbacks of HBase, however, is that it is not built for high availability.

Something to think about is perhaps a hybrid approach. It sounds like you need acquisition of data, do some amount of calculation over it, and then serve it to users. One way to do this, that search engines are very successful at, is to acquire data into a map-reduce ready storage system like HBase. Crunch over it using Hadoop. Then batch the data into a highly available datastore like Dynomite for live serving.

If you try to satisfy all of your requirements with one system, it will likely solve all of your problems poorly, if at all.
I can tell you're British by the way you put together your BS.

American BS is distinctly different.
Who are we?

I'm speaking to the US audience.

Do we?

Yes. I don't have rigorous data in front in me, but as a society we've become wealthier over, say, the last 60 years. In every community I've every been a part of, talked with others about, or reviewed data on, taxes for schools, as percentages of property values, income, or sales, have gone up. Greater wealth * higher tax percentages = more money/resources spent on schools.

And declining since when?

Though I'm skeptical of using test score data, the US results haven't been pretty from at least the 1970s onward. Anecdotally, however, this seems confirmed by consensus opinion. Education quality and loss of competitiveness of the median high school graduate relative to the rest of the world has been a continuing political issue for decades.

And how much?

There's no way for me or anyone to precisely quantify either part.

As for your skepticism, I simply assumed that readers knowledgeable about modern US history could appreciate the truth of my point based on the weight of their own observations.
C#, VB6 (maintenance), and PHP.
My big stumbling block is foreach loops:

foreach (int i in myInts) { ... }

For Each i As Integer In myInts ... Next

foreach ($myInts as $i) { ... }
What? No Windows Mobile? There are more WinMo phones out there than any other smart phone.
Or, for that matter, you can work on free software for no money for the FSF. I've been an FSF volunteer for years.
I think the original Prince of Persia was the most fun swordfighting game I have played.
Texas has fewer students than California, but spends the same absolute amount, so your argument doesn't apply.
but the size of the problem in total bytes may be more of a problem than the actual looking up and indexing of the data.

One cool property of such a large data-set (~200TB) is that you are pretty sure to see a lot of the same data repeated.

It'd be neat to try reduce the overall data footprint by assigning some sort of signature to repeats. I.e. if you are storing the sequences (let's imagine for a sec that these are non-trivial sizes) ABC, ABCD, ABCDE and ABCDEF , replace ABC with say #1 etc. and perhaps save a whole lot of space.
I wonder how much of this is attributable to better diagnosis and an aging population, rather than an increased incidence of dementia.
yet ironically sketching things on napkins is what game companies do. "Let's make another FPS like <name another successful game>." Or, "let's make a sequel." etc etc
The Nokia E71 is a dreamboat. I'm using it mainly for browsing, email, maps w/ gps, and ssh, but the call quality is amazing. Whereas in the past I would have to repeat myself at least once in every conversation, with the E71 I can rely on the fact that the person I'm talking to will hear what I'm saying.

Also, chicks dig it (when you're a programmer, you have to snatch those points wherever you can ;-).
I think a lot of it comes down to initiative. It's the difference between saying "man, somebody should do X" and just doing it.
Great. It takes a long time though to go from curing something in mice to curing it in humans. Diabetes is cured in mice.
Braintree seem like good people, I went to grad school with Brian and he is sharp and seems honest.

I am really curious about how you compared the CHASE paymentech deal with the Braintree deal. Were they interchange-plus rates? Or were they qualified / midqualified / nonqualified? Did you know how the categories compared to each other?

Disclaimer - I have a startup in the space (transfs) so this is market research for me. :)
I had to do a bit of research recently and I was surprised to find that dementia is mostly a misnomer. Dementia is a set of symptons, not an underlying cause. The most common causes of dementia is Alzheimer's. And as cruel as Alzheimer's may be, the other non-curable disorders are adamantly not better.
Do you have examples that follow this model and perform higher? I believe East Asian countries, and Finland at the least in Europe, are very rigid and have success. I'm curious to see the opposite approach too.
I got a job with the hospital on campus, doing custom win32 app development. Nothing huge, just business process facilitation with some windows-native glue apps.

Under market consulting rates, above typical college rates, I walked to the office and set my own hours. Everyone wins. It was hard to beat.
10 years of pain working in the trenches of the software industry.
'yeah, that'll work'

"That won't work", on the other hand, is easy. Here's why "realistic" fights with lethal weapons don't work for, e.g., a Star Wars game: George Lucas can fiat that Obi Wan never eats a lightsaber until it is dramatically appropriate, but in the game, for there to be challenge there has to be the illusion of risk, and if you're risking insta-death lightsaber cuts then a lot of kids and casual gamers (bread and butter for Star Wars) are going to get cut into pieces by mooks. That is going to be deeply unsatisfying to them. Nethack has a subculture who loves it dearly ("Hah, drunk the potion on level one without first testing to see whether it is a cursed potion? Man, you must be new at this. rolls Oh, you're stoned. Time to restart!"), but the "physics" for Star Wars are very, very different.

The reason D&D doesn't have the fighter die after a hit with a giant club isn't because we're capable of suspending disbelief as long as we don't see it happen. Its because, given that D&D is based around having most shots by competent opponents connect (there is another problem with the other way), having one-shot instasquish makes the heroes seem markedly less heroic, and would result in e.g. the defining D&D encounter with a dragon get skipped totally by rational players. (At least until they could find a suitably non-dramatic low-risk way to dispose of the dragon without ever being perceived by him, such as having the wizard Wish him out of existence.)

[On reflection, the "game fiats you do not die when you as the player fail to dodge an insta-kill attack" gets around this objection, although one wonders if you're sacrificing player agency for "making sense", since a player who literally abandons his controller will see the animation system play his character flawlessly for a minute until his stamina runs out. "Wait, what was I adding to this performance?" is not something I'd want to cause a player to think.]
First of all, what is with the recent deluge of poorly written trite?

secondly, this guy is failing to see that being a "leader" in the field is different from being at the top. Being at the top connotates that one is the best; their abilities are superior to others. A leader can simply be him who recognizes a vacuum in power and seizes it. Being a leader in no way implies that one is at the top of their field.

The best programmers/hackers I've met were quite content to hone their skills without the public attention that only the "leadership types" seem to require.
Well, my hat is off to the talented individual who, despite having lucrative opportunities elsewhere, recognizes the importance of what free software means to the world.
I found that the space of "appropriate" problems for C++ had shrunk to almost nothing.
As a fairly low-level programmer (data acquisition, signal/image processing), this isn't quite my experience. I've found that a mix of simple C-style code for the layers that directly interact with the material, with higher-level layers such as the GUI implemented in the usual C++, makes for a very decent mix, and certainly a better one than plain C.

I could also keep the systems stuff in C functions and use an FFI to call them, but I've never found a language that enabled me to write GUIs more easily than with plain old C++ and Qt. Java (or Java from Clojure) is slow and ugly as sin, and most other languages either use bindings to C/C++ libraries or have incomplete, mostly unsupported kits. C# and family is perhaps an exception, but not very portable.
Supervise a computer lab, that was the best gig ever, i made $12 / hour (8 years ago) and could do homework and write code while I was working. It was a lot of fun too.
Blackberry. I would probably have an iphone, but there's absolutely no AT&T signal in our (San Francisco) apartment.
I'm probably crazy for trying to breach this topic on a forum. It probably would take a very well thought out blog post to get my point across. But, this topic touches on a very important issue in software we face right now. Over the lifetime of the software industry we've experimented with lots of interesting business models. Some have panned out better than others. My suspicion is that selling software has been the most profitable. Preliminary evidence from iPhone app store has seemed to agree this as well.

Prior to 2000 IT industry was making large sums of money mainly from selling software. After the dot com bubble lots of companies selling software took a hit too. The bust wasn't limited to the web. Lots of customers realized they weren't getting the ROI from the software they purchased. Since then the industry fundamentally shifted.

The rise of open source software started to replace previously commercial software. In many respects were much better in quality than commercial software. Linux was making serious advances towards Microsoft's dominance in the server market. And, since then open source software has matured and has advanced rapidly into markets previously dominated by commercial software vendors. Once open source software options come into these markets they exert serious downward pressure on the ability for commercial companies to stay viable.

Take for example BEA. In early 2000's BEA was a market leader with it's Java application server. In fact BEA was the fastest growing software company ever, and I think that's still true. Over $1 Billion in sales in under 5 years. Amazing. Then JBoss showed up. In many respects JBoss' application server was superior to BEA's. Once JBoss came in they created serious pressure on BEA's ability to sell their server. Why purchase BEA when you can get a better server for free? In the end both companies were sold. Red hat bought JBoss for around $350-$400 million. Oracle snapped up BEA for $7 Billion.

Open source has proven itself as the best way to build software. However, it's the worst way to monetize it. Selling software works.
Maybe it's your nick, Kevin. :)
Update: the Mojolicious docs are pretty thin at the moment, but a series of blog posts (one is http://www.simplicate.ca/blog/2008/12/using-mojolicious-with...) will give me enough tips to get going.
More like lobbyist interference codified into law.

You got corporate lobbyists, you got religious lobbyists, you got idealistic lobbyists. You got lots of laws.

Could you start a text book company today? Grab the smartest grads from your local University to build a start up. Sell to the local district and grow from there?

No you can't. Too may barriers to entry. Same problem in many other industries. Intellectual property laws and copyright are mainly benefiting the existing players.

Why does everyone want to do internet startups? Because the barriers to entry are not fully locked down yet.
Blackberry Curve
Nokia 63XX - a few hundred bucks unlocked off TigerDirect. Best non-smartphone I've ever had.
I'm confused at the numbers then: 93 - 36 > 24 (current number of votes)
I have seen, over and over again, this idea of making an AdWords account go to a placeholder page to test out ideas that don't quite exist yet. It has routinely been suggested by people who I would otherwise consider quite intelligent and well-informed about marketing on the Internet.

Have these people used AdWords in the last 3 years?

Google is not neutral at all about what content you put up behind the click. If your page is content-free, their automated algorithms which calculate Quality Score, a key metric which determines how much you pay, are going to penalize you. If you use the landing page as a squeeze page (i.e. "sign up for our mailing list for when we release!", the most frequent suggestion) your QS will drop so low you will probably terminate your test quickly, after you start getting charged 10 to 100 times the going rate for clicks.

AdWords will also manually ban an account they catch doing this. It stands to reason for Google: they want people who click on ads to get what they expected, not hear "Psyche! That positioning was just a marketing test. Well, that will teach you to click on AdWords ads in the future, won't it?"