skysight-inc/HackerNews huggingface dataset:
As one of the developers/founders of Openomy (http://www.openomy.com), I'm keenly interested in this.

Our goal is to provide a place where people can store and access all their online files. We want to be the Online File System. We're heading to the "cloud" already and it's becoming increasingly difficult for us to manage our own files. My photos are locked in Flickr, my docs in Zoho, etc.

We want to break down the "data silos" and provide a single interface to all your "cloud storage" as well as provide a single API for developers to code against to access user's data in meaningful and interesting ways.

We're taking a multi-pronged approach to this. One prong (currently launched) is to allow you to upload your files through our API or our web interface and we'll store it (using Amazon's S3 for reliability and scalability). You can access the files and their metadata through the UI or through the API. You can tag them, share them, make them public, etc.

Our next step is to provide many backing-stores. That is, import your files from Flickr/Zoho, etc. We'll keep things sync'd up, so that Openomy becomes your Online File System; storing the metadata of everything you do online. Then, the UI becomes the equivalent of Windows Explorer or Finder, and the API becomes the equivalent of the single file system API provided to you as a desktop developer.

We think it's going to be, at the very least, extremely fun to build. :)
Pig: high level language to process big datasets via Hadoop
Christopher Olston from Yahoo Research presented Pig at the UCB DBMS lunch talk today. It's a framework that translates a high level processing specification into Hadoop jobs. Users only worry about providing functions for specific data parsing and computation. Pig and Hadoop do all the rest.

Here is a sample program (taken from slide 11) that joins two big datasets (Visits and Pages) to find sessions that end with the "best" page:

Visits = load '/data/visits' as (user, url, time);
Visits = foreach Visits generate user, Canonicalize(url), time;

Pages = load '/data/pages' as (url, pagerank);

VP = join Visits by url, Pages by url;
UserVisits = group VP by user;
Sessions = foreach UserVisits generate FindSessions(*);
HappyEndings = filter Sessions by BestIsLast(*);

store HappyEndings into '/data/happy_endings';

Pig is open-source. It's being "incubated" as an Apache project. More details here:

Apache Page: http://incubator.apache.org/pig

Subversion Repository: http://svn.apache.org/repos/asf/incubator/pig

Powerpoint slides: http://www.cs.cmu.edu/~olston/pig.ppt
"... I want to know specifically why the author thought Python was better than Lisp ..."

Just ask ~ http://news.ycombinator.com/submitted?id=lisper
Very cool... I'll have to forward this on to my grandfather.

A few years back I lived with him for a year in Brighton, UK and we'd spend hours discussing technology and engineering over tea or a pint. What I enjoyed the most most about our talks was learning about his life, who he was and discovering that he helped lead the teams that laid the fiber optics between the US and the UK. (im going to have to ask him about the vulnerabilities)

Previous to that he had spent quite a bit of time in Canada, vancouver i believe, working with Canadian engineers developing methods to push the effectiveness of fiber optics past 3 miles. As a parting gift he gave me a fiber optic stub from the factory he was an engineer at - A tear shaped piece of glass from which the tiny fibers are pulled.

This map just sparked so many fantastic memories of my time with him! The great talks as well as being humiliated at the pub as an 83 year old man drank me under the table!

Thanks for posting this.
Dude, it's nothing you can't learn in a few weeks, or maybe even one week. Get GIMP, read a tutorial, fool around with it for a while, and try online tutorials on how to make a web 2.0 button, say. That might take one, two days tops.

CSS isn't rocket science either. It's a variant of html (sort of) where you choose stuff like font and background color (style, I guess) in one master document that all the other web pages refer to. It's like the Slide Master in powerpoint. It has slightly different punctuation than html, but it's not that special. It's just an easier system, and it's probably overhyped by design bloggers.

Visual design: there's all kinds of "7 mistakes begginers make" type articles in hacker news. Some are better than others. Just use common sense, good taste, effort, and knowledge.

That should get you acquainted with all three problem areas. In fact, after a few weeks one can design clones of homepages of Yahoo, Google, YC, delicious, facebook, and so on. Of course, I don't know what level of mastery you're talking about, and there's definitely a lot of fancy things you can do with photoshop that take a lot of knowledge, but the kind of design chops one needs to do a startup aren't that hard to get.
I hope Pig ( http://news.ycombinator.com/item?id=108224 ) escapes too. It looks very promising.
A better ext4
Good question regarding our audience. We don't really intend to make Seekler a place you would visit every day (as you noticed, we have quite a few gaps in our content, so for any niche, you'll probably only find a few lists that interest you right now) - instead, we're trying to make it resource, more like Wikipedia, but for opinions. The idea is that if you're looking for something in particular niche, you'll start your search on Seekler. Our goal is save you time - instead of spending 20-30 minutes finding your next movie, book, whatever, we can make that 5 minutes or so.

Of course, until we get a lot more data, people won't really start their search at Seekler. But we're starting to show up pretty high on Google results for a number of our lists, so we are trying to get our initial traffic that way.

Your point about the widget and FB is interesting. We have a widget that is being tested and we're strongly considering building a FB app to play up the social angle. For instance, right now the 'Best Movies' list merges the lists of all users - would it be interesting to see a list created from just the lists of your FB friends?

Thanks for the questions and feedback. We're learning something new every day...
Yeah, they are definitely similar. The main difference I see is that all the lists look individualize - that is, you can't merge lists across all users to get a comprehensive lists that includes all items.

Thanks for the heads up.
your formating fucking blows.
http://redsultan.com

Amazon Mechanical Turk competitor in the works (it also happens to be my senior thesis).
This guy is wearing too much white - makes it hard for me to take him seriously.
Launching major site next week. You can hold me to that. Will be awesome site that you'll all want to use. Not sure if I'll do it under this alias.
I've used Quicken and Quick Books. With both, I've found myself simultaneously limited by the inflexibility and shortcomings of the applications, and overwhelmed at the complexity and obtuseness. Personally, I'd love to find a small business accounting system that's not aimed at the MBA//Accountant market, and is flexible enough for different types of companies.
Lisp expressions are different from what most people are taught in early math classes.

As a result more people find y=m * x+b easier than (let ((y (+ (* m x) b))) y) until they get the chance to spend some quality time with a REPL.
lisp can do side effects. as i demonstrated. what's the problem?
Don't know of any in AU, but as far as kiva and prosper, that's not really the YC / seed-funding model, that's the Mohammed Yunnis Micro-loan model.

Lending money to somebody and watching their progress is a lot different then what Y C or any seed-fund does.
"... Numerous people have written diff and merge systems for wikis; TWiki even uses RCS. If they used git instead, the repository would be tiny, and you could make a personal copy of the entire wiki to take on the plane with you, then sync your changes back when you're done. ..."

Now that is a compelling idea. How portable is git? ~ en.wikipedia.org/wiki/Git_(software)
Some of the design choices [pg's] made are just bad, too.

Well, name one, and give a reason.
"... some people prefer not to commingle the functional, lambda-calculus part of a language with the parts that do side effects. It seems they believe in the separation of Church and state." --Guy Steele
Comrade, Jeff Smith is the author of Bone.
MapReduce: Simplified Data Processing on Large Clusters
I get that you're becoming a warehouse of categorical "opinion-based" data, but the trick seems like it's going to be how that data can be accessed once it's stored. Lists of things my friends like, seems it'd be more useful than getting a list of what random strangers like.

I think there's a lot more you can do in that direction, and I encourage you to look into it :]
I was not trading BEA when Oracle made the bid to buy them out. Why? What happened to BEA's stock price during these two offers? Did it not move at all? Did it go beyond the offer price? Somewhere in between?
I was just being a jerk. All i know of this person is he made some comment about REST being a good idea 5 years ago.

Perhaps he is, in fact, a great luminary in language design circles. Pretty much though, this seems like some comment on some mailing list by some guy... not much different than some guy at some bar saying something about sports.

Sorry for the confusion. Nothing to see here. Move along. Move along.
It's also the next Unix in that it's kind of hard to use for newbies, and shits all over your data when you do something that seems like a routine action (git revert anyone?), but in fact will tell it to self-destruct. svn is emphatically not this way, which, when you aren't doing anything distributedly and don't have very many branches, is happy.
AJAX powered travel booking process - can page flow be patented? (Musings on travel ecommerce)
This sort of thing certainly helps to explain why Lisp is threatened with extinction in the wild.

Should I become a new member of the Lisp community, where my every contribution will be greeted with insults, derision, and snobbery? Or should I just learn PHP, Java, Python, Ruby, Perl, or C? Suddenly, the decision that most people make seems pretty darned rational.

One of the biggest advantages that Arc has over the myriad other Lisps is that PG actually wants new users.
We've just launched our social news site for women, with a focus on fashion, beauty, and celebrity gossip. That means lots of pretty photos!

Initial users seem to like it so far :)

http://www.prettysocial.net/
My development and design are being outsourced. While that is taking place I am learning HTML and CSS so that I can understand certain things and tweak design.

I'm reading the Head First book on html and CSS. I didn't know anything before I started. It isn't hard to pick up.
http://www.oreilly.com/catalog/hfhtmlcss/index.html
Git is good at what it was designed to do (manage and distribute changes to source code) but that's not what most people need. Non-programmers are a lot better off with simpler things like OSX Time Machine and iDisk.

That said there seems to be a lot of effort duplicated at the file-system level and at the version-control level. I hope file systems of the future will expose nicer api's (e.g. change logging) to help version-control systems become simpler and more efficient.
My friend looked into porting git to Win32 natively and abandoned the effort because Git uses a ton of UNIX-isms, from fork() to sh scripts.
Not that I agree, but here's one from the thread:

http://www.xent.com/pipermail/fork/Week-of-Mon-20080128/0482...
I think it's bad for startups, not b/c it's one less company buying startups, but b/c when MS buys Yahoo it will hurt/kill the open source projects that Yahoo is sponsoring
Not really surprising, given the author...
Awesome, thanks for the advice. This is pretty much our biggest question right now (whether or not to go for the social angle and implement a facebook app), so I appreciate the feedback.
that's really good insight, thank you.
Kudos for doing something with social news that isn't aimed at the nerd crowd! I'll pass it along to my gf :]
The world as a circuit board.

Kind of makes it feel small, doesn't it?
I plan on launching an initial version of my site in March. I started full time in December, and gave myself 3 months to have something out the door. Im pretty well on track. I initally cut some things that were in the original design, to make sure 3 months would do it. That was probably the hardest part of the whole thing - determining at the beginning what to leave out.
Actually I don't believe "20070202T1422" is a legal 8601 timestamp. My read of the partial ABNF below requires you to designate the time zone with a "Z" or a specific offset (e.g. +08:00).

time-secfrac = "." 1*DIGIT
time-numoffset = ("+" / "-") time-hour ":" time-minute
time-offset = "Z" / time-numoffset

partial-time = time-hour ":" time-minute ":" time-second
[time-secfrac]
full-date = date-fullyear "-" date-month "-" date-mday
full-time = partial-time time-offset

date-time = full-date "T" full-time

If your data is going to stick around a while (and possibly loaded into some other system in the future) you should be sure to encode the timezone.
My Secret Plan to take over the world
.
Oh, right.. two much python in my head, not enough lisp. Now I'm interested, what is the shortest lambda in arc that just returns the first argument? If I'm getting this right the best I can come up with is [progn _], turning my example into:

(keep [progn _] seq)

And thus I see the need for trues =P.
>two much python

and two little english
I started working on a similar product in 2001. Got it done, but it's tough.
That's exactly what I'm afraid of. Bye bye YUI tools...
"Internet users in Asia are growing by 882% per year."

Really? That seems on the high side to me (not to mention unsustainable for more than a year or two into the future).
I use our own service at http://www.i2drive.com not only to store files but also to edit files on the remote server. it uses webdav for back-end which lets you map a drive from windows or mount from Mac and Linux.
If this is true, then webapps should use git instead of databases.
For sure in some cases - yet I don't see any metadata and query capabilities. Are there?
Hmm I bested it... ['(_) 0] beats [progn _] by one character, but it sure is ugly.
-C protects if you create an empty directory and then use it as the argument to -C; saves you a command line cd and a cd back to where you were, though it doesn't save much if you use your mkcd command to make the directory. (I'm talking about GNU tar where any leading slash(es?) is stripped off of the path of any files in the tar (and .. is disabled))
Does [1 _] work?
You are the first person I've ever heard to pronounce < as "ascending". Really, gt/lt functions are my biggest tripover point when doing prefix-everything expressions, because they work in the opposite way of how we were taught as children to interpret them. a<b is true if b is greater than a, and we can evaluate it visually by looking at the expanded side of the operator versus the pinched side of the operator (the former being greater than the latter iff true). In (< a b), the expanded side is pointing at exactly the element which it does not represent, meaning that we can't use those nice optimized mental pathways which are devoted to visually evaluating the expression.

It's logically consistent with how the rest of the system works, but it sucks because having to unlearn anything sucks. Personally, I just imagine it being rewritten as infix.
IP laws gone awry: Implementing algorithms described in a book could be illegal
you could just rename the operators to lt and gt, so there is no visual cue either way.

you could also change the argument order, but that's probably not a good idea :)
I think you're right:

http://www.internettrafficreport.com/asia.htm

"Internet traffic report shows that Iran has been completely disconnected from the internet..."

Looks like this is Bush/Cheney's last hurrah before they leave
Denture Hacks: How To Get Your Teeth Fixed in Mexico
Ask YC: What's wrong with getting linked to from blogs and prominent sites?
Honestly, I think getting linked to a major blot such as TC or any other is not worth it for most startups. Unless your product is so superior and so obviously necessary, I can't see how getting covered or linked to will produce anything more than short-term success.

Maybe a discussion of how many major sites have grown 'organically' vs. getting major coverage..
design is important ..but not something that will take more than a week to figure out ...i think the only thing that you need is to learn a bit of photoshop (2 days) and read the book "don't make me think"...heck you can even leave photoshop ..just read the book and that should be good enough
I so want this to be the case because I can't see MSFT doing anything but wrecking all the parts of Yahoo worth keeping.
For me, it's most important that Arc's been a lot of fun so far. I get to coerce and zap things. There's not much standing in the way of me doing. It's not a bureaucratic language.

Of course, this is coming from a person who was overjoyed only hours ago when she figured out that Ruby can evaluate strings as code. Maybe it's not pragmatic. But I don't code to be pragmatic.

So far it been a lot of fun. I get a heck out of blurring the line between data and code. I like the hackerly feeling it get when I figure out how to use closures in some crazy way. I like being able to easily write my own languages or language elements, in the flow of writing the larger program. I like that I don't even have the impulse to reach for the docs, like I would with Java or C# or Python or even Ruby. When I think of something cool that I hope the language can do, everything encourages me to just find a way to do it myself. It's an empowering language.

I've only explored a bit now, but I think I see where things are headed, and I like it.

It might be that it's a language for people who like writing languages. I know I do.
This has nothing to do with their address book. It builds off previous social meta-data efforts, especially FOAF and XFN.

It's more an extension of their search expertise, and it looks like they should be able to extend it with some of the latent semantic indexing stuff that they already do to make it a useful and powerful tool.

What's innovative about it is the scale at which they are doing it.
A Giant Bid That Shows How Tired the Giant Is
"git is so powerful, that when you put light in it, it can't escape"
Well the secret part is right...
I've got a startup that brings college bulletin boards online for students to buy and sell their used textbooks. It's eBay meets Craiglist for college textbooks. Check out our alpha project at cocunderground.com (COC is for college of the canyons, the school that this project is directed at). All feedback is welcome!
Is this something designed to replace Quickbooks entirely? Something like this would be nice for my situation. I can't run quickbooks from Ubuntu, and I don't like booting into Windows whenever I need to do anything.
GIMP is good, there's no doubt, but I've found that 90% of the graphical development for my project can be done in about an hour on Inkscape.
"Who will be the first (or does it exist) to do a YC in the under-developed world?"

I suppose politics are in the way - most underdeveloped countries don't have capitalism, or am I mistaken?
Try GNUCash? It's not bad.
check out ikiwiki - http://ikiwiki.info/ - it backends to both git (and svn) to do this - git allows offline work.
News Corp. Scrambles To Bid For Yahoo
I think trying to create a product that is superior + obviously necessary should be the goal for most startups... Getting on TC is then just a resulting symptom.

That said, top-tier blogs are a good way for many in the startup scene to really jumpstart their project, but getting a writeup there is certainly not a requirement to success, and there's plenty of echochambering that goes on in them too, so I can see why many choose an alternate route all together.

Perhaps you'll find this post relevant to your question: http://redeye.firstround.com/2008/01/after-the-techc.html

For us, getting on TechCrunch//GigaOm//etc was a strong benefit. Those blogs have a big reach. It got us a lot of other coverage, investor interest, and established some level of credibility out of the gate.
Yes I've used it, and it is pretty good. The problem is I need a platform independent application... Or a dedicated windows computer to replace the other partition on my notebook's hard drive.
Well put. I guess it's a chicken and egg complex.
Does the success bring the bloggers, or do the bloggers bring the success?
"... what Arc gets right ..."

Release early and often. Users sure are commenting on it for good or bad.
Yeah... I can't say I've tried it, but I think you can get older versions of Quickbooks working in WINE. And there is always a virtual machine.

There's also a number of web based accounting software, like the online version of Quickbooks, NetSuite, and Intacct.
Get an EIN online, for free
A what?
"... Wondering if there are any seed funding orgs like YC in Australia? ..."

Ahh let me see. There is easy funding for real-estate, mining, farming, construction, dredging, highways and those new fangled finance schemes. But I doubt it. The VC's in Collins or Pitt St see Startups as too risky and do not understand them.
You need one for a US company, it's the tax ID for a company. The individual equivalent is a social security number.

If you don't know this website, a lawyer or a corporation in a box website will charge you money to fill out the form for you.
It definitely won't hurt your startup. It will bring in some sales/users, but it's not a magic bullet. At worst, it's a free link from a high pagerank, relevant site.
I for one think it is a good idea. Maybe with Microsoft's start page, Yahoo search could get more traffic and the recognition it deserves. Microsoft Online Services could be scraped completely, nobody would miss them.

Server tech would be a concern, but who knows - perhaps rather than switching everything to Windows servers, Microsoft might take the chance to become acquainted with Linux and BSD. It would make sense if MS would in the future simply release a theme for Gnome, rather than waste resources on another OS that nobody wants.
I think a TC post is worth nothing to most startup unless

1. you are targeting thier audience

2. you are after hype to get some VC's interested

3. you are targeting a mainstream audience and hope that other bloggers or mainstream media will pick you up after seeing you on TC

You are better off targetting blogs aimed at your target audience.

A blog with 2000 readers is going to have more value and be easier to get coverage on.

Anyone know of any where that list these b and c list bloggers by niche? I would pay for that list.
good point about the pagerank and could be helpful if you are in a crowded market.
Here a very superficial comparison of 'Qi' and 'Arc':

> (< (length "Qi") (length "Arc"))
> T

Off-by-one!
I have been toiling away for what seems like ages on a source code review app, aimed at helping cube dwellers who can code sort out some of the horrible outsourced code they have to review (something I have to do in my cube too often, and was frustrated with the tools out there to do it).

Its just a part time thing for me these days (still stuck in my cube), but I hope to get a beta release out there in a month or so. Things that are making me nervous/holding me back:

* Can't come up with a decent name for thing!

* While I don't think my app is ugly, its certainly not anywhere near as shiny as many of the other web 2.0 apps out there

* I keep tweaking and adding features instead of polishing what is there and getting the beta finished.

I am pretty pleased with what I have go so far, so at the least I will have learned Rails and created a tool I can use in my current day job.
Yeah, that would be a great list. I think you could compile one by browsing Technorati a whole lot.
CL learns from Arc