skysight-inc/HackerNews huggingface dataset:
Skip both of them.

Just use memory with a (very frequent) periodic write of the entire dataset in a file named after the time you saved it. Just write/read a block of memory (big array) to/from disk in a file named after the relevant time. What's better is that modifying or accessing this data is normal straightforward memory accesses - very likely less work than writing SQL queries and very very likely easier to unit test. So this should not only be faster, it should also take you less work.

You probably only care about ~10,000 US stocks & bonds, maybe 50 numbers per, all of which are represented well by a 64-bit double (most probably fine in a 32-bit float... yes even US bond prices - they trade in 32nds - binary works fine). 10,000 instruments x 50 numbers x 8 bytes = 4M bytes. All 3 dimensions could increase by an order of magnitude and the answer would probably remain the same or very similar.

You mentioned two requests, and a primary goal of SPEED.

1. Data for a stock/bond - it's in memory. Heck, there's a good chance you can fit the entire current dataset in your CPU cache. You just can't beat that (well you don't need to - put down that VHDL tutorial).

2. Data for a certain day/time (across a bunch of stocks): Just read the file off disk for that day/time. Even if this isn't immediately up to your needs, normal optimizations should buy you tons of performance here - buy RAM and disk mirrors. The data should be contiguous on disk, so you should have very minimal seeking (which is normally the speed killer). You could come up with something that beats this by pulling out fewer than all the stocks at once, but your complexity would skyrocket. This will should at least beat most setups with MySQL or SQL Server 2005 without a metric ton of tuning.

I'm guessing you forgot to mention that you need a time-series for price and volume information (if only for a graph to throw up alongside some boring numbers). Consider just handling that separately, especially as all the other numbers you're interested in don't change anywhere close to as often (and those that do are based off the price/volume and can be trivially calculated on the fly). Some headaches to watch out for though: if you try opening 10,000 file handles at the same time, something's likely to get mad at you. Also note that if you're thinking of opening and closing 10,000 files a lot, you're likely going to be sending the disk seeking around like crazy updating meta-data on disk.
Reddit was called "Y2Combinator?"
An interesting business model, and they've got a great team around it as well. Speaking as a college student myself, they definitely offer a good value proposition.
It's funny that you think a lifestyle is immoral if it doesn't accomplish its own "internal goals." I think that's wrong, and not just in pathological cases. I might be an ultracompetitive misanthrope who lives to be on top (gordon-gecko-ish capitalist). But the way I get there (startup? investing?) might end up helping lots of people; maybe helping them surpass me. Was I immoral, because I didn't accomplish my goal to be number one? I'd rather have more of those people than more couch potatoes complaining about the immorality of powerful people.
There tends to be a tradeoff between novelty and time when it comes to research -- more novelty requires more time. Where is the sweet spot for publishing undergrad research papers?
I know pg says you need co-founders, but I have a feeling that partnering with a stranger will hurt your chances of success more than it will help. I remember Jessica saying a major cause of death in startups was founder disputes (link: http://www.grid7.com/archives/189_podcast-28-jessica-livings..., around the 14 minute mark).
It's great that they're doing so well, but their punishment for failing to meet their service agreement is negligible (10% off that month, I think). The downtime's effect on your site's reputation and income can be much more damaging than the 10% discount will fix. The fact that they seem to understand this and are committed to trying to provide 100% uptime gives me a lot more confidence in them.
That was a brief summary and one can say it more carefully. Most importantly, immorality is not a boolean thing. If you aren't accomplishing a goal, it'd be an improvement (more moral, a better way of life) to either change your approach, or change your goal. It is less moral (a worse way of life, immoral) to continue with a lifestyle that is failing by its own criteria of success and failure.

That needs the caveat that we only mean goals you actually intend to accomplish. We don't mean vague goals, and we definitely don't mean the sort of goal you would be happy to partially achieve -- in that case, the real goal is just making progress towards ... your "goal". (It's the same word, but it's a different concept than the one I mean).

Back to your points, you say that even if you fail by your own standards, you might help others, and the net effect of your life may be positive. That's absolutely true. However, it'd be even better (more moral) if you did the same actions, but had wiser goals, which those actions were achieving. Then you'd help others, but also consider your own life successful.
i'm a programmer fresh out of college in bergen, norway (planning to relocate to boston or bay area). i'm 34, and i'm building a neural network to do speech to text. i need another programmer, preferrably someone with more experience than myself, especially with browsers and low-level signal processing.
I can't believe we're still having this discussion.
It's nice to make friends but this isn't how you start up companies
From the responses on this page you appear to be correct.

What would you suggest?
The only problem is such a system will tend to award popular ideas (or humor) as opposed to unique, albeit possibly controversial ideas. When I go digging into the comments I generally look for two things... a large thread of comments and particular users whose opinions I regard. These are not necessarily, although they often are, the posts with the highest points. Occasionally I find gems near the bottom.
Exponent. Its page is at:

http://www.thinkcomputer.com/software/exponent
i just spent past 7 years working with companies doing your #1 & #2 ideas. i'm a product mgr, not a programmer person so I am happy to share with you my perspective. send me an email:

http://www.linkedin.com/in/mgmcmahon
mgmcmahon at gmail
The pairings that are made today would be best to apply for future cycles later in the year. Most people here you talk to will at best end up as an employee in your eyes, it's going to be hard to treat them as equals when most people are just throwing out soft skills they want filled.

My partner and I have known each other since high school but he went off to college while I went into the military. After reconnecting we've been working on small projects on and off for the past couple cycles to learn each others quirks, styles, and most importantly trust.

And after attending our 5 year high school reunion it's really surprising that people we haven't talked to in awhile turned out to be doing very similar tech things as well, even if they're not in the area. It surely doesn't hurt to look up Facebook some old friends and see what they're doing or what they're working on. Maybe some hate big corporate culture as much as we all do here.

A better way would be for people to list out their work experiences or market they're analyzing for their startup.

For example, one of the failed ideas I was researching for was event planning and building an evite competitor. After 4 months I dropped it but I've kept plenty of notes about trends, other websites, feature lists, specs, and so forth. If anyone wants to discuss it feel free to get in touch with me at alaska.miller@gmail.com. I also worked at a big corporation working with CMS tools, search tools, publication, enterprise software for PLM processes. If you want to build software to optimize those fields, again, get in touch with me.

Likewise my current idea is dealing with Flash video and Flex environment, and accessibility. If anyone has experience with that, please email me.

Start emailing people and sharing with them your ideas for feedback, then ask them for another referral to someone else you can talk to until you've literally have no one left to talk to. I've been keeping track of all my conversations and people of interest in iCal and 37s Highrise and building my business network. I'm also inviting local people to lunch to pick their brains.

But it's completely youthful naivety to think this is the right way to cofound a business together.
Highly downmodded comments stand out to me almost as much as highly upmodded comments.
This is bad. Anyone should be able to say anything. That way everyone can see what is being said - good, bad, sad, left, right, or wrong.
Remember, a democracy is two wolves and a lamb deciding whats for dinner.
Fuck lamb for dinner!
Miami, FL... 25. Doing hardware stuff...
Alan Kay said the best way to predict the future is to invent it.

I wish he would have gone further and said that if you aren't creating the future you have no business predicting it.
Go with what you have experience in. If you build it and it is not fast enough, then throw caching at it. There probably won't be a huge difference between most databases, if used effectively.

If you have special requirements, such as true isolation in transactions, be aware that this is "leading edge" for SQL Server 2005, and probably for MySQL. On the other hand, Oracle has been doing it for years.

If the data is being streamed in, and timestamped, and stored, you may want to just hold it in memory (32 Gigs holds a lot of data) or write to flat files. For simple data types, that are used in a predictable way, not much beats well-written custom code.
Aside: The elegant symmetry with which this metaphor crosses three cultures is absurd, baffling, and truly humbling. What lead you to conjure that up? What is it that you are reading?
I got some interesting ideas. What is it you do? I'm in Indianapolis and have a couple sites. http://www.cornbrain.com and another which I'm developing.
Oracle has a native driver for .NET.

Google for ODP.NET. If it's anything like as good as their pure Java JDBC drivers, it will perform pretty well.
Thanks. Already been doing some of what you suggest. This just looked like an interesting thread.

In the meantime, back to work.
Be as novel as you have time to be.
Chicken Farms on the Semantic Web
David Lawee New Head Of Google Corporate Development
> my friend Dikshit Gupta

Please tell me you're kidding...

> got far fewer callbacks than Dave Gupta, even with identical resumes (both are obviously Indian).

Maybe "Dave" implies a certain level of cultural assimilation? (Or at least a willingness to sell out ;-)
No, too many of my notebook doodles are valid Perl...
I've even got you stored in my little database as the ERP guy
An Arc-Tangent
Mustn't goals themselves be good or bad, regardless of whether they're achievable? Or would you say that goals, such as Hitler's, were bad only b/c eugenics ultimately would be bad from an evolutionary point of view?

Then, there is a the problem of heroism, which is defined by a person's courage to pursue a noble goal even if its achievement is very unlikely.

Finally, by your definition, I could be one of the most moral people by making my goal "do whatever I want." Everyone is always doing whatever they want, at least at some level. This would pretty much render all talk of morality pointless. But, I suppose I'm being too literal with your definition.
Offset-Based Linked Lists
George Harrison's 'Set on You' song actually about starting a start up
I know this is rather inane - but Itunes shuffle has been giving me the chills. First Wyclef's version of Another One Bites the Dust followed by the Queen version (from 2532 songs!) - and now this?!

"This time I know its real.
The feeling that I feel.
I know if I put my mind to it.
I know that I really could do it."
Entrepreneurial synchronicity defined (or "I love it when a plan comes together")
The HTTP Sweet Spot
http://www.snopes.com/politics/satire/revocation.asp
If chains of trust could be widely and rigorously established online, most (maybe all?) of the big problems we have today would be gone.
Human-level AI is always ~20 years from "now" (whenever "now" happens to be).
It's more than kind of sad, it's pathetic.
The religions by language breakdown is not terribly revealing, but the opposite, languages by religion, is very interesting.

Atheist like Python, C, C++ and Java. A very practical toolkit. Most of the major large religions, Christian, Muslim and Hindu also seem to favour widely used languages. Buddhist ranks Lisp very highly. Jewish coders like web languages like PHP, Perl and Javascript. Pagan's must all be running Apache servers, because Perl, PHP and Python dominate. Unitarians, true to there reputation, go for an eclectic mix that includes oddities like Groovy, D and LOGO. I think people's deepest held beliefs about how the world works does seem to lead them towards certain languages more than others.
I have just read the introduction, and it does seem that the author is looking far ahead, but I don't think it's too early to reflect on the ethical aspects of lethal robots.

Science moves faster than the field of ethics (e.g. cloning) and the US has already lethal robots on the battlefield. They are not autonomous yet, but we're certainly going there. When the robot 'assists' the decision by providing additional data (say an sensor detects a hidden weapon on an otherwise civilian-looking guy), it already skews the decision process. As the author mentions, there will be a point when the military will start to think that their robots are better at making the shoot/not shoot decisions...
Norvig doesn't think so: http://www.news.com/8301-10784_3-9774501-7.html
Thanks. I never thought of describing myself that way, but who knows, maybe I can find 2 chicks who dig it.
These are just starting points. I do think goals are themselves good or bad, but it's much harder to explain how you can objectively make assertions like that, so I wanted to make the lesser claim, for now: there are ways to explore morality objectively.

I'm definitely not claiming consistency of this sort is the only criterion of morality. Only that it's an important and objective one.

I don't mean to be a tease, but if I say too many things at once, I won't be understood as well. On the other hand, threads here go stale fast (usually in under a day), so I'm not sure how to ever get very far in explaining, here. By contrast, on another forum, I am in a thread that has been going for 4 years. And it's only 180 comments long -- so around 1 comment per 8 days.

I'll keep posting here if anyone replies. Or contact me, curi42 on AIM or curi@curi.us

So for depth, my best idea so far is to link longer, external writing. Here are two things I wrote about morality which explain my views a bit more:

Essay: http://www.curi.us/blog/post/1252-xii

Dialog: http://www.curi.us/blog/post/1169-morality
It would be a good thing. If anyone wants to apply to YC with that kind of idea, we'd be into it.
While it would be interesting, my quick glance through the list showed there to be very little difference between the religions. Yes, atheists prefer Python while Catholics prefer C, but the difference is really quite minimal. My guess is the differences are not statistically significant. I would run the numbers but at the moment I'm feeling lazy. Just like every other moment.
The way you get admitted to good grad schools is by impressing specific profs. It's much more personal than undergrad admissions. And profs I know say the thing that impresses them most is good recommendations from other professors they know and trust. So talk to current profs who like you and ask them who they know at grad schools you want to go to who would make good advisors for the kind of work you want to do.
Ah, good. I figured I wasn't getting the whole picture. Something similar to objectivism makes sense to me. I think there is such a thing as human nature, so everyone is ultimately made happy by the same things, at a certain granularity. Morality's objectively good goal, in your framework, is to maximize happiness.

However, at this point I have to veer into territory considered "religious," because such a claim requires at least an element of non-materialism to make sense of our moral intuitions.

At any rate, my views are not rigorously defined enough, and I'd benefit from critical, constructive discussion. I'll check out your links and see if I can participate.
Shouldn't it be another type of combinator?
It's been a really long time since I looked at this kind of stuff in any detail. The librarians I worked with at university would go on and on about semantic markup, and there were some very dedicated users that seemed to be willing to slog through the SGML nightmare. But, as soon as HTML came out it swamped the other technologies because it was so easy for people.

Has anything changed since the SGML days? Rhetorically, it all still sounds like a bunch of librarians complaining woefully that they can't do search properly if no one marks up their data properly.
I had thought facebook would be the beginning of this, but it seems to be a flash in the pan, and too locked in to really be effective.
But News.YC prides itself on having both a terrible user interface and a terrible user experience.

What kind of site calls a feature "noprocrast" without giving any detailed information about what it does? Only news.yc!

--edit

I wonder why I am being modded down. Maybe I said it too sarcastically. News.YC really does have a terrible interface. Look at the account settings page-- imagine yourself as a user who has never been there before. It's confusing and poorly documented. I love this site for the content and the simplicity, but Paul Graham is a programmer and spent very little time on the user experience, which makes sense. I don't blame him.

But it is still a bad interface. I'm pretty sure 99% of people who fill out the "email" field expect their email to somehow be visible to other users. It's right next to another field that IS visible.
Recession, Burst and Lay-offs - a poem
So apparently Paul can't keep trolling from happening on News.YC afterall.

But, as anewaccountname shows, at least the trolls here do it with style. Well played, anewaccountname.
My favorite (upmodded 26 points as of 9pst):

Regarding News.YC: "Your ancient ritual has summoned me to News.YC so those tight-assed holier-than-thou fucktards can be shown the error of their ways.
Maybe the reason there aren't trolls is because NOBODY FUCKING VISITS THEIR SHITTY SITE."-- terwin, Reddit
From reading the application, that still looks likely. "Only founder" is in a list of apparent deal-breakers. Perhaps the 5 pg mentioned had co-founders in-waiting, some special connection, history of success, or finished app.

---------------------

Are any of the following true?
(a) You are the only founder. (b) You are a student who may return to school in the fall. (c) Half or more of your group can't move to Boston for the summer. (d) One or more founders will keep their current jobs during the summer. (e) None of the founders are programmers.
FP for the rest of us
I was thinking more along the lines of Sony giving a $50 discount to people who exchanged HD DVD players for Blu-ray players, just to solidify their victory.

Do you guys think Sony will even consider this route?
Cross-holdings muddy Yahoo, Microsoft drama
What kind of site calls a feature "noprocrast" without giving any detailed information about what it does?

A site for the kind of people who'd try it to find out.

Actually, it is explained in News News (http://ycombinator.com/newsnews.html), but maybe most people don't bother looking at that.
I am going to be in Leeds, UK in 2 months time working on a p2p database/webserver. If you are interested, around the UK, and know Java, let me know!
Half the entries are something to worry about. Wake up USAish people.

PS: "Americans" describes a lot more people than just the citizens of the USA. It's about time you choose a suitable word.

PPS: Today is trolling day, in case you didn't notice.
http://philip.greenspun.com/careers/women-in-science

It's not what you asked for, but if you haven't read the likes of it before, you should. I wish I could have read it when I was your age. I wonder if I would have believed it.

If you still need advice on graduate admissions, I can tell you that pg and cperciva are on the right track. Undergrad research is particularly great, as it gives you lots of contact with an adviser who will write great recommendation letters for you.
Yeah, I'll add a note about that.
But Microsoft has been declared by the courts to be a monopoly that has abused it's market position. Every acquisition must be considered in that light. So they won't be buying up Silicon Valley anytime soon. (and certainly not with a Democratic administration coming to power).

OTOH they could diversify: buy up biotech & nanotechnology firms, etc. and put part of the $44 billion into VC investment.
It surprises me that people can indict others with the adjective "holier-than-thou" and not feel ashamed of themselves. The word screams, "I'm going to bring you down to my level!"
Hagakure.
THIS IS A GREAT SITE!!!! but it's got some kind of anti-troll field protecting it. The most perfect trolling oppertunities always seem to degenerate into civilized rational discussion.
He is saying that the article says nothing new and it's more or less pseudo-journalism. I tend to agree, having read the book he wrote on this same topic 9 years ago.
Damn, I was actually fooled by that. Whoever wrote it is pretty good. I laughed out loud when I got to "American brands will be referred to as 'Near-Frozen Gnat’s Urine.'"
pg didn't like the name 'Reddit' in the beginning and encouraged them to change.
But when comparing the cultures of humans and chimpanzees, that's where the real differences are visible. "That's the promise of what humans are. It's not the individual intelligence but this collective intelligent culture," he said.

Wrong.
I run http://www.colr.org, which is powered by web.py. It has survived several digg / lifehacker / other high-traffic incidents.

If you are running web.py as a fcgi process, you'll handle a digg effect just fine.

If not, you'll peg the server and your site will be largely unavailable.

Happily, switching web.py from fcgi to non-fcgi mode or back again is a single-line change :)
I am not exactly looking for a co-founder right now but I'd definitely like to make friends with ppl with startuppy interests.

Location: Seattle, WA.

(Proximity not a requirement)
They will, when you get acquired.
Aviary - Creation on the fly
Name another commercial hosting service that allows for non-negligible punishment in the face of downtime. I think the point is that Amazon is doing pretty well, considering the scale, the cost and the nature of the service that they're providing.
how do you turn a stranger into a partner? You try working together and see if it works out...

What's the reason you don't have a cofounder? because you haven't done what's stated above.
The midwife of miserabilism
Yahoo's Entire Design Innovation Team Fired
Artistic License 2.0 and ... REM?
Our CS program here is more like "programming trade school." I doubt recommendations from those profs would be worth much -- what about from the math department?
Optimal answer :)
http://www.cs.cmu.edu/~harchol/gradschooltalk.pdf
Precise advice. Nothing -- absolutely nothing -- beats having a published paper at a reasonably respected conference / journal, at least for admissions in CS-related areas.
Taking play seriously