skysight-inc/HackerNews huggingface dataset:
I enjoyed reading your article about parsing. Any thoughts on open sourcing your parsing work?

When you mention "lifetimes of research pending to be done in parsing" are you extending the reach of parsing to beyond text - to speech - human conversational analysis, baby acquiring language or maybe even to more abstract communication - like automatically parsing patterns visually.
I plan to add indentation support to my system in the future... it's a very interesting area: gathering information from all available sources and using that to help the user.

I do believe using the history of edits, as the article describes, has a much, much greater potential.
Stanford Course On How To Build iPhone Apps Will Soon Be Available On The iPhone
All carbs require insulin. Fructose and complex carbs need other "processing" first though, which spreads out their absorption, reducing spikes in insulin levels.
Social Actions and the Microphilanthropy Movement
Hands down, my favorite presentation at PyCon
really? Your sure about that? You try it. Put yourself in a life where you have no bank account, no or bad credit, and try to take the check you just received from "low income job x" and try to find a retail branch of the bank it was written against. If you're real lucky, there is a retail branch near you. When you go in, you will be presented with process barriers the bank uses to keep you from cashing the check. Are these barriers legal? Doesn't matter. You still will be sent out the door without your money...and then you go back to the loan sharks.
I like that this is one race where Google has to catch up to Amazon.
On the idea that problems which seem solved to some people are far from solved, indeed. Of course.

I suspect, however, that the idea that there has ever been some kind of consensus that all problems in software were solved, or all problems in Physics were solved, is BS. Fun BS because it is always fun to ascribe a single personality to a large group, and then point out the foolish inconsistencies or limitations to what "they" say as though they were a singular personal pronoun.
This article goes something like this:
* I thought being a freelancer would benefit me.
* In fact it benefits my boss.
* Therefore I was wrong and it doesn't benefit me.

The author assumes that employment is a zero-sum game, when in fact it would be entirely possible for an arrangement to improve the situation for both the employee and the boss.

It's nice for your boss to save money on office space. But it's no skin off your back, because you don't gain anything from your boss spending money on office space (unless you like working in an office).
Free Stanford course on developing iPhone software [itunes]
"Future of Democracy at Stake"- music labels on darknets
Why not?

That's my whole point. Whenever someone complains against Google, Microsoft, take your pick, they've done pretty much everything they could have to first privately resolve the issue.

Saying that 37Signals have the right to completely skip to public flogging of another company IS fanboyish. That's all I am saying. If this is supposed to be a forum of objective readers, why are they completely discarding that part?

I am not a fanboy of GS. Until this, I've only heard of them maybe three times. This is about objectivity and bullying. Jason Fried has a big enough of a forum to bully another company and he's done it.

Why is that that can't stand on its own, divorced from the argument for/against GS?
For a mainstream TV show/station, I'm not sure I entirely agree. What would be more exciting to watch, web/tech with thousands of faceless customers they would likely not interact with outside of email/twitter for at least a year... or tangible product & biz development that a mass audience can see, understand, and relate to easily?

I think web/tech is more exciting to live and build for the intellectual types, but much like computer gaming, it's just not meant to be a spectator event. 90% of footage would be pale faces behind a glowing computer screen, where as retail is a lot about vendor selection, sales and deal making.

That said, I think you could replace those products with anything — what's interesting is watching them overcome challenges in their respective industries. Who knows, could learn a thing or two. :) Some startups need to stick their neck out of the bubble once in a while and breath something different... Perhaps there wouldn't be less me-too-point-oh.
> Careful here: are you _really_ suggesting that Twitter would do a full rewrite
> without making sure that it would increase throughput first?

No. I say this Scala vs. Ruby/Python/PHP rant sounds farfetched and unfounded.
In other words...

MapReduce implementations have to take into account locality, i.e., the dataNodes have to be rack-aware for optimal I/O speed. Therefore, if their dataNodes are actually just cpu cores reading/writing blocks to S3 (other racks) then they are effectively getting kicked in the face.
Thanks for the kind comment. Nowadays, my products are my source of funding, so no chance to open source anything. In the future, I'm sure some things will make their way into more open forms (not totally sure about what form).

Every computation can be seen as an exercise of parsing. The cases you mention, and many other ones.

I'm working in a new computation model, a new programming language and a new type of VM. But I don't want to talk much about it until work is more advanced.
Assuming no ill intent and simple (if major) ignorance being GS's problem, having someone around like Fried with an understanding of trademark and branding would have probably prevented the whole issue.
Yeah, pretty much like a Starbucks opening right next to your coffe shop. Suddenly every neighbor wants a cup of coffe, and you'll see increased attendance.

Cloudera offers are more like consulting. People need help setting up Hadoop jobs, understand MapReduce, or don't want to send data to S3.
The main cause of early death was injuries and infections following injury. Disease was relatively low because the population was dispersed. Contrary to much belief, startvation was also rare, it was an anthropologist who coined the term "the original leisure society" for hunter-gatherers; Google the term in quotes for many references.
Can you not fix the media problem? Putting nginx up front to serve media would relieve a lot of huge Apache/PHP processes.

Are you actually hitting the limit in practice? So you have 50+ concurrent requests at once? Is this a "real business"? If so you shouldn't be so afraid of putting in the time to setup nginx or something else... you'll literally _have_to_ to different media serving and load balancing if you plan to grow whatever_this_is.

As far as actual RAM usage, you'd probably save some space and lose very little (depending on what your app does) on a 32-bit OS. Sadly, Slicehost only offers 64-bit... you might check out Linode or one of their other competitors. I'd only suggest this option if you're really broke, though. You'll still have to do the work mentioned above on Linode eventually... the 64->32 would only be a small stopgap.

I don't do any PHP anymore, but I think thread-safeness largely depends on what external C libraries you use, and generally isn't guaranteed.
Possibly a lot of the lessons learned in their shoe / salsa business will be applicable to a web business as well.
Terrible metaphor: a critical blog post is not aggravated assault.

You seem to have a bit of a fixation on criticism as some sort of injury.
I'll give you that point.
Reread my comment. I didn't say he wasn't a founder, but Mr. Shorr's contribution is underrated in the article. He's only mentioned in that paragraph.

I constantly see mentions of boy geniuses in the popular press. He's undoubtedly bright, but it's his access to connections and money, and in particular Mr. Shorr, that made the difference between a mom and pop payday check cashing operation and growing to sell a business after two years. Readers of HN should recognize that it's not through his brilliance alone.
This makes it clearer than any Taleb article I've read that black swans aren't just an external danger to investments; they're invited, even mandated, by the very structure of the market. I hope someone can come up with a better solution, though; I can't see money management being outlawed in anything less than a zombie apocalypse.
I've listened to the podcast and it was a treat. Ed was also able to isolate the chemical that ants lead behind when they're tracing back home from a food source. When he dabbed it at the entrance of an ant colony, they exploded out of the burrow.

For fun, he likes to write his name with the chemical, and the ants will follow it, tracing out his name.
That doesn't make much sense. I've written a lot of blog posts and a lot of email and I can't say that the latter takes longer than the former.

The irony in the "this could have been a lawsuit" is that their reply could have been too, since I suspect they'd have a pretty good case for a defamation suit.

http://en.wikipedia.org/wiki/United_States_defamation_law#De...
Yep I mean, in 60s it appeared like it was a-few-years matter.
Have you written more about the Digg fiasco? I'd love to hear about it.
> they're going to remain the center of attention for awhile.

Everyone on the Internet has a soapbox. Jason Fried/37S has a much bigger soapbox than the average disgruntled blogger/user because of their huge reader base.
I received a call from a recruiter last summer. It was about flying to Chicago to interview with Cash America. They needed a director of IT and had a lot of RoR stuff on hand.

I politely made myself unavailable.
Maybe its because I spent a decade, the 90s, building financial apps at large banks and insurance companies. Like so many other IT guys, I know I had a hand in building some of the machinery that has destroyed America and those that trusted it.

But even if you had never worked on the inside of Wall St. firms, you should be able to spot a crack dealer when you see one (what do you think those call center reps were doing?). I don't care if they are using Linux and pay well. There is a reasons usury laws exist. The same reasons drug laws exist...usury and drugs both eat at the bottom of society and destroy it.
I agree that the Ruby 1.8 to 1.9 transition has caused headaches. But the 1.9 version does have some interesting features that have been long being cooked to get to this point, and perhaps more interesting is that the 1.9 version has motivated even more developments in Ruby-land as found in MacRuby that uses the 1.9 version or in yarv2llvm that recompiles the bytecode to llvm or something.

Let's say that many folks might have had higher expectations that have been hard to meet given the reality of it all.

The Merb guys helped with further explorations themselves which probably helped with making Rails advance more decisively, given the promissed "merging" of the Merb ideals with Rails.

Also, with things like Passenger, the deployment of Ruby met an all-time high in features.
Nice recommendation!

The book description sounds a lot like data mining. Isn't it? Why didn't they use the buzzword anywhere?
> Java scales just fine thanks. FWIW, Mibbit handles a good number more messages
> per day that twitter. On a single VPS with 1.4GB ram, running Java.

I wasn't talking about that. Message passing is trivial and scale on any language, or at least it should be. Well, unless you do Java and XML. That doesn't scale, from experience. Unless something changed in the last couple of years I'm not aware of. (Just taking your red herring further, to show it doesn't make sense and it has nothing to do with what I said.)

Third time lucky, I hope. My point is I seriously doubt Scala is much better at scaling than Ruby, Python, or PHP. And the author seems to have based his opinion on his experience refactoring Twitter components previously coded in Ruby in Scala. Caveat emptor, I wasn't at the conference and this rant is second hand via a questionable source, The Register.
"Why not?"

Because publicly criticizing wrongful, damaging behavior is not "bullying" in the eyes of most people. That, in fact, is exactly why it's embarrassing for the people complained about in cases like this.

"Saying that 37Signals have the right to completely skip to public flogging of another company IS fanboyish."

Well, I'll cop to being a total free speech fanboy.
* edit * dumb question. Sorry
The bacteria don't eat the enamel. They feed on sugars (and other carbs, but especially sugars) in the mouth, then excrete an acid that attacks tooth enamel. They live on teeth, especially in the crevices where they won't get knocked off, so that's where the acid is concentrated. Diet can help reduce the incidence of decay in the first place by reducing sugars in the mouth.
Once upon a time, people made the same comments about Ruby and Rails: "it's a fringe language", "the tools are immature", "you shouldn't use it because you can't find Ruby coders", etc., etc.

I find it honestly hilarious to hear it suggested that one of the early high-volume Rails shops should stick with "safe" language choices because they won't be able to hire enough developers to work with a new language.

I've read through the sources to Kestrel, and found them quite easy to grok. If anything, most real-world Scala code seems to be essentially very clean Java, with much of the syntactic cruft (redundant type declarations, inner classes, 'public static final' declarations, etc.) removed. Any experienced, effective developer is going to spend far more time getting to know your in-house APIs and coding style than they are the language.
Yeah thats what I meant, they could have went with some electronics company building phones, TVs, etc instead of salsa, and nothing would really need to be changed. I dunno salsa is just boring.
Saying that someone is trying to extort money from you and that their behaviour is "mafia-like" and tantamount to "blackmail" is very heavy-handed criticism, especially on a public forum. It is intended to injure, if only their reputation.

Publicly whipping someone in a town square will cause more mental anguish than physical pain. It's intended to embarrass that person, as well as scare anyone else watching. Please point out where I have gone wrong with the drawing of these parallels.
HN is getting lamer than ever with this downmodding of anybody who doesn't conform. It would be interesting to know the overlap of opinion-based down-modders and Reddit/Digg bashers.
Actually average life expectancy has consistently been inaccurate. If half our species died at birth, and the rest lived until 80 years old, our average life expectancy is only 40 years. Anyone notice a problem?

The length of a human life has usually rested around 70 years, which is actually seen in nomadic people today. Their life expectancy can be appallingly low due to their environment, it doesn't mean everyone's dropped dead by the time they're 40, it means lots were lost at a ridiculously young age.

During the industrial revolution 75% of children in London died before 5 years old. This meant that life expectancy was only around 30 years, despite the fact that poor houses were predominantly old people. The UK instituted the Old Age Pension Act in 1909 to deal with this problem and the claiming age was 70 years, which means people were expected to work until this age. If people were really only living until 30, why would people have invented a pension for 70 year olds?

IIRC the breakdown for the Paleolithic was, at birth life expectancy was ~33 years. If you lived to 15, your life expectancy was, on average, actually between 39-55 and I wouldn't doubt that if you lived to 55 you had a solid chance of making it to 70.

During the Bronze Age life expectancy at birth was 18, this was due partly to wars, population densities and essentially the plagues caused by such (this is the essential reason why Europeans had diseases that wiped out 70-80% of Native Americans). At about 1000 BC we hit our average of 30 years again, although for Greece and Rome it actually ended up being 20-30 years because again of wars and some diseases (although by this time we were more disease resistant).

Another example of this is the USA. People in the US have a lower life expectancy than many other countries, which aside from my problem with privatized health care (which is primarily a quality of life, not a quantity of life issue), is mainly due to the US having an appalling infant mortality rate... however if you actually survive infancy you're good to go.

IMO stating life expectancies is very misleading, due to the fact that it creates the impression of a Logan's Run environment where you're dead at 21. Only in modern society has life expectancy surpassed 40 years, but Genghis Khan died at age 72 (IIRC) due to an infection after his horse bucked returning from war. Apparently this was no mean feat in the Mongol armies as many of his generals were as old, if not older, than him as they'd been with him since the beginning.
But I think that the problem is that it was harder than it seemed, but that doesn't mean it's not doable.

Developers are always optimistic when estimating how long it will take to do something they (we) want to do.
I'm sure you will find some crack dealers that are real sweethearts as well.

No one involved in any form of usury has high moral standards. The cash loan business only exists because state politician were paid off to build loopholes around existing state usury laws. Who do you think pays off these politicians to create such a loophole? Someone that doesn't make money off the industry? I seriously doubt it.
Who Should be in Charge of U.S. Cybersecurity?
I'm defending their choice to move to Scala (a "niche" language) after running into performance problems with Ruby. I don't think it's a straw man to mention Java and C++ as representative of the non-niche languages that perform significantly better than Ruby. There are plenty of faster language platforms they can move to, but which ones offer a powerful and elegant language? As far as I know, you either pick a "niche" language that few developers know or you pick an old and clunky language like Java or C++. In this case, you have to weigh the ability to hire new people, which would be better for the more popular languages, against the ability to retain your existing Ruby developers, which is better with an elegant, powerful language.
If you know and like Tomcat, may I suggest you have a look at JRuby deployment for Rails apps? With recent versions of jruby-rack and the warbler gem, it's as simple as listing your dependencies in a config file, and then running 'rake war' from the top level of a Rails app to get a shiny new WAR file ready to drop into your Tomcat deployment directory.
The use case is handling 50 concurrent connections ...
How many people have dedicated hadoop clusters available to them in the first place? The point of the pattern, framework and product is to scale linearly and on demand, not to do the fastest computation per-machine possible.

As to IO - HDFS is going to divvy up the workload from S3 when the cluster bootstraps. After that the data will be local to each compute node, so they won't be getting kicked in the face. There is no reason to suppose that S3 replaces HDFS - its just used to load your data in.

Also, one would hope that their Hadoop setup IS rack aware - why shouldn't it be?
Brings to mind the airline version:
http://www.seatguru.com/
Ok ok, I should have provided more substance in my post. Thank you for spanking me HN. :)

Certainly I don't feel 37signals customer support is broken, but there is a point in that quote: Would it have really been that difficult for Jason to have his people monitor a customer-created Get Satisfaction page too?

Honestly, he's probably wasted at least two full work days writing these posts — which could have been direct-from-the-founder responses to a lot of customers while he picked up the phone and had a talk with GS team. After all, all the link-backs from GS to 37signals is only going to boost their SEO overtime. There are a lot of benefits to GS that 37signals ignores in favor of an ego-driven tantrum that has turned me off yet again.

For the record, I am both a 37signals and GS customer.
The simplest bookmarking service in the Web
Manage your bookmarks the most convenient and easiest way.
It's absolutly Free. There is nothing superfluous, only that you really need. Try it and make sure in this!
I commuted on the subway in NYC for a few months last summer, but I never noticed a hierarchy beyond "sitting > standing" -- probably just because I wasn't paying enough attention and wasn't there long enough. I'm going to watch for this when I go back.
Language Shootout: Scala vs. Ruby
The item similarity code example that uses the api has 4 python streaming mapreduce steps with custom reducers:

http://elasticmapreduce.s3.amazonaws.com/samples/similarity/...

AWS just put a pointer to the code bucket on s3, I'll add a link to a zip file with the full source including a script for converting the raw netflix data to something more friendly to Hadoop streaming.

The article is here:

http://developer.amazonwebservices.com/connect/entry.jspa?ex...

Check out the resources link at the end, the Cloudera Hadoop Hacks site has a few good intros to custom reduers in java, ruby, and python.
Again: not an anti-Scala person, just turning the economics of this around in my head. That said:

Ruby is our house language. We've never hired anyone that came in the door knowing Ruby. We have a very high bar for hiring (typical first project: "write a programmable symbolic debugger for closed-source binaries on platform Y"). It takes people at least a week to become facile with Ruby --- a famously easy language to learn. Reading isn't writing.

If you have a house style, easily figure another week to pick up the right idioms, so that your code survives in the repository and isn't just refactored out.
I was considering using Get Satisfaction until I found out they would create pages for companies that hadn't asked for them.

Even if the service is good, I don't support companies that don't match my values.
In the article, the image is cropped so you can't see "Position 1". Here's the full image: http://meish.org/wp-content/uploads/2007/08/tubecarriage4.pn...
Once you make your goals clear and explicit, much of the argument over how to educate becomes moot or even silly.

It's interesting how many controversies evaporate if you stop assuming that the labels on things are accurate. For example, if instead of the "education" system we spoke of the "child processing system", the world might make more sense. And instead of "health care" we could talk about "disease management". And so on!
"you'll literally _have_to_ to different media serving and load balancing if you plan to grow whatever_this_is."

I guess my point is that it's a mod_php issue and I may not need nginx. My plan is to try a threaded apache without mod_php and use mod_fcgi for php. If that doesn't work then try a threaded apache with mod_php and see what happens.

Another solution is find another provider that offers more memory and similar service for the same price. But it will be hard to match slicehost's service I think.