skysight-inc/HackerNews huggingface dataset:
Tynt: What's being copied from your website right now?
Building and Scaling a Startup on Rails: Things We Learned the Hard Way (by Posterous S08)
It looks like his blogging engine is removing some of the code. A few lines are not valid Haskell, and GHC indeed complains about them.

But anyway, a "working" version is not much longer than this. Haskell's threads (or rather, MVars and Chans combined with purity) are really a joy to work with. If you haven't tried writing a small app like this, you should.
Having BSD-derived userspace (like ftp.exe and probably more) does not imply having BSD derived TCP stack. And the internal architecture of windows networking is significantly different from any unix (one could probably also call it "broken") so I find it hard to believe that there are major parts of BSD-derived code in there.

Vista is probably first version of Windows where Microsoft actually advertises the POSIX part, so it's there and probably it's here to stay.
This might be a good post to ask this particular question: I am very unfamiliar with the nuts and bolts of how the Internet actually works. I understand DNS resolution, ports, HTTP protocol - all of the higher-level abstracted stuff that you need to know to program within the environment. However - route-maps? BGP? ASN? Whatever else is under the hood that makes it tick - no idea. What's a good book to learn all this stuff?
http://vxsbill.com/
>> If some Wall Streeters move on, perhaps other, more innovative professionals will take their place and build something a bit more durable..."You might get down to the core group that actually likes finance, and lose people who are coming just for the money,"..

Is this not precisely what happened when the last big bubble burst, the dot com era? People who had no real business being in technology or had no passion about programming left the industry leaving behind the ones that actually cared and were much more motivated and driven leading to greater innovation.

Taking a page from history, I say this is a good thing for Wall Street in the long run.
android market JUST started paid apps. he shouldn't be drawing any conclusions right now.

once we start seeing real retail developer products on android we'll see competition
Don't insult SGML!! SGML is actually quite nice. With omitable and case-insensitive tags, its pretty easy to work with if it is used in its intended domain: document markup.

YAML is great for exchanging data but SGML and YAML target different problems.
First, Barack Obama is not Santa Clause. I'm really indifferent towards the man at this point (don't really like him, liked McCain even less) but the one thing that really bothers me is this idea that we should just all sit down and let President Obama do everything for us.

If the author here really cares about what he's saying he should write a lesson plan that teaches programming and send it to schools. Put up a website that asks for support of his lesson plan. Solicit donations for computers in classrooms. Overall just get off his own @$$ and try to make it happen.

Finally, I don't see what DRM and banning Cell Phones has anything to do with teaching technology to kids but I can say his first point is largely wrong. I grew up in public funded housing (aka the projects) and even we had computers in our classrooms. If someone can find me the high school that is completely without computers in the U.S. I'd like to see it becasue I think it's just a myth.
The article does not use the buffer overflows to conclude anything about the algorithm -- it concludes that writing software that manages memory correctly is very difficult. This is why you should all stop using C, and people should stop telling other people to use C.

(Anything I would use C for, I now use Haskell for. Everything runs as fast, and the language and libraries are much better.)
I may not be directly participating in this conversation, but I don't appreciate your invocation of a false dilemma. The situation is more complex than you are implying, and the whole "Do you approve of affirmative action or are you a racist?" line has no place here on HN.

And yes, it is what you are saying. I'm not fooled.
That's what I plan to do, yes.
Norwegian Minister Wants to Legalize File-Sharing
How is this data being collected? (I looked at the "pinch media" website, and it looks like they are some sort of ad network. So I doubt their analytics cover the apps that don't request ads from their servers, making this data very close to useless.)
If the parser allows the programmer to be sloppy they may not know they're making a mistake. If they try to use another parser that doesn't handle their mistake the same way, it will break.

Worse, in the case of browsers, each browser vendor wants to be sure every major site works in their browser, so they have to go out of their way to program in all these special cases. See tolmasky's comment above.
From their about page:
Tracer inserts a java script tag into the html code of your website to non-invasively track users interactions with your website content. The only change the user sees is when copied content is pasted into an email, blog or website, we automatically add a link back to the originating site at the end of the content.

What happens if the user just deletes the link, as I have with the above quote?
Feds Propose Storing Internet User Data for 2 Years
> Do you approve of affirmative action or are you a racist?

I don't believe in affirmative action, and my advice of "get used to an unfair world" is consistent with that view. Thanks for putting words into my mouth though.

> I'm not fooled.

Fuck off, chuckleface. Furthermore, don't tell me what has "place" anywhere.
99% Flash Player Penetration – Too Good to be True?
What dropbox is about? Screencast here: https://www.getdropbox.com/screencast
the Gale-Shapley algorithm requires an authority (like the medical school establishment) to make it work. in real life, if you want to get married, you face a much more amorphous situation: a series of prospects, and with each a decision, "are they good enough"? i can't find it anymore, but iirc the answer was, to estimate the length of the game (say 20 years on the marriage market) and then just date for the first 1/e fraction of this time. and then say "yes" to the next person as good or better than the best you could have had during your trial/dating period. can anyone name the theorem or is this apocryphal?
Steven Rudich has a good lecture on this: http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15251/di...

http://www.discretemath.com
Lazyweb Request: Why would PHP be faster than HTML?
Let's keep a sense of perspective here. The pay limits only apply to companies that accept government aid. He who pays the piper calls the tune; there's nothing wrong with that.

You also need to understand that these policies are set based on factors beyond pure economics. The various financial sector bailouts are hugely unpopular among US citizens. If ordinary people don't see bankers sharing some of the pain then we could actually see riots in the streets at some point. Look at what has already happened in some parts of Europe.
All Thawed Out - Debt Sales Now at Record Levels
There's a lot gamesmanship that goes on. For some sbir grants you can subcontract out 50% of the work.

So you get a grant to work on some subcomponent of one of the major defense contractors (Raytheon, Lockheed, etc). Then you turn around and subcontract that piece right back to the defense contractor, up to 50%. Then you take the remaining work and give it to ex-Defense employees who become your employees for a bit.

So the small business wins b/c they get some % profit plus can do work at a high dollar amount per hour. The big defense contractor wins b/c they get essentially free R&D work on their subcomponent.
Fearing 'Cyber Katrina,' Obama Candidate for Cyber Czar Urges a ''FEMA for the Internet'
The real problem is a systematic inability to tell good managers from bad managers - the people who are leveraging up bull runs, or betting on 30 numbers on the roulette wheel, versus those actually generating excess returns. So really, most of the smart people in finance, especially quants, are just there to signal intelligence rather than having testable skills. This is the waste. Otherwise we would need much fewer money managers.
My dad always wanted me to be a finance geek

Funny - my family had similar ideas. I went off and did computing instead, landed in finance by ridiculous accident and only realised years later that I'd wound up doing something quite close to those plans.

When 'managing' is valued higher than creation, we have problems. Big problems.

Managing is creation. If you find a creative way to do a neat hedge that solves risk problems for a bunch of customers who have farms and manufacturing plants and workforces distributed over different currencies you've created value because they can spend their time doing what they're good at instead of stressing about how they're going to meet pay in the Swiss division because the Euro has strengthened. This aids open trade which itself supports specialisation. Financial flexibility is ridiculously important.

A case in point: when I moved to London I needed to pay a deposit on a flat. Unfortunately I had no bank account set up, and no pounds to my name. When my father had the same issue forty years ago he had to go to great inconvenience and cost arranging an unsecured short-term loan to cover himself. He was aided because of family links between the people he borrowed from and friends in his own country. Whereas I just put the transaction on my credit card and the problem went away. Less management fee, no credit checks (on that part of the transaction), zero hassle. When I was contracting over here years ago I didn't even bother getting local currency, I just lived off the card. Internationally recognised short term credit isn't even a status symbol these days - it's a commodity item.

There is a popular idea that unless something is physical and produced with sweaty work it doesn't have value. Manufacturing is considered to be inherently better than office work. Farming is the holiest pursuit of all. It's all a fallacy. See Galbraith, _The Age of Uncertainty_ for a more thorough treatment.
Its not anti-banker populism. Its the bailout mentality that these bankers have that's disconcerting. A startup that fails disappears the way its supposed to, but when a big bank fails we all pitch in the bail them out. That's the problem.
My Ph.D. advisor wanted the same thing for me. This was the mid-90s. Perhaps I should have listened.
Another half-baked notion: broadband + subscription tv + set-top box
Sell a subscription service and a simple set-top box (media PC/PVR) package. New customers will create their new subscription account using the set-top box, which will request a strong UID from the server.

From the set-top box, subscribers can select from current or past television shows and movies. Let's say that a movie deducts $2.00 from their account, and a TV show deducts $.50. Subscribers pay into their account as their balance dwindles.

The data is encrypted using the set-top box's UID. Yeah, DRM sucks, but you'd have to do something to get the studios on board.

The downloaded content is ad-free, which the subscribers would appreciate (a la pay-er-view).

Content providers get a large chunk of the revenues, plus actual (aggregate) viewing data that they can actually use, instead of the garbage ratings system currently in use.

Just occurred to me as some relatives were talking about a movie on pay-per-view.

What do you all think?
Would it be a good idea to combine link placement, reporting back to the server on every highlight and searching for the content later be a good combination?
The problem of finance isn't that its results are intangible, they are tangible. So I agree with you on this point. Its that it's an industry with strong political ties, like automakers, that is simply not allowed to fail and is propped up unnaturally by taxpayers. Failure must be part of the way we do business, bailouts cannot.
I'll take it into consideration.
I am loving The DailyBeast more than Huffington these days
Super clocks: More accurate than time itself
Pretty good run down. Actually, a very good run down.

Sphinx is cool, but there's no reason to be afraid of solr/lucene. It takes _very_ little java knowledge to get it up and running and it's very very, really, crazy fast. Like, hundreds of thousands of searches a day on millions of documents and it's totally stable. knocks on wood

Also, Passenger is indeed better than all this mongrel + god + tweaks they're talking about it. AboutUs.org (my employeer) is the largest site on Passenger (that we can find) and we've had 2 actual crashes in the last 6 months, and they were fixed with rebooting Apache.
For those who don't think it's possible to argue about such technical subjects (with a threshold for expertise) as JavaScript, take a look at any recent conversation involving mathematics and the infamous JSH on the sci.math USENET group. (I'm not posting a link, but just Google "sci.math".)

JSH is an amateur pseudo-mathematician who believes (falsely) that he has solved numerous difficult problems in mathematics...he has his own proof of Fermat's Last Theorem, he has (most recently) proposed a (wrong) solution to the integer factorization problem, he has attempted to solve the Traveling Salesman problem, etc.

My point is not that JSH is not very good at math, but rather that there are people who hate this guy more than Hitler. People waste hours bashing him, trying to diagnose psychiatric conditions that they believe he has (the consensus seems to be "narcissistic personality disorder"), and generally tearing this guy to shreds. Over math.

Here's a brief excerpt from a recent conversation about JSH's factoring algorithm:

"Do you realize you are full of s---? You have nothing! You can't demonstrate remarkable claims on a worthwhile problem. That is [sic] classi[c] snake oil salesman, charlatan and cheat. It speaks of your narcissism! Get your head out of your a--! You have zero!"

Anyway, does anyone really think that mathematics is a part of these people's identities? I found the essay thought-provoking as well...but anything can be controversial. The people involved in this thread DO have expertise, and they definitively know that JSH is wrong. This doesn't stop them from spending hours hashing out their arguments. Why can't they just ignore him?

I think, quite honestly, that some people just find it fun/stimulating to argue with and insult each other.
Thanks for the suggestions! Will probably explore Solr down the road. I've written search on top of Lucene before, just didn't have time to work too hard on getting search going in between all the other features we're trying to push out.

Passenger I have heard great things -- been meaning to try since it does fix the mongrel queue problem.
How Government Can Stop Killing Innovation and Start Encouraging It
Blast, it was fine just two hours ago.

http://74.125.47.132/search?q=cache:iqMjX8uGgroJ:www.ailab.s...

Google cache, to whet your appetite...
the first tcp/ip illustrated volume provides a very solid foundation for the low-level protocols, though there is probably some newer stuff and best-practices not covered in it (it was published in 94).

i've never read any books on the stuff, but i used to work at an isp and learned about larger-scale routing concepts just by gleaning it from cow-orkers. unless you're going to work at an isp or do network administration for a large network, you're probably not going to interact with bgp, eigrp, asn's and other concepts much, if any.

that said, you can subscribe to the nanog mailing list (be prepared for lots of boring "can someone from xyz network contact me off-list" threads), play around with the various public route servers available, and if you're really interested, setup a test network with openbgpd or quagga.
thats only a nice algorithm if you can go through the process several times, because i think that its a bound in expectation rather than point wise.
Essential Web-Based tools for JavaScript Developers
I proposed doing this with a graphics person for a from-scratch project that had a very short release timeline. Project got scratched though so I don't know how it would've worked out. But, it seemed logical for me to be coding up in HTML/CSS/JavaScript the various elements that they were mocking up, as they created them.
I do think that fb's current approach makes sense when you want to share photos to people who are not on facebook, and that the marginal return from that being possible outweighs the "information leakage" of the current setup
Great article.

Background/deferred job processing has been immensely painful for us, and I have no idea what the accepted best job queue is. We're still stuck on backgroundrb, which is a nightmare. Nanite looks like overkill, and has too many deps.
Warren Buffett’s bio “The Snowball” and lessons for startups
I don't know jack about Rails, but there is some good general advise here too. I would have liked to see some DB commentary that didn't choose MySQL as a foregone conclusion. I can't think of many instances where I would recommend it in general.
The name of the game at Facebook is building a giant company over the course of a decade. They're looking to get billions using facebook. So, any user that quits facebook now, taking into account that the growth is based on the current number of users, actually means that many more users won't join in the future. If you're planning things around a decade of growth, then you come to think of one user leaving facebook now as 10-100 users leaving facebook by a decade from now.

My guess is the ad system they have now is just to test the waters... get the team learning the ins and outs, improving the system's algorithms and reporting capabilities, etc. Running branded ads is much more of a significant move: they're clearly ads, and not so much some sort of pseudo-relevant text ad. In other words, branded ads are much harder (as a user) to ignore, and would lead to a stifle in growth -- the absolute last thing that Facebook wants.
> It's stunning that someone in Florida can live off of the profits of mines in Madagascar, fast-food restaurants in California, steel mills in South Korea, and insurance companies in Connecticut, all without spending more than half an hour a week reading brokerage statements.

Precisely.

Spending a half hour reading machine-prepared brokerage statements isn't adding value to the process. Said investor has little knowledge of the actual operations of the mines, restaurants, mills, and insurance companies. Their gains have come from general market trends, hidden risk, and luck.

Basically, over-abstraction has encouraged investing based on market conditions rather than company fundamentals.
I've been living in China for years, have Chinese friends and employ Chinese people.
How To Catch The Oscars On The Web
One thing to note is that on a large/complicated site, you can have 10's or 100's of possible engagement loops(what most people call this) which this metric applies to. The article tends to talk about optimizing a single loop. The most common engagement loop, I've seen, is making users confirm their email address.
Interesting concept, but this is not designed to find where your content has been copied onto other sites/blogs. That would be an cool service.
Awesome video, I was coming here to post it and found this.

The thing that bothers me about the whole thing is this:

Why did the borrower have investments that were basically slices of his, and other peoples, mortgages. I mean honestly, 4 or 5 middlemen just so my RRSP (401k) can own a portion of my own house? Sounds like a pretty inefficient investment/tax scheme to me. People should have an emergency fund, but pay off those debts before buying assets, especially assets that are just repackaged debts.
Twiddla: Zillon Percent Better at UI Design
I'm 'stuck' on backgroundrb too. Until now I thought it was the default and best way to run processes in the b/g for a Rails site.

I'm eager to see if anyone else can recommend alternatives that they've had good experiences with.
That service is called Google.
Workling has worked well for us -- Rany Keddo is a phenomenal open source guy.

Tobi's deferred jobs also looked totally solid to me. That one's proven because it drives Shopify. We use his liquid plugin, which is also stellar.

That underlines a big issue with Rails dev even today -- its really hard to know what's good / what works, and what is just some weekend project for someone.
From my experience (and I have used Sphinx, Lucene, Solr and Xapian in production) is that Lucene/Solr have a pretty bad perfomance compared to Sphinx or Xapian.

My Lucene setup began to throw deadlocks and memory exceptions pretty early on. Searching on "deadlock lucene" on Google yields 25000 results. I have later rewritten the system to Xapian where it has run without any problems.

For live updates I would recommend using Xapian. For fairly static indexes I would recommend Sphinx (as it's _really_ fast for both indexing and searching, but it does not support live index updates yet).
What's at Stake at the Butcher Shop
Lose money != destroy value. If you bought something that earned $1 million/year, and the next year it earned $900K, your acquisition may have destroyed $100K in value -- but it's still making nine times that.
Great post with obvious real world experience behind it.

I think it's a fantastic point that you should focus on optimizing your database before you start adding caching. If you can tune your DB with the right indexes and give it enough RAM to fit the whole dataset, you've got a great cache right there!

(BTW, I have been using PostgreSQL on my latest project. I'm impressed so far. It has a much query optimizer and better indexes than MySQL.)

I also like using Solr/acts_as_solr. I haven't used Sphinx but from what I've read about setting it up it sounds incredibly fiddly. Solr, by contrast, is quite simple.
Hey, what do FDA and DWT stand for? (Sorry, that link doesn't work for me.) Thanks!
Every day: Work email, Gmail, Google Reader, HN, and FriendFeed. If I have spare time: Reddit, Friendfeed, Techmeme, Twitter.

I also subscribe to a lot of mailing lists that I have a filter set up to automatically label and skip inbox on, that generates a couple thousand extra emails a day that I skim through and read 1% of.
When people are forced to patronize a service, they generally resent it.
I said that the acquisition loses money, not the company itself. In your example, the acquisition loses $100K, even though the company still makes $900K.
Really? What size of index/documents where you doing searches on?

We're doing live updating, probably close to 1 update per second.
They also provide tracking service completely independent of the ads. http://www.pinchmedia.com/developers/

So while it doesn't cover all apps in the app store, it covers more than just apps that use their advertising platform.
If you find a creative way to do a neat hedge that solves risk problems for a bunch of customers...

I definitely understand where you're coming from but "creative financial instruments" sounds awfully familiar. And hedge funds closed doors at record rates in Dec. 08. Computer models will only get you so far when COMPLEX (read: disguised) financial instruments rely on past history, not future performance.

The idea that one can CREATE LASTING VALUE through exploits, arbitrage, and 'creativity' in bloodlines as important as the financial markets seems reckless.

There is a popular idea that unless something is physical and produced with sweaty work it doesn't have value.

That's not the argument I was making, and I agree with your counter-point.
Quickest way to $30,000 profit after taxes?
I'm a programmer employed full time. I need to find a way to earn 30,000 dollars or so after taxes to pay for some health related debt I incurred last year. What kind of site or online activity would you start in your free time to make that kind of money?