skysight-inc/HackerNews huggingface dataset:
Write a crawler: Make it nice, so it doesn't hammer sites, convert relative urls to absolute, pull out relative data from html, cycle urls back through crawler for more crawling. Make sure your ISP doesn't have a cap on your monthly bandwidth (i.e. don't use a residential-scale ISP). Take into account pages that are dupes of other pages, so compare MD5s of all content to detect urls that generate duplicate content to what you've already pulled and ignore those. Integrate sitemap detection for smarter crawls.

Index the data: Figure out a way to index the massive volume of data that's not a live query system (lucene starts to gag at queries > 50GB or so), so generate all possible results for all possible queries in a database and update those results occasionally as you pull in more data. When someone queries for a search string, pull up the results from a pre-generated query from a database, don't do a live search of all of your terrabytes of data, or the query will take days.

Generate interesting results: find a niche. Don't plan to take on Google, Bing & Yahoo on a personal scale. People put together good engines, but target main sites, and front pages, or a shallow-depth crawl. Don't plan on indexing every forum and every blog on the internet.

I've been impressed by this guy's search engine: http://gigablast.com/

In short, unless you've got a TON of money, machinery, people and time, don't try to compete with Google. Find a niche like shopping search or movie search, or be human-powered like Mahalo. Google's got a dedicated computer for every possible search query out there or close to it, plus a team of 500,000 Chinese people making sure that popular results are relevant (Google does human-validated results for many of the most popular queries, not like an error message query).
Yes, D shows a great deal of promise. Maybe I'm short-sighted, but at the moment it's the only language I can see being a viable replacement for what C++ does well. D was invented by Walter Bright, after he had written a C++ compiler (and somehow kept his marbles). I think it was his take on a better C++. Quite a few other clever people have jumped on board to help with the design since.
It's official. Textpattern CMS 4.20 is available.
Nobody can beat this masterpiece of classics:

int main(void) {
printf("Hello, world!\n");
return 0;
}
A frozen one ? :)
More importantly, it ought to have been part of the required reading regimen for the authors of said study.
Don't fall into the Turing equivalence tarpit. It's indeed possible to rewrite recursive code in terms of iteration and vice versa, but the most natural expression of many algorithms is recursion. If you can convince the compiler and/or runtime to manage your stack for you, you can concentrate on the interesting parts of the algorithm.

It's like the difference between using objects in Smalltalk and writing your own object system in C.
Startup idea: a Wikipedia for small businesses?
http://news.ycombinator.com/item?id=790361

Compare the comment ratio here ...
Key Developer Position 
Hey guys,

A year ago three friends came up with a great idea and have spent every waking hour of every day developing it. All while keeping the concept of the site a secret; we quit our full time jobs, sacrificed time away from our families, and are willing to do whatever it takes to make sure this site becomes a reality. We are looking for the missing piece of our team and need an experienced developer, willing to embrace the role as our technical leader. You will be welcomed with open arms, and will be given the greatest thing we have to offer, our idea, this project. You will be an equal member of our team and being such will make you a partial owner of the company. If you or someone you know is interested in getting more details, please email me jason.helm55@gmail.com

Jason Helm

P.S. LAMP environment
I think we're 80% of the way there
This just goes on to prove that "The way to a man's heart is through his stomach" :)

As a side note I got to say that I would've stopped going to the lunch with a person who'd kept insisting on paying all the time. If not for any other reason, but for the fact that I just feel plain uncomfortable with this kind of arrangement as it clearly puts me in debt to that person. Paying back with a friendship feels wrong and "He is that kind of person" is not an excuse enough.
Leo Laporte is going around shooting people using the word twit? Wow, he sure lucked out with the advent of Twitter and its lexicon.

Edit: Leo Laporte actually has rights to the trademark “TWiT” in relation to audio-visual performances. So, anything that’s outside of the realm of video are not a valid target for him.
I agree. But then, I might be a little biased.

Ben Dilts
Founder, LucidChart
Google Analytics Blog: An API Integration To Measure Significant Change
try benchmarking with apachebench

ab -n 10000 -c 10 http://127.0.0.1/

or install siege (or some other tool).

Use passenger. If you really need speed use rails metal to write your response, rails metal can certainly handle 150 requests per second on a single instance. But should one client be laggy, it will take down the whole server :)
The names are silly, and Apple isn't leveraging blocks nearly as much as they could be. The very first thing I did is add an option monad, plus additional collection methods (including the nice and short map and filter) via categories.

My only complaint is that The [[]] messaging syntax gets way too verbose when chaining together sequence comprehensions.
Use the Apache Benchmark "ab" program, and measure the number of requests per second for your application.

It is number of requests per second that should interest you.

If you do not have ab installed, try:

sudo apt-get install apache2-utils

then:

man ab
Chimps caught in meat-for-sex scandal
no way you can get a rails app to send 6k requests per second on a single machine. A full page cached page, yes, but no way on rails.
I would rethink things fundamentally. That requires a lot of abstract thinking about ontology, but that's how the past revolutions in search began.

Yahoo treated the web like a phone book or directory. Altavista relied on self-categorization efforts in meta tags. Google treated links as votes.

You need to come up with a new (and hopefully better) way of thinking about what the web is. Come up with an inventive way of thinking about what linking means, what DOM structure means, how to think about non-standard types of content, and so on.

If you start with the same premises about the web that Google started with in 1997, you'll never surpass them much less carve about anything more than a toy niche.
Using a pattern is writing more or less the same thing over-and-over, with variations. That isn't necessarily bad or good in and of itself.

Code that has repetition will not be as compact as code in which every symbol is new, original. The later will require much more thought to both write and understand and so is not necessarily desirable.

If we use the term pattern loosely, any programming involves a series of nested patterns with variations. That is how it should be. That is essentially how good writing works as well.

So, actually, patterns and repetition are good if they make the original parts clear. So, as far as language goes, it seems like a language which allows you to include what you need to include and exclude what is irrelevant would be desirable.

Any program that expresses your ideas in the most compact fashion possible will incomprehensible to anyone else and incomprehensible to you next week.
Use http://selectorgadget.com to help make the selectors, too.
Citation needed. Here, let me Google "prison recidivism non-violent" for you -- a 1997 study which suggests that only about 15% of nonviolent offenders are returned to prison within three years for violent offenses [PDF]:

http://www.ojp.usdoj.gov/bjs/pub/pdf/pnoesp.pdf
I can relate to feeling guilty when monies tight and you eat out. You sound like you do it for all the right reasons, good karma man.
And of course anybody who hacked your HN account couldn't possibly have hacked your Twitter account (assuming they are both really u, and u'r the Google Matt Cubbs). :=)
Politicians don't write bills. The law doesn't say they run the Internet. But you're right, when asked, 72-year-old sponsoring Senator John Rockefeller was unable to give a coherent summary of how information security works.
Mac OS X v10.6 provides support for block objects in C, C++, and Objective-C.

Huh? Isn't this a feature of a compiler, not an OS?
We're in the Midst of a Literacy Revolution
In other news, use of Real Player has pushed the date of the Singularity back by 5 years.
I would try to build something like clusty.com, except I would try to identify themes in user search requests and track these for each user over time, using click throughs per user to keep search results on theme.
Turing machines are Turing complete.
In the example of editing wikipedia it mentions:
"This imagined paradox is caused when the percentage is provided but not the ratio."

We see this so many areas. China's GDP is growing 12% every year and the US has grown just 2% type of arguments.

I haven't been able to improve on my yearly salary raise of 300% in the last 15 years when I went from 5$/hr to 15$/hr
I'm all for modernism but their font-size is too damn small. One of the very, very few times I've ever had to use font zoom.

Anyhow, after a few hiccups installing SL on my Mac Pro today (MacBook was flawless) — I can definitely notice some amazing improvements under the hood. It's really shaping up to be a pretty stellar OS.
I think this is a good idea for two reasons:

1. It's cheaper for businesses to start their own Business-pedia article than to pay for the development and hosting of their own website.

2. It may stand a chance of keeping a lot of the business spam postings out of the real Wikipedia, making it less work for admins to delete non-notable articles.
Murdoch attack on 'dominant' BBC
Clamato: A Smalltalk Dialect for Javascript
I just finished upgrading my desktop and laptop, and I have to admit Apple wasn't kidding about this being a snappier release -- almost feels like I'm working on a Linux box again, in terms of snappiness.

I'll be interesting to see how the underlaying changes impact me, though, as everything I'm doing right now is on the JVM...
0.

So, when you look at something like this, I think you have a choice to make: you can put on the tinfoil hat and concede any relevance you might have to the discussion, or you can recognize the real weaknesses of this bill and the process that is producing it and comment rationally on whether the government is capable of legislating improved security for its own systems when those systems are by necessity constructed from COTS pieces created by unregulated technology companies.

1.

The thing that everyone is going to talk about here is the definition of a "nongovernmental critical information system". The term is defined broadly in this bill: the President designates them. But I think the intent here is pretty clear: private industry operates the E911 system, the cellular phone network, all our financial exchanges, and a good chunk of the power grid.

Most of these systems are in some way connected to public networks: for instance, a generic Cisco VPN vulnerability could get you a telco, which would get you to private leased lines. Before you shrug that off, read up on "Operation Sun Devil", and the state of the art of teenage hacking in 1991.

I think it's hard to say that the NSC, given a secret update that, say, all Cisco IOS versions were vulnerable to a pre-auth generic TCP remote code execution vulnerability, should NOT have the capability to ensure that exposed power grid systems were locked down.

On the other hand, I agree that the wording is overbroad. I'm interested in what HN people think good wording would be for what would qualify as a nongovernmental critical information system.

2.

What sucks about this situation is this:

The broad intention of this bill, to improve "cybersecurity" across all of US industry and government systems, is going to fail. You can't legislate it.

But narrowly, this bill is going to define what it means to work with systems at DOD, law enforcement, and energy. And I don't care that much, except that the existing processes in these areas are arcane, arbitrary, and exclude a lot of talent and ideas. Relative to financial services, DOD does not have excellent security.

But since everyone is going to get ratholed in the meaningless broad intention of the bill, nobody's going to get into the nitty-gritty of secure software accreditation, procurements, certification of personnel, funding for technology and technology grants, and so on. Those topics are boring, but they're more important than whether you can outlaw insecurity.
I.e. they are the ones that have to back the decision legally.
And CALEA narrowly impacted on the tiny minority of systems that directly worked with voice communications. The tinfoil hat interpretation of this capability would cost hundreds of billions of dollars.
like you can compete with cuil
The Facebook Exodus
Think its already been done to a certain point. http://www.crunchbase.com.
This is one of his that I often reread, as there is a lot in it. Thus, easy to miss that particular aspect of the last paragraph.
A minor point, but C# will generate simple getters and setters for you automatically with syntax like "type Property { get; private set; }". As soon as you want to do anything interesting when get or set, however, you're back to playing human compiler.
In reality it is just a discount code

.. which is exactly what I was trying to say.

You want to run your own private Exchange server!?
None of those examples actually demonstrate anything useful (except maybe the last one which I don't understand).
Audio White Paper - Global Distributed Service In The Cloud With F5 And VMWare
FeedBooks has less than 3000 'official' books (mostly public-domain titles) and around 600 user-submitted books; over a million Google Books can be read on the Sony Reader. However, the OCR quality of ePub content from Google is spotty. I believe all books on FeedBooks have been proofread.
So I am wondering how we got to the opinion of "because one of their fundamental tenets is that patterns cannot be represented in code". Seems quite odd.
Every millisecond counts | Facebook engineering discuses website speed
I wonder how many of those hulu users were on Time Warner's internet service?
You often get colonoscopies at your local McDs do you?
Have you read the first sentence under the title?

This chapter guides you through the creation of a simple OpenCL application that performs calculations on a data set.
Does this fact not strike you as sad? According to http://www.time.com/time/magazine/article/0,9171,1653653,00.... if we are generous in our estimate of how much is spent on gifted programs, we spend 10 times as much to educate the mentally retarded as the gifted. Yet which group poses greater potential returns for society?

Gifted kids should not have trouble with school. Yet they drop out at the same rate as everyone else. And frequently are pressured to under perform. I consider this a tragedy.
Obviously the real concern is that "such machines that do the sort of jobs people find dull, dirty or dangerous." will get angry and unionize.

We must prevent this or all is lost.
If you're elitist, it's in a good way. I agree.
It's true, and the APCs stationed on every block of my neighborhood are damned annoying, as at the Lt. Col's constant requests to quarter soldiers in my house. I'd be fine with it, except they want they beds, and not just the couch. SIC SEMPER TYRANNUS!
Like an old worn-out sweater?
Yes, I know. I just could not resist making the itemization complete.
Google Irvine had an open house two years ago, before the troubles. Similar feel as this one, it seems, and worthwhile for anyone interested.
So you've Upgrade to Snow Leopard? Post Dev Feedback Here
Those developing on the mac generally have lots of added libraries and tools... like Fink or MacPorts, different Python or Ruby versions, etc..

In my case, I don't use the base install of Apache and run my own, with mod_wsgi. I also run my own build of MySQL. Because of things like this, I tend to shy away from all of the automated crap (such as the migration tools, and upgrading a point release rather than a fresh install) and I don't think i'm alone. I upgraded my older Macbook Pro to Leopard by formatting and installing fresh, then manually moving data.

Because Snow Leopard is a smaller release and more of a glorified patch or service-pack if you will, I am wondering if I really need to do this. I'd love to get feedback from all the Mac hackers out there who have upgrade to Snow Leopard. What kind of problems have you noticed, if any?
Oops. Right you are. It was a typo!

Then again, he never said he was using SSH keys, so maybe ...
I'm not sure why you got down voted to 0 for asking a legitimate question. The block objects are part of Grand Central Dispatch. A programmer tags an isolated unit of work as a block. The OS then takes these blocks and schedules them into execution queues. While the compiler adds support for defining the blocks themselves, OS X 10.6 provides purpose to these blocks. Just like you can have a VM which supports threads on an OS that itself is not multi-threaded, the feature would be pretty useless if only the compiler were updated as it has been for 10.6
I haven't bothered checking Politico since the 2008 U.S. election. They were horrifyingly fixated on covering the horse race, with a big spot on the front page dedicated to who "won" the day. Iirc, they also build the partisan split directly into their information architecture and staff, another big no-no as far as I'm concerned. This sort of thing is what's ruining our politics.

In theory, I would be interested in a site with a tight focus on low-level politics, but Politico ain't it.
On the other hand if morbidity gets compressed [1] we may need fewer caretakers.

[1] People suffer from age-related illness for a shorter time. They stay healthier for longer. Some statistics point in this direction.
Well, thank you for your opinion.
I especially enjoyed Stephen Adams' talk. The other two were very interesting, but much broader overviews of Google technologies. Adams really drilled down into the nitty gritty tricks he used to decrease the size of Chrome updates.

In contrast, a large portion of Chee Chew's talk was basically a Google engineer walking through how to attach documents and use video chat in Gmail.

I really hope they do more of these talks. As a UW student, it's thrilling to be able to hop on a bus and go rub shoulders with Googlers.
The Anatomy of Hadoop I/O Pipeline (Hadoop and Distributed Computing at Yahoo)
Lies! Facebook will never die, or be eclipsed like Myspace, or Friendster or Xanga or Orkut or.................
Nice work. I just put two of our sites, getmiro.com and miroguide.com in, and and excited to see what it tells us.

Is this really just an internal project in the way the blog post positions it, or do you have business ambitions for it? The phrase, "try it for free" might imply that you're planning to charge for it at some point.
I can't decide if I want to yell at you for writing "shud" or praise you for conserving bandwidth while preserving all information.
Consider trying readability :: http://lab.arc90.com/experiments/readability/
Oh my, this rocks. What would be even cooler is a library to use these functions. That would beat me to it, since I was thinking about a general data interface for websites too. If I could implement your effort into my code (PHP) that would be so awesome. Anyhow, nice idea though.
Why not spreadsheets? The end-user would not be bothered with this stuff.
That is confusing work ethics with implementation strategy.

One could argue that it is more ethical -- your duty to your investors, employees, employers, customers, etc. -- to focus on a more practical solution ("80%") than to seek perfection to satisfy your own personal standards. But if your responsibility (whatever it may be) dictates near perfection, then clearly that is what needs to be done. (Ex: 80% safe mass transportation system?)
OS X includes its own compilers.
One step at a time :)

What you suggest would benefit from the performance/feedback from what the OP suggests.

This is basically what I am interested on working on currently , using "clues" gleaned from what the user is doing (a la RescueTime) to reorganize/reformulate a user's search query.
http://github.com/fizx/parsley/tree/master is a C library which represents the core of the parsing functionality. A PHP binding is quite possible, and indeed, there already are bindings for Python and Ruby. In fact, I might just go look at how PHP bindings are done in general...
During college I designed a processor that could do that. It would read like this (assuming A, B and C were constants representing memory addresses pointing to integers):

$300: $(A) $(B) ADD $(C) MUL

And the result would be at the top of the stack.

Not all assembly languages are created equal. ;-)
This is one reason why I like the old NeXT term "system software" in favor of "OS". (The former is all inclusive.)

Blocks are a language feature, and do need compiler support. The version of the compiler that you need to use them in GCD comes with v10.6 system software.
I think you just need a couple years of programming under your belt. Books won't really help with the proper grokage.
Yup, if you're anywhere near Mountain View. :)
Don't you bring the dinosaurs into this.
Well...the thing is that I normally DO pay for my male friends.

If I call them up and say ask them to go out, they're usually all broke (mostly students). I'll pay because, to me, its worth the extra few bucks to have somebody to drink with...

(or are you saying that people are just being irrational? Your last sentence makes me believe so.)
I don't think I've ever used print that way. If I need to print more than one thing, it's clearer to use regular string formatting. Thanks for clarifying that though, interesting.
It is easy to confuse activity with productivity.

For example, spending four hours on non results-producing activities like paperwork instead of sales.
Caml trading
Dear blog (on anonymous meanness)
I'm sorry, but is this really just a handful of anecdotes about individual people who have stopped using Facebook? And this proves what, exactly? Every successful site has had people along the way that stopped using it for one reason or another.

Come back when you have some data on either the number of people leaving or the impact of their absence.
Crime expert backs calls for 'licence to compute'
Carbon Emacs works fine. Ruby works fine, although some gems that have native extensions needed reinstalling.

X11 seems to be totally broken though. The Gimp and Crossover both don't work at all.
Wolfram Alpha to open data feeds
I 2nd you. The only thing I've consistently been able to use Posterous or Tumblr is for my photo blog. The iPhone to blog feature is just awesome.

Yet, for anything else, even for the simplest of blog entries, there comes an occasion you HAVE to post something a little advanced. With both Posterous and Tumblr you end up hitting a wall. I can understand their perspective too: they want to keep it dead simple. But I'm also hoping one day that they can build an obscure page somewhere with all the advance features generally found in a typical wordpress blog.
Maybe someday your super sophisticated fraud detection software could be made to check and see if someone got slashdotted.

I understand this would involve knowing the content of billions of websites and figuring out which ones link to each other.

Who could possibly be expected to keep track of all that information?

So keep your pants on everyone. The scope of this problem is insurmountable especially since there are only thirty days in the month (or six) before you get paid for your clicks and impressions to discover fraud.
Even if you were hitting on her, for her to ignore you, it probably wasn't your problem.