skysight-inc/HackerNews huggingface dataset:
Former Archive employee (& still occasional contract contributor) here. This was one of my 1st questions when joining in 2003!

Some Wayback Machine queries require sorted key traversal: listing all dates for which captures of an URL are available, the discovery of the nearest-date for an URL, and listing all available URLs beginning with a certain URL-prefix.

Maintaining the canonically-ordered master index of (URL, date, pointer) – that 20TB second-level index rajbot mentions – allows both kinds of queries to be satisfied. And once you've got that artifact, the individual capture lookups can be satisfied fairly efficiently, too. (A distributed-hashtable would then be something extra to maintain.)

Also, the queries aren't random: there are hot ranges, and even a single user's session begins with a range query (all dates for an URL), then visits one URL from that same range. Then loading nearest-date captures for the page's inline resources starts hitting similar ranges, as do followup clicks on outlinks or nearby dates. So even though the master index is still on spinning disk (unless there was a recent big SSD upgrade that escaped my notice), the ranges-being-browsed wind up in main-memory caches quite often.

There's no doubt many places that could be improved, but this basic sorted-index model has fit the application well for a long while, avoided too much domain-specific complexity, and been amenable to many generations of index/sharding/replication/internal-API tweaks.

BTW, the Archive is hiring for multiple technical roles, including a senior role developing a next-generation of the Wayback Machine: https://archive.org/about/jobs.php
Here, in the very temple of PG, you speak such blasphemy?

;)
There was a discussion about this a couple of months before: https://news.ycombinator.com/item?id=6461936
PSA/ranty thing: Just because something is archived in the Wayback machine, do not trust that archive.org will keep it there for all time. If you need something, make a local copy! A few months ago TIA changed their stance on robots.txt. They now retroactively honor robot blocks. Now any site can completely vanish from the archives.

Let's say I died tomorrow. My family lets my domain slip. A squatter buys it and throws up a stock landing page, with a robots.txt that forbids spidering. TIA would delete my entire site from their index.

I've already lost a few good sites to this sort of thing. If you depend on a resource, archive it yourself.

edit - Official policy: https://archive.org/about/exclude.php

If I am reading it properly, once blocked they never check later in case of a change of heart? No procedure for getting re-indexed at all?
I'm definitely a fan of Python's strategy. It's much better at detecting bugs early and often. That, and there's a distinction in Python between identity and equality, with the `is` and `==` operators respectively.
Also covered in the NYT about a year ago:
http://www.nytimes.com/2013/06/11/health/new-rapid-malaria-t...

The Gates Foundation has been funding development of a very similar technique by another group since 2010:
http://www.malariaworld.org/sites/default/files/mwjournal/ar...

It appears to be an active and promising area of research as several other groups have published work back to at least 2008:
http://www.nature.com/srep/2013/130312/srep01431/full/srep01... (2013)
http://www.malariajournal.com/content/9/1/207 (2010) http://www.cell.com/biophysj/abstract/S0006-3495(08)70272-8 (2008)
Book Review: Learning with Big Data
I still think all the Quora bashing about their signup interstitials is overstated. They've determined the optimal experience is for people to sign up to see an answer. I know that seems antithetical to an open and "easy-to-use" Internet, but this isn't Wikipedia. Quora is about answers but also about opinions. Their email newsletters, although tweaked recently in a not-so-favorable way, are still amazing. You get a ton of value becoming a user, but unless you sign up (or are semi-forced to sign up) you'd never know that.

While I keep saying I'm not sure if it'll be a good business, I love Quora. They've done something truly unique in the Q&A space and I'm happy they have so much funding that they'll be around for years to come regardless of whether it's a "business".
This is similar to a project I did for my visual/data analytics class

http://twitterhealth.herokuapp.com

Gives you a tweet in the US for you to categorize as being related to veing sick or not. Kind of like crowd sourcing classification of sick tweets. It's pretty interesting just looking through the tweets..
Bleeding edge requires a bleeding edge browser.

If you want cross-device compatibility, use a mature API like Cordova.
For explanation, see this thread: https://news.ycombinator.com/item?id=7540530
Master Currency Counterfeiter Prints Millions, Says ‘Screw You’ to US
You can find intro videos in the documentation: http://ankisrs.net/docs/manual.html#intro-videos
This gist was what helped me finally learn Haskell, after failing on my first two tries.
I would buy this if it had a front facing camera.
The real answer is : it doesn't matter, unless you set it way too high. The economy will adjust until the value of basic income is equivalent to a particular value.

Now of course, what could happen is that you set it too high and we get massive inflation and dollar devaluation for a few months/years.

The reverse should also be true. Because fitting within basic income will massively expand client base for companies, there might actually be devaluation if it's set too low.

I think it's pretty obvious that one cannot choose the level of basic income. You can only do damage by choosing wrong. Given that it's currently zero, if you'd want to do this, start at a very low level (but still high enough to make companies want to provide services at that level), work out the kinks, and increase it over the course of a few years slowly until you see undesirable phenomena, then stop. Ideally, make this a state thing so regional differences can efficiently work.
"coddinghorror" gives it away though :-)
A most user friendly competitor to omsignal?
IYAI Common lisp's quicklisp implements and uses cdb https://github.com/quicklisp/quicklisp-client/blob/master/qu...
For HN audience, probably the most useful plugin is the syntax highlighting plugin [1]. It uses the Pygments Python package so it can handle almost every known programming language, and makes it easy to save snippets into the cards!

[1]: https://ankiweb.net/shared/info/491274358
Note that the above comment is his first comment in 4 years :)
That raises the question - why bother taking those people's money in the first place?
An important factor has been overlooked in this analysis: synergy. Individuals with low income live with other people. When a group of people all start receiving a BI, their spending power will be more then the simple sum b/c they share and buy in bulk. Therefore the BI does not need to be quite as high as one might otherwise think. I have thought about the appropriate level of BI for years. The best I have been able to determine is that it should be pegged to half the minimum wage, i.e. equivalent to a 20 hour job at minimum wage, which works out to about $600/mo. I believe this strikes the right balance between work incentive and poverty prevention and also lies within the potential budget of the government at approx. $1.5 trillion per year.
Can $12k/year actually replace medicare? Medicare is health insurance, so at the very least we would have to have a competitive market for self-health-insurance from which retirees could buy coverage, before Medicare could be scrapped.
Quora is awesome but lack of enough users who can actually deliver "answers". As a student I visit Quora for three things: find out some "secrets" about a company written by former and current employees, find out secrets about cities and history and find out just about anything interesting.

StackExchange has a strong, super strong moderation. You can't just start a random question and not expect someone close it in ten minutes. SE's divisions makes asking question and finding related questions very difficult. Sometimes you don't even know where to ask the right question...

If Quora has the active user base that SE has I will sure people will flood there.

Quora to me is the modern Yahoo Answer but with the hope of quality control.
I also didn't like Kangaroo, it tasted like dry. Perhaps the stuff we get in the USA is fresh.
High School Hackers Meetup in Philly
The YIQ block is a reference algorithm using a simple model of color brightness. It decides if the color is "bright" or "dark" and then makes the text black or white according to that. It's there to demonstrate how the neural net can (if trained right) outperform the simpler model.

But yes, the background colors are random.

I agree it could use a bit of explanation.
Without trying to be too flip, there is a poverty industry who benefit from every dollar that passes through the social service programs. Its naive to think that these bureaucrats NGO officers are going to sit idly by while their livelihood is taken away.
Cutting Medicare to give people BI would leave the people who need Medicare in the lurch, because there's no way in hell that you're going to get the medical treatment you need on that $12k.

Medicare provides for the sick by levying a disproportionate cost:benefit on the healthy. Distributing BI to the healthy and sick alike in lieu of Medicare eliminates that.

You might also notice the "Total Federal Spending" line in that chart you linked is $3.6 trillion...a full $200 billion short of the $3.8b mark.
Medicare expenditures would in no way decrease as a result of the implementation of a BI (unless you plan to pay for BI by causing the deaths of most of our senior population, as they couldn't pay for medical services or medication on $12K). Also, tax revenue would decrease as at least a portion of the population would simply drop out of the workforce.

BI, like socialism and communism, sounds good on paper to a certain set of people. It won't work in the real world. I have always wondered why otherwise intelligent and successful people keep bringing up ideas to give others' money away, and then I recently heard casino mogul Steve Wynn's interesting explanation for it (that they believe that everything should be given to everyone, because everything has been given to them):

https://www.youtube.com/watch?v=tCitAufBvu8
That's cool. But I feel like I'll totally forget about it and lose reference to this (in case I have future interest).

Where's a github mirror?

A google search reveals some entries from the language implementations.

Go: https://github.com/jbarham/go-cdb
Java: https://github.com/malyn/sg-cdb
Haskell: https://github.com/adamsmasher/hs-cdb
Can someone explain the meaning and value of IPv6 (and the hurdles for using it)?
I can live with condensing numbered lists and removing the blank lines between items. Unwrapping plain text paragraphs is worse but not the end of the world. We start to toe the line with unwrapping my blockquotes. You went way past that line when you removed the empty line at the end of the file.
And the people earning more than $1000/hour can pay their $30/hour assistant to do it for them.
the argument is because that translates to tax credits, which basically means the exact system we have right now that everybody wants to scrap. So clearly we can't do that.
Great list of design tips that increase conversions.
> The Federal minimum wage is $7.25/hour and is widely considered to be too low.

That's comparing apples and oranges. The Federal minimum wage is the wage below which it is illegal to pay an employee. If you're unemployed, you still get no wage. Basic income would, obviously, get paid to every single person regardless of employment. It makes sense for it to be lower than the wage someone would earn for working.
This was a rhetorical question, it is clearly ridiculous to toss things out just because they are 40 years old
FEATURE
Media : Ability to use device
speakers with the FM radio app

Really mozilla? Sorry, I would happily donate $99to Mozilla foundation than buying
> An enormous tax increase only if you figure there would be no compensating decreases in government expenditures.

Which is a safe assumption, since I rarely observe governments decreasing their expenditures.
​Facebook kills Snapchat clone Poke and Facebook Camera
Just to be clear: the issue is not that the economy is more important than ecology. I don't think it is. The issue is that the economy is a proxy for how serious the ecological issue is.

To wit: if pesticides are wiping out the honeybees, and honeybees are in very high demand to pollinate crops (they are: honeybees are trucked from Virginia to California to work the pollination circuit), the economy should clearly indicate that ecological problem happening.

The point about the Varroa mites is tangential. Or rather, two tangents:

1. Compared to even double-digit overwintering losses, pesticides are nothing close to the problem that mites were; mites eradicated the wild honeybee population! According to the EconTalk interview (this blew my mind and I wouldn't be surprised if it's a drastic oversimplification), if you see a bee in your backyard, and it's actually a honeybee, it's domesticated; its home is some apiarist's colony nearby.

2. The honeybees we're concerned about are themselves not native to the US. Honeybees are a species people introduced to North America.
My grandfather told my that during the WW2 occupation, fisherman occasionally used to catch & eat swans. Even during the war, the meat was not socially accepted for human consumption (ie. in western culture would consider it on par with "dog meat"), thus not very popular.
Amazing what ~40 years can do. I still find it astounding that my Snapdragon 800 powered phone, a Lumia 1520, is ~800x (130 GFLOPS vs. 160 MFLOPS) faster than the Cray-1 and ~1/24000th the mass (5.5 tons vs. 209g). Truly a "supercomputer in your pocket". Which plays various flappy bird clones. Or Wordament.
Depends on how you define "contrast". If you define it as "difference in amount of energy emitted", then neither method is using "contrast". The YIQ method is using a simple perceptual model that assigns different coefficients to red, green and blue because of how bright we perceive each of those colors to be. This has to do with the distribution of photoreceptors in the typical human eye.

The neural network approach goes a step further and accounts for how the brain processes color. There's no reason to consider that notion of "contrast" less valid than the brain-agnostic eye based model. In fact, in the context of readability, the psychovisual notion is far more useful.
You get this money without even working. Any money you earned while working (there would be no more minimum wage under this system) would be in addition to the $12,000 you got from the basic income. Also the basic income is presumably not taxed, as it's welfare income.
Kiwi Company 8i Puts Viewer In Movie
your conclusion doesn't match your statement.

BI as a supplement to wages and other resources already exists in the form of welfare and the like, its just a matter of reducing the conditions to nothing.

somebody mentioned 600 a month for BI, you could do something lower without needing to quadruple tax revenue
Average SS monthly income is around $1500, or $18K annually. That kind of puts a floor on Basic, because if you propose a pay cut to all retirees, you would have a mob of cranky oldsters on your back.
Back in the bread boarding days, hard wrapping was using a tool to physically wrap a hair thin insulated wire from one pin on an IC to another pin on an IC.

Not using the PCB foil electrical path but rather a direct short path as possible wire connection between two points.
Always is a bit of a strong word but yes, it's always true.

Virtual functions in C++ which allow some form of dynamic behaviour are slower than static function calls because they inherently involve another level of indirection. Static calls are known at compile time, they can be inlined by the compiler, they can be optimized in the context in which they're called. Now, nothing prevents the C++ run-time from trying to do the same thing at run-time but you can relatively easily see that it'll have to make some other compromised to do so. Nothing prevents a C++ program from generating C++ code at run-time, compiling it, and loading it into the current process as a .so. Now that's a pretty dynamic behaviour but there's again an obvious price. You can also write self modifying code. At any rate, static languages are capable of the same dynamic behaviour that dynamic languages are capable of but you often have to implement that behaviour yourself (or embed an interpreter...).

Fundamentally, a dynamic language can't make the kinds of assumptions a more static language can make, it can try and determine things at run-time (ala JIT) but those take time and still have to adapt to the dynamics of the language. The same line of code "a = b + c" in Python can mean something completely different every time it's executed so the run-time has to figure out what the types are an invoke the right code. Now the real problem is that if you take advantage of that then no one can actually tell what this code is doing and it is utterly unmaintainable.

To compound the problems facing dynamic languages is the fact that CPUs are optimized for executing "predictable" code. When your language is dynamic there are more dependencies in the instruction sequence and things like branch prediction may become more difficult. It also doesn't help that some of the dynamic languages we're discussing have poor locality in memory (that's an orthogonal issue though, you could give a dynamic language much better control over memory).

EDIT: One would think it should be possible to design a language which has both dynamic and static features where if you restrict yourself to the static portion runs just as fast as any other statically compiled language and still allows to switch to more dynamic concepts and pay the price when you do that.
Im not sure yet if the SE fragmentation is a good thing or bad yet. From one perspective I see where they are coming from and why they are doing it. But in today's age of IT where the specialists is dead and people are more-or-less required to be an expert generalist it is very frustrating to use SE. I find myself more or less giving up when a question gets migrated to a SE Im not a 'member' of yet (they should really just make all users IDs members on all their sites whether people use them or not). Whether you are a webmaster, server admin, programmer, general PC user or just want to ask a general question about how SE works, I shouldn't need to go so many places. It's really 50-50 in my mind if they are doing a good thing or bad (and it probably doesnt matter in the end)

But I have never used Quora, so I know nothing about it beyond getting the blurred webpage when you are not a member there. Ill check out that video now.
All I can think of at your response is http://en.wikipedia.org/wiki/First_they_came_. A few years ago I would have thought it to be be simply unthinkable that FCC would be considering such a proposition. Now, I'm all but certain that this would be the natural next step.

> They don't use much bandwidth, wouldn't pay much before just shutting down, and are very sympathetic.

Wikipedia consumes a lot of bandwidth. Okay, not as much as Netflix, but what about the Wikipedia of tomorrow - which houses lots of high-quality media? It feels like there might not be one if net neutrality is beaten. Or the future of education that's video lectures for the masses. Etc.
How We Get Tall
What is the meaning of YIQ?
Canada tried this as an experiment for a few years. While the Wikipedia[0] article is light on information, there are links to meatier sources.

[0] http://en.wikipedia.org/wiki/Mincome
Seems to me that not only did you provide a quick recipe for an implementation, but also provided a name for it!
My conclusion does match my statement (that BI is generally a bad idea, and wouldn't cause significant decreases in other social programs).

As far as the rest of your comment, by reducing the conditions to nothing, you dramatically expand the expenditures while lowering the tax base by encouraging a portion of the population to not bother being productive. People like to say that welfare acts as a form of BI, but that is far from the truth. Most welfare programs in the US contain requirements that recipients at least look for a job, stay drug-free, etc. With BI, people that don't want to work will simply move in with each other. $12K/yr isn't really enough for 1 person to have a good life; however $60K/yr for a household of 5 with no taxes taken out comes alot closer.
WOW! do I love the internet. Go you guys (and gals :) I couldn't imagine connecting like this even 10 years ago.
Wayback Machine for the Brain.js link on HN (2013)
"In Michigan, however, which has the highest population of mute swans in North America, the creatures are considered pests. According to the Michigan Department of Natural Resources, the statewide breeding population increased from about 5,700 to more than 15,000 in just ten years. The birds attack people in the water and on shore, particularly children that wander too close to their nests."

Sometimes when I get excited that an animal population gets to the point of potentially excess, I have to remind myself that it might be that we've adapted our lives to the absence of the animal and can't tolerate its presence even in modest amounts. Yes, swans are aggressive, but just because they are annoying doesn't mean they are overpopulated. Or concerns that some wild cats might take dogs. That sucks for the dog owner if it happens, but is it really okay to have the expectation that your animal can be in nature without any natural repercussions?

That said, once swans become common we shouldn't be afraid to push back against their aggressiveness. Nature does have away to balance behavior, and if humans think they just have to put up with asshole swans or else kill the swans I think we're missing out on some obvious behavior correction opportunities.
The Sims You Left Behind
Oracle wins copyright ruling against Google over Android
Agreed. It really is awesome to see highschool students coming together and organizing an event like this.
Which explains why this bit FTA isn't strange at all:

"The 2014 US Poverty line is $11,670/year for an individual, and ~$4,060 for each additional person.3 It is strange to assume that some humans can live off of $11,670 and others can live off of $4,060."
It seems that this already exists: http://en.wikipedia.org/wiki/IBM_RPG#Platforms. The company may or may not be dead; consider sending them an e-mail.

(Of course, you're free to reimplement "for fun"; just thought you might be interested in a faster path to working? software.)
http://en.wikipedia.org/wiki/IPv6
I started the HS Hackers group in September — it's been an incredible ride seeing it grow from 12 high schoolers at PennApps 2013f to a 1,000+ member strong Facebook community.

If you're a high schooler and you're reading this, join us :)

https://www.facebook.com/groups/PennAppsHS/
Huh, jars within jars... sounds a lot like .war files!
Assuming this ensures BI not to overshoot the correct value, how do you ensure BI not to undershoot the correct value?
Might've been cooked too long. Best quickly seared and then rested.
Show HN: Spirit, modern django based forum software
I finally got something worthy of posting here.
Spirit is a modern forum software written on python employing django's framework.

It has many of the features available on more known solutions. It is easy to integrate to preexistent Django projects, since it works as any other app. I think it is worth a look for any Pythonist out there.
Also, it has over 250 test cases!

Marketing site: http://spirit-project.com/
GitHub: https://github.com/nitely/Spirit
Congressmen ask ad companies to pretend SOPA is law, break anti-trust
Is there a .pdf, .mobi or .epub anywhere? Reading this online might not make economic sense.