skysight-inc/HackerNews huggingface dataset:
Given a background in programming, you can ramp up on this in an afternoon. Here's an intro "crib sheet"

Your application believes that it has all the RAM to itself. This is a lie that the operating system and hardware tell your application to decouple the physical RAM addresses and the ones your application uses (virtual RAM addresses). Learn more about virtual memory here: http://en.wikipedia.org/wiki/Virtual_memory

In order to keep this mirage working, the computer needs to map from virtual address to physical address. Instead of tracking every single address, it tracks spans of addresses. So, the address your application sees as 0 to 4096 will map to physical address 5000 to 9096. Keeping this map using fixed-size spans keeps the size of the mapping down and the performance fast.

This article is about using bigger spans (0 to about 1 billion) instead of the standard 4kb. The advantage of this is that the mapping from virtual to physical is stored in memory as a tree and bigger spans mean you need fewer nodes in the tree. Fewer nodes means you have less traversals/indirection to find the node you are looking for. Less work means faster performance.

The details about the caching and the counts of TLB in the processor has to do with how much dedicated space is in different parts of the CPU for this mapping information.

The details about offsets and changing how the memory was accessed in order get positive / negative performance in the tradeoff of 4kb vs 1gb have to do with wether the mapping information was in the cache or not. it is similar to alignment: http://en.wikipedia.org/wiki/Data_structure_alignment

A lot of the obscure parts of the code are just how the author is calculating addresses to read using pointer arithmatic http://en.wikipedia.org/wiki/Pointer_(computer_programming)#... and bit-shifting http://en.wikipedia.org/wiki/Bitwise_operation

Finally, in order to use these 1gb maps instead of 4kb maps, the programmer has to leverage special way of allocating memory from the operating system called mmap http://en.wikipedia.org/wiki/Mmap
A/A testing should be used to get accurate estimates for within-sample variance. If you run an A/A/B test then you can calibrate the A/B component to be sensitive w.r.t. the tolerances of real data.

And then yeah, I'm sure a lot of successful A/B tests will get washed.
"Far out in the uncharted backwaters of the unfashionable end of the Western Spiral arm of the Galaxy lies a small unregarded yellow sun. Orbiting this at a distance of roughly ninety-eight million miles is an utterly insignificant little blue-green planet whose ape-descended life forms are so amazingly primitive that they still think [smartwatches] are a pretty neat idea." - A Hitchhiker’s Guide to the Galaxy, Douglas Adams
Books are typically quite a bit out of date if it's a rapidly progressing field.

More stuff ends up in journal articles, but about a year or so behind the actual work, and at a level of detail which is often problematic to reproduce. You typically have to treat it as a reverse engineering problem - but once you know the core ideas, that's often possible.

Then there's the mountain of "trade secrets" which never get published anywhere. A lot of compelling work in e.g. graphics falls under this case - although you might see a demo and overview at SIGGRAPH.

Publishing tends to be a net positive for serious commercial research work, even if the release has to be delayed. It's a solid process for documenting internal knowledge. It's also one of the main things that's going to interest other researchers in working for you.
Congratulations Sam :) Looking forward to more big things from you.
New technology for the Deaf and Hard of Hearing Community
Hello,

My name is Jibril, I am a Biological Scientist and an entrepreneur. I would like to share with you my technology start up I am doing for the Deaf and hard of hearing community. Our I See What You Say acts as a device for use for voice to text translation.

As a Deaf person, this is like our new "pen and paper" for public places. (We, Deaf, use pen and paper to communicate between hearing people) I would have to wait until someone finish writing which is not productive. Or in the other room where I am being called by my parents and not responding because I can't hear them. This is a new technology that can bridge communication for both hearing and Deaf/Hard of Hearing individuals.
Cogent has been trying to upgrade their connection to Comcast and Comcast has been stalling working with Cogent for months. It's pretty clear cut in this case. A rule when the CDN and the ISP already have a direct connection is not overly unreasonable. Comcast customers are requesting a large amount of data from Cogent, Cogent tries to work with Comcast to improve the existing connection, Comcast refuses.
Your patches were broken. People spent plenty of time trying to help you fix them.

Your response is to tell the Internet that Vim is wrong for not accepting broken patches?
Nuke the interstitials.
I love that if whose condition stretches over 15 lines with inline ifdefs, and then the body of the conditional is just one line without surrounding braces..
Your attitude seems contradictory to me.

Firefox OS is very, very heavily focused on the use of HTML5, JavaScript, and other web technologies for the development of mobile applications.

Yet as you found out recently, apps or websites built using these technologies are often extremely inefficient, even when highly tuned by very experienced developers.

To get good performance on limited devices, the best thing to do is to move away from HTML5, JavaScript, and related technologies. Applications written in C, C++, Objective-C and even Java vastly outperform HTML5/JavaScript-based apps, especially on devices with limited capabilities.

Yet this is completely contrary to what Mozilla is doing with Firefox OS. So it seems really unusual to me to support them and their efforts, when it's clear that their approach flies totally in the face of what you'd like to see happen.
Tests are calibrated on their false positive (alpha) and false negative rates (beta). If you have a lot of financial/upside/pain information then you can start to determine the relative pain of each of those kinds of failures and calibrate accordingly. At the end of the day the best choice is some complex function of the cost of false positives, the cost of false negatives, the cost of each new observation (which is probably non-linear), the upside of a discovery, and the prior likelihood of finding a discovery.

Which is to say you can definitely pick better choices for alpha, but it's really hard so everyone just picks whatever their field agrees is "OK". In science it's often 95%.
Who is our allies? Who are our enemies? Do we spy on them equally?

Is the information yielded from spying only used for valid national concerns?

The answer is of course you cannot see why you should care, since any duplicity is hidden behind the cloak of good ole patriotic national security there is nothing for you to see. Continue on brave blind soldier.
His talk is long on sales and short on science. If you're looking for a detailed discussion of how this works not why-you-should-be-super-excited, you can skip the video.
I've never done that, no. That's not very nice, but it's also bad business. People who unsubscribe have already told you they aren't interested. Email them again and you're far more likely to provoke an angry response than a useful click/open.
> DoS of the high-work-factor algorithms actually is a thing. The bad part is the attacker doesn't have to wait for a response, so he can actually initiate a bunch easily.

Thankfully, rate-limiting login/registration forms is pretty straightforward (although the metric choice, not so much) and is good practice. If you can HTTP 429 them before they can actually submit a password to your service you're mitigating a large part of the problem.
I too am a fan of rapid prototyping, but the example app here is incredibly simple. More complex apps require more time to refine & prototype.

That said, this sort of flat prototype should be do-able in a few days for even the largest of apps.

So, I'm in agreement with OP — except on the timeframe :)
How to build smarter mobile apps?
Hate to keep being the voice of sanity here.

1) The loss of privacy and anonymity on the internet is horrendous. The NSA is the poster child, not the problem. We don't need to "stop the NSA". We need to fix the problem.

2) Yes, even allies spy on each other. Spying on each other is a good thing. It prevents misunderstandings, it allows for back channels. It's been expected ever since there was diplomacy.

As far as Obama saying he was going to stop spying on Merkel? That makes no sense to me. Somebody should ask him.

In all of this, I get the distinct feeling that people without knowledge of (or who don't care to understand) how intelligence and diplomacy work are being used as pawns for other intelligence agencies who I can guarantee will have no Snowdens. This is not an optimal state of affairs for stability in the world.
This was from 2009; it's more prescient than ever, and no less terrifying: http://tctechcrunch2011.files.wordpress.com/2009/10/net-neut...
I don't like to, but every time I point out why node sucks, the node guys cite LinkedIn, so now I include it.
I've come across people that believe The Cloud is one specific place.
I work in a totally different environment -- a very large enterprise IT environment with 200k+ users and probably 2,000 physical locations.

When we do business with cloud providers or other third parties, we usually start with internet based access. When the relationship gets bigger or needs to serve a large portion of our base, we typically either peer with that provider (in the case of big cloud providers like Amazon, Microsoft, etc) or establish dedicated connectivity between our networks.

I think Comcast, Verizon, TWC, etc has a point here -- as a relationship grows with another party, you should have a more robust connection. Netflix hasn't wanted to do that -- it prefers to use a shitty ISP (Cogent) for cheap, and augment it by co-locating content on end-user ISP networks for free. Services like Netflix and Youtube stretch the net-neutrality argument, because they aren't good citizens.

That said, the end-user ISPs cannot be allowed to discriminate, which their monopoly power will almost certainly enable them to do.
"Modernized" means JSON-rpc IPC? If so, no thanks.
No openings in NYC listed on their site
Best reason for a startup: "I want to solve problem X"
What I don't understand is why Comcast refused to put NetFlix's CDN gear into their network/DC setup. That would solve the problem...
The US (and UK) have no right to operate listening posts in Germany either. Just because they have some secret contracts or whatever doesn't make it a right.

Contracts can be canceled. The current affair would be a very good reason to do so, and close all the military bases in one big swoop.
not really. i'm unemployed and don't qualify for any financial aid that i'm aware of. Thank goodness for my family taking me in, or i'd definitely be homeless.
Smartwatches are the greatest idea ever conceived. Just think about the NFC smart tag fever it will release upon earth.

It's not about apps, icons, UIs, it's all about presence.

Mere mortals still don't get it, but they will very soon.
Call it a clone, refactor, rewrite, fork, whatever.

If it gets subsumed into Vim I'll use it without having to choose.

If it gets traction as a second thing, I'll try it.

If it's better than Vim I'll keep it.

Let them write, good luck, have fun, I'll try it if I hear from HN that it's good enough to use.

I don't think it violates "don't rewrite" because these aren't the Vim people. Vim itself won't "suffer" from the ill effects of a rewrite.
Those goofy diagrams are wonderfully effective at explaining the situation.

Worth noting is that Cogent is small potato and basically has no leverage. It barely does $300 mil in revenue. Compared that to most Tier 1 providers which do $10 bil+.
Evidence-Based Entrepreneurship – How to Build a Startup
I am getting tired of those news websites requiring you to register and login to read their content
Actually, its going to be pretty awesome. We're going to start shipping our coal to China, since we'll have switched aging coal plants to combined natural gas turbines. The price of natgas will then go up in the future (duh); at that point, we'll either have to try to outbid China for the coal or drive more money into renewables.

I'd like to think we'll go the right route, but this is America. We don't do the right thing until we've tried everything else.
Unless Grande and Google eventually decide to do the same thing. Why not, from their perspective?
reminds me of something my coworker put together some time ago
http://hipsterstack.herokuapp.com/
Style guideline of using curly braces with ALL if statements, even one liners, would have gone a long way to prevent this.
Would be interested to see patio11's feedback on this one.
7 ways to boost your business just by asking
And then we all grow up and realize that little Johnny can't run his competition on his competition without paying protection money.

All joking aside, there isn't much you can do. comcast owns the tubes. The world where they have to bend to the masses is a scary world.

Is this better? I dunno, only if Netflix goes out of business. This only really shortens comcasts rope long term. They just created a new market with this strong arm.

Grab your popcorn folks, this is going to get interesting.
I also found it a bit hard to swallow. The article mentions entire towns without access to electricity. Its hard to imagine a digital currency being preferable in those conditions.
I've seen that, too. One of my clients redid their marketing site 3x in one year, each time claiming incredible improvements. The incredible improvements turned out to be local hill climbing, while the entire site's performance languished... 3-4 years ago there were a ton of blog posts about how a green button produced incredible sales when compared to a red button. And so everyone switched to green buttons...

By contrast, I've evolved multiple websites through incremental, globally measured, optimizations. It's a lot of fun and it requires you to really understand your user (I've called AB testing+analytics "a conversation between you and your users"). But, as you point out, it can be tough to get statistically relevant data on changes to a small site. That's why I usually focused on big effects (e.g. 25%), rather than on the blog posts about "OMG! +2.76% change in sales!". That's also why I did a lot of "historical testing", under the assumption that week-week changes in normalized stats would be swamped by my tests.
IMHO, I don't know why people keep confusing UI with UX like they are the same thing, do you actually design the "UX" base on your own intuition without even know or test what the audience you are targeting for the specific product?. This more a wire framing to me, since this "prototype" is far from finish. I wait to read the next chapter since you it seems that it will address some of my concerns above :). Cheers
But yes, you're right of course: like all fiat money schemes, it only works because people want it to work. IMO people will want this to work, and furthermore: it's in each individual person's self-interest that it works.

Without taxation, what leverage does the government have over demand for the currency? Fiat money works because it can be used to pay taxes at the very least. With taxes out of the way, the government-issued currency would be no better off than a so-called altcoin.
We should spy on all foreign states equally. We should be actively performing industrial and national-security espionage against foreign companies.

Please define what you consider a valid national concern.

All of this said - I am wholly opposed to any and all domestic spying without proper legal and constitutional authorization. Further, information learned about America citizens through foreign espionage must never be used and should be destroyed as soon as it is discovered.
Yeah, I should have replied to your parent.
The way it's looking it's going to happen one way or another. It's probably better that the move towards isolation happens intentionally rather than be forced by screw-ups like these.
Exactly — start with intuition then get feedback and fix what you were wrong about. And it's definitely far from finished. Thanks!
I think this is how most people work, unless you're a consultant that get paid by the amount of deliverables you produce. :-)

Prototyping to solve problems is often the quickest. Pulling other people's teeth to reach that point is what takes weeks. :-)
Horribly out-of-date in parts. The 'substance' has evaporated.

"I thought it would be useful if I explained what a nerd was.
What I came up with was: someone who doesn't expend any
effort on marketing himself."

I always considered Paul Graham a 'nerd' despite his excellent essays marketing his intellect to nerds. (A positive is that he seems very humble; he does not recurse very well, and so had to create a Y-Combinator for this purpose. For somebody that doesn't appear to believe in marketing this man sure knows how to signal to me.)

I also don't see dressing informally as a very positive signal anymore. Many nerds have lost their right to wear formal clothes and they've just not woken up to that reality yet. Companies want their executives to wear suits and ties but their engineers have a new uniform: "no uniform". You can tell you're wearing a uniform, because there is a reputational cost to rejecting it. My slightly rebellious slant is to just pick the opposite of what everybody else is doing and do that.
Except they bought an even more worthless pile of stock with it. Compare that to Google, who gutted Motorola for its IP and then flipped it for a significant chunk of Lenovo.
My understanding is that M-Pesa is used in Kenya and Tanzania, which mathematically can't be the majority of the continent.

M-Pesa uses local currencies, which frequently lose their value in non-OECD countries.
I am curious just how credible this high-ranking official source is. These are pretty explosive claims and if true, have the potential to jeopardise US > Germany relations and force other countries including Australia and New Zealand to investigate and question if the US has been doing the same elsewhere.

If this is seriously true, then the US are most definitely spying on everyone. This is the kind of stuff that starts wars, it's actually quite scary what the NSA has been and continues to do.
Indeed. I don't use gmail, but Google still knows almost everything I write since half my correspondents use gmail.

If Germany really wanted to retaliate (they're already talking about amping up their spying on the US), they could publish what they hear while spying on the US. Provide transparency to the world, as a public service.
"Its silly to pretend that the abstraction is the reality"

No more silly than pretending the the abstraction of "files" and "directories" is reality. No more silly than pretending that your computer is running a Python program, when in reality there are sequences of machine language instructions being run. Computers are useful because of abstractions.

"There is no Internet, just a bunch of private networks connected together"

You might have missed the definition of "an internet:"

https://en.wikipedia.org/wiki/Internetwork

There is, in fact, an Internet that we use every day. The overwhelming success of the Internet is due to the abstraction it presents, which is best-effort routing of packets from one host to another -- without regard for what networks the hosts are connected to, what arrangements govern their connections to those networks, and perhaps most important of all, what applications those hosts are running. On the scale of positive improvements to human communication, the Internet is up there with the printing press and written language.

"Now that real money is in play, the fiction of the internet is crumbling and the physical nature is becoming exposed."

No, what we are seeing is the impact of years of systematic attacks on the Internet by companies that are desperate to protect obsolete business models. Comcast is worried about the imminent demise of cable TV and all the power and profit that came with it. AT&T is worried about the demise of long distance fees, roaming charges, SMS charges, and all the power and profit that came with controlling the phone system. The recording industry and movie industry worked hard to leave the Internet divided into "consumers" and "services," even though the system itself was designed to be far more general.

In short, had we told these companies to adapt or die rather than tolerating their efforts to roll back the clock, we would not be in the situation we are in today. This has nothing to do with leaky abstractions, physical limitations, or the design of the Internet. It is just a power grab, a systematic effort to destroy a good thing that threatened the entrenched players.
Another interesting related article in The Guardian: http://www.theguardian.com/society/2014/feb/23/europe-11m-em...
Do you have any examples of sites that allow display customization? I'd be interested to see this in action.

Browsers give you limited options, but how many people use or know about them?

You can't please everyone. If you try, you're doing it wrong.
Your post mixes together wireframing with prototype hacking. But I guess this is to be expected when programmers are doing design, a designer would look at this very differently.
Growing up in an environment that was in certain regards ignorant, even hostile against science and technology I remember hearing about DNA for the first time I had this uneasy feeling that DNA is pretty much like computer code.
Great read.

What bothers me about A/B tests is when people say, eg."there was a 7% improvement" without telling us the sample size, or error margin. I'd rather hear: On a sample size of 1,000 unique visits, the improvement rate was 7% +/- 4%
> All joking aside, there isn't much you can do. comcast owns the tubes. The world where they have to bend to the masses is a scary world.

Less scary than the one where they don't have to.
That's the part that makes me suspect the talk about this harming new entrants, startups, etc. is overblown. Afaict, the situation for small players is the same as it's always been: you pay for transit from an ISP with good connectivity, and you get good connectivity. If you have some colo racks somewhere with high-quality connectivity, your customers on just about any network have good access to your content. If not, you find a better colo facility.

The controversy here seems to be over what happens when you're a massive player and looking to get your bandwidth for less than typical retail transit prices. Then it enters the whole game of peering politics, which has changed considerably over the past 10 years. But if you're a relatively small startup with some colo racks, I don't see how that game is any more relevant today than 10 years ago: you still just buy transit. I mean, I personally have no trouble delivering my modest amount of content to Comcast users, and I've never paid Comcast.

I do think there is a general barrier to entry on the internet, because large players (YouTube, etc.) get free transit from peering agreements while new entrants don't, so any YouTube competitor is at a huge transit-cost disadvantage to Google. But that's a fundamental problem with the way the internet backbone has been built out of a mesh of private peering agreements, ever since it abandoned having a single neutral backbone run as a utility (originally by the NSF). I think going private was a mistake and benefits both big players and politically savvy ones, at the expense of new entrants paying commodity, but that's a (very) big issue to fix at this point. It's not an issue of "net neutrality", though: the traditional peering system is inherently non-neutral, not utility-esque.