skysight-inc/HackerNews huggingface dataset:
you can write an entire web scraper with just a url using http://scrape.ly

With scrape.ly I can just do this to crawl the entire HackerNews site across pages and grab the urls and extract any data from the page it lands on without defining any fields (it discovers them on it's own) and so doesn't require you to 'relabel' fields when the site changes layouts. It also generates new IP addresses on the fly so you don't get stuck and launches multiple threads for you to speed up the process. It works fully with ajax sites and single page apps. Flash support is coming too.

http://scrape.ly/s/{https://news.ycombinator.com/}
{next:More}{Space Monkey dumps Python for Go}*{fields:'Auto'}

Honest question (I don't mind downvoting if you disagree), but why would you want to waste time writing web scrapers, maintaining it to run and fixing the code? Multiply it by 100 or 1000 different websites and it becomes a full-time job. For me, I want to get the data I need with the least possible of overhead and as soon as possible and I don't really want to be bothered with setting up environments and hosting for it to run and fixing bugs when sites change layout.
Works fine in Chrome on my Nexus 5.
F-35 C++ coding standard [pdf]
Did you release your faster version of the Protocol Buffer library ?
Solutions to prevent/combat email header spoofing have been gaining quite a bit of traction and are quite prevalant. And yes, if that means taking the domain away from people that don't implement said solutions, then poof you've lost the domain. It would accomplish getting rid of domain owners that enable spammers.

So if I don't even use my domain for email, I'm liable to lose it because I don't have the knowledge or even awareness to set up the proper SPF records or whatever solution du jour? That's a terrible tax on people who just want to have their own small website.

And what for? If the receivers can check for the existence and SPF records of the domain, they can also just block email from domains without any valid SPF records. There's no reason to create such draconian policies.

There is a big difference between censorship and removing junk from your network. Perhaps it is something we should worry about, but we should have thought of censorship BEFORE we allowed the US to maintain control of ICANN, don't you think?

Not really, because they don't usually censor. It should be something we should worry about if we gave you the control of ICANN.

Probably not the common definition, but if it pollutes the domain/tld namespace with useless/junk domains then yes you can call it spam.

No, because "useless" is subjective.

Hence why I suggested we either treat ICANN like a for-profit private enterprise with NO government monopoly or privilege of licensing, OR we damn well make sure it behaves like non-profit acting for the public good. I.e. no lucrative money grabs such as the one in this article.

I don't see why is this TLD scheme against the public good in any way. Transferring money from big companies to an organization which provides such a core service seems great to me, and all the downsides I've been hearing about are mostly people's aesthetically sensibilities being hurt.

At the very least they're better than what we have now.

Adding censorship is never better.
Design decisions behind Twitch's stats pipeline
I would be interested to see how this effect affects the conversion rate. I could imagine that it could work very well on physical products.
The main problem is it takes some amount of time to figure out how to navigate. It should just be inherent.

The webmaster needs a copy of Steve Krug’s “Don’t Make Me Think” … http://brad.me/1pot03u
Show HN: My HTML5 multiplayer fast action game experiment
The problem is that you have to think at all. It should be inherent and automatic for you to navigate where you want to go as a user. If you have to pause to figure out how to use the UI then the designers have failed.
Tim O'Reilly Reddit AMA
My short bio:
You probably know me from O'Reilly books & conferences, including the Open Source Convention (OSCON) and the Web 2.0 meme. My original business plan was “interesting work for interesting people,” and that’s worked out pretty well. I'm also a partner at O'Reilly Alpha Tech Ventures and on the boards of Safari Books Online, Code for America, Maker Media, and PeerJ.
Lately I've been intrigued by the emergence of "Internet of Things" and the opportunities and challenges in this area. You can see me talk about it in this interview: https://www.youtube.com/watch?v=zhUEvsq1xOo.
I'll also be talking about it more at O'Reilly Solid, a new event we started about the Internet of Things: http://solidcon.com/solid2014.
This area is getting a lot of attention now, but I'd like to go beyond the hype and talk about the real ways that IoT can change how we live and do business. I’ll be answering your questions live, starting at 10AM Pacific time.
My Proof: https://twitter.com/timoreilly/status/456178718424432641
We did initially use m1.xlarge, a few years ago when we first started with HBase. I no longer have the hard numbers available, but while I agree HBase is memory hungry, our m1.xlarge servers were constantly pegged for CPU. For a while we ran both types, and eventually phased out the m1.xlarges entirely.

We run pretty frequent compactions due to our read and write patterns, so of course if you do not need to compact as often you might get much better mileage out of the m1.xlarge.
It seems you are trying hard to find something in my statement with which to refute. I was merely pointing out that the GP's conclusions are probably incomplete, given the data we have on lead.

Most of my previous comments were quite non-committal. For instance, the phrase "lead-soil levels seem to have some negative effect" was cherry-picked for your analysis, and given much stronger meaning than intended. First, let's examine the word choice of "seem". It means "give the impression or sensation of being something or having a particular quality." This modifies the statement, to imply that the data "gives the impression" that there is "some negative effect". This is a much weaker statement than a hypothesis on the effect of lead-soil levels on poverty-stricken neighborhood. Perhaps, that statement would have been more clear if written as, "lead-soil levels seem to have an effect on crime". The implication from the sentence that follows is that the value of "some" is "crime".

It very well could be that lack of political power caused lead-producing activities to be concentrated in low-income neighborhoods. However, the issue at hand is not that lead-soil levels cause poverty; it's that lead-soil levels increase the incidence of crime. This point may not have been clear, as conceded above.

Indeed, correlation does not imply causation, but we know much about the health effects of lead [1] and its impact on decision making. We know that the correlation of crime rates with increasing and decreasing lead-levels, in a variety of situations, throughout many policies and governments, holds. We know both a pathway and have a strong correlation. Dismissing this as happenstance is unwise.

[1]: http://en.wikipedia.org/wiki/Lead#Health_effects
We had about 3 or 4 guys full time. On my silly little i3 laptop, the whole codebase builds in 18 seconds - I suspect most of that is our C extensions.
I'm confused if a low-enough Reynolds number implies that laminar flow is still happening, how would the dimples on a golf ball even create turbulent flow given they have such a low Reynolds number?

EDIT: Ah, the velocity determines increases so does the Reynolds number. So at around 55 mph, the Reynolds number is high enough to induce turbulent flow.

This article is fascinating and now I understand why golf balls have dimples.
Are you saying that good engineers can't have a sense of humor?
We're planning on open sourcing a bunch of stuff. We'll have some follow up posts.
I wonder how many lines of code they ended up with afterward, although admittedly lines of code is a pretty silly metric. I think it is useful in orders of magnitude when describing the size of a project, 100 lines, 1,000 lines, 10,000 lines, 100,000 lines all speak volumes about effort. So I guess what I'm wondering if you got about a roughly similar sized code base or significant change difference?
I've never used a site like Weebly, Wix, or Shopify, though I have seen Squarespace's interface when someone was building a site one time and my dad used Weebly after I recommended it to him (seemed easy to use).

I'm curious their main sources of customer acquisition. I assume a lot of its word-of-mouth and organic, but I'm curious how much marketing they do. Squarespace has been huge in the podcast space doing advertising and did that big Super Bowl ad splash. Wix seems to have been around for a while so I see many of their out-dated designed sites.

Edit: I did just get retargeted via Facebook ads exchange after visiting Weebly's site, so they are doing at least some advertising :)
You send your burst a few milliseconds before an event. It chokes up C (all of those depth updates sit in his queue and he's busy for the next 50 millis churning through them). Then both you and A are free to pick him off.

The thing is, you know to ignore the burst on the public feed, since you know it is coming (you created it). You still have to process the order book updates, but you can skip your decision logic. So any participant who has an internal latency (packet decode + orderbook update + decision logic) greater than 1 exchange cycle, and who doesn't have an "out" clause to stop making decisions if it is falling behind, will be vulnerable.

Like I said, I don't buy it, but it is possible in theory.
Does anyone know if there's a way to hire a person or company to build a custom weebly template?

I think weebly offered this service themselves a few years ago but now I can't seem to find the option.
I had the same thought... how this effect could enhance porn picture sites. Given the dominance of "tube" porn sites, I doubt it will be as big as it could have been when picture sites dominated. However I could still see some niche sites using it to "get a better look" or on-page to attract clicks.
College can be tuition-free.  We did the math.
C extensions because something you wanted wasn't in go? Are you using cgo to build?
Well it is kind of obvious that a compiled language is going to be faster than an interpreted one, especially the way how these interpreters work (https://wiki.python.org/moin/GlobalInterpreterLock). You can fool yourself with Twisted (or in Ruby with EventMachine, Goliath etc.) but it gets so just a bit ahead. The surprising fact for me is that Go is not as much faster. I was expecting a bigger gap in the performance between Go and Python. Understanding where your bottlenecks are is crucial and it is not super hard in Go. https://www.datadoghq.com/2014/04/go-performance-tales/ I guess the SpaceMonkey guys might further improve the performance just by doing a thorough analysis on their code.
> At TalentBin, I refined my own approach to strip mining networks for the best possible people. It comes down to three things:

Yeah, so, I have my own personal EPA, and it's banned companies from strip mining my networks, period. No, no, and NO a third time. If you want me to recommend the company to everyone, you have to make this company freaking awesome on everyone's dimension. That's really not going to happen.

Worse, unless it's a condition of hiring, I'm not going to want to sit down with you and let you chew through my social networks for recruiting leads. That's unethical (although I'm not sure quite off how to quantify that).
Unless there is some reason to disbelieve the line-by-line claim, can't we assume it's all due to the more performant implementation?

Python is a joy when it's fast enough, but it's not surprising that it isn't good at throughput on small devices.
I came up with the same thing a while back. I was hacking on a side-scroller and spent a lot of time trying to get the feel right. The detailed explanation:

To start a jump, the user presses the jump button while the character is standing. There is a "grace period" where you can still jump if the character isn't on the ground now, but was a few frames ago. This is important to be able to jump right as the character reaches the edge of a platform. Without this, you have to jump earlier than you feel you should.

Once a jump has started, we track how long the jump button has been held down. Every frame, the character's Y velocity is modified by the "jump acceleration". This acceleration starts at some high value, and ramps down linearly the longer the jump button is held down.

For example, the first frame of jumping may add -10 to velocity, then -8, -6, -4, -2, until finally zero. I found this acceleration ramp was the key to getting the jump to feel snappy yet responsive. The first frame gives the biggest velocity boost, so the character immediately leaps into the air. Then later frames give a diminishing effect so that you can more precisely dial in how much oomph you want the jump to have.

Gravity is then applied on top of this. In code, it looks like:

// Start jump.
if ((Actor.LastOnGround <= JumpAllowedFrames) && UserInput.JumpHeld) {
jumpFrames = 1;
}

// Accelerate upwards while holding jump button.
if (jumpFrames > 0) {
velocity.Y -= JumpSpeed * (MaxJumpFrames - jumpFrames) / MaxJumpFrames;

if (!UserInput.JumpHeld) {
// Stop accelerating if player releases button.
jumpFrames = 0;
} else if (jumpFrames >= MaxJumpFrames) {
// Can only accelerate for so long.
jumpFrames = 0;
} else {
// Keep jumping.
jumpFrames++;
}
}

// Apply gravity.
velocity.Y += Physics.Gravity;

I was really happy with how this turned out. Actually, now that I dig into this several-year-old code, it kind of makes me want to resurrect the project. :)
I built a similar thing, in Go on App Engine: https://github.com/imjasonh/simply-put
Yeah, I was going to quote that same part. Getting more users is great, but you're still left with the question of how to monetize them.

I'm a pretty hardline user experience idealist, I hope to never host an ad on any site I run, but still, you have to be able to monetize to keep the business afloat. Whether Slate is doing just that, or actually being greedy, I don't know, but I understand the reasoning for enhancing the experience for paid users, which unfortunately inherently results in a somewhat gimped free plan.
Kids don't change that. My little girl is 5 weeks old and I find much solace behind my editor during quiet times :) It's reinvigorated with the hope of whatever I learn I can pass down in the future.
Speaking of which, anyone has experience with testing flat design against older audiences? Like, 70 year old non-tech users? I'm quite worried that skeumorphic design is more intuitive for this type of user, and latest move to flat design, especially in iOS, will leave them behind. However I don't have enough data to go on, so find myself uncertain and indecisive. Thanks for any help!
That's actually kind of shocking. I thought it would take way more work to port from Python to Go, and a 400% speedup is way less than I'd have expected (though this is string handling, and the Python was already brutally optimised with C modules where it counted).
Powdered Alcohol Cartoon
> You clearly have some opinions about shortcomings of the American labor market.

Not that are relevant to this discussion -- I just have a belief that there is a difference between being middle class (the class between the working class and the capitalist class, at least, in the usual definition applicable in post-feudal economies) and identifying as middle class.

Historically, the working class being larger than the other two combined is the norm and America's brief period with a very large middle class was something of an aberration resulting from the rapid application of agriculture by a small population to a large land area before that agriculture was effectively consolidated by large commercial entities.
I added some information about the Factor language to the blog template - great idea, thanks.

Concatenative languages are quite interesting, and I'd encourage you to try it out. You might find it helps your thinking about certain problems.
> better suited to work towards education and not towards entertainment

HSW's podcasts aside, which I think have their own separate entertainment value, whether the site itself serves a non-redundant purpose is questionable.

My wife and I were listening (for the first time in many months) the Stuff You Should Know podcast, and we started talking about the HSW site itself.

Who is it for?

I can't think of one reason any search-literate adult would ever use it. (Except incidentally, in the way that one finds themselves suddenly on about.com every 6 months or so and thinks, "Wtf? Am I lost? Why am I on about.com? Ok, might as click through a few pages before bouncing, regrouping, and heading to a destination worth my time.")

HSW is a compilation of quickly researched articles mainly based off secondary sources. It's more or less an edge-case content farm. Lots of topics. Lots of links. Quickly produced. All of it disposable.

It's not trash, of course. But it's not good entertainment (again, arguably excluding the podcasts), and it's not good educational material. For anyone. Even kids.

I quickly pulled up

http://health.howstuffworks.com/diseases-conditions/death-dy...

This particular article is about how hanging works. You can cite this article. There's a button beneath the infinite scroll that gives you a citation slug. The implication here is that the material on How Stuff Works is worth citing. It's not.

The article itself cites three sources. The links to two are broken, and the third is to an ABC News journalism piece that, in connection with the hanging of Saddam Hussein, spends a few paragraphs talking about the physics of hanging. This is a news piece and, as such, doesn't cite any sources itself.

So is what this how stuff works article about how hanging works really how hanging works? Fucked if I know. Now that I've read the article and checked the sources (or really, the one non-defunct, non-source), I still have to do the leg work to go track down more authoritative, primary sources.

That means (instinctively), going to wikipedia (which I could have done initially, skipping the whole HSW excursion) and tracking down sources there. Or doing whatever else that has proven reliable in the past. Encyclopedias. Journals. Whatever.

This is true about HSW for kids, as well -- though I'm not sure whether to a slightly lesser extent, or to a much greater extent. The question there being one of pedagogy: is teaching how citations work and what they imply important (or not) early on?

Yes, my wife and I (infrequently) enjoyed the Stuff You Should Know and Stuff Mom Never Told You podcasts... but I always found it odd that the podcasts were tertiary to the whole HSW content thing, and what the hell is the deal with that?

/off-the-cuff ramble
Could you give some examples of the social truths? I'm not saying you're wrong, but I'm finding myself becoming less and less liberal and want to know what exactly you're talking about.
It's on LTO (link time optimization), which was previously not much optimized for compilation speed. Most projects probably don't need LTO so this won't matter for them, but for projects like Firefox that do, faster LTO compilation is a great thing.
Palcohol: the powderised alcohol that turns water into rum
Is anyone else getting SSL errors on the site?
For me that would suffice - but its not the answer, I'm afraid. Management/IT wants to:

- have workflows, so documents can be approved among other stuff

- have document lifecycle, so they expire and must be reviewed after time x

- solve the Dropbox problem - how do you collaborate with external entities without compromising security by having users uncontrollably put stuff on Dropbox, because there is no other way?

- have automatic versioning

- be able to checkout/lock a file during editing

These are just a few aspects. You can do most if this by gluing stuff together yourself, but these are the constraints:

- be able to achieve all this with semi-competent IT staff

- seamless integration into the existing Windows infrastructure is a big plus

- be able to blame someone else if something goes wrong

So - what do you propose?
They wrote a naive, line-by-line translation of their python code in Go. There's probably still a lot of low-hanging optimization to be had.
But that's not apples-to-apples. How about building for OS X using gcc?
This article doesn't address one of the biggest benefits I've noticed in switching from Python to Go.

I've been using Go for almost two years now. Like OP, I am/was primarily a Python developer. Like OP, my first Go project was a time-sensitive rewrite[0] of a project from Python (tornado) to Go.

Even though I was an experienced Python developer, the Go rewrite was marginally (~20%) faster[1]. But the real benefit came from the subsequent development - refactoring, rearchitecting, and maintaining the project on an ongoing basis. Go was designed to make it easy to scale the maintenance[2] of software, and on this one axis, it absolutely blows every other language and environment I've used out of the water.

For a fresh project, I'd say Go is about 10% slower to write than the equivalent Python[3] for the average case. But the time/cost savings are very quick to come thereafter.

[0] I would absolutely not recommend doing time-sensitive rewrites in general, but that decision was a separate matter.

[1] Some of this is due to the nature of rewrites in general, but the fact that it wasn't slower to use a language I'd never used before says something about the language.

[2] Scaling software development as teams grow is very different from scaling software as users grow.

[3] Assuming comparable familiarity with both languages, which is rarely the case, of course
Presumably, because they were doing it line by line, and the Python version used C extensions (for performance reasons).
7 Lessons Learned Bootstrapping Multiple Successful Companies
That launch facility was for polar orbiting shuttle missions, which were never executed.
> Actually, the U.S. Supreme Court has a formal definition of freedom: "The right to be left alone."

[citation needed]
Agreed, there's nothing creative or inspiring here. If anything, it reminds me why I don't want to create a site with a washed out fullscreen background, with white text over top, since there are a million other sites using the same format.
Congrats David. The new homepage looks super clean. Excited to see what the future has to offer for Weebly.
> Worsee, unless it's a condition of hiring, I'm not going to want to sit down with you and let you chew through my social networks for recruiting leads. That's unethical (although I'm not sure quite off how to quantify that).

Who would you be more amenable to it, or the action be more ethical, simply because its a condition of hiring?
Why is that a poor choice of API and ownership semantics? That's entirely natural. You use a range in order to provide the maximum convenience to your callers, and you use unique ownership to provide efficient memory management and reduce the amount of copying of data that has to be done when the backing store is resized.

(Also, your equivalence is not quite correct: `&[T]` is more like a Boost range. In particular your transcription makes it look like the original code only accepted Vec<T>, which is not the case.)

> That's just one symbol... how horrific can we make something without noticing with just a few more params and symbols?

You've pretty much covered all the type-level symbols in Rust, except for * for unsafe pointers.