skysight-inc/HackerNews huggingface dataset:
This article is actually much narrower in scope than the title would seem to suggest. If you don't let the first sentence sink in, it barely even makes sense.

> Here at Factual we apply machine learning techniques to help us build high quality data sets out of the gnarly mass of data that we gather from everywhere we can find it.

So really, this blog post deals with the topic of "principles for applying machine learning techniques to data cleaning".

Clearly this is an appropriate topic for this company, as they're product is essentially API access to pre-cleaned/curated data sets. However, the post itself is lacking depth. The work I do involves a lot of time cleaning my company's internal data sets, so I definitely recognize the pain of the corner/boundary/special case. However, anyone who has worked with data cleaning (read: everyone who has worked with data) would know this pain, they wouldn't need a blog post to point it out.

I would, however, be interested in knowing what sorts of machine learning techniques they're applying to the problem. When I clean data, the process is largely manual, probably in part because I'm not working with as large of data sets. Maybe they don't want to reveal their secret sauce, but I think a more technical blog post could serve to highlight how good their data cleaning is, and therefore how high quality their product is.
I hope you take my feedback constructively but I am going to criticize most of this. But let me tell you the positives first: it is refreshing, different and definitely very confident.

Now, your cover letter, even though trying to stand out by being personal, does not offer any value to me as your potential employer other than the fact that you are cool and trying a new approach. Reason is that you are telling me a lot of good stuff but not showing me what you have done already. What are your accomplishments as a coder in your own words ?

For example, you say "to me, there is nothing more gratifying than fixing that last little bug". Sounds good. How about you give me an example of when you did something like this which helped the team?

"Give me a problem to solve and I won’t give up unitl it kneels before me"

Great. But again, give me specific examples. For example, if you say "I solved the problem of performance in the system by doing xxx which saved the team yyy in terms of time and efficiency" would be effective.

So overall, make it specific with examples. You can keep your personal tone but by adding specific achievements (small or big), you get the attention of any employers.
Great post and great story. I've lived in the Twin Cities for 10 years now and we got a pretty good set of tech companies running around this area. I'm glad you were able to find a good home out in San Francisco man.
Agreed.
I have been working from home for more than a year now and I do feel isolated. When I am working on something interesting this doesn't affect me much, but when I am stuck in a problem it is easy to feel a bit anxious and depressed.

To cope with it I exercise and try to see my friends as much as I can.

Running and weights seem to be more effective against isolation induced anxiety. Swimming helps me relax when I feel frustrated with a specific problem.

Reading HN helps as well.

I am much more productive working at home, but as soon as I finish my current project I am going to try to get contract work that involves some office time before I loose my sanity.
This is dumb, because he's taking the price per gigabyte for some low tier where Glacier's pricing is tiered. In fact, for an amount of storage as large as his thought experiment assumes, Amazon asks the user to "contact us" for pricing information.
Grew up in KC (Shawnee, specifically). If you are from a large coastal city and want to get acquainted with life in the Midwest, KC is a good start. Certainly less of a shock than, say, Omaha or Wichita.
Obviously the functionality of it is terrible, I don't get it at all. But how something so large would be overlooked is near impossible. Not a single developer/tester had a dual monitor setup? Unlikely.

Just because you don't like how it works, doesn't mean it's a bug.
Hmm thats odd. I don't have an account and I have no trouble reading it.
How would you market one product over another without admitting their differences?

Subjective experience. Most people don't compare horsepower & engine volume & torque & etc. in cars, they just walk into a showroom, try out a few, and go "I'll take that one."

Apple is winning despite not "admitting their differences" in speed, capacity, etc. vs HP vs Dell vs Dell. Apple's opposition have to resort to mutual fights over specifications precisely because they don't have anything better to appeal to; give the user specifications so good they don't care, refine the end-to-end experience, cut costs without cutting quality & experience, and buyers will flock.

Anecdote: I loved Sony products. Raved about them for years. Finally gave up because enough corners (hardware, software, UX) were cut that my enthusiasm was shredded.
No way! The people that play video games are the ones that argue about playing video games? I completely expected those not playing video games to be the ones being chided for their game playing.
What about Pownce ;)
Do you have to use LaunchRock? If it's holding you back form presenting a great first impression, then dump it.
> Hmm, too bad you need a Facebook account to read this.

That doesn't seem to be the case, I have no facebook account and I got in just fine.
I like Cmd-[0-9] to map to my bookmarks, I like Safari Reader, I like iOS-like zoom (chrome only fakes it, and does so not very well), I like the rendered text on Safari a million times better (it's not anti-aliased on Chrome on most pages - at least that's the miserable state Chrome was till v19 I think), I like iCloud tabs and I like Safari's UI much better.

That's why I use Safari. I couldn't possibly live with Chrome even if Safari was worse that it - I absolutely hate the stupid download bar, or "Downloads/Preferences" not being standard windows but silly webpages. And don't get me started on custom contextual menus Chrome team thought were cooler than standard OS X menus (even if they were cooler, they're at least 5 times slower. So what's the point of them?)

That said, it all comes down to what your personal preferences are. The difference between browsers are not huge. Some like Safari, some like Chrome, and some like Lynx.
Ditto to HNers in South Carolina, particularly the Midlands. I'd love to have an HN meetup in Columbia, but would make the trek to Greenville or Charleston for one.
I feel like this same effect could apply to most things that are "addicting". When something is taking time away from a relationship, it is most likely going to cause problems.

Things that cause similar problems in excess:
Porn,
Sports,
Friends,
Alcohol,
Drugs,
Consistent but time consuming hobbies like surfing,
Startups
...

What I am getting at is that basically anything could cause similar problems, though the exact side-effects could be slightly different because of differing situations.

Growing up in a small surfing community, I witnessed surfing to be a pretty common relationship problem. Many were so addicted to surfing that they wouldn't spend enough time with their families.
Not really... if you write an app that's native, then in-app purchases are controlled by Apple and they get a 30% cut of those sales. If you build an app using UIWebView then you could by pass those in-app purchases and get 100% of the sale.

ref: https://developer.apple.com/appstore/in-app-purchase/index.h...
Ugh. I feel sorry for the founder left out in the cold. Especially if he was actually pulling his weight and just isolated from the rest of the team due to personal or political conflicts, or if the acquirer was just being cheap and said "We're only willing to pay for n-1 of the n founders, the CEO has to pick somebody to leave behind." Even if the guy deserved to be left behind, it seems like he shouldn't walk away empty-handed when all the other founders got something.
I don't agree on some points here. I don't think anyone reasonable is claiming that the 1% is bankrupting the middle class. The observation is that middle class earnings have been stagnant and even deteriorating over the last decades. That's just an observation.

You claim that the middle class hasn't produced. What exactly do you mean by that? If you mean that workers for large (and small) corporations haven't produced as well as they have historically, then why are corporations making record profits? Certainly a small number of executives can't be making the difference.

How have the wars in Iraq and Afghanistan destroyed wealth? On the contrary, U.S. defense contractors (some very large employers) have made quite a bit of money as the defense budget has swelled. Hence the outcry over sequestration.
This post is very light on information, and is confounded by the use of idiomatic terminology.

Here's my summarized rewrite of this blog with my added commentary:

1. Worry about the outliers. If you are working with a billion instance dataset, make sure to worry about the 1000 hard examples (0.0001%). Personally, I think this is terrible advice. You should only worry about getting 99.9999% accuracy if the problem really merits it. Otherwise, you're focusing your energy on diminishing returns.

A more generous rewrite of this point 1 is: Don't use a model that is only good at modelling the easy data points. If you can't correctly model many of the data points, use a more sophisticated model that can correctly infer generalizations across more of the data. This good point is obscured by the suspicious claim that you should focus on 4.5 sigma events.

2. Pay particular attention to examples that are near the margin. i.e. Examples that the model is unsure about. This is good practice when doing exploratory data analysis. I'm not really going to go near the half of this point that is non-information.

[You should also do exploratory data analysis on examples that the model gets wrong. One of my key ML principles: You learn about your model by breaking it.]

3. Do exploratory data analysis to see if your model isn't fitting some class of examples, and figure out why. Essentially a restatement of points 1 and 2.

4. Listen to the data. Good advice, and a restatement of 1-3.

5. I'm not going to touch this.

I'm going to do a quick pass at my 5 principles for applying ML techniques:

1. Figure out what accuracy is acceptable, and be honest here. If you don't need 4.5 sigma accuracy, then you can solve the problem as needed and move on to different problems you have. Train a baseline model and see if it's good enough. If not, then proceed.

2. Choose a model that scales to a lot of examples and is semi-supervised.

3. Find unlabelled data sets that make your data set much bigger, and do semi-supervised training.

4. Do exploratory data analysis. Break your model to understand what kind of mistakes it makes. Figure out if you would need a far more sophisticated model or feature set to get marginal returns, or if you can make a big impact with small changes. If the latter, go to step five.

5. Use expert knowledge to improve the feature set, and capture information that a human expert would need to do the task themself.
Fitocracy Member Spotlight: super_luminal - Fitocracy Blog
https://www.facebook.com/notes/facebook-engineering/under-th...
Looks pretty impressive, thanks for the link!
That was not the point at all. Criticism is totally fine and often valid, and it certainly aids in improvement. The point I was trying to make is that people are quick to criticize, but slow to compliment. We share our honest opinions when they're negative but not nearly as often when we have something positive to say.

I'm always open to criticism and am willing to improve. Sorry if that wasn't clear enough in the post.
Spending more time doing X is correlated with spending more time talking about X?

Spending more time talking about X is correlated with spending more time arguing about X?

Spending more time arguing is correlated with higher relationship conflict?

All completely obvious results, strung together with logical fallacies to conclude that video games are correlated with relationship conflict. Although I do have to give the article credit for not implying causation—wait, they do, never mind.

For example, I can assure you that: 1) men who cook more also talk about cooking more, 2) men who talk about cooking more (all other things, including amount talked about in other areas, equal) argue about cooking more, and 3) men who argue about cooking more (all other things, including levels of argument in other areas, equal) exhibit more physical aggression towards their partners. Using the exact same logic, I can conclude that cooking causes relationship abuse.
HN has changed my life, too. Ironically, moving from a pre-seed stage startup (where I was more of an entrepreneur in residence) to one of the biggest tech companies out there.

More importantly, it is continuing to push me on my own projects.
This is really cool. I would love an option for a repeating monthly donation. Maybe send me an email with candidates either based on random draw, treatments which just need a bit more to get over the top, or something similar.
Well, all homes, whether they were overpriced or beyond a person's means, have gone down in value. As a home is an average's person's largest means of equity, their assets have decreased as a result of the housing bubble crashing.
No, look at the data more closely. The top 20% numbers are skewed by the fact that the top 1% is included. The real story is the top ~1% pulling away from everyone else, NOT huge gains in the top 20% as a whole (another way to say that is that someone in, say, the 81st percentile of the income distribution has not seen large income gains)

edited to add link to CBO inequality report:
http://www.cbo.gov/publication/42729
(shows that 1979-2007 top 1% income grew by 275%, the next 19% grew by 65%, next 60% grew by 40%).

As I said before, there is certainly a structural component that means that education and skills can command even more of a premium than they always have, but that's not the majority of the story behind this inequality growth.
I did that in highschool... It's not very hard
Yes please. And maybe credits that are not allocated within N days automatically get directed to a profile that Watsi chooses.
Samsung opens new Apple store in Australia
> Don't iOS blocks natively perform tasks asynchronously?

The blocks themselves don't, no, they're just anonymous functions.

I'd hope NSOperationQueue[0] and maybe GCD composes most of what they mean by "use background threads".

And as sandyarmstrong notes, using things like operation queues will still impact your performances more than separate threads, as the main thread's runloop will still have to go through them, stuff like that.

[0] Which can take a block if an NSOperation is too much overhead for the work, see addOperationWithBlock:
The flaw in this argument lies in this sentence: ...after all we eat mostly the same food and while he can fly first class that is not going to get him to the destination much faster than me in coach.

What makes you think that someone who is unemployed or making minimum wage, can afford to fly coach? Contrary to your implication, there are in fact people in the US living in shacks without running water, and their number is growing.

Wealth is created and destroyed, but it is a fact that the upper incomes have been growing faster than overall economic growth for many years now, while the middle and lower income levels have stagnated or regressed. We can argue why that is so, but it's clear that the top 1% are consuming a greater and greater share of an economic pie that is not keeping pace with their accumulation of wealth.
Not unless they are clicking on the ads to ensure that Twitter can make money. More likely, they're clicking because they wish the purchase the good or service the advertiser is offering. Consumption is not subsidy. You might as well argue that the chattering masses are subsidizing a viral marketing platform for the consumer goods sector.
Supercoder 2000 Keyboard
And they do show the best way to use the router. Rendering content is not related to the library at all.

How you write your handlers and display your views is completely up to you. Why should a simple router dictate how you write the rest of your code?

I agree that it could negatively influence code that more novice programmers might write with it. They should replace the code inside the page handlers with a comment indicating that the user should render output there.
That link also requires me to log in
Illinois Intern Takes on Silicon Valley
It only end ugly for the water companies. On the contrary, it ends beautifully for the customer.
Absolutely amazing, love the idea, love the way you implemented it.

I'll definitely be signing up and making some donations.

My main concern though is as this scales (which I'm sure it will because it's awesome) you will run into the scammers.
Lots of artists and tech people in a small community where it's possible for everyone to know each other an work together on interesting stuff helps.

Add in a bunch of rich people who don't have to spend a lot to live luxuriously and get bored easily means a lot of cutting edge stuff gets funded that you wouldn't expect.

And then you have a culture of giving back that includes the Kauffman foundation, which is the 4th largest foundation in the US and focused on education and entrepreneurship. They us KC as a testing ground for a lot of their pilot programs in teaching entrepreneurship. So the city has more startup resources than just about anywhere else.

But the city has long been a testing ground for new products and services over the years due to having a lot of basically isolated areas that mimic demographic conditions in all other parts of the country. That was part of the reason for the google fiber coming here, and the fact that as it spreads across the country it will end up following railroad right of ways to keep expansion costs down. And KC being a transportation hub in the middle of the country makes this a natural match.

You have to realize that it's basically the entertainment, financial, and technological hub for about 30-40million people living within an 8 state radius. So there are a lot of things centered here that normally couldn't be supported in a city of the same actual population.
Magellanic Clouds-Milky Way System's Twins Found
Dyson and McClure: Two Investors One Theme -- VC as Auteur
The middle class is virtually bankrupt.

This "relative wealth" you speak on is an illusion, because it is based on debt. Debt enslaves you.

The true wealth of the top 1% is their freedom.

One miss-step, disease, misfortune, and you will cherish the power of money. You are standing on a thin, melting ice sheet and below is the abyss.

When you are below the poverty line, you will realize that this relative difference in wealth is not relative but absolute.
We don't need any breakthroughs. Electric cars could handle the entire peak load with today's technology, the right incentives, and capital flowing in the right place.

The peak load for South Australia was 1.8 gigawatts. There are 1,275,041 motor vehicles registered in SA.

If you could get all of them turned into electric cars plugged into the grid, it would be a 1.4 kW power drain per vehicle, which is less than the power drain of an ordinary kettle here (240V, 10A, ~2.4 kW).

The cheapest Tesla Model S has a 40kW hour battery meaning it could sustain that power drain for over 28 hours (a total of 51 GWh). It's got a 10kW charger standard, which is more than enough.

So if everyone had the cheapest Model S, kept it plugged in with the ability to send power back into the grid, and didn't mind that their car was sometimes down to 20% charge, you could power the entire grid on solar alone.

That's obviously a set of unrealistic assumptions, but it does indicate that it is feasible to solve the problem like this.

A more realistic set of assumptions:

* 1/10 cars are turned into a Model S base model equivalent
* Through financial incentives, the owners are willing to keep them plugged in
during the day and using up to 20% of the battery to push back into the grid.
* Cars need to shave 0.2 gigawatts off of peak load for four hours to bring
it in line with the rest of the day

That's 800 MWh of power to satisfy peak demand, at 200 MW.

We have 127,504 vehicles, can can use up to 8kWh and 20kW each, giving:

1 GWh of capacity and 2.5 GW maximum power draw.

So it's just enough. Alternatively, 5% allowing 40% usage works, etc. The power draw is insignificant.

Owners can be heavily compensated for pushing energy back into the grid. I won't run the numbers here for length and time reasons, but knocking out peak power usage is incredibly profitable. You're literally decomissioning a large percentage of power plants. It would be feasible to make all the energy your car uses free; likely the lithium ion battery packs, too.

The question is; would 5% of the driving population buy a $50k car if they no longer had to pay for fuel or battery packs? Financially that would probably put it closer to a BMW. I think it's feasible. What about in 10 years when the cars are $20-30k? Undeniably. In 20 years when you only need 2% of the population to be in the scheme due to battery increases and the cars cost $20k? No brainer.

The system would require a smarter grid (so you can plug your car in at work and have it all taken care of), 10 years of Moore's Law for batteries, etc.

But I think the numbers check out, and it means we could go crazy with solar (the explosion in solar must continue to charge these vehicles during the day). What's needed is the will to make it happen.
I doubt it gets that ugly. The per gallon charge for municipal water in the East Bay is ~$0.005. People like flushing their toilets and taking showers, so conservation only eliminates so much.
5 Tips for Managing Social Media for Clients
I found it interesting that the hosts of that show were suggesting that the system would be self-correcting.

And so it may be, but nature often has a way of self-correcting that is not very hospitable to human life. Tectonic plates shift suddenly, causing earthquakes. Revolutions break out, leading to the deaths of millions. It might be prudent to use our powers of reason to consider how to make a soft transition rather than blindly letting nature take its course.
It's vague because it's a problem in many areas. It could be a documentation issue. It could be data returned. It could be naming conventions/parameters. I don't like being surprised by a result coming from an API.