skysight-inc/HackerNews huggingface dataset:
Ask YC: A tough problem - how do I pick African children for a tech scholarship?
My opinion has been asked for a unique task - how to select which children should get a scholarship based on ability.

The situation is this - there is a large school in Africa with about 1500 students from ages 6 to 12. The students come from wildly differing income groups, and some have no problems paying the fees, and some can just barely afford the fees. There are various programs in place to help the really poor off, but the school has to maintain fees to run.

The school would like to encourage students who are talented, but don't have a lot of money by giving them scholarships - i.e, academic scholarships.

But there is a HUGE problem - the scores of all the students is exactly the same as their family income. The richest students come out at the top of the class, and the poor end up at the bottom. So simply setting an exam and giving the scholarship to the best always ends up with the scholarship going to those who don't need it. All tests always follow the same curve, and are always directly proportional to the income of the family.

The teachers cannot be trusted to give accurate assessments of the children, because a lot of the teachers have family/relatives in the classroom.

How can this problem be solved? How can we identify students who have ability, but are not particularly rich? And using a system that is somehow fair and not completely arbitrary?
Mac OS X targeted by Trojan and backdoor tool
Yeah, I read that the other day ... blanking on the site/url but I remember it being possible. I guess you're screwed either way.
Don't you think if people came to the site at certain hours and there was a page saying "get back to work, we'll reopen in 1 hour and 14 minutes" it might interrupt their procrastination in a useful way, and make them think, "Ok, I'll work for a couple hours, then read the news?"

I find when I've been procrastinating and there's some kind of reward coming in an hour or two (e.g. dinner) that often makes me buckle down and start working.
Have you tried the simplest possible means, merely asking them? Ask students who want the scholarship to apply, and discourage those who already have the means to pay from applying. Then simply award the scholarship to the most able from the group of applicants. Am I being naive in believing that this trivial solution works reasonably well in most other such situations?
Upcoming Ruby Enterprise Edition improvements thanks to sponsorship campaign
Not at all surprising given the large number of disks involved. Here is a very interesting analysis of actual Google failure rates: http://storagemojo.com/2007/02/19/googles-disk-failure-exper...
They will all say yes. It's free money in a poor country, who would say no?
This is an excellent article and a good reminder that when it comes to the economy, we are all only feeling up part of the elephant.

Just remember that the people who produce financial porn, and who get paid to describe the deals and the play by play are not in the business of telling you the truth; they are in the business of selling ads for financial services.
Can sorting be compared to matrix computation?
I have a question about two articles recently on HN. In one of them a physicist says that

"There is no computer on Earth that could possibly store such a big matrix in its memory"

On the other Google says that they sorted "1PB (10 trillion 100-byte records) on 4,000 computers" in six hours.

I know one is talking about matrix computation the other sorting. But is there a way to compare these two to verify if what the physicist claims is wrong and that google could actually store and make that computation?

Thanks

http://www.newscientist.com/article/dn16095-its-confirmed-matter-is-merely-vacuum-fluctuations.html

Virtual quarks make the calculations much more complicated, involving a matrix of more than 10,000 trillion numbers, says Stephan Dürr of the John von Neumann Institute for Computing in Jülich, Germany, who led the team.

"There is no computer on Earth that could possibly store such a big matrix in its memory," Dürr told New Scientist, "so some trickery goes into evaluating it."

http://googleblog.blogspot.com/2008/11/sorting-1pb-with-mapreduce.html

It took six hours and two minutes to sort 1PB (10 trillion 100-byte records) on 4,000 computers.
Are you sure that's not a bias you're bringing to the situation? I fail to see how it being a "poor country" comes in to the matter at all. You could say the same about any country, that people aren't going to turn down "free money" but in reality they do, on a daily basis. Do you have any evidence to suggest that this is the case, or is that just a knee jerk reaction?
Actions speak louder than words. And not just your own, but others' as well. Observe what people do, not just what they say (or write). Learn from it.

To do that, you have to participate.

Participating is challenging. Learn to stand up for yourself.

Take martial arts. Not to be a badass, but to maintain your integrity around jerks who use physical intimidation.

Actions speak louder than words. If you feel bad about what you are doing, likely it is impacting others negatively as well. (Even if it's just talking.)

We learn emotions, not just ideas. If you stay stuck in bad emotions, you will reinforce their patterns.

Two things you have in finite quantity and which, once lost, may not be recovered. Time, and your health. Guard both strenuously. I dont' mean that one should live in isolation; quite the contrary. But don't let your time and health be wasted or needlessly, stupdidly compromised.
"Speak up, change things or get out."
Unless, of course, changing direction would get you headed out of hell, rather than in deeper.
:D 2 firewall wouldn't help at all. Use one - the best one .
Well if the scholarship is being given out based on ability like you say, you have no choice but to give it to the students who perform the best.

If there is a need based component, it would be different. But you do not seem to indicate that there is. I'm all for leveling the playing field, but if the person who created the scholarship wanted it to be talent based, that's what you have to go on.

EDIT: We also have a relationship between income and test scores in the US. It's a fact of life.
Flip Title ( ǝlʇıʇ dılɟ) : write upside down
Adding Your Own Methods to jQuery
This is certainly a case where the loss is efficiency due to regulation is outweighed by the cost of a market failure. In all honesty, essentials like food, education, clean water, and so on should never be left up to the free market.
I think the "not available" page could have a lot of good uses.

A poll of "what are you going to do now?" would add to the experiment. Sure, not everyone will answer honestly but a good number probably will.

How about some kind of stats? Show a user list and "off-time" visit count next to each name, or let us enter a small string to say what we're going to do instead of HN. That might get abused but you get the idea. I think what we see while the site is unavailable is a great opportunity for quick-response community building.
If people saw that hypothetical page once, I think they might respond to it the way you describe.

But once people subconsciously know what that page looks like, I doubt they'll actually read the text -- they'll just come here, see "oh, news.yc is broken right now" and bounce off somewhere else.
Not necessarily, you're looking at time and dimensionality in a linear light (ba-dum-dusch!). In all seriousness, by traveling through time he could just be traveling to a parallel dimension (universe?) that has already progressed to a future time.
These claims don't contradict each other. At any point in time, there is no way Google stored any significant amount of records in memory. The scientist is not saying it's impossible to multiply the matrix, he is just pointing out that you need to use special algorithms that don't require the entire matrix to be loaded into memory at once.
Buy all the two-letter, three-letter and dictionary word domain names possible and retire before going to college.
Fast + Furious: How to Use Internet Marketing to Launch Your Startup - SlideShare
Micromessaging at work
The Benefits of a Movie Inventory
Albert Einstein's Blackboard
They mention that they did some extra optimizations ("stragglers") for petabyte test (that were not there yet for the terabyte test).
Sounds like a very informal study.
I really don't get why he declares that a &rest variable is of type list... Of course it is.
Interesting. I wonder if a bias or law against price gouging could be a problem, here. I mean, if you know there is, e.g. a 20% chance of a bad harvest that will let you sell food at twice the normal price, you probably have an incentive to store some yourself. But if you know that, in the event of a bad harvest, someone is just going to take your food -- or tell you that you can sell it, but only at an artificially low price -- you might not bother.

The usual question applies: can this complaint about the market be restated as a business plan? If so, why not do it? If not, is it really a solvable problem?
And the comments commenting on the meta discussions on the petty semi-scandalous incidents on the internets.

And the comments commenting on the comments regarding the meta discussions on the petty semi-scandalous incidents on the internets.
LG MG295 Unlocked
LG MG295 is a mobile phone in the clamshell form factor. The phone's dimensions and weight are: 90 x 45.6 x 19.4 mm and 80 g. The device is designed to operate on the following networks: GSM 850/1800/1900. It is necessary to note the presence of a digital camera 1.3 MP.
*I'll have my girls, you have your startup. Smiley face.
Yes, I have a lot of experience, and it most certainly won't work. Not only that, it's children between 6 and 12. They are not going to make rational decisions.
Well, the point of the scholarship is to give academic scholarships, but to those students who need it. I.e, encourage a culture of learning, but avoid giving to those who already have. It's not to be based solely on ability, but also on need.
I wanted to share a personal experience with status updates. We have a Campfire chat room and then wire almost every activity to send notifications to the chat room. When there's a commit, the chat room gets notified. When I'm playing phone guy, my updates to Highrise (CRM) show up in Campfire. So our status is usually represented well throughout the day.

Most people focus on the information sharing aspect of status, but I've found that the steady stream of accomplishments gives me more momentum. It creates some pressure since I don't want to be seen as slacking. More importantly, it feels like a small reward for completing a task since I know there's a slight bit of recognition of the accomplishment. It's like clicker-training a dog. With every positive step I take throughout the day, I get a reward.
1 failure per 50K in 6 hours is the same as 1 per 34 in a year, or a 3% annual failure rate. I think this is the same number Google reported in their disk failure paper.
Must be nice to live in a world where govt fails less often than markets.
As I understand it, your position is:

1. they will not turn down free money

2. not turning down free money would be rational

3. they are not going to make rational decisions

Surely all three of these cannot be true? Or do I misunderstand you?
Speaking of electricity, at one point Google claimed that during a Web search your PC uses more energy sending the request, waiting, and displaying the results than Google uses to perform the search. So the next time you need to sort 1PB of data it may be cheaper to send it to Google. :-)
Interview with Rusty Klophaus on the Nitrogen Erlang Web Framework
Oh my word, that's awesome..

Snore.
Actually, the government doesn't fail that much more than the market.

If you want to talk small scale, the government makes little mistakes every day. But the market fails every day too. Any transaction in which one party has incomplete information is a tiny market failure.

Large scale, neither fail that often. But when either does, it has drastic implications.

Honestly the idea that the market works the vast majority of the time is misguided.
Reminds me of a recent HN conversation about Mitt Romney being the best guy to "turn around a sinking ship."

So when you're going through hell in a sinking ship:

1) Keep on going.

2) Turn around.

3) Rearrange the deck chairs.

4) Ask Mitt Romney for help.
Don't look at test scores, look at how test scores change.
"create strong passwords"

I'm not sure it's that strong.
(: ¿ʇuɐʌǝlǝɹ sıɥʇ sı ʎɥʍ

"encrypt your download links to prevent leechers" -- "encrypting" links with this would be a pain in the ass when you can't perform the reverse process.
i'd suggest a simple split:

- select the highest scores on the exam without question
(these students will make your program prestigious)

- normalize actual scores based on "expected scores", select highest normalized scores
(these students will highlight your program's accessibility, rewarding those who do much with little)
This would have the added bonus of being the kind of decision that keeps the pipes from redditing up.
We also have a relationship between income and test scores in the US. It's a fact of life.

The original poster said test scores were 100% correlated with family income ("the scores of all the students is exactly the same as their family income"). I suspect this is an exaggeration; however if it's true then the correlation between income and test scores is vastly greater than in the US.

If the correlation isn't 100%, the OP could for each age group, rank-correlate all the children by family income and by test score and award the scholarships to the ones with the biggest difference.
Okay, don't try to confuse me with logic :) Fact is - asking the students does not work. It has been tried, it does not work, they just take everything they can get.
What about taking the override option off noprocrast?
Yes, the correlation is not 100%, but it's very close. Another issue is that the family income is not exactly known either, it's just estimated from the appearance. Also, the system has to be obviously fair - if you have someone estimating family income and grouping people in particular groups, the system is too easy to game.
Freelancers: 5 ways to make a bad economy sting less
Do you agree? Disagree? What else do you do to help the slump?
This can easily be gamed, as students will simply drop their scores one term, so they can get the scholarship the next term when they improve.
> Actually, the government doesn't fail that much more than the market.

If govt doesn't fail much more often than the market, why would one want the govt to be responsible for all essentials?

> Honestly the idea that the market works the vast majority of the time is misguided.

It's just better than govt.

I note that very few markets have set up death camps or waged significant wars. (The exceptions being folks dealing in illegal goods.) Markets don't try to throw me in jail for saying the wrong thing, markets not having jails and all.
Who decides what the 'expected score' is then? This would have to be the teacher, and as I mentioned in the post, the teachers cannot be trusted to be fair.

Also, it's just 2 scholarships per class of 30 people per term. So one just has to pick 2 people from the class.
Yeah, for a fee, you can find historical registration info and other goodies:

http://www.domaintools.com/
1. There is a big difference between understanding something and appreciating something

2. Remember that you can make the right decision and get the wrong outcome
> Large scale, neither fail that often. But when either does, it has drastic implications.

Large scale market failures are less common because there's almost always some diversity in a market.

Govts and regulation have no diversity. When the US govt's banking regulation fails, all regulated US banks are at risk.

That's why govts are especially prone to black swans.
New honeycomb tire is 'bulletproof'
> If govt doesn't fail much more oftenI note that very few markets have set up death camps or waged significant wars. than the market, why would one want the govt to be responsible for all essentials?

The article addresses this; because the government can be more consistent. Hence the need for them to be in charge of essentials.

> I note that very few markets have set up death camps or waged significant wars.

You're mixing apples and oranges here. How about a market that stops human rights violations?
1) Syllogisms are mostly used for manipulation
2) Everyone hates manipulation
.`.) Everyone hates syllogisms.

...nope, doesn't work for me.
If income isn't known exactly, then yes it is too easy to game. I suspect that what you are trying to do may well be impossible.

Since it is a tech scholarship, maytbe you could give it to those kids who have an interest in tech subjects over other subjects?

Or maybe, as another poster has suggested, you could award some of the prizes to those who make the biggest improvement on test scores. Of course, some kids may respond by deliberately doing badly in one year then doing well the next year.
"A final difference: After the 1929 crash, the nation had to wait more than three years for a president who simply wasn't up to the job to leave the scene. This time, we've got to wait only two more months." Golden.
This is what those OLPC laptops are for.

1)Pass one around. Use some flash based game, which is actually an aptitude test in disguise and measure performance on that.

2) Have a Webcam conversation with each (if 1500 is too many for your team, farm this task out. * ). See how they answer non generic questions and look for evidence for analytical abilities or quick thinking.

I think some internet-based live-streaming solution is the way to go as there is no corrupting intermediary that adds noise to the evaluation process.

*Would be cool to have a website where you can sign up to interview a student, something like MTurk, but limited to invited candidates?
Government and market are both human institutions. Market is really a convenient shorthand for "distributed, decentralized, uncoordinated economic transactions between consenting parties with an interest in the outcome", while government is shorthand for "centralized decision making made by disinterested economic actors". Please note that "disinterested" is a value-neutral term - it implies that the decision makers don't personally & directly benefit from the outcome.

Once you have the definitions right, you can see the benefits and pitfalls for both approaches. If you believe in maximizing human freedom, it is very clear that leaving people free to transact as they wish is the best path for all but those rare edge cases.
I can't believe he would be so foolish as to cite safety nets as proof that this won't be so bad. The biggest crashes so far have been in mortgage-backed securities (created by government entities, owned by government entities, insured by government entities, subsidized by taxes) and banks (heavily-regulated -- it's not like there's a Fed equivalent for oil companies, that would give Exxon millions of barrels of oil at a huge discount to market prices if they asked nicely). Meanwhile, loosely-regulated hedge funds are outperforming almost every other asset class.

Safety nets do not prevent crises; they delay and concentrate them. People take safety nets into account -- if we knew, for example, that the government wouldn't let stocks drop more than 20% without bailing us all out, we'd all leverage our portfolios four to one (not strictly legal, but you can do it fairly easily with derivatives). This would make the 20%-drop insurance plan far too expensive, and make stocks so overpriced that such a drop would be inevitable. GSEs, deposit insurance, social security, and unemployment insurance all have similar moral hazard issues.

The effective insurance plans are not the ones that reduce risk (assuming behavior is the same). They are the ones that reduce actual risk relative to apparent risk -- which means they could be the rules that increase risk. Unfortunately, "Effective, but superficially harmful" is a perfect description of a policy that will never be enacted by a democratically elected government, so we're stuck with what we have.
This is ridiculous. YOU decide when to waste your time and not the web.

And think of Europeans (like me) who eagerly read Hacker News.
> Any transaction in which one party has incomplete information is a tiny market failure.

That's an absurd definition because no one has complete information.

However, if we're going to use it to criticize markets, it should be noted that it applies at least as much to govt actions.

It's an odd sort of comparison when one points to a problem with one alternative while ignoring the same problem with at least the same magnitude for the other alternative.
Regression analysis. Then factor out the expected effects of wealth.
It sounds like a great idea to let government regulate food. A very successful country did just that, the USSR. Where people stocked up on such nonessentials as butter when the factories finally could produce it.

Point being, arbitration and market forces will always ensure there are more resources available because the government is a monopoly with no challengers. In any markets with competition prices are lower, and more output is available. Besides, does anyone really think that governments can better judge and hedge the risk of various crop failures etc than the market?
Nobel laureate George Smoot on the design and simulation of the universe
Dostoevsky said, "All happy families are alike; all unhappy families are unhappy in their own way."

The same can be applied to the economy. We aren't likely to repeat the mistakes that led to the Great Depression. Instead we find new and interesting ways to screw up. I doubt that this will be anywhere near as bad as the Great Depression. However, the claim that this time is different shouldn't be as reassuring as it sounds. We simply can't evaluate this depression(or recession if you prefer) in terms of the last. Its apples and oranges.
Don't all students who want to study technology in Africa need it? I would imagine there are not too many opportunities.

If that isn't the case, you could give the scholarship to the middle of the pack. This would maximize the need and intelligence of the people that the scholarship would help.
You cannot do this from your armchair.

Go to the school; you'll figure out what to do once there.
You can ofcourse unflip it by typing the flipped text in the first box.
No but it would be hella profitable. The rest of life would be more fun with an unlimited source of funds.
> It sounds like a great idea to let government regulate food. A very successful country did just that, the USSR. Where people stocked up on such nonessentials as butter when the factories finally could produce it.

No one is proposing "regulation" on the level of the USSR.

> Does anyone really think that governments can better judge and hedge the risk of various crop failures etc than the market?

Does anyone really think the government can better judge and hedge the risk of mortgages than the market?
A cheap shot. FDR ran on a platform of cutting spending, balancing the budget, and repealing prohibition. Guess which one of these he actually did!

Guess which President was the first to decide that a program of massive aid, make-work, and bailouts would be the best way to end a recession (hint: his initials were not FDR). Guess which recession was the first to which the US responded with such policies! Guess which recession was longer and more severe than any before it!
That doesn't sound unscalable. It's pretty linear scaling, they already store information per logged in user anyway.