skysight-inc/HackerNews huggingface dataset:
So in a free market, all entities inherently act in the best interest of their customers, regardless of whether or not there's competition? And they wouldn't use their position to do anything anticompetitive to maintain that position and keep the "free" market effectively closed?

My point is, without some entity establishing rules of fair play, the market isn't a free one, because one player can gain enough power to effectively close the market to any competitors.
When your key is a phrase, for instance?
A hash of { phrase => count } is a pretty common data structure.
Adriana Lima : Sexy Adriana Lima 2011 desktop calendar
This article was really inspiring to me. I realized that "I don't do enough of what I do", reading it.

...the lock picker practiced disabling every type of lock in use in all the major hotels in Dubai.
What's the name of the law firm? This is criminal.
Freedom of speech, the right to drink from any fountain you want or the right to perform any other human action is not given to any man by a GOVERNMENT. Do you think man was not free to do any of these things before they elected a president? The only thing that GOVERNMENTS have done is provide a selected group of men with the power to deny actions to the rest, by using "regulations" that restrict actions. Don't forget that segregation was sanctioned and made possible by the GOVERNMENT. No individual has the power to institutionalize something like racism, only the GOVERNMENT has that power. GOVERNMENTS have done it in the past, and probably will try again in the future as long as we keep giving them the power regulate us.
<nitpick>456</nitpick>
Lying isn't right, so I tell my children. I am sure consumers don't really understand the technology, but clearly stating something you don't have is material the FTC should be investigating.
A note of caution: If you choose to read the actual suicide note as I did, be prepared for a rather intense emotional trip. I don't mean to say that to dissuade anyone from reading it, but just that... I was not expecting what I ended up reading and it really made me sit back and, I dunno, just be sad for awhile. Such a shame.
Closest to Econ or Finance. I think you're in a pretty good place to be relative to e.g. Social Work.
Something like this? http://xkcd.com/576/

In fact, it turns out somebody has already built such a thing for TradeMe, the local New Zealand equivalent to eBay - see the Twitter feed of its purchases here: https://twitter.com/trademe_xkcd576

Now that I actually check the Twitter feed, it seems the same guy has created a public version at http://mysterybox.co.nz/
Unfortunately there's a hard cap on the number of potential users in the world. There's about 7 billion people in the world right now, even if we're generous and suggest that 50% of the population are potential users you're limited to a 3 billionish maximum. They currently claim over 500 million active users, which means they're already at 1/6 of their maximum. Furthermore, the law of diminishing returns suggests that their growth will slow as they get close to their population limit. So unless they can figure out how to monetize those users in some significant way I can't see the potential for revenue being close to what the current evaluations are suggesting.
One wonders if Google is becoming the new Yahoo. If so, a big opportunity for the likes of DuckDuckGo and other nimble searchers. Today's upstarts can also run on the cloud, sidestepping the need to build Google-scale data centers (at least initially).
Ask HN: Is my concept and current application viable?
Hi

I have recently finished the first launchable product of my "startup" for want of a better word. http://jv.binkd.com

The basics - it connects businesses to join together to form joint ventures to help increase both of their businesses.

I am struggling to get constructive feedback on this application and whether it is actually ready for me to take further or needs a different approach.

The responses from people testing so far have been, looks good, good concept and it will be useful once you have lots of people. Not exactly confidence building.

So my question, is it just the people I require to increase it's usefulness and am I ready to hit my foot on the accelerator?

Currently this is funded by my other business activites in Binkd (Jvee is a small off shoot), I am not new to business but am fairly new to web based startups and the startup community in general.

I would be very greatful for anyone who could offer constructive feedback on this application and concept.

Thanks
Adam
Ask HN: How to scrape and is it legal?
I want to put together a website with a collection of data from other websites. I'm simply putting all the info in one place instead of many.

What's the best way to scrape this data?
UPDATES: What if the data changes, will I be able to set it to rescrape or do I have to do it manually?
Is it legal? I know its someone else's data, but is it the same as copy and paste?

*I have no background in scraping and only a little to make the web app, so any advice would be very helpful!

Thanks
Can anyone get profit/revenue to valuation ratios on pre-IPO Google so that we can do some more rational comparing?
well, I don't pay for Dropbox and I am quite happy with how they have solved this problem

so I guess to answer your question... i would not be one of your paying customers and my interest in having to setup a service on all of my machines that is anymore complicated that Dropbox would also be near 0
Irony is no fun when it is explained.
I imagine what you're looking for most is people for reliability and/or scaling?

The jobs link is down.
The same place as the magical $15B valuation Microsoft set way back when.
I don't get it, are you trolling? You're talking about over a decade ago, back when NT on Alpha was relevant. A lot changes in 10 years.
Apologies. I skimmed through the first few paragraphs and thought that it was a summary. I was attempting to help HN cut back on multiple posts on the same old content. Ah well, I guess I won't be doing that any more.
UConn cardiologists uncover new heart attack warning sign
Munster Says Google Has 'Given Up' on Social Networking
Linkity!
http://jv.binkd.com
Indeed perhaps time to give duckduckgo a try. They seem to actively filter out all ad sites. I've been going nuts in the past couple months with google searching for technical solutions to problems, command clicking on links to open in tabs and discovering that most are clone ad sites from a question on stackoverflow.
I've seen something that is exactly what you describe. I can't remember the name though. Not that that should discourage you from creating this.
Do you have any insights into why then x86 chips consume so much power than the ARM equivalents?

Btw, I tend to agree with you,and think Intel's engineers are just being lazy wrt power. I expect this Nvidia chip to get crushed technically when Intel's engineers gear up and really work on power (like they did when Transmeta's Crusoe came out).
They never said the ping pong balls had to stay as ping pong balls.

I'd be thinking about filling the tires with liquid ping pong balls too, but maybe I'd be missing the spirit of the question...
Next generation nothing, barring a major and sudden economic turnaround I expect this to happen next year, by which I mean, this next fall school year. Somewhere in March I expect a few little nervous merpings to trickle out about how college applications seem to be a bit down this year but we're talking ourselves into believing they'll pick back up by the start of the school year, and somewhere in June or July for this to break as a major national news story/obsession after it doesn't pick back up. The obsession will center on how this is yet more proof of the inevitable decline and dumbening of the US and how important it is for high school graduates to apply anyhow. And it will all be wrong, because a return to some cold, hard cost/benefit and affordability analysis is not merely what we need, but in fact will be utterly unavoidable, and this will simply be another consequence of that broad (but quiet) trend.

I think I'm going out on a limb a bit here, but it's plausible. I feel much less like I'm going out on a limb to say that if the economy is still sputtering along in 2012 that this is almost certain to happen. (To avoid this we must not merely tread water but see noticeable improvements in our economic outlook.) I'm also curious as to how the bubble pop in higher ed will manifest, because none of the historical precedents for bubbles like this have the college admission cycle to contend with, and most of them involved resellable goods, even if the value did tank. The housing market can collapse overnight because anybody can sell a house any time, college can't be sold and can only be entered at certain times. Still, fireworks will occur somewhere and sometime, for better and for worse.
Most websites have terms and conditions saying that automated retrieval of their content is prohibited.

It is generally all up to the Terms and Conditions of the site.

If a site wanted to share it's data it would normally give you an API.

You could email them and ask for permission if they do prohibit it, depending on your purpose.

And scraping does have its drawbacks such as changing site design and having to change your code with it.

The best and only way to scrape is just to retrieve the webpage in your application and parse out the results you want. You could do things such as XSS but it all depends on the complexitiy of the data. Normally the split and substring and indexof functions are what you will use.
Four good reasons to skip the Amazon Appstore
Thanks for the candid response, it's tough to get that from friends :)
At it's highest, I recall Google's was around 100:1. This puts Facebook's at 250:1.
I didn't make the original claim, so I don't see why it would be my job to prove or disprove it.

That's sort of beside the point though. I'm just tired of the ridiculous amount of misinformation in the diet, nutrition, and exercise fields. It's not like there isn't a long history of claims that are "just common sense" turning out completely wrong ("eating fat makes you fat", "eating high cholesterol food gives you high cholesterol" (mostly genetics), "diet and exercise are equally important to losing weight" (reality is that it's mostly diet), etc.)

Bottom line, if you're going to make a claim that certain food improve your brain's performance, you should have just a shred of evidence.
Well said. Corruption in India is indeed a systemic problem. While there is no denying that the system of governance is corrupt, it is also true that citizens are ready to bribe as long as their work gets done.

It is in this context that IPAB believes in a systemic approach to tackling corruption. We do not target individuals as they are dispensable and replaceable. It is the system and processes which need to be changed. Alongside focusing on procedural change in the government we also focus on greater knowledge and understanding on the part of the citizen. We work on the belief that citizens who are well-informed of their rights wrt govt services are less likely to indulge in paying a bribe.

To this end, on our site we provide Citizen Charters which are documents of service timelines published by the government. Armed with this document, citizens are better enabled to stand up against corruption. On the govt-facing end of our campaign, we have come out with Janamahiti report for the Transport Department of Bangalore which analyses bribe reports received on our site, pinpoints the most corrupt processes and puts forward suggestions to tackle corruption rampant in this department. We are also briefing the Transport department this week on the Janamahiti and the recommendations it provides.
Gucci Bags
what is the process for making a link active in HN.

Do I do html? BBCode?
It's a scam of the pump-and-dump variety, plain and simple. Look forward a few investors and investment banks making out like bandits and the rest getting screwed.
links in the body of your post will not become a hyperlink. only in comments do they become clickable.
It's more like Facebook. Minus all of the content about your single friends drinking and all of your married friends talking about how their newborn baby just pooped/peed/barfed on them. Add people asking smart questions with domain experts answering (I've seen Ev Williams and Dennis Crowley answer the same question.).
http://www.quora.com/What-is-the-process-involved-in-launchi...

It has real identities like Facebook. The information density and structure of Wikipedia where each page updates in real-time. The follow model of Twitter where you can follow smart people and topics.
You might not know me as well as you think you do.
You would like this thread: http://news.ycombinator.com/item?id=1540621

:-)
Unless you use HTTPS, which currently depends on the appspot domain.

In any case, thanks! Very welcome upgrade.
And I regret not blogging about it on my personal blog and linking that here to generate traffic and potential subscribers there. LOL.

Of course, it may not have actually been a wasted opportunity as the (google.com) URL part of the title probably accounts for a large percentage of clicks.

P.S. I upvoted your comment to generate a little more karma as compensation.
SONY VAIO PCG-7Z2L battery
SONY VAIO PCG-7Z2L battery
5200mAh, 11.10V, Black Li-ion SONY VAIO PCG-7Z2L battery replacement
SKU: SN036B
Type : Li-ion
Just fresh out from factory, Brand New
Capacity: 5200mAh
Grade A cell, 1-Year Warranty
Voltage: 11.10V
Overcharge and overheat protection
Dimension:
Fast Shipping to the World
Net weight:
http://www.laptopbattery-shop.co.uk/sony-pcg-7z2l-battery.html
The sad thing is that for a lot of this sort of propaganda to be effective, it don't have to prove anything; it suffices to just raise the question and spread fear, uncertainty, and doubt.
You beat me to it. I also bought the same bundle because Tweetie 2.0 made it a really good deal. Then boom Twitter bought the iPhone app and employed the dev and we where left in the dark. Now if it's free, I'll feel a tiny bit cheated. If it's not and they don't honor the deal I'll be pissed.
On the contrary, I wish javascript had more Rubyisms.

Don't you get sick of typing function(){..} when just {..} would do?
The State of Business Strategy in the Mobile Phone Market
This could be useful: http://www.reddit.com/r/bachelorchef/comments/ewcua/pdf_thou...
How would this be rational? Facebook is not Google. Google's primary market (search) is nothing like Facebook's (personal communication).
Ajays, we beg to differ. IPaidABribe believes that corruption in India is systemic and only procedural change can make a difference. Merely booking corrupt individuals is a band-aid solution to a deep-rooted problem as they can easily be replaced by another corrupt officer. Instead, if there is a transparent and accountable system of governance in place it would naturally reduce the degree of discretion which officials enjoy today in India. This, we believe, would help build a corruption-free India.
Why Do Schools Fire Losing Coaches But Not Bad Teachers?
There most certainly is something about Greece.

If you have Firefox as a web browser, I suggest you have a look at the little bar that pops up when you hit "Ctrl + F" ... note the two citations, one for Der Spiegel and the other from USA Today, in the below section...

""
Goldman Sachs between the years 1998-2009 has been reported to systematically help the Greek government to mask its national true debt facts.[72] In September 2009, though, Goldman Sachs among others, created a special Credit Default Swap (CDS) index for the cover of high risk national debt of Greece.[73]
""

By assisting Greece in hiding this information, the apparent level of risk of the debt was lower; people who bought Greek debt as a consequence under-priced the risk. This can be demonstrated by reality, as when the info got out, Greece's cost of borrowing went up rather than going down.

Therefore, GS' involvement in this matter led to investors losing money they otherwise would not have, which is ... unethical.

Helping a country or company withhold information that is known to be material from investors, is unethical.

No different than telling someone the tires they are buying are rated to last 60K miles when actually they are only rated to last 30K, except you have blotted out the big sticker on the tire that states that.

That the SEC, as for example in the case of Madoff, was told explicitly for years beforehand of the fraud, and did nothing, is a matter of public record.

A more skeptical person might therefore wonder what else the SEC is remiss at investigating. However you have the right to be incurious if you so choose.
Let me guess, you do all overall design work, train staff, stay late. Then someone else gets the credit, because his best friends with the manager?

That was my last mental breakdown.
Microsoft Surface 2 is real and it looks amazing
Ironically that was sarcasm anyway.
Neal Stephenson's novel Anathem has a section that talks about how the 'reticulum' (the internet, in the book's fictional world) was overrun with false copies of documents with slight changes made to them. 99.99% of all of the information on the internet was spam.

A huge industry of commercialized systems connected to the internet for the sole purpose of filling it with spam, and then the corporations would sell back filters and knowledge of which documents weren't spam to customers. Eventually, the algorithms used to modify documents developed a malicious edge, so that the thousands of spam copies of an original document would be deceptive in ways that would harm people (e.g., in Marco's electrical plug wiring example, the document would have been modified so that it could get you killed by telling you to touch the wrong wire or something.)

Inevitably, it spiraled out of control, and a sophisticated system of social trust and ranking was put in place by IT workers and systems administrators, which are a caste and race of people in the fictional world.

Good book. Prescient, even.
This is much ado about nothing. Google has a few search problems, and they always have, and they always improve.

Also, if Marco is going to list some problems, how about listing some problem searches? I search for what he lists, and the top result is fine in most cases, and debatable in others.

You folks think Google sucks? I don't. It's awesome, and I rely on it more everyday.
Both of them market in advertising. The delivery method is different, but it's still advertising.
Powerset worked well enough that Microsoft bought them for $100 million.
Is China closer than thought to matching U.S. fighter jet prowess?
Steve Ballmer Kicks Off CES 2011
Holy cow, you're right! I knew Dalvik was built with the help of Harmony, but for whatever reason, the broader implications / possibilities of that never really clicked. Thanks!
So it's nothing like Facebook.
Interesting point about the fragmentation. Perhaps integrating all these HN services could be helpful...
Wait... what? They're making it so that overbooking will happen more often and so that people will get less compensation for being bumped from their flights.

I'm curious as to why you think that's good for me, the consumer? More "effective" pricing means that the price I pay will get closer to the maximum price I'm willing to pay. In short, it means that I'll pay more. The article discusses this, with the airlines taking away as much of the "consumer surplus" as possible--i.e. getting people to pay them more money.

I'm really, really confused right now as to why you think that having to pay more would make me want to fly more? Surely I would want to do things where the benefit to me is higher than the price, not where the benefit and price are as nearly equal as possible. After all, I can do only so many things, so the opportunity cost of doing that instead of something with more benefit (e.g. taking a road trip instead of flying, or taking a cruise at sea) makes flying a losing proposition all around.
What really irks me in the last few month is that google increasingly doesn't actually answer my questions. More often than not, none of the results on the first page contain all of my search terms, and most of the time it is the most specific term that it is missing everywhere. Or the big G has replaced that term with something completely unrelated. I have to prefix every search term with a + if I want to get a result quality that is even remotely similar to what use to be the default.
Let's say that they have quadrupled that, so $3.2 B revenue, and that they have greatly reduced their operating expenses (new servers, etc.) - so let's say they are making 50% net income, which would be almost impossible - that is still at best-case scenario $1.6 B a year, which hardly justifies $50B .

Even the most optimistic growth targets are going to bump up against hard limits: people can't spend more than x hours a day online, there are maybe 5 billion users worldwide as a potential market.

Unless the CIA is going to pay them a couple billion a year for data mining access (CIA's VC arm was an early investor) since we have all so nicely classified ourselves and our social graphs for "TIA" purposes, I can't see the valuation sticking.
As much as I like seeing MS getting shafted, Google was pitching hosted email solution whereby they would run a "government only cloud". Telling them No was a very sensible thing to do. I'd rather have the government use an Exchange server than share its internal documents with Google. Apparently the govs are of a similar opinion which is quite telling.
If I'm understanding your methods correctly, you only have data for earnings by occupation. So when you say "Computer Science majors earn x." it actually means "People who do the same jobs that computer science majors tend to do earn x."

If that's the case then the data is not really actionable unless we also know the probability of getting a job with and without some degree.

For example, if computer science majors all tend to get job A, but having a computer science degree makes you no more likely than people without computer science degrees to get job A, then the value of the computer science degree is 0.

Of course having a degree does usually provide some advantage at getting a job, and in some fields it's impossible to get a job without a degree.

But still, for this data to be truly meaningful and actionable you must multiply the value of each degree by the amount that the degree aids you in getting a job, which is a hard value to determine.

Edit:

Here's an example of why this is important.
Let's compare physical therapy and programming.

According to the data, a Doctorate in Physical Therapy has a value of about $1.2 million, while a Bachelors in Computer Science has a value of around $1.5 million.

You might conclude that the computer science degree is a better value, especially since it costs less than the doctorate.

However, you cannot be a physical therapist without the degree. So the value of the P.T. degree is $1.2m * 100%, or $1.2 million, because without the degree you have a 0% chance of getting the job (and assuming that having a P.T. degree guarantees you a job).

You can be a programmer without a computer science degree. So the value of the degree is going to be $1.5m times some percentage less than 100. In order for the C.S. degree to be more valuable than the P.T. degree you'd have to be 80% more likely to get a job than a programmer without a C.S. degree. (1.5 * .8 = 1.2)

It doesn't seem like people with C.S. degrees are 80% more likely to get jobs. Especially when you factor in the opportunity cost of going to school. That's a lot of time you could spend working on open-source projects, building websites and businesses, etc. which are all things that count more than a degree.

So the value of a C.S. degree could very easily be less than the value of a P.T. degree.