skysight-inc/HackerNews huggingface dataset:
I looked into using some kind of NoSQL solution at work, and one thing puzzles me. Assume a key/value database, where values are JSON documents or something similar.

Using such a database, with the key being customer account ID, and the value being a document that contains all the information about that customer's account would be convenient and useful. Information about the account would include things like name, address, email, status of the account, when their subscription to our service expires, re-billing information, all their purchases, and so on.

This would definitely be a lot more convenient than our current SQL-based solution for two of the three things we do with our customer data.

One of those things is display a customer's information for our customer support people when they are dealing with the customer on the phone, or working on a ticket in our help desk. With the NoSQL, it would be a simple get the JSON document for that customer, and display it nicely.

The second thing is providing information to the installed software on the end user's computer. Our software provides a service that can partly be done locally, and partly requires back end support. All the back end information needed to support a given customer's installed software would be in that JSON document, so again the key/value store would work great.

However, I'm at a loss as to how the third thing would be handled efficiently: reporting.

For instance, suppose I'm asked for a quick report on how well a new SKU has sold over the last week. With our SQL database, this is a trivial select from the "orders" and "line_items" table. It is almost instantaneous. With the NoSQL solution, it would appear that I would have to go look at every single customer's entry.

All in all, we've got a couple dozen regular reports, many running hourly, and a few being interactive on-demand via a web interface, that are fairly straightforward and efficient in our SQL-based solution (some might involve joining 6 or 7 tables, but it is still straightforward and efficient).

Thus, at first glance, it would seem that going to NoSQL would be a nightmare for my reports. Many times an hour, it would appear that the database would get slammed by reports that would have to be accessing every freaking record in the database. That doesn't sound like something that's going to make me happy.

How do people deal with reporting when using NoSQL?
The idea--which I am rather partial to--is that my email address is my information, not yours.
His page at http://people.ischool.berkeley.edu/~hal/ says he's retired from Cal and now at Google.

I've bookmarked his page of academic papers. Unsurprisingly, since his getting affiliated with Google about 8 years ago, he's written on economics of Search: http://people.ischool.berkeley.edu/~hal/people/hal/papers.ht...
and on the Google Library Project.

Sure wish he (or someone) would be funded to do an update of this (2000 pub date) study. I wonder what additional variables might need to be factored in by now.
I don't know, it won't make a change to the way the place is administered to reduce the budget, it's mostly going to affect the amount of money scientist can use to advance their research. Making the quality slowly decline more.

Restructuring NASA is a hard problem maybe even impossible. But I do think it's still producing good enough stuff that removing it would be a mistake. I'm a believer in public research. It has produce some incredible benefits to society in the 20th century and hopefully will continue in the 21st.
The point is that you need someone to prevent name conflicts, and that person/organisation is a single point of failure who can be pressured by governments/corporations to revoke names they don't like.
People always hate the redesigns. Then, three months later, they revolt against the NEW redesigns because they love the current one so much. Facebook's kind of abusive like that.

They also usually cycle between adding new functionality, and then stripping down the designs to its essentials. This is more of an Essentialist design, which I like; I've been waiting for a redesign since their last one added all the weird things.
Wow.. my feed (primarily non-technical) couldn't be more different. I'd estimate it abouts 80% status updates and comments on the same.
I have an Entourage Edge and it does have a nice big screen, I I know it supports PDF's I use them all the time. Not positive about table of contents support, but I can try that out in a bit.

It does have a lot of issues though. It has two screens, one e-ink, and one LCD. It uses Android 1.6 right now and has no access to the Android Market. The company that makes it has minimal resources and they are really not supportive of developers. The LCD screen is resistive touch and its responsiveness isn't great. You can get a very long time on a charge if you make sure to turn the LCD off while you are reading.It isn't the worlds greatest e-reader, but it might suit your requirements where nothing else does.
Thought experiment: imagine you are going to port Bugzilla to a noSQL solution. How would you do it? There are some pretty obvious relationships between bugs, authors, commenters and comments.

Imagine a bug that has 100 comments from as many commenters. A normalized soluion with no joins is going to require at least 200 round trips to look up all comment and commenter information, so the need to denormalize is pretty obvious.

So let's say you enormalize all the way and save all relevant information in he bug document. Each bug has comments embedded in it and each comment has all of the commenter's information in it. Now each bug requires one lookup, which is awesome, but what about the fact that commenter information can change? Sure it's a somewhat rare event, but users can have thousands and thousands of comments. When a user changes their email address, do you tell them, this might take a few hours, in the meantime I hope you still have access to your old address? What kind of load do those massive updates place on your system? Unlike reads which are fairly steady and regular, these high load queries will be sporadic. How do you plan for and accommodate the spikes?
Added to wishlist.
Your review table can still only contain the user_id, book_id, and rating.

While using an RDBMS, instead of using joins, you want to get the user and book as separate objects anyway. The three objects (user, book, and review) should be cached separately, so you can use them in the duplicate places around your site. Of course, where you have a list of items to get users or books from, you will want to query your cache and DB using multi_get or in (id_1, id_2, ..).

The idea behind these scalable sets are that we end up caching our data differently than how we store the same data in a relational set. Later, our code is checking our cache, first, in one fashion and then the relational store differently; why not query both the same? We can now "simplify" how we think about and consider our data models, whether it be a k:v, document, graph, or otherwise. We end up building these models on top of the RDBMS anyway, in our caching layer.
Gotcha. I didn't realize he meant growth.

Thanks.
I think it's becoming politically incorrect by means of an activism campaign. That's an uncomfortable thing to stand against. The trick here is precisely what you point to: it's very easy to take the question to ad hominem territory. If you say that hard work is not enough, they will say that's because you're an elitist.

About the public school system, I didn't mean to put its origin in socialism (I believe it predates socialism), but to describe a mindset. In fact, I believe it works. Soviets powered the chess in the education system trying to prove that they could surpass corrupt capitalism... and they succeeded. I mean they succeeded dominating chess. But IMO that's not enough to prove anything :-)
Embrace Your Losses - They Will Make You Stronger
From my perspective this is more structuring of data. If you look at the profile in the example its a very clear organization step.Advertisers will now have a better and easy targeting mechanism than the usual demographics info.This I think is more of user training where people enter the data on how facebook wants to see it; why would a friend of mine who has known me long enough need information in this structure?

Facebook is now going to be able to sell super targeting techniques to marketers. Google relied on machines to do this, Facebook just asked its users to do the same;needless to say its a master stroke :).
Potential NVIDIA/Intel settlement could mean good news for Apple
Ask HN: What is the difference between the terms "The Grid" and "The Cloud"?
Why not keep a count running in the SKU's document?
Internet Privacy: Firefox Makers Look at Hiding Online Footprints
Since you're so interested in technology and science you might consider earning a PhD in biomedical research/biomedical engineering, genetics, genomics, or something else that interests you. The amount of actual science involved in the day to day practice of most doctors is negligible.

With your interests I'd suggest looking into MD/PhD programs - granted, that's a long time to spend in school (7+ years for both doctorates, then residency or postdoc or whatever), but if your interest is really in medical science, it can be a great opportunity. Also, these programs generally pay your tuition plus a stipend the whole time you're in school, so you can live comfortably instead of acquiring hundreds of thousands in debt.

Good luck!
You forgot: Make sure you stay logged in constantly, checking for changes, just in case Facebook change their privacy settings again and make stuff public that previously wasn't.
Could you explain this ? I am confused now.. I was under the impression that your email address is not exportable if you do not display it http://www.facebook.com/settings/?tab=privacy. So is not similar to me manually writing it down on paper ?
My main interest in a decentralized DNS system is to make it easier to do business with countries when the US decides it wants to steal their oil, and creates sanctions You couldn't purchase .ly domains until we stopped bullying Libya in 2004. If we build decentralized systems outside of the reach of the government, maybe we can even build a world free of petty politics and corrupt governments (like our own). It's a pipe dream, but it's the best dream I've got.
Considering that currently about 1 in 3 Americans die from mistakes made by doctors, limiting their independence might not be such a bad thing.

[1] http://www.ourcivilisation.com/medicine/usamed/deaths.htm
I really hate the photo scroller. I would love to hide the whole object, not just individual photos. Every time you hide a photo it just fetches a new one. Really annoying.

Update: I had to press hide on several hundred photos for this to go away. Nice update.

Update2: There appear to be facebook partisans that think not having choice is a good thing? Really?
Secret santa anyone?
I don't know what 'I need the data to be transactional' means. One way to get isolation is to put a transaction in one document. Often this is easier and more flexible that trying to take the 'car to pieces' everytime your park it. It is also more efficient in network bandwidth.

Everyone wants their data to be correct for some definition of correct. Normalized data is incorrect if the expectation is that relationships are preserved as the author created them, rather than re-created as they would be given the current database state.

For example, say a financial transaction record has an account holder and their related address.

For a naive implementation, if the address on the record should stay as it was when the transaction was created then:

* The normalized implementation is wrong
* The de-normalized implementation is right

If the address should be resolved at query time:

* The normalized implementation is right
* The de-normalized implementation is wrong

A lot relational tools (Foreign Keys, Unique Keys) become more complex for temporal schemas, which address these kinds of issues.
I think a single day is a poor timeframe in which to look for improvement. You can't better yourself in all subjects that interest you in 16 hours.
I just can't sympathize with this viewpoint at all. Your email address is the first thing you give out to your contacts, so that they can contact you. Before Facebook, anyone who wanted to contact you via the Internet would need to know it. A "friends list" is totally useless without this one vital piece of information.
I would consider using dev chromium so you have the option to hide the system title bar and push the [x] [_] [[]] down inline with the tabs. Otherwise great links!
So "friends" need your permission to email you ? I am comfortable with the idea that my friends will store my email address in their personal address book (or data store)...
In the past I have attracted older people who want to partner, so although I welcome advice on form your feedback is flawed.
Web Crawlers and Privacy: The Need to Reboot Robots.txt
Can't you just run a bunch of API calls, then zip the result?
Hudson is great. It's one of the few open source CI servers (that I know of) which can be configured without mucking about with reams of config files.
I was thinking about peoples' usage of facebook last week and wondered about how often people used the news feed as their go-to page, as opposed to their own profile. I'd contend that it would shed some light on the extent of peoples' narcissistic tendencies.
What I mean is that making reviews happen is not a cost borne by publishers, thus is not a cost that should be recovered by publishers in the form of subscription fees.

Finding reviewers is hard: Good academics don't like to do it, because it doesn't contribute to promotion; bad academics are happy to do it because for them it does (at XYZ state), but they are at XYZ state for a reason, which means they aren't the best reviewers.
Might be worth trying to make an infoproduct: http://unicornfree.com/

(like e-books, screencasts and things like that to help teach people things, distilled information certainly has a value)
That would be a better definition of "passion" than of "talent".
Yeah, same here. It seems a bit buggy; a couple more examples: it's impossible to say you live in Paris, France (only random cities elsewhere in the world named Paris), I can't turn on chat while looking at any of the profile pages (but can turn it on on any other page), etc..
It seems to me that Amazon has set a dangerous legal precedence for the hosting industry in general. Except for DMCA takedown notices, ISPs have been able to uphold that they do not own their customers' content (the earliest big case I can think about was when the Scientologists sued Netcom, and everybody else in the world).
Is it just me or are the advertisements when viewing a profile larger and more in focus?
Two Thanksgivings - Controlling Workaholism
I should have been more clear - I meant that marketing expenditures on SEO and SEM are still growing year over year, though not at the rate they once were.
In 10 years:

There will be cures for a few more things than now, but the average person won't be able to afford it (if you live in the USA).

A ride in an ambulance will cost $50k instead of $10k today (if you live in the USA).

An aspirin in a hospital will cost you $500 instead of $100 today (if you live in the USA).

Somehow the masses will be convinced to still fight against single-payer and defend massively profitable insurance companies and hospitals (if you live in the USA).

But if you want to be a doctor to help people and can maintain that attitude for a decade, then I applaud you, for what it's worth, you are truly a noble human being.
Why do you hate it so much? People can just as easily get your photos from your Photos library instead. Is it simply an aesthetic complaint?
Ask HN: What Technology does Optimizely use?
I was playing with Optimizely's experiment creator today and I really want to know how they grab any website and then allow the user to interact with it.

I want to do something similar, but instead of editing the code, I want to allow the user to select a region of the page and take a screenshot of it.

If someone can comment on how they do that, I would really appreciate it.
Julian Assange's Barrister speaks out
If an SQL (any) database is fast there are a few possibilities:

1. There is an index
2. The database is in memory and a scan is running in process
3. The query result was precomputed
4. The query was not that hard

With couchdb you could create a view on sku and week, and reduce to a count.
Any references to previous work (eg. deviantart, etc.)?
I'd be happy to contact you in the future.
I thought it was going to be just the dev build, but in Chrome stable it doesn't load either. So, not sure why saying so provokes downvoting...
I've started asking for non-tangible gifts recently. Things like piano lessons, sailing lessons, rock climbing gym membership, Flickr Pro subscription, etc.
Hehe, yup, I checked too. :-)
This is nice - can we get the iPhone version UI to look this good?
Clickable Link: https://optimizely.appspot.com/edit#url=http://tabtrick.com
Medicine faces a dystopian future.

Doctors are being paid less and less while patients expect more and more (and they expect it to be free).

B/c of HIPAA, most of the obvious IT innovations will be stymied by bureaucracy and stagnation. The recent backlash against personalized DNA testing shows that regulators are looking forward to squashing that as well.

Waits are long now and will be longer in the future. Doctors are harried now and will be more harried in the future.

Most speciality medical care (other than plastic surgery and other borderline areas that people willingly pay out of pocket for) is a financial racket akin to the banking fiasco. The AMA keeps supply of specialists low to keep salaries high, and battles are fought over regulations that lead to more billable procedures, gaming of the medicare system, etc.

Many commonly used procedures and drugs have very little effect and drain money from society. If you think critically and read the studies you will realize this. Then consider the Billions being paid the respected specialists who do these mostly useless procedures. Consider the influence they have in hospital policy and government policy.

Medicine in the US is a phenomenon of the American psyche. We have too much faith in the latest buzzword or technology to solve our problems, and we want to cheat death. Meanwhile, medical diagnoses are the only validation provided to so many desperate souls whose miserable lives are spent in constant complaint and agony, defined by their illness.

So many doctors traffic in the misery of these people, and the institution of medicine legitimizes it (for profit).

There are some niche areas where science is actually making amazing progress (some kinds of cancer research, mostly) but overall the medical class (those enriched by the status quo) are exploiters of human vulnerability, hope, and malaise.

Yes there are some amazing doctors and most I've met are incredibly hard working, caring people. But there are so many lies so deeply ingrained into our system that it will be hard ever to change it.

If you're a 60 year old man with significant arterial blockage, it's considered normal. You'll be sent home on an expensive pharmaceutical and told to switch from steak to chicken. The population of aging adults (55+) are the human batteries that power the medical matrix.
I like it, but the Education info seems buggy. I added half a dozen classes to my college, and now have half a dozen college entries each with one class. :-(
Nope. In the FAQ, it says "You will not be able to switch back to the old design after upgrading."
In the ideal world we would have hypertext as Ted Nelson intended it, but in this world contributing to such a thread does not pay the bills for a scientist. He is financially better of by writing a three page paper that is mostly filler around a valid argument. So, that is what he will do.
Makerbot :)
It modifies the existing rules, which are already complex and require knowledge (or diligent research) of the tax code.

Edit: OK, I suppose it's not that obvious. but if you keep reading, you'd see the next part of the law adds a 5 year carryback for certain small business investments, which is basically a retroactive tax credit allowing some small business investors/owners to get a refund of some taxes paid over the last 5 years. Like, say, capital gains from investment in a small business.

That's not a full explanation (or even a meaningful one), just an example of how different provisions can interact to give results that are no obvious when they're taken in isolation. You should read the whole tax section, there's a lot of interesting stuff in it. Reading bits of laws in isolation from their context often gives a misleading or incomplete picture.

Here's some accessible and more detailed information provided by an accounting firm that provides a better overview of ways this law might benefit startups and self-employed hackers: http://www.mohlernixon.com/informed/2010/09/focusing-on-the-...
That's my basic response. I have a single public Amazon wish list that I keep updated with little things in the "buy later" category and I send that to those who want it.
I have been inundated with an amazing response, I know I need to update "webcrunchers.com", My "wiki", and my Facebook page. But due to my lack of being able to type at normal speed, I'm forced to use Mac Speech Dictate, which breaks when I get a skype call, or type with my thumbs. I can use a few more volunteers...
"
I think chronic disease is really interesting, and I have a particular interest in using vitamin D as a potential treatment for certain chronic illnesses. Much of the vitamin D research has been observational research, and while it shows tremendous promise in preventing heart disease, certain cancers, and diabetes, more needs to be done to establish whether these diseases are caused by a deficiency, if so to what extent, and whether correcting this deficiency has a treatment effect.
seconded. if you can, read '4 hour work week' too, to figure out what might make you the most cash with the least time.
Using,Understanding,and Unraveling The OCaml Language
I worked with Eduardo (the author of the post) at Sun and I can tell you that when he says that Kohsuke is among the top 3-5 engineers that he has worked with, that REALLY says something.

Eduardo was a Distinguished Engineer at Sun and as such, he worked with Sun's top engineers. He was also a senior technical member of the team that created JavaEE.
He says: "However, I don’t believe that Mr. Gayle’s post really warrants a direct response..." and then does that in the following paragraphs.

Has anyone else had enough of hearing opinions on internet celebrities?

Also,
> I think the guiding principle for Mary...should be that she doesn't type anything into a web browser that she doesn't want publicly known

That was Eric Schmidt's answer a while back too, but I'm not sure it's adequate anymore.

As PG has remarked, more and more of the tools we use to interact with the world are becoming software (and as all software is increasingly web-enabled & socialized, that's often indistinguishable from "typing it into a browser" for purposes of this discussion.)

We're essentially telling people who have very legitimate reasons for wanting to protect their privacy that their other option is to become a Luddite hermit. Is that really the best we can do?

It's a tough question to answer as an entrepreneur, because it probably is a much smaller opportunity, and by definition doesn't enjoy the same viral loops - - but I wish it's one that we as an industry would spend more time on.