skysight-inc/HackerNews huggingface dataset:
"They all have trade-offs. Key-value stores eliminate your ability to access data by anything other than its key. So, you have an article with an id of 5. What if you want to look up the articles by author 9? "

For cases like that you don't have one source of data, you do almost like an AFTER INSERT trigger to update other flattened tables of data that you need.

For instance, where you would normally perform a JOIN you would just after save update a JOIN table or a VIEW that is created by incremental updates.

So then you could filter by many other things and really aren't limited too much by space to make those duplications.

It happens with large RDBMS systems anyways after about 5GB of data, same type of scalability decisions have to be made.

But yes the article is off and you are right, most likely systems will approach this with multiple solutions for specific problems.
Most of the discussion seems to ignore the "in this space" (scripting existing software packages) part of your question.

I submit that Python is more popular in this space because:

- many end users will be users of the package first, and programmers very much second. In this case a very gentle learning curve is more important than ultimate expressivity for hackers.

- In the same vein, it superficially looks more like BASIC than Ruby does

- The C API may be simpler to use than the Ruby interpreter, though never having used Ruby's C API that's speculation on my part. Never understimate implementor laziness as a technical driver.
Facebook internal presentation re:Casandra DB
Also known in startup circles as "release early and iterate"?
A feed reader is already an aggregator. This is a meta-aggregator--and when people come out with different flavors of this type of thing, we'll have meta-meta-aggregators.
This article was linked to recently on Penelope Trunk's blog. For those that are familiar with her then this kind of article shouldn't be much of a surprise.

I'm posting this because it's an extremely well written article with intelligent social commentary. Very thorough and articulate reasoning.

If you are going to comment please do so on the social nature and implications - ignore silly comments.
Hedge funds are leveraged to the hilt. Works great when you make the right bets but when the whole world gets scared and starts calling in their loans, you're SOL.

I wonder if you could relate financial hacking to code hacking. Once any system grows so large, you can no longer keep it in your head. Many financial instruments didn't exist when Buffet and Soros were making their money.

Of course we're now back to Taleb and his black swans.
A. That is fine until the first time somebody has to write a report.

B. With the right product, administration is easy. MySQL is completely ubiquitous and something like SQL server is easy. This certainly conflicts with point C.

C. Most of these people aren't doing high-volume anything. If you are doing high-volume stuff (like Google) then of course it makes sense to use something very specific to your task. But that doesn't mean Bigtable, for example, makes for a good generic solution.
The colon-period pairing to indicate a block is just as clever. That way, the colon actually serves a purpose instead of just being syntactic noise as it is in Python. The ability to use tables as arguments also gives you the advantages of keyword arguments in Smalltalk, but does it folded in with a generic mechanism. As a result, the language is bigger than Smalltalk, but it feels just as small.
Actually, for a while I was spending quite a bit of time in #python. I didn't come across "arrogant children".

Looking at some of the drama and prima donna action in the Rails community (from a distance) just made me sad and grateful that my corner of Python (scientific computing) doesn't have those sorts of issues.
Grand Pianos
Modern upright and grand pianos attained their present forms by the end of the 19th century. Improvements have been made in manufacturing processes, and many individual details of the instrument continue to receive attention.
Ask HN: Best way to redesign an existing webapp?
Hi,

I run an online music community with decent traffic and usage. When we started out, I had no background in web (and usability) and have learnt a lot in the last two years. I can now tell whats lacking in the current design (aesthetics/usability/interaction wise) but still don't know if I can design from scratch. I do have a designer (to help with graphics) but he is good only with skinning so I have to do the usability and interaction design work.

What is the best way out of this situation given that I can't hire external help (for financial reasons of-course). I have been thinking of reading some good books on web design patterns and following standard practices. Does that work well?

Thanks in advance

PS: I have a coding background in case its not obvious why my design/usability skill aren't that great :)
Any Keith Thompson is the REAL Keith Thompson, and neither is the real Slim Shady.

Don't get your value EQUALITY and pointer equality or IDENTITY confused, hacker!

http://www.nhplace.com/kent/PS/EQUAL.html

When speaking with arab-speaking programmers, I find myself using the translation of words like "congruence" more often. It actually sounds better in arabic too:

"yusawi" --> equals

"yuwazi" --> is similar/congruent to
Yes gnip rocks, but it seems they provide the business service for companies to provide this. I want an app as a consumer. Gnip could build this.
Speaking as a commercial Python developer, I can honestly say that there is a tremendous amount of interest in and usage of Python across a wide spectrum of commercial and enterprise applications. This is everything from realtime high-frequency trading to oil exploration to mixing toothpaste to processing Hubble images to manufacturing SATA and PCIe driver chips. The very active, very professional community around the language, combined with the ease of integration with legacy codebases in FORTRAN, C, and C++ makes it very popular in the commercial sector. Consequently, there are numerous institutions and companies that have developers spending time on improving the langauge and its underlying libraries. Guido is a sharp guy and is sensitive to all the places where Python is used, and he takes these into consideration when approving or disallowing language changes. This understanding and the overall process makes the language "mature".

On the flip side, I am not aware of any significant impact of Ruby in commercial development that is not web-driven. When we bid on projects and talk to prospective customers, we're typically making a case against Java and C++. I can't recall a single instance where someone seriously asked us about Ruby. (We've had more inquiries about KDB and K, that abomination of a language.)

So, just my $.02 as a scientific software developer.
A Hacker's Clay: Expression through Electronics
Maybe everyone else is underconfident... Who's to say what the optimal amount of confidence is. Tenured academics? Non-entrepreneurs? Salaried lifers in big firms?
I'd thought about building this but threw it out when I figured I couldn't get my various devices to talk through it to Twitter. Computer, sure that's easy: fix your host to api.twitter.com, but fixing all the iPhone apps to point to this service instead of api.twitter.com? Not likely.

Unless you harnessed the existing applications then it wouldn't be useful.
> So, you have an article with an id of 5. What if you want to look up the articles by author 9? You're just unable.

Yes you can, at least in CouchDB, just create a view that emits the author id and you're done.
FWIW, there are lots of Python folks moving to git and hg as well. But I don't understand what VCS choice has to do with language culture. What's next, a critique of editor choice? :)

Python is such a large tent that it's quite inaccurate to say that "the Python culture" lacks a sense of experimentation and adoption of new tools. What you might be getting at is the fact that there are already so many existing libraries in Python for doing most things that there's not a lot of public reinvention of wheels going on.
Nice. Ideally, with more cell phones getting GPS, eventually it'd be possible to set up one's cell phone to communicate one's position to the central lighting facility, and have it turn on the lights as I approach them, and off the lights after I've passed them by.
Mobile Application 
Red Chery is a GPRS-based mobile application used for receiving mails from any of your free e-mail accounts / company email account and presents it to your mobile phone. In its near-future versions, Red Chery will be designed to support, more enhanced features like data synchronization, customer relationship management features and Global Services Amalgamation (GSA), a first-of -its-kind global service which is a patented mobile-based service technology supported by Red Chery.
Axis 216 MFD Network Camera – A Best Spy
Axis 216MFD Network Camera is so intelligent that we usually say it is a combination of computer and camera. Its high performance 1.3 mega pixel camera is basically made for eyeing on banks, government offices, airports shopping malls, schools, general stores, high security areas which captures and transmits live photos directly over an IPv6 in addition to IPv4 which is easily handled from any laptop or PC anywhere through Internet, Intranet or LAN so that it is difficult for thieves and burglars to steal anything. The reason behind that is, its dome glass is fixed from inside and the casing is secured due to tamper-proof mounting screws. Its adjustment is versatile by tilting, panning and rotating high quality lens to any angle. So get this secure deal of Axis 216MFD Network Camera at Buyelect.com.
1) Don't mess with the database. Back it up, remotely.

2) It's good to have a semanticly sane code base with all the common pages elements factored out into templates, but it's not a requirement. If it ain't broke.

3) Involve the community. Have a poll asking your users what to fix first, and start from there.

4) With a copy of the database, you can start developing the site using better technologies and processes; invite a few users to test the redesigned site, preferably on another machine. Report back to the community, publish a newsletter, show screenshots, etc.

5) Don't fuck with that database. Don't lose the server logs.

Good luck.
Indeed. I avoided the term engineer purposely.
New Logo For MySpace: No Longer A Place For Friends
A potential buyer might be interested in knowing how many users you have as well.
How wide is the world's digital divide, anyway?
Ian B., aren't you occasionally one of those crochety naysayers? :)
I am sorry to say Caitlin Flanagan is one of the the reasons I dropped my subscription to the Atlantic. I find her views are so coloured through her own experiences and opinions (with which I invariably disagree) that they completely distort any chance of objectivity in her writings. So I guess we differ about the intelligent social commentary part - it's opinionated, but I never get the feeling it moves the debate forward.

As to the subject matter: personally, I think stuff like this is just a symptom (not a cause) of the fact that in large sections of society we are no closer to gender equality than we were half a century ago. The damage has been done long before girls become old enough to engage in sexual behaviour, and focussing on sexual behaviour is a red herring, though it can make for dramatic reading and heated conversations.
If you anticipate having to write a report, obviously you should use the right tool. If you don't, use what's simple and gets the job done. It's not incredibly hard to switch later.

On the low-end, administration for MySQL is certainly not easy; for an app you build in a day or two on Rails or Django or Sinatra (or as a CLI tool for that matter) you can easily spend more time doing sysadmin work to provision, configure and maintain MySQL than you do writing software. (MySQL shared hosting isn't everywhere.) There are plenty of throwaway webapps out there that just aren't worth setting up a database server for.

On the high end, it's not a matter of what's easy; it's a matter of what's possible.

These are different use cases, and there are different software packages being advocated for them. No one who needs a disk-backed hash table is using Hypertable, and no one who needs massively parallel analytical capability is using Tokyo Cabinet. And no one's advocating reconsidering RDBMS use as a whole; just RDBMS use as the default choice.
For the same reason you would store json in there. It's when you want a schema-less structure. I chose XML because the platform that the app uses prefers that.
That's a brilliant quote.
Never ask your users what they think before you ask your logs what they do.

Users lie. Users misremember. Users are inexpert at describing their own use of your software.

Logs don't lie. Logs don't misremember. Logs are experts at describing your users' use of your software.

If for some reason you didn't build your web app with an eye to getting useful information from your customers, start adding Google Analytics (or one of the numerous copy/paste Javascript competitors) and give it a few weeks to start collecting data.
This preso is about Hypertable, not Cassandra, and there's no evidence inside the presentation that it was presented to Facebook.
Won't it be nice when we have refactoring tools that are trusted enough such that you can remould code (even legacy code) without the pain of writing unit tests?

Moving from C to Java you really appreciate tools like IntelliJ and Eclipse which can automate some refactorings with a fairly high amount of certainty that it's not going to change the behaviour.
It's a running joke where I work how bad Hibernate's SQL is compared to hand-written by experienced developers. Maybe it's "good enough" for some applications, but we don't really get out of bed for less than 5000 transactions/second...
You'll still be able to use SQL. You will still use RDBMses for things they are good at. You will stop using SQL for things SQL is not good at.

You will start collecting much more data. You'll be able to throw as much processing power at your problem as you want to, for short periods of time. You'll be able to effortlessly pull in disparate data-sets and work with them without the effort required to import new data and pre-define relationships and indexes in a SQL schema. You will be able to query unstructured data in multiple formats including video, images, audio, html, plaintext, etc. with more structured data. You will live in a culture of data, where large numbers of PCs are brought to bear on all of your data whenever and however you want. You will discover new relationships you did not know were there. You will get increased value from data you didn't know had value because you didn't bother collecting it, because your DBMS wasn't good at working with that kind of data.

All these are new opportunities that exist in addition to traditional RDBMSes, not in place of them.
A Limb Regeneration Mystery Solved
Cash-On-Demand.co.uk
Cash on Demand Pulled in £87,958.61 for us in just 7 weeks. 18 Months ago and heavily in debt we stumbled upon a fantastic way of making money which was developed by a reclusive gentleman who used it himself to become a Multi-Millionaire
This article is really terrible, and it's sad because most of the pieces of software discussed are at least interesting. Several are indispensable to anybody in the business of dealing with PB-scale data. (I'm pretty sure Hadoop is seeing healthy enterprise adoption.)

That said, I think that calling the whole event NoSQL was kind of asking for it; of course people are going to try to squeeze some drama out of that stone. (I would have gone to the meetup anyway, though, if I hadn't already been booked.)
Thanks for the reply! I have infact been talking to users as well as watching the logs and analyzing google analytics data. I do have some insights on whats missing but as I said, the real challenge is in designing interfaces that can solve these problems. For example how do I display primary and secondary information on a profile page. What should be emphasized and de-emphasized. Since these are common problems, I was wondering if there is a book or something that I can readup to get some answers/pointers. Profile page is one example. Pages like account setting or stats page are other common pages that should have some standard design patterns.
Ok, seriously? You're asking an RDBMS to index into XML data you've shoved into a column?

Quoting Microsoft & IBM on database design is like quoting GM & Chrysler on vehicle design. All are years behind and focused on an under-informed and declining customer base.

2 things:

1. Postgres supports both Regular Expressions and indexing on expressions. If you can't index your table on a regex expression, you are probably doing something wrong.

2. Reconstructing full XML documents from a single column based on a query is a lazy anti-pattern. Either a) store the document in a way that it can be retrieved on demand, or b) structure the data and the query in a way that is performant. Jamming a verbose textual representation of data into a single column of a table and then demanding performance is a fundamental misunderstanding and misuse of an RDBMS table.
and what language/platform, hosting costs, etc...
In Perl you have "wizards" and in Ruby you have "rockstar ninjas"... No other language communities do this.
Our company provides and supports commercial builds of Python along with 40+ other packages (including scipy, numpy, PIL, wxPython, etc.) on Solaris. The Sun compiler is not great, but that has very little to do with Python.
This is an amalgamation of lots of different Twitter apps. More importantly, it's an amalgamation of this guy's favourite Twitter apps. It's his killer app. However, many other people will want a different set of apps in their killer app.

What this guy is really saying is, "Build me a custom version of this thing." Sounds like enterprise software.

Some progress may me made by rolling up some of the popular/complementary apps and, thus, satisfy a large enough group of users.
This article is the biggest load of fucking bullshit I've ever read. That's all I'm going to say about it. The author couldn't tell his ass from a hole in the ground.
"FWIW, there are lots of Python folks moving to git and hg as well"

It's not remotely as universal as it was for Ruby developers. When working with Python you still can't go a day without hitting an svn repo, but with Ruby you'd easily not touch one for months.

"But I don't understand what VCS choice has to do with language culture."

A lot, and I already described it. Ruby developers and the Ruby language overall are less conservative and more open to dramatic change than Python developers or the language. Each model has its benefits.

"'Python is such a large tent that it's quite inaccurate to say that "the Python culture'"

It's a large tent, but the common thread is Python and Python's principles, and they define the culture.

"there are already so many existing libraries in Python for doing most things"

But individual developers don't do most things, each works on certain things. Python has lots of libraries, but that doesn't mean it's the best choice for any specific job. As a web developer, there are a huge number of Python packages that don't matter at all in that domain. If sheer number of libraries mattered, we'd all be using perl.
tl;dr it's not what your neon sign says, it's the fact that you have a neon sign.

Good article, but in the case of Understanding Media it is very much worth the time and effort to read the original. I cannot praise it highly enough.
The fact that Tweetknot is extremely derivative is crucial to the story.

If Twitter had pressured a more unique, independent product about using "Tweet" or "Twitter", one might reasonably draw the conclusion that their goal is really to protect the terms "Tweet" and "Twitter."

But when Twitter attacks Tweetknot through this charge, the most obvious explanation is that they have a particular problem with Tweetknot's general (in my opinion) rip-off of Twitter's aesthetic, and they're using "Tweet" as an initial concrete and legally solid means by which to force Tweetknot to stop copying Twitter. I do not see any evidence that Twitter is going to start going after people who are using their trademarks with unique and useful products and services.

The blog post you linked delineates this policy from the horse's mouth. Your original article, on the other hand, obscured Twitter's intentions in order to present a more compelling and alarming story. In retrospect, given how derivative Tweetknot was, Twitter's email looks absolutely reasonable and makes their intentions totally clear.

(Reference: original TC story -- http://www.techcrunch.com/2009/07/01/twitter-grows-uncomfort...)
Michael Jackson First Artist to Sell Over 1 Million Downloads in a Single Week
Zoho Projects Goes Social - Integrates Twitter
Cool but unpractical without gestures and huge monitors or projections.
I don't think you understood the article. Overconfidence is a well-defined and measurable term, a systematic tendency to estimate one's future performance in excess of expectation. In one widely quoted study, 95% of a cohort of MIT students expected to graduate in the top half of the class. (That might well be apocryphal, but it serves to illustrate the point just the same.) It also illustrates the point that while you may not always be able to determine that an individual is overconfident, you can point to a group and say that its members are overconfident on average.

This is completely unrelated to the question of whether or not overconfidence is a good thing. For example, there are studies showing that students who are slightly overconfident -- but not too much -- improve the most in the long term. (Not able to find a link right now, unfortunately.) As another commenter pointed out, the consequences of failure have changed dramatically (for the better) in recent times. In my opinion, a lot more people should be taking up entrepreneurship than the current levels.
There seem to be two quite contradictory arguments for NoSQL. One camp says, we already have what we need in the form of [programming language] objects, so why would we take everything apart and map it to something as reductionist as relational tables?

The other camp says, relational tables are soooo complicated, we need something even more reductionist and free form, so key-value pairs are just right.

I can understand the second argument, but the first is a step backwards unless your programming language is something like lisp, which has a reductionist data model itself.

Object oriented models are fine if you write one application. But they tie functionality to data very closely and that makes repurposing of data for seperate applications much more difficult. So how do they analyse their data or use it in more than one application?

They have to write a lot of code using an application specific object oriented API instead of a general purpose reductionist data model. That's horrible but how horrible it is only shows a few years down the road. It's a disaster in the making. A whole army of programmers will be required to extract data from all those big APIs of legacy apps.

I'm convinced that data has a different life cycle than procedural code and therefore needs to be expressed in a simple, uniform, reductionist model independent of all application code.
Eh, I agree with all the problems people have pointed out...but it still kicks ass. I can't properly articulate what it is, but it's the same thing I liked about Google 10 years ago.

It knows what I want. I love the interface, in fact, it's superb. I will use this every day, and I am very hard to move on search.

Faults: search text too big, looks fisher price. Does not respond dynamically to my selection, should refine further. Still, awesome.
Overstock follows Amazon, Drops Affiliates in Four States