skysight-inc/HackerNews huggingface dataset:
Isn't this what the cloud is for?

Surely you just rent an instance of the computing power you need for an hour or two then upload the data, a script and wait.
Once the app I'm buying is tweaked and running smoothly there shouldn't be too much to do. -> sorry, but that's a terrible, terrible misunderstanding of how these things really work.
I thought that was the C-One: http://c64upgra.de/c-one/
I'm pretty convinced that self driving cars can fix this. Reaction times will be much faster and they can network together to spread out the braking and acceleration. Possibly even merging at high speed.
Who do you use for execution? I wish I could use Interactive Brokers for server-side but, I think, that requires 100k and FIX connection.
When I first started meditating I found the book The Relaxation Response helpful. It has a one page guide to activating the parasympathetic nervous system and is non-spiritual in its application.
Apart from what other commenters already said about the cost of software complexity, is there a variant of Amdahl's law that could be used here? 10x 100 GB servers working on a problem together will probably never be 10x faster than 1x 100 GB server. Perhaps just the increase in the order of magnitude of the distance information needs to travel is already sufficient to set some higher bounds... So you may need to buy more than 10 of them to match 1x 1 TB server.
decrease the 20-30% margin -> You're effectively planning to use your own money (!) to go in a price war with Uber (!).
The Himalayas are younger than the dinosaurs - they've grown in the 66 million years since the dino extinction.

Oh, and in Europe the mountains/hills have eroded to be rockier on the slopes, so vineyards are planted there; in much-younger New Zealand, the rocks are in the valleys and the slopes have more soil - so vines are planted in the flat areas.
> Complexity, sure. But cost? I thought a single 1 TB RAM server is more expensive than 10x 100 GB RAM servers.

Sure, but in the latter case you'd also have to pay for the manpower to build a cluster solution out of a formerly simple application. And people are usually more expensive than servers.
I like your 'burning zombie'!
Looks like 1.5TB RAM with 15 cores costs $50K. But it shouldn't be just about RAM. The problems I'm working on requires 250 cores on similar amount of data. If there was an option to get say 150 cores with 2TB RAM, things would fly for sure.
I love it. I was just doing some Fermi estimates for a friend on the data for a project he has in the pipeline. I was curious whether or not it would be cost efficient for his project's budget to go with NVMe SSDs or have to stick with traditional SATA ones, and turns out it doesn't even matter (for now) because at least the first three months of data will fit in 256GB of RAM, even allowing for a 2.5x factor stemming from some (estimated) inefficient storage or data structure use in a scripting language like Ruby or Python.

Edit: And after those first three months he'll know more about the use and performance demands of the project and will be able to make far more accurate decisions about storage categories.
Startups are looking for talent like yours
That really depends on your usecase. Not all analysis is 'one-shot' and not all businesses are free to upload their data into 'the cloud'.
> generalizations are always dangerous

I've got to remember this one
Outside of scale-out, scale-up there is also solution: scale-in. Optimize your memory usage, so your data occupies less space.

I work on something like that.
"...AMD's chief gaming scientist..." this is some fancy position
Then I think I misunderstood what jasode was asking. Specifically, the example (our link) already names groups if you hover over matching text (group #1, etc) - I thought they were just asking for the highlight color to correspond to the group # listed.
From all that I read, the game itself performs very well on AMD cards. The only exception is the Hairworks feature. But that has such a large impact even on high-end Nvidia cards that this is the first thing that one disables. It is probably the single most expensive setting in the game, and for a relatively small benefit.
Apostles as a hackers – haters gonna hate
But if everyone uses the phrase that way, then it's not dishonest for this person to do it too. That's how language works. If I tell my student, "the early bird gets the worm", it's idiotic to call me stupid because I confused a human with a bird. I didn't in fact do that. I used a common idiom in the language that we all understand. The fact that you can recognize there's a literal meaning that wasn't accurate is pointless. I'm not dishonest or a liar either. I used language in a non-literal way, just like all of us do hundreds of times every day.

When someone says "company X is using your data", there's a common understanding in play. We all know what they mean. They mean "data they've collected about you". If you want to call everyone names because, taken literally, it's incorrect, then go ahead, but you're beating a dead horse here. And I'm not actually accusing you of pummeling the carcass of an equine. I'm saying you're spending your energy on a pointless task, but doing so in a way that relies on you understanding that the meaning of a sentence isn't strictly determined by the meaning and order of the individual words in it.
We actually have to compare with heat pumps. They provide 3KW of heat per 1KW of AC electricity consumed.
Spitfire P9374
Ask HN: Why has Chrome's rendering performance got so bad?
Chrome's rendering performance on high res displays is absolutely abysmal. Both on a brand new 13" retina Macbook Pro and my Core i7, high spec Nvidia card Windows desktop with a 4K monitor are dreadful. I get checkerbox rendering when scrolling and horrible repainting.

Safari and Firefox have none of these issues. I've had a look on the Chromium bug tracker but the ones that seem relevant say 'fixed'. Any ideas of how to improve this?
Tales from an Era When Airlines Knew Good Design
"Little free memory" means that Chrome has eaten all my ram though.

Chrome is really lagging behind Firefox in this, as it has no backgrounding / unloading process that I'm aware of to maintain a reasonable memory footprint.

It also attempts to render every tab at once when restoring after a crash - another obvious problem.
Lidars need to get cheaper - so do decent RTK GPS receivers and IMUs. They're all expensive at the moment because the economies of scale are so limited.

But you know what industry is great at economies of scale? The car industry. You'll be able to get a sensor that outperforms that $100K sensor for $100 when they're fitted as standard on every car.
Interesting.

I'm licensed through a company called Shirlaws Group - Australian HQ is where I work in Brisbane; global HQ is London. Google Maps pins 'me' at Shirlaws motorcycles in Aberdeen, Scotland.

So it's smart enough to make some connection "Jacob Aldridge = Shirlaws" but must be missing some wider data or geo information.
If you go by the Dell site linked from the article, a PowerEdge maxed out with 6TB RAM (4 sockets * 24 LRDIMMs * 64 GB) will set you back US$444,000 (or thereabouts). They also list 3.2 TB NVMe PCIe cards for US$11,000 each.
It's probably worth extending "Your data fits in RAM" to "Your data doesn't fit in RAM, but it does fit on an SSD". So many problems will still work with quite reasonable performance when using an SSD instead. By using a single machine with an array of SSDs, you also avoid the complexity and overhead of distributed systems.

My favourite realization of this: Frank McSherry shows how simplicity and a few optimisations can win out on graph analysis in his COST work. In his first post[1], he shows how to beat a cluster of machines with a laptop. In his second post[2], he applies even more optimizations, both space and speed, to process the largest publicly available graph dataset - terabyte sized with over a hundred billion edges - all on his laptop's SSD.

[1]: http://www.frankmcsherry.org/graph/scalability/cost/2015/01/...

[2]: http://www.frankmcsherry.org/graph/scalability/cost/2015/02/...
Another 4 to 6 years and that should be a reality.
A more powerful potential mechanism is that they can be programmed to respect do not enter directives from a traffic control system (keeping human drivers off a freeway would take a lot of infrastructure). So the amount of traffic on the road can be regulated to keep it below the capacity implied by the safe response time of the vehicles.
A very impressive achievement for a private group. Is there anything it's doing that wasn't done first by IKAROS, though?
In my experience the problem with Drupal is that on the one hand it's too customizable, and on the other it will never beat a proper code-based framework in actual ease of customization.

Because it's so customizable, it's admin interface is too complex for many users. It has to be to be as customizable as it is. And to get a site going you often need to do a whole bunch of work up-front: configuring views, creating content types, taxonomies, rules, workflows, etc. This is often not needed with Wordpress because you just pick a theme, dump in a few plugins if necessary, and it 'works'.

On the other hand, if you really do need all that customization, you're often better off with a proper framework like Ruby on Rails. You can build features in code rather than clicking around the CCK/Fields/Views interfaces, and keep them and other configuration in version control. Most importantly, if you're capable of building complex views and workflows in Drupal, you probably have the ability to do so in code as well. And the latter is generally better in every way if you are capable of it.

Drupal strikes me as the kind of thing that looks good on paper, and that project managers would like, because in theory it can be customized by anyone, and in theory you can build anything you need with it, all without writing code!

In practice, however, you end up paying a lot to get Drupal developers/consultancies to fix whatever mess you're left with. And what do most of these shops do? Use things like the Features module to store everything in code/configuration files, which kind of goes against the whole point of using Drupal in the first place.

The end result is that nobody is really happy, other than perhaps the Drupal consultants who happen to know the ins and outs of the crufty framework/CMS hybrid, and can charge a premium for their knowledge of the dark internals of Drupal.
If you are programming in R, you sure better hope it does!
Could you give a concrete example?

If your working set is small, say 1TB -- and so fits in RAM -- for what kind of problems would using SQL be so much of an overhead that you need a different approach? And what would that approach be?

I suppose you could have a massive set of linear equations that you might be able to fit into 1TB of RAM, but would be difficult to work with as tables in Postgres?
Cohort Analysis for Startup
Making Business Decisions Easier with Data Visualizations
In my opinion the correct answer is 255Gb. (i.e. AWS r3X8 High Memory instances ).

While one can purchase servers with larger memory most likely you will run into limitation on number of cores. Also note that there is at least some overhead in processing data, so you would need at least 2X the size of raw data.

Finally while its a good thing to tweet, joke about and make fun of buzzword while trying to appear smart. The reality is that purchasing such servers (> 255 Gb RAM) is costly process. Further you would ideally need two of them to remove single point of failure. it is likely that the job is batch and while it might take a terabyte of RAM you only need to run it once a week, in all these cases you are much better off relying on a distributed system where each node has very large memory, and the task can easily split. Just because you have cluster does not mean that each node has to be a small instance (4 processors ~16 Gb RAM).
There was an article back in March wherein a Stack Overflow (or Quora?) user comment about the hilariously porous boundary between game engines, drivers and actual hardware on GPUs. I'll see if I can find it because as much as I love a good ole shitshow of industry skulduggery, I can completely see a best-intentions effort leading to these sorts of unintended consequences (doing their best to get tessellation working, and so forth).

Disclaimer of non-fanboy-ism: I don't play video games anymore, and only have graphics via Intel on my machine. No stock in Nvidia or AMD either.

Edit: Found it, and my memory completely sucks. It was an article on gamedev.net about lower-level graphics APIs: http://www.gamedev.net/topic/666419-what-are-your-opinions-o...
Have you got hardware acceleration switched on?
ADB port forwarder to connect local adb client to a remote adb server
Some ex soviet countries have city central heating systems. I am from Vilnius, Lithuania, we had that in our old flat. Maybe it might be a good idea to create data centers around there?

The only unsolved (as in I don't have an idea) problem is, what do we do with all the heated water in summer?
Why #NewSpotify is doing it right
You should post that if it hasn't been posted already, it's a much better way to make the case than the current link.
> Further you would ideally need two of them to remove single point of failure.

That's assuming that everything needs to be 'high availability' and buying two of everything is a must. This is definitely not always the case. In plenty of situations buying a single item and simply repairing it when it breaks is a perfectly good strategy.
And i though self driving car is a hard technological problem.

Silly me. It's a design problem.
I'm a poor source of modern information like current consumer ones, and never did build my own.

I used to follow the openeeg mailing list, which has tapered off in the last couple years:
http://sourceforge.net/p/openeeg/mailman/openeeg-list/
(http://openeeg.sourceforge.net/doc/)

building one yourself was usually around $200 in parts but there were always interesting ideas to do simpler devices. You will find references to consumer ones people were considering, and responses that are helpful for figuring out which specs matter. I would also look for the names of new projects people are developing and maybe some of the hobbyist communities that are still active and debating the minimum specs for a budget device today..
Not only that, but uploading a 1TiB dataset implies a certain quality of connection which not all businesses want to take on...
Although we can theoretically handle up to 2^64 bytes of RAM (16 exabytes), the practical limit is much lower. I think someone on Wikipedia said it's somewhere around 8TB, but I imagine the performance of random access into 8TB RAM is much worse than a motherboard designed for up to 32GB RAM.

It's not as easy as just buying more RAM. You'll have to pay more attention to how you make use of the various caches in between your CPU and RAM.
Show HN: Kangaroo – a tool that helps create automated tests with a few clicks
“I understand now”
Take a graph, you could use an SQL database to store it and do your graph analysis using SQL, or, alternatively, you could convert your graph to an extremely compact in-memory format and then do your analysis on that. Much better efficiency for the same size problem, bonus: you can now analyze much larger graphs with the same hardware.
"These mountains today. ... Get off my slope!"
How often are you planning to do that? The bandwidth it takes to send data that fits in RAM somewhere else is somewhat expensive.
It's precisely a C++ issue because if multiple return types were natively supported by the C++ language, none of this would be needed since no one would be trying to export a STL object across shared object (or DLL) boundaries.

It's only the workaround (tuples) of this C++ issue that platform-specific issues are allowed to arise, like Windows as you observed, which is why I view the platform-specific issues as symptoms of a deeper problem.
I laughed at this "Think about what you did" - I've been a bad WordPress dev :(

No mention of backups, but that can at least save you from some of the freak out - assuming your site isn't changing more frequently then your backup scheme.

Daily backups are nice, but the problem might not be detected immediately - so snapshots (week, month, 6 month) are advisable.
I imagine that on a motherboard with 96x RAM slots, the access time between the first one in row and the last one will be actually quite different, due to the physical distance between them.
That would translate to ~$60k for 6TB of RAM. Plus the cost of the server itself ($10k?)
RegExr is my go to regular expression playground. I've been using it for many years, back when it was on http://gskinner.com/RegExr

It's evolved a lot since then :)
If AMD's platform is really as good as they claim, why does everything run on the nVidia one?
Also, I don't think this is an issue worth getting upset about, especially if you can just turn the feature off in-game.
If the particular engine I'm using runs better on nVidia cards, then of course I'm going to prioritize their platform as a base for my hair simulations or whatever fancy extra stuff I want to have in there for people with high-end GPUs.
Solution: AMD optimizes HairWorks. Reality: complaining is easier.

NVIDIA have always claimed that no exclusivity deals are signed with GameWorks. Others have [supposedly] always been welcome to assist game developers during the phases that NVIDIA gets proactively involved in.

It's just an extension of the systemic AMD problem: they [very likely] have the best hardware but their Windows drivers are horrific and not only do they not have developer support; they actually play the victim because others take the initiative. For crying out loud, even Intel has game developer support/code. Correction: it turns out that AMD do actually have some open source libraries for game devs.

The problem here isn't NVIDIA. NVIDIA set out to optimize the game as best as they could for their hardware. AMD didn't set out to do anything: once again they sabotaged themselves by not getting involved. If I'm lazy and I get left behind that is my fault, not the fault of the other guy who did what I didn't do.
#equality #rayromanfilms #vizcaya #miami
Done: https://news.ycombinator.com/item?id=9582060

I originally saw it on HN, but almost two years ago. Old comments: https://news.ycombinator.com/item?id=6398650
That's like saying "you can fly first class".

If you don't have money, you can't. Very few people can afford it.
This is classic case of "Algorithm/Problem Selection" if your algorithm/problem is tailored to a task such as PageRank, surely a single threaded highly optimized code will beat a cluster designed for ETL tasks. In real organizations where there are multiple workflows/algorithms, distributed systems always win out. Systems like Hadoop take care of Administration, Redundancy, Monitoring and Scheduling in a manner that a single machine cannot. Sure you can "Grep" faster on a laptop than AWS EMR with 4 Medium instances, but in reality where you have 12 types of jobs which are run by team of 6 people, you are much better off with a distributed system.
Previous comments (2013): https://news.ycombinator.com/item?id=6398650

Also worth a read, "Command-line tools can be 235x faster than your Hadoop cluster (2014)": http://aadrake.com/command-line-tools-can-be-235x-faster-tha... | https://news.ycombinator.com/item?id=8908462
No, that's a very expensive server, and Dell will charge you a hefty premium for the memory. There are quite a few options that will cost you less than that (of course the maximum capacity will vary).

It would be nice to see an article comparing all the high RAM machines side by side with specs and prices.

The largest machine I have right now will hold 512G and was a run-of-the-mill machine, it was about $5K, I'd expect the more exotic ones to be substantially more expensive but probably not as expensive as the machines linked here.
After reading the title I was sure there was something about R in the comments.

You can program R in Spark you can now program in R http://blog.revolutionanalytics.com/2015/01/a-first-look-at-...

Now you can work directly with SQL Server as announced this week by MS. http://www.computerworld.com/article/2923214/big-data/sql-se...

I have had a ton of arguments about R's "biggest weakness" being that it uses RAM. I haven't once in the almost 3 years of working in R that I ran into this road block, but I am sure others have. Which there are several good distributed choices that will keep getting better and better.

Using RAM instead of Distributed is better in R as well as really any other language in terms of complexity and flexibility.
Well, the reason exceptions should be reserved for exceptional cases
is because they are gotos. There is no way around that, a function
that both returns values and throws exceptions is more complex than
one that doesn't because you are jumping through call frames.

I once worked with a billing system that threw a BillingException when
a charge didn't go through. It works ok, as long as your only response
to such an error is to spew an error message to the user. But the more
you need to handle it, the less an exception makes sense. The code
that tried to handle the BillingException was a tangled mess of try
and catch statemetns mixed with retry loops. For example, if there was
a temporary error at the payment provider, you would just retry a few
times, a permanent error, try with another account, if the customers
remaining funds were to low show an error message or if they were just
a few dollars of, charge them that and be happy we got something from
them.

On the other hand, the billing system could also throw a DbException
in case there was something wrong with the database connection. That's
a different kind of problem and fatal for a database-backed
website. Nothing to do, except log the error and crash.

The point of exceptions is not so much that you can catch and handle
them, but that it separates exceptional situations from in-band normal
error handling. Now what is in-band and what is exceptional depends on
the circumstances which is why, as you say, Boost.ASIO provides both
variants. If your system is supposed to handle the errors, use the one
without exceptions. If not then use the one with exceptions.
Off-heap direct-memory data stores
Thanks for mentioning the tooltip popups. I didn't realize RegExr used mouseover/hover events on the matching text since the other regex sites don't do that. (It seems like non-obvious UX/UI but I now notice that the bottom left corner does say "Roll over a match or expression for details" -- but who reads the instructions?!?! Nobody's got time fo 'dat.)

Nevertheless, I prefer explicit color-coding highlighting the boundaries instead tooltips displaying the offset numbers.
An arabic social blogging site that helps content creators to write in arabic
There are a team of folks in NYC working on combined heat and computation as well – Project Exergy (http://projectexergy.com/)

Seems like the model is something that will become more familiar in the future.
I see backups more suitable for a slighlty different topic. Something like "how to take care of your website" where there would be db otpmization, recurring backups, daily & monthly snapshots, etc.
Have you heard how it is convenient to use Back endless caching API?
Then surely any person who delays the bus for many future passengers down the route is a much bigger arsehole?
From what I read, AMD doesn't have access to the GameWorks source code. These features are also designed with the capabilities of Nvidia GPUs in mind, you can't really optimize around fundamental differences in the GPUs (e.g. the newest Nvidia cards are significantly better at tesselation).