skysight-inc/HackerNews huggingface dataset:
Its not about having two of everything at all times, but rather about having a capacity whenever you need it. At 244Gb you hit a sweet point where you can have access to large capability at a flexible price (Spot Market / On Demand / On Premise). This is what separates engineers with business acumen from run of the mill "consultants" with a search engine.
Emotion web tools for Your website
If I need help with WP I'm taking a long walk off a short pier.
> 0. Spreadsheet is all you need.

I HATE when people use Spreadsheets to do anything besides simple math.

http://lemire.me/blog/archives/2014/05/23/you-shouldnt-use-a...

TL:DR your work is not reproducible and we can't see what you did to get to your numbers. A million examples of why this is bad.

Also

> 1. Python script is good enough

You mean Python with pandas and numpy?

I use R which is also a great choice

> 2. Java/Scala is way to go.

For you but the vast majority of Data Scientist don't use either and their choice for people is not universal. Julia looks like a great new comer. I again mainly use R.

> 3 & 4 are good points.
PRIVACY: Australian laws attempt to criminalise the teaching of “encryption”
Can I be rich@pubmail.io too?
If I select 1KB, why does the link point to an HP server with up to 6TB of RAM? Linking to an 80's PC seems more appropriate :)
I remember reading that link and thinking about how the standards are not working.

It also concerns me for the future of the open source graphics stack. If the proprietary drivers are being updated to fix game specific problems, how are the open source drivers going to fair when the general rule for open source is not to have hacks within a code base.

I can only hope that the Mesa 3D stack comes to Windows in a meaningful way to ensure a common open 3D graphics library. Vulcan might shift the problem to the game engine from the driver but that's a long way off yet.
No, it probably doesn't fit in RAM (and probably never will).
How to create personalized videos and send them by Email with Zapier (Tutorial)
IMO bret is a truly revolutionary thinker in the field of CS, really a genius of our time. understudy of the similarly gifted Alan Kay. Inventing on Principle (https://vimeo.com/36579366) profoundly changed my perspective on life and software when I first saw it years ago, much like the Hubble photos. I wish our culture and society cherished and gave more credit to these individuals.
I'd appreciate an explanation of where I've been inaccurate.

I agree that I raised a concrete objection in my initial post and did not mention the abstract motivation for raising the objection, so no need to pursue that aspect of it.
Does anyone know the word for this?

Physics must have a word for how energy in forms such as electricity can do work, but once transformed into thermal energy it can no longer do work (a temperature difference is required to perform work, and then only at carnot efficiency — a temperature alone cannot perform work).

I asked on reddit once, and someone suggested "exergy" but I think that word is explicitly concerned with heat, so couldn't be used to say things like "1000w of electricty provides more exergy than 1000w of gas", or "An electric space heater destroys the exergy of the electricity it consumes, while a computer uses it for computation".

So is there a word?
And if your data doesn't fit in a single server's RAM, just buy some more and run Apache Spark [1] on them. It's an in-memory computation engine that's really nice to program for: you don't have to worry about low-level clustering details (like MapReduce). And it's way (10-100x) faster than Hadoop.

[1] https://spark.apache.org
Indeed, it is just another useless look-at-me/my website post… Stupid simple calculation everybody should be able to do from their head
You're probably right, this is a far more realistic scenario. I have a bit of a soft spot for dystopia.
You could start with a book like "Practical Data Science with R". This review http://www.amazon.com/review/R3IOXEI69G6044 gives a very good summary of it's contents and the intended audience
What do you think the "dummy equations" are?
Lithuanian Parliament legislated Internet censorship
Scalable graph systems have been around for many years and have never taken off.

Most businesses doing big data (like ours) often have multiple disparate data sources that at the start of the pipelines are ETLing into some EDW. Trying to consolidate them into a single integrated view is very difficult and time/resource intensive. Having billions of disconnected nodes in the graph would be very hard to reason with.
Very cool, thanks for the reply!
4-6 months and you'll have a Knight's Landing Xeon Phi with at least 72 cores and 288 hardware threads, with vector instructions, and you'll be able to stick 3 of them in single blade.
VDiscover: large-scale vulnerability discovery using Machine Learning
I am affraid the in our research lab we didn't have 10 000$ up front/200$ a month to get a pc with 1TB ram ... we did have a large computer hall and BOINC though :)
I stand corrected, all four in concert is ideal.
This of course ignores that it's much easier to get your hands on a cluster of average machines than one massive bloody server, and all the non-performance-oriented benefits of running a cluster (availability etc.).

Much easier to request a client provisions 20 of their standard machines, or get them from AWS. People don't like custom hardware, and for good reason.
Ditto for computationally intensive work: if it is CPU dominated, more CPU's calculating in parallel will be of advantage, even if the data could fit some RAM.

There's no a single simple answer, but sure, whenever less computers are enough, less should be used.

The recent problem is, some people love "clouds" so much today that they push there the work that could really be done locally.
Moving Out? Tips on Hiring a House Cleaning Service
Pick a technology that can grow with your dataset. Take a look at Cascading or if you like Scala, scalding. They work on your laptop and on a 2000 node cluster the same way.
Do you think you have a clear reason why you find a rebuke from a convention organizer more worrisome than a rebuke from, say, a coworker or friend?

It seems just as easy for a statement made with no CoC in place to have consequences.
One of the comments I've heard about how to start addressing some of the harder aspects of autonomous vehicles is changing the environment to help them in various ways. I suspect radio beacons communicating various information to complement vision systems or to hand off control (within some defined interval) are one of those things. Of course, they'll have to operate in mixed environments for many decades so they can't be dependent on, for example, coordinating with other autonomous vehicles.
Spark is fast becoming the default tool for big data.

The recent addition of SparkR in 1.4 means that now data scientists can leverage in memory data in the cluster that has been put there by output from either Scala or DW developers.

Combine it with Tachyon (http://tachyon-project.org) and it's not hard to imagine petabytes of data all processed in memory.
It doesn't go in to detail about how it's free to use. Presumably as well as a fibre optic cable it needs to be plugged into the grid, so do you need to invoice them for power costs at the end of each month?
Is creating another task and project management app just stupid?
> you mean Python with pandas and numpy?

Actually, I would bet that some 50% of time people are importing numpy or pandas they really don't need it

Like for calculating the square root of a number. Or the average of a short list
Jeri Ellsworth is a wizard! A huge inspiration to me.

I always loved this floppy drive reverb she made: https://www.youtube.com/watch?v=Xpr7B-7BFP4
> It's just an extension of the systemic AMD problem: they [very likely] have the best hardware but their Windows drivers are horrific and not only do they not have developer support

You are strongly misrepresenting the actual situation. AMD is painfully aware that their drivers lack compared to Nvidia but they disagree with the reason. AMD does not want to put nearly as many hacks into the drivers as Nvidia does. They went quite far (by making Mantle) to make the drivers leaner and more predictable.

Nvidia traditionally detects misbehaving game code and patches it up with various degrees in the driver.

> AMD didn't set out to do anything: once again they sabotaged themselves by not getting involved.

AMD made the right step and backed the only thing that will evolve the situation from the status quo: Mantle and now Vulkan/DX12. Otherwise they would always end up playing this game of magical drivers that NVidia supports but nobody likes.
Don't graphic drivers already optimize themselves for specific games ? I don't really know what's optimized underneath though...
yes, I only knew about the hovering because of the instructions, which I did read. FWIW, my reaction was it would probably be better if they were all visible at once, who has time to hover over stuff.

anyway since they have the logic to already know what to put in the tooltip, making that number determine a color seems like it would be simple, almost as simple as appending it to the tooltip. (if indeed that's all that's missing.)

email the author your suggestion :)
Here in Nova Scotia, we have the right to refuse overtime.... does that exist in the US?
What happens when you lose the phone?
> why does everything run on the nVidia one?

What is "everything"? PC Gaming is nVidia centric because they put a lot of money behind it. AMD supplies the chips for all next gen consoles so by pure market penetration in the gaming market, I'm pretty sure they are doing quite well.
This is a great thought provoking article about the intricacies and paradoxes of our current society.

It's a little OT but I want to ask HN a simple question. The vast majority of my friends and acquaintances completely despise philosophy and they have one single question: what is the usefulness of philosophy?

I am often inspired by reading philosophy related articles, as I was inspired by my philosophy teacher in high school. Unfortunately/fortunately I was in another country with a completely different educational system. In this country most people only meet philosophy in a single semester in college and it's only a rush to memorize the most things possible about philosophers. How do I explain the importance of philosophy for the development of our mind?
Google translation is a bit poor. So, basically, it legislated blocking selected gambling companies which are not licensed in Lithuania (not some shady ones, but including major ones like Unibet). These companies were dominating the sports betting market here.

The main concern in the country is that it might be the beginning of wider censorship since the technologies will be already in place ('politically incorrect', etc). Will see how it turns out in practice. But that's a worrying start.
Startup Vows to Make Mobile Service Free. Rolls Out Android Device
I don't see why AMD would need to source code to be able to optimise their drivers. Sure, it would be nice, but with a good GPU profiler (they should have one), one will be able to pinpoint bottlenecks and optimise for them.

Your point about hardware differences makes more sense; I don't expect AMD to admit that their hardware is simply not as suited for the task as Nvidia's is.
Arguably Numpy/Pandas is just as performant as Scala/Java and it certainly beats R hands down when data becomes more than a say 10-20 gigabytes after which I find R slows to a crawl.
That's even worse. They will just keep together, and the bus at the back (which was the initial full bus) will have to wait for the bus at the front without any good reason.
Actually Fortran and C++ would be better options than C.
What do you see as a replacement for Awk, then? And what does your Linux whiz use instead of sed and Awk for getting things done?
Hi Lewis.

I really liked this page, but it is consuming too memory and CPU power.
Heh, thanks. That was sort of the start, my first major project outside of programming, outside of anything I had actually been trained in.

I'd always had hobbies of a wide variety, and people always like to parrot that tired "jack of all trades, master of none" line. Seeing Ellsworth's work, it made it feel normal to not only be interested in and doing a bunch of different things, but to try to be good at it all, too.

She's always seemed extremely brilliant, yet ultimately accessible. She's always not only created awesome things, but tried to explain them as well. To try to bring other people along into the fold. We could all use more of that.
But you're assuming that memory is the only thing that matters here.

10x100GB will have 10x the computing power of 1x1TB server.
I believe the idiomatic way to solve multiple writers would be to give each writer it's own channel, and multiplex the output of those to a single channel. The problem is, without generics you need to create a new multiplexer for every type of channel payload.

Also, if semantics make life complicated, they should be changed. Since there's already an 'ok' return flag for channel receive, I have hard time understanding why there can't be one for channel send.
This seems like an interesting, but inefficient solution. (What do you do during the summer when heat isn't needed? How much more do these units cost over a regular computer at a data center? Who is paying for the electricity and bandwidth for these machines?)

Is there some sort of complementary process that could use the heat generated by the data center at the site itself? (energy production, manufacturing, etc)
Flow: Microsoft new chat app based on Email leaked
Useful Modules for Creating Your Own Node.Js Command-Line Interface
Regex 101 is fantastic, I love how it provides a plain english explanation.
Graphics Feature Status

Canvas: Hardware accelerated

Flash: Hardware accelerated

Flash Stage3D: Hardware accelerated

Flash Stage3D Baseline profile: Hardware accelerated

Compositing: Hardware accelerated

Multiple Raster Threads: Enabled

Rasterization: Software only, hardware acceleration unavailable

Threaded Rasterization: Enabled

Video Decode: Hardware accelerated

Video Encode: Hardware accelerated

WebGL: Hardware accelerated
Good luck optimizing for shader code you have no access to, because that is (was?) the situation, at least for some previous games that used it. Not even the developers of the games have any access to it.
Gameworks actually prevents AMDs to optimize paths in their drivers. Exclusivity does not matter a bit.

Note that AMD also has a program that help developers write shaders and optimize their engines to work better on AMD cards. But that does not prevent NVIDIA to see the code and optimize their own drivers accordingly.

So, to me the problem here seems NVIDIA's attitude to sabotage (for no technical reason, except economics, see the physX on x86 debacle when they refused to compile code for something else that i386) performance on other platforms for their own gain.
Like physX, Gameworks is already showing itself for what it is and will eventually fail and be forgotten.
And people will believe that AMD is at fault for the next NVIDIA "magic technology" that brought no gain whatsoever to the gaming world.
So this is the screenshot with HairWorks enabled: https://cdn.arstechnica.net/wp-content/uploads/sites/3/2015/...

Is there a comparable one with it turned off? Because that untextured hair looks completely out of place next to all the other nicely textured & shadowed parts of that scene (well, the chainmail does look amusingly flat where it goes over the shoulder). The hair might move nicely in gameplay, but it looks plain, garish, and low tech in a still.

I suspect this works much better with dark hair, where the texture is not as noticeable as the silhouette.
I took a look around for "high-ram" servers, and it seems one I can buy today, is HP ProLiant DL580 Gen9. With just 256 GB of ram, it clocks in at 540.995,- NOK (71.5k USD). It has 96 ram slots, and I can't seem to find anything bigger than 32 GB DDR4 RAM, and rounding the price up 96x32GB comes to roughly 672.000,- NOK (~90k USD). Adding that up (throwing away the puny ram installed), gets us to a little over double the original price, or 1.212.995,- (~161k USD). This has 4 18 core E7s (72 cores) clocked at 2.5Ghz -- and 3TB of ram (half of max, because of 32 GB dimms).

It is true that while the jump from 256GB to 3TB is "just" ~2x -- I could get a server for 1/10 of the price of the original configuration -- but only with 4GB of RAM, and nowhere near even 18 hardware threads.

If you are CPU limited (even at 72 hw threads) you might need more, smaller servers.

But such a monster should scale "pretty far" I'd say. Does cost about half as much as a small apartment, or one developer/year.
> ...as many hacks into the drivers as Nvidia does.

That sounds like an absolute nightmare to maintain.
Cool idea, how is re-authentication handled beyond the initial sign up? Do you get a text message each time?
I'm not so sure. When shit goes wrong, backups are not directly part of "solving the problem" but they are a part of the process, an essential one that can mitigate losses and provide developers with a clear path forward.

Good article though - and you're right backups probably deserve their own focused article.
Here's the best one I can find: http://ersilia.net/origami/docs/vicroads.pdf

Up to 58% increase in average speed in one test, which was probably the figure I was thinking of (it was a while ago!)
I really enjoyed that book and Harris' approach to meditation. Since Harris is a neuroscientist and an atheist, there's no mysticism in the book. No talk about "feeling the energy".

In addition to the meditation information, he also goes into some neurological theory of thought generation in the hemispheres of the brain that I found quite fascinating.

[edited for clarity]
Installed. Thanks!
You can easily deactivate it. Would I use this for sensitive data? probably not as I use a password vault, but for non sensitive sites this looks like quite a slick approach.
Seems a bit naive, saying 2.1PB probably doesn't fit in ram, "but it could"...

I get who this is aimed at, and why, but just saying that it fits in RAM isn't as useful as it could be. This is an opportunity to teach, not just snark.
Re. Backups:

check out rsync's "--link-dest" option. It's amazing.

Alternatively, this is one thing that version control could help with. Set up a cron job to check for changes every day and commit if needed.
> . If the proprietary drivers are being updated to fix game specific problems, how are the open source drivers going to fair when the general rule for open source is not to have hacks within a code base.

I'm an outsider, so it's easy for me to sit back and make blanket statements. That being said, the responsibility should lie on game developers to insure their games behave properly on hardware. Making proprietary drivers to address specific games is a bad development practice. Hardware markers should be responsible for insuring their drivers are accurately supporting standards.

It just sounds like the whole games industry has fallen into some really bad development practices.
There is a saying that springs to mind, “Never trust a statistic you didn't fake yourself”.
I have the same number, do you think there is a reason why they are not all valid?
EDIT: actually the number I calculate is 2796. Also using Python 2..
Складные межкомнатные двери книжка – шпон
Other solution: don't use a CMS that mixes plugins code with the core code with absolutely no sandboxing or even a separation between the core and the API.
you do realize that bitcoin needs an internet connection to mine?
I disagree. I don't mind if man pages continue to contain all the content they currently have. Just add a section at the top with basic examples and you halve my effort to start using them. That makes them better. Now, if we're picking about the definition of the word 'documentation' - well, I don't care. I just want utility.

It's especially better you lower the barrier to entry into the world of command-line-fu. There's a huge class of people in the world who will never, ever touch a command line tool - it's the "non-DevOps" people in the article. Or you can just call them 'non-engineers'.

I don't blame these people for avoiding them, either, because ours tools are actively hostile to new users. (in my opinion) The linux community frequently takes a stance of being actively hostile to new users, by intentionally making simple tasks difficult and refusing to provide the most basic of hand-holding.

And, of course you can look these things up on the internet, but by that argument we should do away with local man pages entirely. Nah, I'd rather just make the local documentation better.
Our data fits in ram but it proved to have no speed benefit. So the ram just sits there, being empty.
"Your body shape? What doctors don't want you to know"
Fire is not one thing. You have quite different ideas people would have grappled with like observe naturally occurring fire (eg. from lightning, in terms of light and heat properties), tend fire, store fire, make fire, cook on fire, differentiate properties of cooked and non-cooked food, differentiate properties of heated/burned, discover heating's effect on various materials, etc.
Amazon offers some bloody huge servers... 32 core, 256GB RAM, and 48TB HDD space. d2.8x large
What economy? Some new or parallel economy that businesses want, that consumers have no use for.

This data is almost always collected as a side effect of providing a service or product that a consumer wants. That's the economy that exists, and that consumers want.

The data collection is an annoyance inflicted on consumers, that business want to use to drag consumers into an economy that they have no use for.

As the article describes, the data creates an unnecessary risk of data being mis-used (spamvertising) or sold (more spamvertising).

The data can also be stolen, and the more it's sold the more it can be stolen, and the more it's stolen the more it can be sold. And stolen. And sold.

Sell me a fucking toothbrush and leave me alone. If you can't make a profit selling me what I want, please go out of business.
Push notification to your phone. They don't have iOS support yet which counts me out from trying it out.
Hit the Toughest Part First
Users understand the Username/Password workflow.

Every attempt I've seen at improving this fails by introducing some new unexpected workflow complexity such as "sign in with google, fb etc" or "use your phone"

I think devs should instead focus their efforts on making that standard Username/Password flow work as effortlessly as possible.
Depends on outside temperature, if it's cold enough heat pumps become less efficient than simple resistance heating. The transition point where they become useless is generally around 0F which is not all that cold. Granted if you live in a warm area then it's generally not an issue.