skysight-inc/HackerNews huggingface dataset:
Avro files have a header which has metadata including the schema as well as things like compression codec (supports deflate and snappy) and all of the implementations that I have used (java and python bindings mostly) just does this in the background.

Another fun thing is that avro supports union types, so to make things nullable you just union[null, double] or whatever.

But one of the best things about avro (and parquet for that matter) is that it is well supported by the hadoop ecosystem
Very much worth watching. Agile is not a noun. :-)
E should like this
The first time I came across Pieter was when he was interviewed about ZeroMQ on FLOSS weekly. I've come to find his writing to be engaging and informative, and thank him for that.

His series of articles on psychopaths and the havoc they wreak is well worth reading, even if it takes a while to take it all in.

My best lesson I learned on programming from Pieter was to use code generators effectively. The advantages cannot be overstated.

Thank you Pieter, you will be missed.
Apple App Store: How to avoid app rejections
Or they went from 9 signups to 10.
What's so horrible about confessing to your kid that you don't have all the answers?
Likewise in Chicago, the Sears Tower is now technically the Willis Tower.
I've just had this debate with several friends (trying to convince them not to go for PhDs), and overwhelmingly their goal is simply to avoid the job market.

Totally anecdotally, it remains very hard to get an entry-level job unless you have some hard skills. A PhD offers the promise of avoiding the grueling process of applying for jobs and they often (incorrectly) think that it will make them much more employable.
Why Be Serious? ™
I am always mystified by how a country with such diverse geography and socioeconomics can be represented by just two parties.
I think the answer is very simple - a doctorate still packs a lot of prestige. It says that one is Smart, especially nowadays when more people than ever hold a bachelor's.
Agile principles, in an academic sense, are great. However, for agile to work best, it's my belief that the team needs to be of a similar skill level, trust each other implicitly, and have the same value system (i.e., "we believe in writing software for the long-term", "we believe in being disciplined", etc.) Too often this is not the case. Many things are good in theory but once applied to real human systems, fail.
The problem for me is that "much simpler" is not good enough.
With DO for example when I go to their panel I never used it before it will take me tops 1 hour to find all the possible options, all perks and cons, everything. In just 1 hour.

I know, I know, AWS and Google Cloud is not DO and I agree. But would it hurt to introduce DO or Vultr style UX and make it easier for regular user to launch what he wants without all the expert language and need to jump in to documentation for hours?

Would button "Simple" and "Advanced" UI be really that hard to make?

I, as startup owner, dont really have time to focus on all the noise from large cloud providers, its easier to just launch 5 VPS with DO or Vultr and create my own infrastructure. And while I grow I doubt I will move my business to any of the cloud providers because I dont have time for it. Its easier to just build infrastructure on VPS's for $5 or just go for dedicated servers for backend.
> what makes one consciously decide that a set of rules made up by someone else, which may or may not still be relevant, is the best way to conduct one's life?

That may not be the best. But good enough given the circumstances (time, community, energy, freedom, incentives, capacity, will, goals)?

Analogy: what makes you decide that using a fork is (not) the best way to eat some sorts of food? What makes you decide that a pair of trousers is (not) the most appropriate dress for a person? What makes you decide that nudity is (not) appropriate in a given context?

That's a piece of practical, ornemental, societal, cultural refinements made by a set of people, over centuries. And it IS practical/ornemental/societal/cultural. But what if there was an other, universal, better way? What if not? Or what if, but not (yet) in our reach?
PhDs in the UK are different from US PhDs. At least in CS, a US PhD in computer science is probably more like a UK PhD + a postdoc. That means the time-investment for a UK dissertation is lower.
I've wondered this for years. I got a nice job at a University Research Lab. I have lots of creative freedom, can write grants, and am well funded. Most of my colleages are Phds yet most of our customers assume I do. I did an honors undergrad with thesis and a masters. I chose a good adviser and after those experiences, I feel I can easily keep up with other phds here but without having to go through the stress of comps, etc.
You're overthinking it. Also, "interesting" is most neutral I believe. Alternatives: "fun" - too sarcastic; "sadly" - too dramatic.
The reasoning "because we will scale it off-chain, on-chain scaling is a waste of time" must stop because it's so stupid. They are not mutually exclusive.
> Intelligence is useful, so evolutionary pressure has increased it in humans, dolphins, corvids, octopii, etc. Perhaps it just takes time for the pressure to give results - and we just happened to be first?

I buy that. It follows logically. Similarly the argument that we're the dumbest possible creatures that are still able to create science and that it doesn't mean our level is some kind of hard intelligence limit - it's just that at some point we've reached the necessary capabilities that allowed us to start a secondary, much faster evolutionary process - that of technological advances.
So that it can be an integer with an arbitrary length, or a float/double without precision problems. You can also let our own integer classes do the parsing (which might even be able tonhandle complex types).

After all, everything in JSON is a string since it doesn't have a binary format and it shouldn't cause a huge overhead to do the parsing yourself (that might depend on the library, though).
An iPhone app to create lists of map locations
What about doing it in the cloud? E.g. AWS instances which you create and only spin up when you need them...
While some people certainly are motivated by interest in doing pure research, I don't think that's the dominant factor.

From my anecdotal experience, far more are turning to PhDs simply as a way to avoid the brutal job market. They're choosing it for financial reasons, even though their reasons are backed by faulty information.
Isn't this a fundamental change in the economic expectations as proposed by the system's creator, who planned to increase onchain capacity as far back as 2010?

https://bitcointalk.org/index.php?topic=1347.msg15366#msg153...
The most popular talking vehicle, in the US at least, is named Thomas.

This isn't a particularly convincing argument for a silly name. If kids will accept Thomas, surely they'll accept a more serious name than Boaty McBoatface.
3 most important motivations are:

1) Love for science (it is hard to do full-time funded research without a PhD)

2) Prestige

3) Desire to work for the government (some high-profile public service positions make it easier to get in with a PhD).

Nearly nobody is doing a PhD for "employability" (except specific target positions from #3) or financial reward.
Name a few companies who have switched their large data centers off x86 to something else?
Democratizing Kaplan-Meier
I wonder if its part of Panama Leaks counter measures. Make US and SA look bad and make them hostile towards each other to cover up the leaks.
Taking your own life is seen as a mortal sin by most Christian traditions, mortal sin as in there is no forgiveness for this and you go straight to Hell. Not sure about other traditions but in the case of Orthodox and Catholic Church you are, for example, forbidden to be buried in sacred ground if you take your own life.

If you are curious you should read a bit about the history of Christianity, e.g. a Eastern Orthodox Christianity tends to be more entrenched in tradition than modern Catholic Christianity. You don't have to be a believer to read about history.
Maybe they want to explain their kids all details of the illness, all details of dying and suffering - maybe they think it's acceptable for toddlers. Or maybe they are just too young/stupid/naive to understand it.
Oh, so now you decide whats good for HN and whats not?
HN is "democracy", users choose whats right.
If your NPM installs are mysteriously slow in Docker, try adding this line before 'npm install':

RUN npm config set registry https://registry.npmjs.org/

I don't know why it works, but it does.
This is exactly the point - I use forks because I was brought up that way, but I am open to the possibility that they're not the best, and in fact, I'm actively experimenting (in many areas of my life) to find "better" ways.
I often want to delete what seems to be obsolete code, but I am never sure it is going to break something somewhere done the line. My tactic these days is top comment it out, put a data and explanation next to it. If i come across the same commented out code a while later then I can safely delete it.
You mean you want x86 to only have a "legacy software user [sic] case." Your claim is not backed up by reality whatsoever--just ask every Windows user.
Is anyone seriously shocked by this?
It is always worth keeping Bremermann's limit in mind when anyone talks about general AI being impossible. We are so far from it (as humans) that to even have an opinion about what is possible seems silly.
Good thing you read the book and came back with a solid criticism instead of launching a superficial attack on my lack of credentials. Nice work.
Is it just $300 million scientific research vessels that should be given deliberately stupid names (that even the originator disowns), or everything?
You could also tell someone to just run a marathon, I mean it's easy: just move your left feet and then the right, and repeat that for next 26 miles. Where's the problem? Problem, of course, is that most of us lack the physical stamina to run a marathon. Likewise many people lack mental stamina/strength to "just do the work", or "just eat less", or "just quit smoking/drinking/doing drugs"... why is that such a hard concept to grasp?
> different from something you ARE (a fingerprint, a retina, a brain)

How is "something you ARE" a useful classification? A fingerprint can be easily changed. And retina patterns could (probably already can) be faked too.
Unfortunately, this will not surprise many folks here
Wow I didn't even notice that when I commented. That's like a Michael O. Church level of "hey everybody look what I wrote!"
Freddie Mercury: acoustic analysis of voice frequency, vibrato, and subharmonics
LiveLeak.com – Channel: stgschlmk03
Check out this guy's mod of a Game Boy to be a fully functional Super Nintendo
No other entity is recruiting young undergrads like academia does.

The military does.
Learn interactively how to create interactive graphs using GoogleVis and R
Apply for jobs is way easier than candidacy and comps, not to mention a dissertation.

I stopped after my masters, bought a house, and had 2 phd candidate friends move in with me (we live in a city with a university). They tried to tell me that grad school was harder than the real world. I argued that its easier citing my experience in grad school and now the real world. I put in a solid 40 hours a week. In grad school, I was at the office roughly 50-60 but it was largely unproductive. I find that graduate labs encourage bad habits like staying up too late to fix a problem that could be easily fixed in the morning after a good night sleep.

To prove to my friends that grad school was easier than the real world, I offered to drive them to university everyday and pick them up on my way home. I noticed that neither of them worked while at home. After the first week of a straight 40 hours, they were exhausted. After a month of this, they found another ride to school because they said working a straight 8 hour everyday was too much!
I get a decent chunk of the sale price in fact, as I'm my own publisher. Some people like paper books, some like ebooks, some like PDFs.
I'm going to do the thing where I quote sentences from your comment and respond to them individually, which IMO is normally pretty aggressive -- just wanted to say I'm doing that for organization's sake, not for any other reason.

"Most people have several stores for clothes..." I'd argue that most of those are acting like FILO queues anyways, since you typically don't need random access to clothes in a washing machine/from the to-be-washed pile.

"On days that I need to be out of the house particularly early, I will "cache" my next day's clothes..." you're still spending the same time overall ((sum over all day's clothes(time(access clothes)) + time(get dressed)) to get your next day's clothes, so preloading them just reduces the time spent when in the mornings when deadlines are tight. That's a totally valid technique, but it isn't caching as described in the article, which is described as a technique for mitigating the cost of frequently accessing high latency resources, since you (probably) only infrequently access clothes to wear. In that case, the time spent overall is (probably) dominated by the lookup times for single articles of clothing, so excellent organization is (probably) the best way to cut down on overall time. You'd need to profile to determine whether it could be of use, though.

"I've got a cache of stuff that I need on a daily basis..." that's actually quite valid! I had mentioned b-trees ]1] jokingly, but they can be analogized to what you're describing (feel free to skip the next section if you're already familiar with them). Idea in a nutshell: since hard drives are slow and return data in blocks (you can think of it as retrieving shelves of data at a time instead of individual items), it makes sense to pack as much relevant information as possible (spatial locality) [2] into each block (which b-trees do by having a large branching factor and including data inside of nodes). The relationship to placing items in discrete groups around the house is quite similar: physically moving (following a pointer) is quite slow, so once you're somewhere (a block), you want everything of import to be nearby (high locality, no need to follow more pointers). This is _still_ not caching, since you weren't going to sit down and perform some access intensive operations on your wallet and keys in the morning, but it does have some distant relationship to a CS data structure :)

Or, at least, that's my take on it.

edit: I'd just like to mention that I don't follow any of that advice and just use a roughly sorted laundry bin to keep most of my clothes in a single place, if not in any particular order.

[1]: https://en.wikipedia.org/wiki/B-tree

[2]: https://en.wikipedia.org/wiki/Locality_of_reference#Types_of...
Ask HN: Which analytics do you use on apps?
Hey, I'm using google analytics, but often UTM are invalid. Give me advice please.
The timing couldn't be better.
I was curious about a source, especially since it seems like it was declassified almost a year ago. The same papers are available from .gov as:

https://www.archives.gov/declassification/iscap/pdf/2012-048...

The index page,

https://www.archives.gov/declassification/iscap/pdf/2012-048...

calls it "Saudi Notes". There's one other set of "Saudi Notes" and some "Saudi News Clips" on that page.

Does anyone know what the declassification process was for these / why they were declassified?
Apply HN: Prime finds your best time to post to social media
Problem: Platforms such as Twitter and Instagram have average engagement rates lower than 5%. That means that all the time you spend preparing amazing posts and building a huge following is put to waste, because many of your followers are never actually seeing your content.

Solution: Prime’s patent pending algorithm analyzes tens, even hundreds, of thousands of personalized data points at rapid speeds. This is specifically useful for brands and companies with large international audiences across many different timezones.

Prime's currently launched on iOS for Instagram (www.primeforinstagram.com). We've seen engagement rates organically increase by over 30% for users who post at their "Prime time" as calculated by Prime. Prime was featured on the home page of Product Hunt and has since also had some really great reviews on social media marketing blogs alongside platforms like Buffer (blogs including PH Creative and Netted). We are now really focused on expanding our user base and working with more global brands.

We're looking for initial funding to build our team and expand the reach of our product; namely, to Twitter and a business based subscription service (which would include deeper analysis, more frequently updating information, and monthly reports for businesses as a form of monetization). We're also talking with a number of the biggest social media platforms about licensing Prime's powerful algorithm.
Google is at least testing OpenPower in their data centers. I don't know how serious they are, but people are starting to explore x86 alternative more.

http://www.nextplatform.com/2015/03/18/google-looks-ahead-to...
Apple is a Microsoft OEM
This is hard to parse without any commentary. What should I, as an average person, take from this? Should I start to think Saudi Arabia as a terrorist country or is there room for interpretation?

I'm very sure, by intuition, that this is important but probably would have been more valuable with some opinions from people who are knowledgeable on the subject.
> Will it still work when I'm drunk?

Not working when drunk a potentially useful feature in some areas.
FSJS #46 – Best from JavaScript world (weekly digest)
I did my PhD in theoretical chemistry/biophysics in the late 90s. I was funded throughout so didn't have to take on any debt (I was married for the last 3 years). I learnt a lot during those years including how to really think about a problem, design experiments, and collaborate with the bench scientists on the team. I also fell in love with protein structure, got to understand spectroscopy, the electronic structure of vision, and got to run stuff on some of the biggest supercomputers in the US. PhDs require (IMO) a love of your field, lots of discipline, and tons of patience. I never wanted to be an academic and worked in the scientific industry for 8 years at startups and BigCos (5 of them in product management). The last 8 years I've done something else (although a lot of my interest came from my HPC experience) but have never lost touch with my roots.

Regrets? None. Every experience has been completely worth it.

Many of my friends have done PhDs and postdocs. Some are academics, some lead large research teams, others R&D teams at biotech startups, and some have moved on to different fields like me. Almost none regret doing what they do or the path they've chosen. Sure there is a lot to change with the system and too many PhDs in some fields, but a doctorate was a rewarding experience for me and for a good chunk of those I know who have one.
Great video to help OO programmers begin to think functionally
Not calling it Boaty McBoatface will just be a massive missed opportunity for effective science outreach and a crappy PR move.

You say it's going to be doing important research? Great! Add a social media presence, pull in people with the "funny name" and keep them for the exciting and interesting research.

But you're not going to that, because you're too important for that, ... because you're snobs.
Mixpanel along with internal dashboard to monitor custom tracking URLs.
I'm not from the 1st world, so it adds a bit of a darker attitude. To me it looks pretty insane that you earn so much money, have so high quality of life and yet you don't invest nearly as much as possible into life extension. Instead you buy big overpriced houses and other weird stuff.

Also looks like the mods banned me.
Bangladesh Bank exposed to hackers by cheap switches, no firewall
Or your provider, or Opera, or own server with VPN. For majority of people setting up own server is too complicated, so it's much better than nothing, especially when you are in place where you can use only non-secure wifi network.
And penicillin is a classic "recognize what you have found" example, scientists prior to Alexander Fleming had noted and wrote up observations of molds killing cultured bacteria, but just treated that as a curiosity. He, at least, was actively looking for anti-bacterial agents after his experiences in WWI battlefield hospitals.
> My best lesson I learned on programming from Pieter was to use code generators effectively.

Likely among other places, Pieter discusses code generators here: http://download.imatix.com/mop/introduction.html

> Thank you Pieter, you will be missed.

Agreed. I posted my thoughts on his blog, although the hundreds of comments on this page will also stand as a testament to Pieter's work and the lives that he touched.
This video provides an excellent snapshot of what Prime does: https://www.youtube.com/watch?v=EbOugpVhR6w

What's your Prime time? We'd love to hear your feedback!
I actually have the opposite reaction to the spring. When I get 5+ minutes extra per day of light, I'm ecstatic, happy all the time. October and November are always brutal on me.

I'm only at 43-ish these days, but I've lived as far north as Edinburgh, Scotland (56ish). I had to explicitly get outside every day at lunch to get as much sun as I could.

Seasonal Affective Disorder is apparently a real thing.
This document is different from the 28 pages that remain classified. For more info on that, read here:

https://28pages.org/faq/
So they are trying to make it learn how to go on a killing spree. Doesn't sound like such a great plan.
> I think that we can build a device that can capture a brainwave et replay it.

I want such a device that can play back into my own brain.
Let THEM omit frame pointers for a decade or two, it is in the broader national interests of America.
I hope when $NAME_OF_VESSEL crosses the Arctic Circle, they refer to themselves as Boaty McBoatface, at least for the day.

https://en.wikipedia.org/wiki/Line-crossing_ceremony
This fantastic iFixit (Step 10) has all the details you need:

https://www.ifixit.com/Teardown/Dell+XPS+13+Teardown/36157#s...
In the experiment, they used only 50 subjects and they were able to distinguish between all of them. For me, it's not clear that the "100%" will be maintained when they use a larger group.

Also, in the previous experiment they used 32 subjects and got a "97%". I'm not sure what that means, but I guess it's 31/32. So perhaps they reduced the error rate or perhaps they were lucky and didn't get the problematic subject / collision.
Well according to this other article, Intel and AMD's patent cross-license expired in 2015[1]. I'm not sure if that was the only agreement they had relating to x86 IP, but if so that makes this deal more likely to go through, if they can get past the export controls.

1. http://www.eetimes.com/document.asp?doc_id=1329517
10 Best Tools to improve your content marketing efficiency
For more Read at: http://www.inspirationblogs.com/tech/10-best-tools-improving-content-marketing-efficiency/
Apes can be taught sign language (they are anatomically prevented from developing speech), and not all human languages have the same power (probably no recursion in the the Pirahã language).

[0] https://en.wikipedia.org/wiki/Pirah%C3%A3_language
Well, we knew almost immediately after the attacks that 15 of the 19 attackers were citizens of Saudi Arabia.

From skimming the document quickly, I don't see anything that implies definite ties to the Saudi government, which would be interesting. They knew people who worked for the Saudi government, but I don't think this document says anything more than that.
OpenTSDB RCE