skysight-inc/HackerNews huggingface dataset:
We've struggled with the same issues. We also keep a "base" schema in Redshift as a close mirror to source systems (tweaked a little for distribution keys). In our case we control the extracts but schema changes have to be manually added. If a new column is important and we miss it then we have to run time consuming backfills. Then only way we've been able to ameliorate it somewhat is to be more involved in the upstream release process but that is time consuming in itself.

The solution I'd like to implement is having the extracts and loads be built dynamically whenever a change is detected. E.g. have a process that runs separately checking the schema of our sources for changes, dynamically updating the ETL processes and then publishing them to Azkaban. Looking forward though our biggest data sources are going non-relational so that approach will entail a great deal more complexity.

Regarding schema knowledge / metadata, this is a perennial problem. I once worked on a huge Teradata DW system at a global bank who had a large internal website for just this purpose which was hugely out of date at all times.

When I started building out the analytic capability at my current company I made a fork of "Rocco" (https://github.com/joeharris76/rocco) for doing inline documentation as Markdown inside SQL block comments. http://pow.cx/docs/ is a good example of what I was hoping to achieve. However even our smallest components have many, many files and the docs were too difficult to navigate using the "Jump To…" dropdown menu.

So, while this produces some very nice docs, sadly they turned out to be largely unhelpful to anyone who didn't already understand the code. Our current approach is to try and keep individual components small, include a comprehensive README in each component and use Slack comms to fill in the gaps.
Climbing the tether will change the center of mass and also increase the rate of rotation. It's going to be complicated.
sure but you could say the same thing about tweets or 1-5 star restaurant reviews
Time to prepare for the unthinkable
Maybe. But those were the early stages of the language. Python 2 has huge amounts of industrial-scale deployment that many people would prefer not to break. There are also numerous systems in deployment that need to have 2 and 3 running simultaneously on the same machine.
Completely agree: Pinterest feels very much to me like a "list" application that is a huge breakout success.
Yeah, the Apple Notes app is a perfectly sufficient app for making most lists, and it automatically syncs to all of your devices. That's just one fantastic free option.
Ask HN: Fun DIY projects to do with a month free?
For the next month, I have essentially nothing to do, and I was interested in working on a project, preferably one that ties together engineering and coding. I'm a competent coder, but only have a little electronics or engineering experience. There's a hard limit of $350 for the project, though the cheaper the better.

I was thinking of building and coding a magic mirror[1], but I'd love to hear any suggestions you have!

[1]: http://michaelteeuw.nl/tagged/magicmirror
Ah, thanks for clearing this up!
I don't have any specific studies, but I have read about human cognitive biases in general (Thinking Fast and Slow, Freakonomics), have talked to pregnant women going through the interview process, and have myself interviewed as a pregnant woman over the past couple years. I've also read countless forums (Corporette posts[1], the excellent Ask A Manager[2] blog, Laurie Ruettimann [3] that talk specifically about workplace psychology in a non-clinical way.

The hiring manager likes to think they are not biased, but they always are, in the same way all humans are. Be it by names in resumes [4], or because employees experience a "cultural mismatch", we are imperfect as humans and will judge a person just as much as we judge credentials or resumes.

The best hiring managers will recognize that this is a Thing That Exists and try to actively overcome these biases. But the world is not made up of best hiring managers. It is made up of people who may or may not assume that because you are pregnant you will not do as much work, or they don't want to give you leave or deal with FMLA, or they don't want to deal with you potentially not coming back after your leave is over.

This is a constant fear for women, this fear of being judged on our childbearing potential, and comes up over and over again in both motherhood forums and professional work forums, as well as more recently in the national conversation with the likes of Lean In, Anna Marie Slaughter, and others.

I've had conversations with women who are worried that they should remove their engagement and wedding ring in case the employer sees it and assumes the woman is newly married and wants to have children right away, ruining the amount of time she might put into the career. Other women are terrified of losing their jobs after coming back from a maternity leave where men or non-pregnant coworkers handle all of the business and they suddenly feel they are not needed.

To anyone unfamiliar with how hiring and workplace psychology work, these concerns seem far-fetched. To many women working today, they are an unfortunate reality, although one that doesn't exist everywhere and is starting to shift, hopefully, with things like an increasing number of more flexible parental leave policies.

So, don't mention anything until you're far along enough in the process that it's time for both parties to evaluate whether they want to work with each other (i.e. the offer stage). Then it's 100% worth putting on the table.

[1]http://corporette.com/
[2]http://www.askamanager.org/
[3] http://laurieruettimann.com/
[4]http://freakonomics.com/2013/04/08/how-much-does-your-name-m...
Right, that was from 2010, post great recession, and it indicated that the US held 39% even then. US assets have massively increased since then.
There's a lot of grumpy sarcasm in this thread! I wonder, might a violent primate story stir some innate tendency towards violence in humans?

Or maybe it's just monday during holiday season and that has people rumpled. Or maybe too much of the internet is leaking into HN these days ... :(
Rent is high because housing is scarce AND people can afford high rent. Like you say you can't get employees there if they can't afford it, but that also means that if no one could afford it there would be no demand at that price. Landlords will absolutely try to extract every last drop of economic surplus out of their properties.
The quick way to remember this is that hashes naturally consume and supply lists. When a hash consumes a list, it alternates to assigning list items as keys and values. When a hash is supplied on the right hand side of an assignment, it's keys and values are supplies as a list in alternating order. Values for a hash or array must be a regular variable (scalar), or a reference to some item (such as an object, hash, array or scalar).
An alternative one: https://c1.goote.ch/3e2311a93339448a9d19f21c7368aa01.scene/
Fossil fuel subsidies more than four times renewables
I guess there isn't a "serious" Dali.
Yeah, but boards don't really take center stage in Pinterest's UI. Feeds and search results are really an amalgam of various pins and boards.

If I wanted to see a board of "25 Great Christmas Gifts for Under $25", it's not obvious where to look or whose board is authoritative.
Does this have an affect on the ability of wealthy Chinese people to move their money out of China?
Tightening Your Belt
> removing leader unnaturally

Not sure what that means. The matriarch was essentially kidnapped and killed by a large predator, for all practical purposes. That's quite natural, actually.
But Halle Berry has to pay child support to all four of her former hubbies.
All true which make the linked survey results at

http://nchfp.uga.edu/educators/natl_survey_summary.html

pretty ridiculous. I can stuff, but I follow the Ball book / USDA rules. Oddly enough everything turns out great and no one gets sick.

Honestly some of the USDA statistics are so ridiculous that I'd suspect they're getting trolled.
Electric cars have a complex relationship with electrical prices, since they are effectively large batteries with wheels, and this makes them ideal for managing demand response. Particularly overnight, fleets of busses, taxis, corporate car pools and home consumers can charge at whatever hour suits the grid best.
https://delicious.com

Started as a way to make lists for yourself, turned into shared lists.

Heck, reddit's original idea was pitched as Delicious with voting, so you could say reddit fits this group too.
Wow, Google Books is awful on iPad. Completely unusable.
Think of Perl5 and Perl6 as American English and British English. Both languages are English, but essentially they are different languages but with a similar foundation.

> Confusing? Almost certainly.

Agreed. I don't know if it would've been better for Perl6 to not have 'Perl' in its name, but it almost surely would have been better for Perl5 if Perl6 hadn't had 'Perl' in its name.
You are correct, currency and financial systems today are vastly different. That site seems to be pushing the "THE PETROLDOLLAR IS DYING ANY DAY NOW" agenda.
Why on earth would anyone bother paying for a list app, when there are so many ways to do it for free? The best you could do is try to write something really excellent and get bought as a replacement for one of ${GIANTCO}'s existing list products. So best case for a team is probably an acquihire, and it'd require a fair investment for scaling at zero revenue to get even that.
>>You demanded I be "completely removed from any day to day activities at Mozilla" & got your wish. I'm still unemployed. How're you?

https://twitter.com/brendaneich/status/583792067156946944
I know a few people who pride themselves on their unblinking candor. Most of them, in reality, just get a kick out of constantly shitting on people and they use "candor" as cover for their behavior.

Here is a good rule of thumb: Is telling someone something hard going to directly benefit them, you, or someone else their behavior is affecting? The go for it and give them constructive and actionable feedback. Just telling them they blew it, they are pissing you off, etc is just shitting on them.

OTOH, not giving people hard feedback because it is awkward is a betrayal of the relationship you have with them because you aren't helping them be better or at least understand the situation they are in when you could have just to save yourself from your own squeamishness.
so if I use Alooma android SDK, I can push event data straight into RedShift ?

I'm not very clear about how to use Alooma.
We detached this subthread from https://news.ycombinator.com/item?id=10650774 and marked it off-topic.
Why? Because people drink soda at restaurants?
Show HN: Real-time ephemeral file sharing
Real Life Heroes
Show HN: Nodulator – Complete Node.js Framework for RESTful APIs (LiveScript)
This is not, as billed, about building a custom analytics solution.

This is SQL as BI, which is great for small technical teams, but hardly a good or scalable solution.

If anything, this is a vague argument to have a data warehouse. Data warehousing is only an intermediate step in any analytics or BI solution and the hard problems are not in moving data or writing one-off queries.
I understand cobalt is a big cost issue and is not plentiful.
Which both need significant network effects to be useful to people.

How does the list making app get exponentially better the more people are using it?
Nothing here about "what you learned". This is a straight up tutorial on how to use your product!
تور کیش ورود ۱۳ آذر ۹۴ هتل پارمیدا کیش
> There are also numerous systems in deployment that need to have 2 and 3 running simultaneously on the same machine.

There were numerous systems that needed both 1 and 2 in the Redhat ecosystem. I agree it likely won't happen again, but mostly because the Linux distros have likely learned their lesson. It's the package managers and Linux distros with included system functionality written in some version of Python which have the ultimate say, and there are much better solutions for the problem now than when Python 2 came out. That said, it's possible some distro will make /usr/bin/python be Python 3 and /usr/bin/python2 be Python 2, if they haven't already. Weirder stuff happens all the time.
The National Security Letter spy tool has been uncloaked
workflowy is the best list and mostly underappreciated app.
Free Tuition Is Not the Answer
Donations to the Mozilla Foundation support work in the realms of education and advocacy. More specifically, this means gifts support the free, open source tools and curriculum at teach.mozilla.org, which teach skills like privacy, security, HTML, CSS and more. These activities and products are used in classrooms around the world.

A couple more examples: Thimble, our educational code editor (mzl.la/thimble), and Webmaker, our Android app for mobile-first Web users (mzl.la/webmaker).

Donor support also helps Mozilla fight for pressing issues like net neutrality and mass surveillance reform. Mozilla's community of donors provided the resources we needed to support a successful campaign for net neutrality in the U.S. this year, plus supported other advocacy campaigns internationally.

-Kevin (from the Mozilla Foundation)
Definitely go for the interview. You have no reason to discuss your pregnancy with interviewers. If your pregnancy is obvious and the interviewers bring it up (as a kind of ice-breaking "small talk"), don't be alarmed, just play it cool, as if they're commenting on your hat.

Remember, Google is a big company, and unless you're being hired for a very specific position, none of those people who interview you will likely be in your team when you join. So, "What happens after she delivers a baby?" is not really the interviewers' concern.

If you're really worried, you could tell the recruiter/HR person after the interview is over and they contact you. I don't expect anything other than "Oh, congratulations!" But if (God forbid) any one of them is stupid enough to say "Oh really? Now we have to re-think your application.", then you have a story to tell HN, and hopefully somebody will be fired from Google.

(I mean, statistically speaking, none of the people you meet during the interview process will be in the same team as you when you join, so I don't see why anyone would be "worried" about your pregnancy. A competent new hire is still a competent new hire even if she has to delay joining the company for, say, half a year, and Google has so many employees that "one hired person will not be available for several months" shouldn't be a problem for anyone.)

- Ex-Googler

* p.s. If you do get rejected, don't assume it's because of your pregnancy, even if somebody commented on it during the interview. Chances are that such comment wouldn't even make it into the interview feedback (if if they do, they will be summarily ignored). Lots of really great people get rejected from Google for mysterious reasons all the time.
Normalize for the true Yelp rating scale of 4 - 4.5 stars. Filter out anything below a 4 star rating. If a small business is unable achieve 4 stars on Yelp, then something must be really wrong.

My experience is that higher star ratings on Yelp equate mainly to presentation over taste and that higher prices tend to mean a better rating (Out of a few average Chinese food places, the higher priced one will be rated the highest).
Probably the biggest challenge I had in my last startup was setting up the analytics platform. Obviously building an analytics platform wasn't the top priority (probably the last).

So you put Google Analytics on your website, then you need the tracking code from Facebook for the ads. And you add Flurry because it provides different analytics (e.g: Funnels and Cohort Analysis). We also logged on the DB some actions difficult to capture on analytics. Then you need mobile ads, so you add in your app the framework to track the install.
And, I forgot, you have another platform to track your mailing lists.

The final result: for every business question you have data in at least 3 different platform that gives you three different answers. Even worst, when you are still struggling with the product your "hard tech" co-founder could not understand why you push to re-implement the analytics framework (not necessarily building it in house) while your business co-founder decide to don't ask questions and doesn't understand why you want better data. If it looks like a recipe for disaster is because it is.

We really need better analytics platform, easy to use like Google Analytics and MixPanel for the beginning but with the ability to build custom analysis for the growing biz.
Yelp reviews contain more content than any other service, whereas most google reviews I read that have anything more than a star rating come from illiterate or barely-literate foreign reviewers. My wife had a time period when every one of her reviews on yelp was flagged even though she put a ton of effort into her writing, whereas I never had a problem. Their spam/business-owner detection system catches a lot of false-positives, but that doesn't make the ones that do make it through necessarily worse. Verifying an email address and linking with facebook (although not necessary), and having friends helps reviews on yelp not get auto-flagged.

I usually dive into reviews and see what people are saying, not just go by a star rating. It adds to the hilarity when you read a 1-star review for a restaurant because they "are scumbags who serve lion" (actually, a menu item had dandelions on the side) or because someone drove by and saw a balkan restaurant "roasting a dog on a stick" (actually, it was lamb, but I had a great laugh).
It's actually hilarious that your SDK's are just modified versions of Mixpanels'...

https://www.alooma.com/docs/integration/ios-sdk
I was curious about pruno, and found this poem on the way:

http://www.pen.org/poetry/recipe-prison-pruno
Ask HN: Encrypted Email Service. What Do You Use? Why?
Yes, exactly. You'll push events from Alooma's Android SDK and they'll land in your Redshift in seconds.
You can have a look at the documentation here:
https://www.alooma.com/docs/integration/android-sdk
The flip side of this is being able to accept constructive criticism. While Scott learned a lot about how to be a manager herself, the true takeaway here is probably more that Scott was able to recognize and accept Sheryl's valid criticism. Just because someone tells me the truth -- candidly or not! -- doesn't mean I'm going to listen. There has to be a level of respect and trust in that relationship. I have to believe them when they say that.

I'm terrible at accepting feedback like this from my friends, but when I met my old boss for lunch last week, he gave me almost the same advice they did. And somehow it stuck. I'm self-aware enough to realize that I put a lot of stock and respect into what he thinks of me and I trust his opinion, more than I trust my friends who aren't necessarily tech people or managers.

And maybe that's another fault within myself. My friends clearly gave me solid advice, but it took someone else to give it to me again before it stuck. How can I learn from this and possibly take that advice sooner?

I appreciate Scott's story, and I absolutely agree with her conclusions across the board, but I think it's only a small piece of the overall puzzle.
Was going to suggest Google BigQuery (faster, cheaper, more scalable) but then looked at the rest of your workflow, and it looks like ETL would get more complicated, and the visualization options wouldn't be as extensive (although BQ does seem to support Tableau). On balance, not worth the hassle.

Looks pretty neat, job well done!
Also worth noting, unlike Black men, multiple studies show that Black women (across countries) are the least sexually desirable race/gender combo for straight men. I've linked to one source from OkCupid below.

http://blog.okcupid.com/index.php/your-race-affects-whether-...

To speculate, IMO its in large part due to physical differences between races - the small petite Asian frames being advantageous for Asian women and disadvantageous for Asian men, and the large Black frames being advantageous for Black men, and disadvantageous for Black women. Societal views definitely play a role too. It would be interesting to compare across different societies.
Always great to see nostalgia articles that are about technology old enough to feel like it is old to me (Amiga and Atari ST are modern systems to me..).

Also some of us might have a PDP-11 in the closet somewhere.
Children's Imagined Robotos
Silly article, but isn't Trello doing pretty well? They take it to the next level by making lists of lists of lists accessible.
Although it can be overstated, the real question (modulo the fact that, for geographic reasons, some cities and areas are better able to add housing stock than others) is why companies are willing to effectively channel large amounts of money to landlords and home sellers in certain areas of the country in order to be located there. I'm not sure the article makes an especially coherent argument on this particular point other than observing a trend toward more centralized industries.
The second type of list this article mentions, social lists so to speak, is definitely valuable to users and probably isn't receiving the specific attention it deserves. Foursquare as a "public itinerary" for a trip through select art galleries in NYC is a great example. Informal, public education could also be orchestrated with these "social lists"; reading material / exercises listed out in a syllabus and shared among interested parties.

Something that I think is pretty silly is the novelization of the fact that there are not many "single user list apps" that have transcended themselves and become "social / public list apps". I think that's because a whole heck of a lot of people will write lists on platforms which are not list-centric, ie: a physical notepad, text messages / emails sent to yourself, nodepad smartphone apps, or even just notes written on your freaking forearm in sharpie!...

Lists are an interesting topic though, and when you think about it, an enormous portion of the internet really only functions as a list delivery system. Lists of popular links (Reddit, hackernews), lists of friend's pictures (Instagram), lists of friend's life updates (Facebook, Twitter et.al), lists of categorized media (Pintrest, Tumblr), lists of queried results (google, duckduckgo, search et.al)...

But I gotta say that lists as an integral motif of IT is a really cool concept.
I've read all three books listed by the author (K&R, Expert C Programming - Deep C Secrets, and C - A Reference Manual), and although these really help make sense of how C declarations work, the best "trick" that I've come across is the Clockwise/Spiral Rule: http://c-faq.com/decl/spiral.anderson.html

Someone showed me this trick right after college, and it has saved me in numerous scenarios. That and http://cdecl.org/
Visual Studio 2015 Update 1
Efficient Object Localization Using Convolutional Networks (NYU) – At Wikitract
Improving your build times with IncrediBuild and Visual Studio 2015
And sometimes, such as content from a poetry focused Pintrest account, a Pintrest board with a list of pictures of words...
iOS Onboarding Without Signup Screens
'Use More Expressive Words' Teachers Bark, Beseech, Implore